SwadhyAI Notes

All DSA notes

DSA Sorting

Counting Sort

Exam scores are integers from 0 to 100. Why pay n log n comparisons when the entire key universe fits in a 101-slot table?

Counting sort replaces comparisons with direct addressing on a . With placement it becomes a building block for radix sort. As a , it can run in Theta(n + k) when k is affordable.

Algorithm journey

1

Measure the range

k = max - min + 1 must fit memory.

2

Count frequencies

Offset keys into [0, k).

3

Prefix boundaries

Convert counts into exclusive end indices.

4

Stable place

Scan right-to-left into the output.

What actually happens

Naive idea: comparison-sort any integers. Correct always; wasteful when k is much smaller than n log n.

Better idea: count, prefix, place. Domain assumptions are the contract.

It breaks when k is huge (IDs up to 10^9 with n = 100), when keys are not integers, or when you forget stability and feed radix sort a broken digit pass.

Invariant

After the prefix pass, count[j] is the exclusive right boundary for key (min + j); each stable placement decrements that boundary and writes one record.

How to choose counting sort

1. Are keys integers with small k? Yes: ages, bytes, ratings, small categories.

2. Is k on the order of n or smaller? That is the sweet spot. If k is huge, prefer comparison or radix with digit grouping.

3. Need stability for composite keys? Use the reverse-scan placement; it is required for LSD radix.

Quantitative check

n = 10^7 scores with k = 101: time dominated by a few linear passes over n, plus tiny k work. The same n over k = 10^9 is impossible: a 4-byte counter array alone is about 4 GB.

When the model fits

Bounded integer key

Ages 0-120, exam scores 0-100, byte values, ratings, or objects with a small integer category are natural inputs.

Dense enough range

The algorithm touches all k counters even when most are zero. It shines when k is on the order of n, or otherwise comfortably small.

Payload can travel

Records need not be integers themselves. Count their bounded integer keys, then move whole records into stable output positions.

Range check first: sorting 100 values from 0 to 10⁹ would initialize roughly one billion counters. A 4-byte counter array alone is about 4 GB, before runtime overhead.

Interactive: where stability comes from

Step through records with keys from −2 to 1. Watch frequencies become prefix boundaries, then follow the reverse scan that preserves the A/C/F and B/E orders.

Stable records, including negative keys

Frequency → prefix boundary → output slot

The letter is payload. Sorting uses only the numeric key, so preserved letter order makes stability visible.

Offset index

key − min = key + 2

Map the range

Step 1 / 17

Input records

Frequency table

array index = key − min

Stable output

The key interval is [-2, 1], so k = 4. Map key x to count[x − (-2)].

Invariant

No records have been processed; every frequency is zero.

Original visualizer · preserved

Interactive counting sort demo

Follow frequency counting and the reconstruction of values in increasing key order.

Speed:
ℹ️ Ready
Input Array
Count Array (Index = Value)
Output Array

The three transformations

1

Count frequencies

count[x − min] += 1

Offsetting by min maps negative or shifted keys into zero-based array indices. After the scan, count[j] is the exact frequency of key min + j.

2

Accumulate prefixes

count[j] += count[j − 1]

Now count[j] equals the number of records whose key is at most min + j. It is the exclusive right boundary of that key's output block.

3

Place from the right

output[−−count[j]] = item

Later equal records claim later slots first. Moving backward through the input therefore preserves the original left-to-right order among equal keys.

Stable implementation

This version handles arbitrary integer ranges by offsetting with the minimum. For records, replace value with key(record) in the counting and placement steps, and store the record in output.

Python

python
def counting_sort(items):
    """Stable counting sort for integer keys, including negatives."""
    if not items:
        return []

    low, high = min(items), max(items)
    k = high - low + 1
    count = [0] * k

    # 1. Frequencies. Offset maps [low, high] to [0, k).
    for value in items:
        count[value - low] += 1

    # 2. Prefix sums become exclusive right boundaries.
    for i in range(1, k):
        count[i] += count[i - 1]

    # 3. Right-to-left placement makes equal keys stable.
    output = [0] * len(items)
    for value in reversed(items):
        bucket = value - low
        count[bucket] -= 1
        output[count[bucket]] = value

    return output

Java

java
class CountingSort {
    static int[] sort(int[] input) {
        if (input.length == 0) return new int[0];

        int min = input[0], max = input[0];
        for (int value : input) {
            min = Math.min(min, value);
            max = Math.max(max, value);
        }

        // Compute in long: max - min can overflow an int.
        long range = (long) max - min + 1;
        if (range > Integer.MAX_VALUE) {
            throw new IllegalArgumentException("Key range is too large");
        }

        int[] count = new int[(int) range];
        for (int value : input) count[value - min]++;
        for (int i = 1; i < count.length; i++) count[i] += count[i - 1];

        int[] output = new int[input.length];
        for (int i = input.length - 1; i >= 0; i--) {
            int bucket = input[i] - min;
            output[--count[bucket]] = input[i];
        }
        return output;
    }
}

Why the algorithm is correct

Frequency fact

The first scan increments exactly one counter for every input record, so no key occurrence is lost or invented.

Position fact

Prefix count C[j] equals |{x : key(x) ≤ min + j}|. Therefore the block ending at C[j] − 1 contains exactly the records with key min + j.

Stability fact

For equal records i < j, the reverse scan places j first at the rightmost free slot. Then i receives a smaller slot, so i still precedes j.

Scanning the input left-to-right with the same decrementing right boundary would reverse equal records. A frequency-only reconstruction: write key x exactly count[x] times: sorts primitive values but does not preserve record payload order.

Complexity, with every term exposed

WorkCostWhy
Find min/max + countΘ(n)Two linear scans, often fused or close in practice.
Initialize + prefix countersΘ(k)Every key slot is allocated and visited, including zero-frequency keys.
Stable placementΘ(n)Each record is copied once into output.
Total timeΘ(n + k)Best, average, and worst case under the same range.
Auxiliary spaceΘ(n + k)n output slots plus k counters. Excluding returned output, some texts report Θ(k).

The usual “linear-time” description assumes k = O(n). If k ≫ n log n, initializing the counter range can already cost more than a comparison sort.

What the comparison lower bound does, and does not: say

Why Ω(n log n) is bypassed

The decision-tree lower bound applies to algorithms that learn order only by comparing pairs. Counting sort uses a stronger operation: a key directly addresses one of k known slots.

Where the cost reappears

The key-universe assumption appears as the k time and memory term. Unbounded objects, huge sparse IDs, or exact real numbers do not provide an affordable direct-address table.

Extensions and escape hatches

Negative and shifted keys

Use index key − min and allocate max − min + 1 counters. Compute the range in a wider type so subtraction cannot overflow.

Objects and stable multi-key sorting

Move whole records using a bounded integer key. Stability lets a later counting-sort pass refine a previously established secondary order: the engine behind LSD radix sort.

Sparse keys

A hash map can count only present keys, but its keys must then be ordered: O(n + u log u) for u distinct keys. Often a standard comparison sort is simpler and more memory-friendly.

Decimals and real numbers

Scaling known fixed-precision values can create integer keys, but arbitrary floats cannot be indexed directly. Quantization changes equality and must be part of the data contract.

Failure checklist

  • Empty input: do not call min/max before returning.
  • Range overflow: calculate max − min + 1 in a wider type.
  • Hidden instability: test tagged equal keys, not only repeated primitive numbers.
  • Allocation pressure: validate k before creating the count array.
  • Wrong complexity claim: include both n and k, plus output storage.
  • Changing semantics: never bucket floats or timestamps approximately unless the specification allows it.

Exercises

  1. For keys in [-2, 1], show the count array after frequencies and after prefix sums for input [1, -2, 1, 0].
  2. Why does a left-to-right placement into the same boundaries lose stability?
  3. State a concrete (n, k) where counting loses to mergesort on both time and memory.

Primary sources

  • Cormen et al., Introduction to Algorithms (CLRS): counting sort.
  • Knuth, The Art of Computer Programming, Vol. 3: distribution counting.
  • Radix sort prerequisites: stable counting as a digit pass (same CLRS chapter family).

Decision rule

Choose counting sort when keys map losslessly to a compact integer interval and n + k work plus n + k auxiliary storage fits comfortably. Otherwise prefer radix sort for bounded-width keys or a comparison sort for general data.