DSA Sorting
Counting Sort
Exam scores are integers from 0 to 100. Why pay n log n comparisons when the entire key universe fits in a 101-slot table?
Counting sort replaces comparisons with direct addressing on a . With placement it becomes a building block for radix sort. As a , it can run in Theta(n + k) when k is affordable.
Algorithm journey
Measure the range
k = max - min + 1 must fit memory.
Count frequencies
Offset keys into [0, k).
Prefix boundaries
Convert counts into exclusive end indices.
Stable place
Scan right-to-left into the output.
What actually happens
Naive idea: comparison-sort any integers. Correct always; wasteful when k is much smaller than n log n.
Better idea: count, prefix, place. Domain assumptions are the contract.
It breaks when k is huge (IDs up to 10^9 with n = 100), when keys are not integers, or when you forget stability and feed radix sort a broken digit pass.
Invariant
After the prefix pass, count[j] is the exclusive right boundary for key (min + j); each stable placement decrements that boundary and writes one record.
How to choose counting sort
1. Are keys integers with small k? Yes: ages, bytes, ratings, small categories.
2. Is k on the order of n or smaller? That is the sweet spot. If k is huge, prefer comparison or radix with digit grouping.
3. Need stability for composite keys? Use the reverse-scan placement; it is required for LSD radix.
Quantitative check
n = 10^7 scores with k = 101: time dominated by a few linear passes over n, plus tiny k work. The same n over k = 10^9 is impossible: a 4-byte counter array alone is about 4 GB.
When the model fits
Bounded integer key
Ages 0-120, exam scores 0-100, byte values, ratings, or objects with a small integer category are natural inputs.
Dense enough range
The algorithm touches all k counters even when most are zero. It shines when k is on the order of n, or otherwise comfortably small.
Payload can travel
Records need not be integers themselves. Count their bounded integer keys, then move whole records into stable output positions.
0 to 10⁹ would initialize roughly one billion counters. A 4-byte counter array alone is about 4 GB, before runtime overhead.Interactive: where stability comes from
Step through records with keys from −2 to 1. Watch frequencies become prefix boundaries, then follow the reverse scan that preserves the A/C/F and B/E orders.
Stable records, including negative keys
Frequency → prefix boundary → output slot
The letter is payload. Sorting uses only the numeric key, so preserved letter order makes stability visible.
key − min = key + 2
Map the range
Step 1 / 17
Input records
Frequency table
array index = key − min
key -2
0
index 0
key -1
0
index 1
key 0
0
index 2
key 1
0
index 3
Stable output
The key interval is [-2, 1], so k = 4. Map key x to count[x − (-2)].
Invariant
No records have been processed; every frequency is zero.
Original visualizer · preserved
Interactive counting sort demo
Follow frequency counting and the reconstruction of values in increasing key order.
The three transformations
Count frequencies
count[x − min] += 1
Offsetting by min maps negative or shifted keys into zero-based array indices. After the scan, count[j] is the exact frequency of key min + j.
Accumulate prefixes
count[j] += count[j − 1]
Now count[j] equals the number of records whose key is at most min + j. It is the exclusive right boundary of that key's output block.
Place from the right
output[−−count[j]] = item
Later equal records claim later slots first. Moving backward through the input therefore preserves the original left-to-right order among equal keys.
Stable implementation
This version handles arbitrary integer ranges by offsetting with the minimum. For records, replace value with key(record) in the counting and placement steps, and store the record in output.
Python
def counting_sort(items):
"""Stable counting sort for integer keys, including negatives."""
if not items:
return []
low, high = min(items), max(items)
k = high - low + 1
count = [0] * k
# 1. Frequencies. Offset maps [low, high] to [0, k).
for value in items:
count[value - low] += 1
# 2. Prefix sums become exclusive right boundaries.
for i in range(1, k):
count[i] += count[i - 1]
# 3. Right-to-left placement makes equal keys stable.
output = [0] * len(items)
for value in reversed(items):
bucket = value - low
count[bucket] -= 1
output[count[bucket]] = value
return outputJava
class CountingSort {
static int[] sort(int[] input) {
if (input.length == 0) return new int[0];
int min = input[0], max = input[0];
for (int value : input) {
min = Math.min(min, value);
max = Math.max(max, value);
}
// Compute in long: max - min can overflow an int.
long range = (long) max - min + 1;
if (range > Integer.MAX_VALUE) {
throw new IllegalArgumentException("Key range is too large");
}
int[] count = new int[(int) range];
for (int value : input) count[value - min]++;
for (int i = 1; i < count.length; i++) count[i] += count[i - 1];
int[] output = new int[input.length];
for (int i = input.length - 1; i >= 0; i--) {
int bucket = input[i] - min;
output[--count[bucket]] = input[i];
}
return output;
}
}Why the algorithm is correct
Frequency fact
The first scan increments exactly one counter for every input record, so no key occurrence is lost or invented.
Position fact
Prefix count C[j] equals |{x : key(x) ≤ min + j}|. Therefore the block ending at C[j] − 1 contains exactly the records with key min + j.
Stability fact
For equal records i < j, the reverse scan places j first at the rightmost free slot. Then i receives a smaller slot, so i still precedes j.
Scanning the input left-to-right with the same decrementing right boundary would reverse equal records. A frequency-only reconstruction: write key x exactly count[x] times: sorts primitive values but does not preserve record payload order.
Complexity, with every term exposed
| Work | Cost | Why |
|---|---|---|
| Find min/max + count | Θ(n) | Two linear scans, often fused or close in practice. |
| Initialize + prefix counters | Θ(k) | Every key slot is allocated and visited, including zero-frequency keys. |
| Stable placement | Θ(n) | Each record is copied once into output. |
| Total time | Θ(n + k) | Best, average, and worst case under the same range. |
| Auxiliary space | Θ(n + k) | n output slots plus k counters. Excluding returned output, some texts report Θ(k). |
The usual “linear-time” description assumes k = O(n). If k ≫ n log n, initializing the counter range can already cost more than a comparison sort.
What the comparison lower bound does, and does not: say
Why Ω(n log n) is bypassed
The decision-tree lower bound applies to algorithms that learn order only by comparing pairs. Counting sort uses a stronger operation: a key directly addresses one of k known slots.
Where the cost reappears
The key-universe assumption appears as the k time and memory term. Unbounded objects, huge sparse IDs, or exact real numbers do not provide an affordable direct-address table.
Extensions and escape hatches
Negative and shifted keys
Use index key − min and allocate max − min + 1 counters. Compute the range in a wider type so subtraction cannot overflow.
Objects and stable multi-key sorting
Move whole records using a bounded integer key. Stability lets a later counting-sort pass refine a previously established secondary order: the engine behind LSD radix sort.
Sparse keys
A hash map can count only present keys, but its keys must then be ordered: O(n + u log u) for u distinct keys. Often a standard comparison sort is simpler and more memory-friendly.
Decimals and real numbers
Scaling known fixed-precision values can create integer keys, but arbitrary floats cannot be indexed directly. Quantization changes equality and must be part of the data contract.
Failure checklist
- Empty input: do not call min/max before returning.
- Range overflow: calculate
max − min + 1in a wider type. - Hidden instability: test tagged equal keys, not only repeated primitive numbers.
- Allocation pressure: validate
kbefore creating the count array. - Wrong complexity claim: include both
nandk, plus output storage. - Changing semantics: never bucket floats or timestamps approximately unless the specification allows it.
Exercises
- For keys in [-2, 1], show the count array after frequencies and after prefix sums for input
[1, -2, 1, 0]. - Why does a left-to-right placement into the same boundaries lose stability?
- State a concrete (n, k) where counting loses to mergesort on both time and memory.
Primary sources
- Cormen et al., Introduction to Algorithms (CLRS): counting sort.
- Knuth, The Art of Computer Programming, Vol. 3: distribution counting.
- Radix sort prerequisites: stable counting as a digit pass (same CLRS chapter family).
Decision rule
Choose counting sort when keys map losslessly to a compact integer interval and n + k work plus n + k auxiliary storage fits comfortably. Otherwise prefer radix sort for bounded-width keys or a comparison sort for general data.