SwadhyAI Notes

All DSA notes

DSA Complexity

Insertion sort cost

A mobile timeline is almost sorted; each new post lands near the front. Why do textbooks still write Theta(n^2) beside insertion sort?

Because sets the bill: Theta(n + I) typical shift models, with when I is Theta(n^2). Stable, in-place, adaptive.

Algorithm journey

1

Sorted prefix grows

i from 1 to n-1.

2

Pay per shift

Each shift removes disorder locally.

3

Best case

I = 0 implies Theta(n).

4

Worst case

I = n(n-1)/2 implies Theta(n^2).

What actually happens

Naive claim: insertion is always quadratic. False on sorted data.

Better claim: state I. Hybrids keep insertion for tiny/nearly sorted runs because constants are excellent.

It still loses on large random arrays to Theta(n log n) algorithms.

Invariant

After processing index i, the prefix a[0:i+1] is sorted; total shifts across the run equal the number of inversions removed.

How to read insertion sort's complexity

1. Nearly sorted or tiny n? Often the winner.

2. Reverse-sorted large n? Theta(n^2); use a better sort.

3. Inside TimSort/IntroSort? That is where insertion's cost model shines.

Quantitative check

n = 10^5, I = 100: roughly a hundred thousand comparisons plus about 100 shifts. Same n reverse-sorted: about 5 x 10^9 shifts - past interactive budgets.

How it works

1

Start with Second Element

Consider first element as sorted. Pick the second element as the "key" to insert.

2

Compare with Sorted Elements

Compare key with elements in sorted portion, moving from right to left.

3

Shift Elements Right

Shift larger elements one position to the right to make space for the key.

4

Insert Key in Correct Position

Place the key in its correct position. Repeat for all remaining elements.

Note: Best for Nearly-Sorted Data!

Insertion sort is adaptive - it runs in O(n) time when the array is already sorted or nearly sorted! If each element is at most k positions away from its final position, insertion sort runs in O(nk) time. This makes it the fastest simple sorting algorithm for small or nearly-sorted datasets, beating both bubble and selection sort in practice!

🎯 Visual Example

Sorting [12, 11, 13, 5, 6]:

Initial:
[12 | 11, 13, 5, 6] ← 12 is "sorted"
Pass 1: Insert 11
Key = 11
11 < 12, shift 12 right
[11, 12 | 13, 5, 6]
Pass 2: Insert 13
Key = 13
13 > 12, already in place
[11, 12, 13 | 5, 6]
Pass 3: Insert 5
Key = 5
5 < 13, shift 13 right
5 < 12, shift 12 right
5 < 11, shift 11 right
[5, 11, 12, 13 | 6]
Pass 4: Insert 6
Key = 6
6 < 13, shift 13 right
6 < 12, shift 12 right
6 < 11, shift 11 right
6 > 5, insert after 5
[5, 6, 11, 12, 13] ✓ Sorted!

🎮 Interactive Insertion Sort Demo

Click "Start" to watch the algorithm pick a card (bar) and insert it into the sorted hand!

Speed:
ℹ️ Ready
Comparing/Active
Sorted

🎯 Key Insight: Online Algorithm!

Insertion sort is an online algorithm - it can sort data as it arrives! You don't need the entire array upfront. As new elements come in (e.g., from a stream), you can insert them into the sorted portion immediately. This makes it perfect for real-time systems where data arrives continuously, unlike algorithms that need all data first!

💻 Implementation

Python

python
def insertion_sort(arr):
    """
    Insertion Sort - Insert each element into sorted portion
    
    Time Complexity:
    - Best Case: O(n) - already sorted
    - Average Case: O(n²)
    - Worst Case: O(n²) - reverse sorted
    
    Space Complexity: O(1) - in-place sorting
    
    Properties:
    - Stable (preserves order of equal elements)
    - Adaptive (fast for nearly-sorted data)
    - In-place (O(1) extra space)
    - Online (can sort as data arrives)
    """
    n = len(arr)
    
    # Start from second element (first is "sorted")
    for i in range(1, n):
        # Element to be inserted
        key = arr[i]
        
        # Position in sorted portion to compare with
        j = i - 1
        
        # Move elements greater than key one position right
        while j >= 0 and arr[j] > key:
            arr[j + 1] = arr[j]
            j -= 1
        
        # Insert key in correct position
        arr[j + 1] = key
    
    return arr


# Example Usage
arr = [12, 11, 13, 5, 6]
print(f"Original: {arr}")

sorted_arr = insertion_sort(arr.copy())
print(f"Sorted: {sorted_arr}")

# Output:
# Original: [12, 11, 13, 5, 6]
# Sorted: [5, 6, 11, 12, 13]


# Demonstrating best case (already sorted)
sorted_arr = [1, 2, 3, 4, 5]
insertion_sort(sorted_arr)  # O(n) - only n comparisons!
print(f"Already sorted: {sorted_arr}")
# Output: Already sorted: [1, 2, 3, 4, 5]


# Binary insertion variant: fewer comparisons, same number of shifts
def binary_insertion_sort(arr):
    for i in range(1, len(arr)):
        key = arr[i]
        low, high = 0, i
        # Upper bound keeps equal keys in their original order.
        while low < high:
            middle = low + (high - low) // 2
            if key < arr[middle]:
                high = middle
            else:
                low = middle + 1
        for j in range(i, low, -1):
            arr[j] = arr[j - 1]
        arr[low] = key
    return arr

Java

java
public class InsertionSort {
    /**
     * Insertion Sort - Insert each element into sorted portion
     * 
     * Time Complexity:
     * - Best Case: O(n) - already sorted
     * - Average Case: O(n²)
     * - Worst Case: O(n²) - reverse sorted
     * 
     * Space Complexity: O(1) - in-place sorting
     */
    public static void insertionSort(int[] arr) {
        int n = arr.length;
        
        // Start from second element (first is "sorted")
        for (int i = 1; i < n; i++) {
            // Element to be inserted
            int key = arr[i];
            
            // Position in sorted portion to compare with
            int j = i - 1;
            
            // Move elements greater than key one position right
            while (j >= 0 && arr[j] > key) {
                arr[j + 1] = arr[j];
                j = j - 1;
            }
            
            // Insert key in correct position
            arr[j + 1] = key;
        }
    }

    // Binary insertion: O(log n) comparisons per item, but still O(n²) shifts.
    public static void binaryInsertionSort(int[] arr) {
        for (int i = 1; i < arr.length; i++) {
            int key = arr[i];
            int low = 0;
            int high = i;
            // Upper bound preserves the order of equal keys.
            while (low < high) {
                int middle = low + (high - low) / 2;
                if (key < arr[middle]) high = middle;
                else low = middle + 1;
            }
            for (int j = i; j > low; j--) {
                arr[j] = arr[j - 1];
            }
            arr[low] = key;
        }
    }

    public static void main(String[] args) {
        int[] arr = {12, 11, 13, 5, 6};
        System.out.println("Original: " + java.util.Arrays.toString(arr));
        
        insertionSort(arr);
        
        System.out.println("Sorted: " + java.util.Arrays.toString(arr));
    }
}

Invariant and correctness

Prefix invariant: before iteration i, arr[0:i] contains exactly the original first i elements in nondecreasing order. Saving the key creates a one-slot hole; shifting only values greater than the key preserves order and moves the hole left. Inserting the key closes the hole and establishes the invariant for i+1.

The strict comparison is the stability condition: equal keys are not shifted past one another. The algorithm is online because a newly arriving item can be inserted into the already sorted prefix without seeing future items.

Inversions explain the adaptive cost

Shifts

Exactly one shift removes one inversion. With I input inversions, insertion sort runs in Θ(n + I).

Binary search variant

It reduces key comparisons to O(n log n), but elements still move Θ(n²) times in the worst case.

Hybrid cutoff

Production hybrid sorts use insertion sort on tiny runs because contiguous scans and low setup cost beat recursion overhead.

⚡ Complexity Analysis

Time Complexity

  • ✓
    O(n) - Best Case

    Already sorted - only n comparisons

  • •
    O(n²) - Average Case

    Random order - ~n²/4 comparisons

  • ✗
    O(n²) - Worst Case

    Reverse sorted - n²/2 comparisons

Space Complexity

  • ✓
    O(1)

    In-place, only key variable

Other Properties

  • ✓Stable: Maintains order of equal elements
  • ✓Adaptive: O(n) for nearly-sorted data
  • ✓Online: Can sort streaming data
  • ✓In-place: O(1) extra space

Tip: Insertion Sort in Practice

Insertion sort is widely used in production code:

  • • TimSort family: Uses insertion-style binary insertion to extend short natural runs
  • • Introspective hybrids: Commonly finish small partitions with a low-overhead insertion pass
  • • Small arrays: The crossover cutoff is runtime- and hardware-specific; measure it
  • • Nearly sorted: Θ(n + I) makes low-inversion inputs its strongest case

Real-world lesson: Simple algorithms can beat complex ones in the right context!

✅ Pros & ❌ Cons

✅ Advantages

  • ✓Simple and intuitive to implement
  • ✓Low setup overhead for small partitions
  • ✓Adaptive - O(n) for nearly-sorted data
  • ✓Stable sort (preserves order)
  • ✓In-place (O(1) extra space)
  • ✓Online algorithm (sorts streaming data)
  • ✓Useful as the small-run phase of hybrid sorts

❌ Disadvantages

  • ✗O(n²) worst/average case - slow for large arrays
  • ✗Many shifts needed for reverse-sorted data
  • ✗Not suitable for large random datasets
  • ✗Eventually outgrown by O(n log n) algorithms on high-inversion inputs

💼 Production patterns

🔀 Hybrid sorting

TimSort-family implementations extend short natural runs with stable binary insertion; other hybrids commonly use insertion sorting to finish tiny partitions after a divide-and-conquer phase.

The cutoff is an engineering parameter, not an asymptotic constant.

📊 Real-Time Data Streams

A small bounded buffer can remain sorted by inserting each arrival. Once the buffer is large or updates are unbounded, a balanced tree, heap, or database index provides better update complexity.

Example: Maintaining top 10 stock prices

🎮 Game Leaderboards

A deliberately bounded local leaderboard can insert a new score with a short contiguous shift. A global ranking service needs indexed storage and top-k retrieval rather than an ever-growing array.

Example: Local multiplayer scoreboards

💾 Sorted-page maintenance

Inserting a slot into a fixed-size sorted page uses the same “find position, shift suffix” primitive. Database indexes additionally manage concurrency, page splits, logging, and storage layout.

Example: B-tree leaf node sorting

Engineering contract and tests

  • Both implementations mutate their arrays. Copy explicitly when an immutable API is required; copying adds Θ(n) time and space.
  • Python list moves are reference assignments, while Java primitive-array moves copy values. For large records, sort compact references or indices.
  • Python's arbitrary-size integers avoid arithmetic overflow, but comparator/key functions can still be expensive; Java comparators should use safe compare methods.
  • Test sortedness, permutation, idempotence, and stability with tagged duplicate records. Include empty, one item, all equal, sorted, reverse, nearly sorted, and random arrays.

Choose insertion sort for small or low-inversion inputs and incremental streams. Do not use an array-based insertion loop for an unbounded leaderboard: a balanced tree, heap, or indexed data store gives scalable updates.

Exercises

  1. Prove swaps/shifts relate to I for the standard algorithm.
  2. Find a family with I = O(n) that is still awkward for bubble but fine for insertion.
  3. Estimate crossover n where mergesort beats insertion on random data (hand-wavy constants OK).

Primary sources

  • CLRS: insertion sort analysis.
  • Knuth TAOCP Vol. 3: insertion sorting.
  • TimSort notes: insertion for minrun extension.

Short takeaway

Insertion Sort = Insert each element into correct position in sorted portion.Time: O(n) best (sorted), O(n²) average/worst.Space: O(1) in-place.Properties: Stable, adaptive, online, simple.Use when: Arrays are small by a measured cutoff, nearly sorted, or deliberately bounded online buffers.Real-world: A valuable finishing phase inside adaptive and introspective hybrid sorts.