Students Professional
JA EN
Home › Algorithms

◇ FIELD

Algorithms

From complexity as a yardstick to search, dynamic programming, matrix computation and parallelism — the craft of computation underneath AI.

0chapters 19Foundations 4Paper walkthroughs 0Interactive

Work through a volume in order: textbook → foundations → papers → lab.

② Foundations

Articles that assume nothing and build the ideas of the field, in order.

Complexity

Big-O, time-space tradeoffs, and where theory parts from the profiler

  1. Complexity From Scratch — What Big-O Actually Measures FREE
  2. When Big-O and Your Benchmarks Disagree — Caches, Branches, and Memory Bandwidth ★ MEMBER
  3. Randomized Algorithms — Why Rolling Dice Makes Things Faster ★ MEMBER
  4. NP-Completeness from Scratch — Not Unsolvable, but Fast to Verify FREE
  5. Approximation Algorithms — Trading Exactness for a Guarantee ★ MEMBER

Data Structures

Arrays, trees, hashes, heaps — what each choice buys you

  1. Choosing a Data Structure — Arrays, Hashes, Trees and Heaps FREE
  2. Cache-Friendly Code — Why Two O(n) Loops Can Differ by 10× ★ MEMBER
  3. B-Trees and LSM-Trees — The Heart of Every Database ★ MEMBER

Search & Optimization

Exhaustive, greedy, DP, branch and bound, approximation

  1. Dynamic Programming From Scratch — On Remembering Subproblems ★ MEMBER
  2. Linear Programming from Scratch — The Workhorse of Optimization ★ MEMBER
  3. Simulated Annealing and Genetic Algorithms — What to Do When Exact Solving Breaks Down ★ MEMBER

Numerical Computing

GEMM, decompositions, FFT, iterative methods — where AI's compute actually goes

  1. The Cost of Matrix Multiplication — Where Almost All of AI's Compute Goes ★ MEMBER
  2. Numerical Pitfalls — Cancellation, Rounding, and logsumexp ★ MEMBER
  3. How Autodiff Actually Works — Unpacking the PyTorch Magic FREE
  4. Solving Systems of Equations — Direct Methods and Iterative Methods ★ MEMBER
  5. Build Your Own Autograd — A Mini PyTorch in 100 Lines ★ MEMBER

Parallel & Distributed

Limits of parallelism, the GPU execution model, communication in distributed training

  1. Why GPUs Are Fast — The Execution Model and the Limits of Parallelism ★ MEMBER
  2. Distributed Training from Scratch — Data Parallel, Model Parallel, and When Communication Becomes the Bottleneck ★ MEMBER
  3. Concurrency from Scratch — Locks, Atomics, and Memory Models ★ MEMBER

③ Paper walkthroughs

Written from the papers themselves. Every piece links the paper page and its PDF.

  1. Probabilistic Data Structures — Counting Without Counting ★ MEMBER "doi:10.1145/362686.362692
  2. The FFT from Scratch — Why Convolution Turns into Multiplication ★ MEMBER "doi:10.1090/S0025-5718-1965-0178586-1
  3. Graph Algorithms from Scratch — Shortest Paths and Where They Lead ★ MEMBER arXiv:1603.09320
  4. Hashing and Nearest-Neighbor Search — The Groundwork Under Vector Search ★ MEMBER arXiv:1603.09320