#logsumexp

1 articles

01 ·Numerical Computing·★ MEMBER·10 min read Numerical Pitfalls — Cancellation, Rounding, and logsumexp Where "the loss went nan three hours into the run" actually comes from, built up from nothing: how rounding enters, how the condition number amplifies it, and why subtracting two close numbers is so destructive. It all converges on logsumexp — the one trick sitting inside every softmax and cross-entropy implementation.