#caching
2 articles
01
·Inference & Serving·★ MEMBER·11 min read
Cutting Inference Cost in Practice — What to Do First
Quantization and distillation both work, but in most shops there are two or three levers left that cost you nothing in quality. Break the bill into four numbers, then reorder the work: cache, batch, shorten, compress. What each lever actually buys, and what breaks when you take them out of order.
02
·Networking·★ MEMBER·11 min read
DNS and CDNs — What Happens Between Enter and Pixels
In the second or so between pressing Enter and seeing a page, a name gets resolved down a tree, four layers of cache get consulted, and an edge server somewhere near you answers instead of the origin. Here is how DNS walks that tree, what a TTL does not promise, and how a CDN talks its way around the speed of light — from zero background.