adaptive

Definition

Adjusting automatically instead of using a fixed setting. Adaptive optimizers such as Adam scale each parameter's effective step size from its own gradient history, which is why they need so little learning-rate tuning to get started. Adaptive computation spends more forward passes or reasoning tokens on harder inputs than on easy ones. The cost of adaptivity is usually extra state to store, and sometimes less predictable behaviour late in training.

Explainers using this term

• Paper Walkthrough: SecOPD — Grading One Token at a Time to Cut Adaptive Prompt Injection by an Order of MagnitudeSecOPD: Mitigating Adaptive Prompt Injections by On-Policy Distillation
• Paper Explained: FreeToken — Treating Your Own PC as a Single Elastic Inference PlatformFreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution