#constrained-decoding

1 articles

01 ·Inference & Serving·FREE·PAPER·11 min read Structured Output and Constrained Decoding — How to Stop an LLM from Breaking Your JSON Instead of asking a model nicely to return JSON, you can drive the probability of every grammatically illegal token to exactly zero before it is ever sampled. A from-scratch walkthrough of constrained decoding — logit masks, vocabulary indexing, what function calling really does, and why syntax guarantees are not content guarantees.