xgrammar

Definition

An open-source engine for grammar-constrained decoding: at each step it masks the token distribution so only continuations consistent with a JSON schema or context-free grammar survive, making structural validity a guarantee rather than something you check afterwards. The engineering problem it targets is overhead, since computing the allowed token set at every step can dominate latency, which it attacks with precomputation and compact mask representations. It ships as a backend in common inference servers.