gsm-symbolic

appears in 1 paper titles

Definition

A benchmark built by turning GSM8K grade-school word problems into symbolic templates, then generating variants that change names, numbers, and surface details while leaving the required reasoning identical. The diagnostic value is in the gap: a model that has actually learned the procedure should score the same across variants, so a drop indicates reliance on memorized instances or surface patterns. Variants that add irrelevant clauses probe the same weakness from another angle.