sharp

Definition

Sharp describes a minimum whose basin is narrow: a small perturbation of the parameters causes a large jump in loss. Two solutions can have identical training loss yet differ greatly in this respect, and the widely-held — though not universally accepted — view is that sharp minima generalize worse, since the test surface is slightly shifted from the training one. Its opposite, flat, is what sharpness-aware methods deliberately seek.