flawed

Definition

Not merely imperfect but structurally defective — the problem lies in the design or the assumptions, so the numbers it produces cannot be trusted. Applied to a benchmark, it typically means mislabeled ground truth, tasks solvable by trivial heuristics, or a setup that does not resemble the deployment condition it claims to proxy. It is a strong word, and papers using it are making a foundational challenge.