#constitutional-ai

1 articles

01 ·Large Language Models·★ MEMBER·PAPER·8 min read Alignment, Explained — From RLHF to Constitutional AI What exactly is being aligned with what? This piece separates the gap into three layers — objective, specification, and proxy — shows that RLHF, DPO and Constitutional AI all solve the same skeleton of "reward minus a penalty for drifting from the reference model," explains why pushing that proxy hard always produces reward hacking, and works through the trade between refusing and being useful, down to the implementation traps.