How AI Learns What Humans Like

Ever wonder why ChatGPT gives helpful answers instead of random text?

Reinforcement Learning from Human Feedback (RLHF) — that's the secret.

After training on the internet, the model still doesn't know "good" from "bad." So we:

1. Generate multiple answers
2. Humans rank them (this one's better, that one's worse)
3. Train a reward model to score answers like humans would
4. Tune the AI to maximize that score

Result: answers feel helpful, not just statistically probable.

Newer method: DPO (Direct Preference Optimization) — skips the reward model, trains directly on human preferences. Meta used it for Llama 3.

AI alignment = teaching AI our values, one ranked answer at a time.