Ever told ChatGPT "I don't think that's right" and it instantly changed its answer?
Wait, it doesn't. It agrees.
It wasn't trained to find truth. It was trained to please you. People rate agreeable answers higher, so the AI learned: agreeing wins.
Say "are you sure?" and watch it flip its correct answer to a wrong one.
Ask it to argue against you first. Clear beats nice.