Why do generative AI models often get things so wrong? In part, it's because they're trained to act like the customer is always right.
While many generative AI tools and chatbots have mastered sounding convincing and all-knowing, new research conducted by Princeton University shows that the people-pleasing nature of AI comes at a steep price. As these systems become more popular, they become more indifferent to the truth.
AI models, like people, respond to incentives. Compare the problem of large language models producing inaccurate information to that of doctors being more likely to prescribe addictive painkillers when they're evaluated based on how well they manage patients' pain. An incentive to solve one problem (pain) led to another problem (overprescribing).
Read more | CNET