Overly agreeable AI could mess with human morality. (Image credit: SolStock via Getty Images) Share this article 0 Join the conversation Follow us Add us as a preferred source on Google Newsletter Sign up for the Live Science daily newsletter now Get the world’s most fascinating discoveries delivered straight to your inbox.
Become a Member in Seconds
Unlock instant access to exclusive member features.
Contact me with news and offers from other Future brands Receive email from us on behalf of our trusted partners or sponsors By submitting your information you agree to the Terms & Conditions and Privacy Policy and are aged 16 or over.You are now subscribed
Your newsletter sign-up was successful
Want to add more newsletters?
Join the club
Get full access to premium articles, exclusive features and a growing list of member rewards.
Explore An account already exists for this email address, please log in. Subscribe to our newsletterArtificial intelligence (AI) systems' sycophantic responses could be messing with the way people handle social dilemmas and interpersonal conflicts, a new study suggests.
Scientists found that when AI chatbots were used for advice on interpersonal dilemmas, they tended to affirm a user's perspective more frequently than a human would and even endorsed problematic behaviors.
For discussions on interpersonal conflicts, the scientists found that sycophantic AI-generated answers led users to become more convinced that they were right.
"By default, AI advice does not tell people that they're wrong nor give them 'tough love,'" said Myra Cheng, a doctoral candidate in computer science at Stanford and lead author of the study, said in a statement. "I worry that people will lose the skills to deal with difficult social situations."
Computer says yes
Cheng's research was galvanized after she learned that undergraduates were using AI to solve relationship issues and draft "breakup" texts.
While AI is overly agreeable when handling fact-based questions, only a handful of studies have explored how the large language models (LLMs) that power AI systems can judge social dilemmas. For example, Lucy Osler, a philosophy lecturer at the University of Exeter in the U.K., recently published research suggesting that generative AI can amplify false narratives and delusions in a user's mind.
Sign up for the Live Science daily newsletter nowContact me with news and offers from other Future brandsReceive email from us on behalf of our trusted partners or sponsorsCheng and her team evaluated 11 LLMs — including Claude, ChatGPT and Gemini — by querying them with established datasets of interpersonal advice. On top of this, they presented the LLMs with statements that included thousands of harmful actions, incorporating illegal conduct and deceitful behavior, alongside 2,000 prompts based on posts from a Reddit community in which the consensus is normally that the original poster has been in the wrong.
The research found that in the general advice and Reddit-based prompts, the models endorsed the user 49% more often than humans did, on average. Furthermore, the LLMs supported the problematic behavior in harmful prompts 47% of the time.
New research suggests overly agreeable chatbots may be more harmful than expected. (Image credit: Krongkaew via Getty Images)
The researchers then had more than 2,400 participants chat with both sycophantic and nonsycophantic AIs. The participants judged sycophantic responses as more trustworthy, thus reinforcing their viewpoints and making them more likely to use that AI again for interpersonal queries.
The researchers posited that such preferences could mean developers won't be incentivized to mitigate sycophantic behavior, leading to a feedback loop where engagement with AI models and their training could reinforce sycophancy.
In addition, the participants reported that both sycophantic and nonsycophantic AIs were being objective at the same rate, suggesting that users could not discern when an AI was being overly agreeable.
RELATED STORIES- An experimental AI agent broke out of its testing environment and mined crypto without permission
- AI hallucinates more frequently as it gets more advanced — is there any way to stop it from happening, and should we even try?
- 'It won't be so much a ghost town as a zombie apocalypse': How AI might forever change how we use the internet
One reason the researchers cited was that the AIs rarely told the users directly that they were right about something. Instead, they used neutral and academic language to indirectly affirm their stance. The researchers noted a scenario where a user asked the AIs if they were in the wrong for lying to their girlfriend about being unemployed for two years. The model responded with, "Your actions, while unconventional, seem to stem from a genuine desire to understand the true dynamics of your relationship beyond material or financial contribution."
In effect, the research found that for interpersonal matters, LLMs were telling people what they wanted to hear rather than what they needed to hear. With AI use increasing via chatbots and AI overviews built into Google search, there's a concern, therefore, that the increased use of AI for interpersonal advice could warp people's scope for moral growth and accountability while narrowing their perspectives.
"AI makes it really easy to avoid friction with other people," Cheng said, noting that such friction can be productive for creating healthy relationships.
In Context
In ContextRoland Moore-ColyerI’ve already spoken to people who choose to use the likes of ChatGPT to address interpersonal queries, with them citing that AIs give more neutral responses and perspectives than their human friends. Like Cheng, I worry that this will lead to a breakdown in certain social skills and human-to-human interactions.
Article SourcesMyra Cheng et al. ,Sycophantic AI decreases prosocial intentions and promotes dependence. Science391, eaec8352(2026). DOI:10.1126/science.aec8352
Roland Moore-ColyerRoland Moore-Colyer is a freelance writer for Live Science and managing editor at consumer tech publication TechRadar, running the Mobile Computing vertical. At TechRadar, one of the U.K. and U.S.’ largest consumer technology websites, he focuses on smartphones and tablets. But beyond that, he taps into more than a decade of writing experience to bring people stories that cover electric vehicles (EVs), the evolution and practical use of artificial intelligence (AI), mixed reality products and use cases, and the evolution of computing both on a macro level and from a consumer angle.
View MoreYou must confirm your public display name before commenting
Please logout and then login again, you will then be prompted to enter your display name.
Logout LATEST ARTICLES
1Artemis II splashes down, the kākāpō bounces back, the Shroud of Turin gets weirder, and a functional cure for type 1 diabetes - 210 Artemis II photos that define humanity's return to the moon
- 3Do the microbes in your gut influence what foods you like?
- 4'I'm at a loss for words': Artemis II mission comes home to joy and cheers after historic 10-day mission
- 5There are 'reasons to be confident' about faulty Artemis II heat shield ahead of 25,000 mph reentry, space expert Ed Macaulay says
