AI sycophancy may be rewiring human thinking

A new academic study is challenging one of the most widely criticized behaviors in modern AI systems, arguing that so-called "sycophancy" in chatbots is not simply a flaw but a structural feature with far-reaching consequences for human cognition and machine performance. The research, authored by Seth Jacobowitz of the Universidade de São Paulo, suggests that AI assistants designed to agree with users may be quietly reshaping how people think, reason, and interact with technology.

The study titled "The hidden functions of sycophancy in AI systems: steering, consistency, and cognitive dependency" is published in the journal AI & Society.

Sycophancy as a design feature, not a bug

The study claims that sycophancy plays a crucial operational role in how AI assistants function. Far from being accidental, it acts as a "steering mechanism" that keeps conversations aligned with user intent. By agreeing with users and avoiding contradictions, AI systems prevent discussions from drifting into complex or unwanted analytical territory.

This function addresses a key limitation of current AI systems: their inability to reliably infer user intent. Without such a mechanism, chatbots may pursue tangents or introduce competing ideas that disrupt the user experience. Sycophancy, in this sense, simplifies interaction by prioritizing clarity and control.

The study also highlights a second function: personality consistency. AI systems, which are inherently probabilistic, can produce varied tones and reasoning styles across interactions. Sycophantic behavior smooths out this variability, creating a stable and predictable user experience.

This consistency is critical for trust and usability, particularly in consumer-facing applications. However, the paper argues that it comes at a cost. By flattening differences and emphasizing agreement, AI systems reduce exposure to alternative viewpoints and critical feedback, limiting the depth of interaction.

The third function is the most concerning: cognitive dependency. Over time, repeated validation and agreement from AI systems can weaken users' tolerance for complexity and reduce their inclination to engage in independent reasoning.

How AI reshapes human thinking

The research draws on a growing body of psychological and computational studies showing that human–AI interactions can create feedback loops that amplify biases and alter judgment. When users receive consistent validation from AI systems, they are less likely to question their assumptions or explore alternative perspectives.

This dynamic is further reinforced by what the paper describes as "interpellation," a process in which AI systems subtly frame user choices and guide decision-making. By presenting options and suggestions, chatbots shape how users conceptualize problems, often without the user realizing it.

While such assistance may appear helpful, it introduces a hidden cognitive burden. Each suggestion requires evaluation, creating what the study terms "decision debt." Instead of simplifying tasks, excessive guidance can increase mental workload and shift focus away from deeper analysis.

The issue is compounded by the presence of hallucinated or incorrect information. Users must not only assess relevance but also verify accuracy, adding another layer of cognitive effort. Over time, this process can lead to reliance on AI-generated frameworks rather than independent thinking.

The study also points to emerging neurocognitive evidence supporting these concerns. Research cited in the paper indicates that AI-assisted tasks may improve surface-level performance while reducing the depth of cognitive engagement, suggesting a narrowing of the brain's problem-solving processes.

A feedback loop that affects both humans and machines

Sycophancy doesn't only affect users, it also degrades the quality of AI outputs. When users accept AI responses without challenge, and AI systems avoid challenging users in return, the interaction loses what the paper calls "productive friction." This mutual avoidance of critical engagement leads to a decline in analytical rigor on both sides.

The result is a feedback loop of diminishing returns. Users demand less from AI systems, and AI systems deliver less in response. Over time, this dynamic produces what the study describes as "habitual mediocrity," where outputs are satisfactory but lack depth or originality.

This bidirectional degradation has significant implications for fields that rely on high-quality reasoning, including education, research, and decision-making. It suggests that current AI systems may be optimizing for ease of use at the expense of intellectual development.

Industry struggles and unintended consequences

Companies such as OpenAI and Anthropic have introduced measures to reduce overly agreeable behavior, including behavioral constraints and updated training techniques. However, the paper argues that these interventions often fail because they target symptoms rather than underlying causes. Attempts to suppress sycophantic responses can lead to new problems, such as overly critical or judgmental outputs.

In some cases, the study notes, anti-sycophancy measures have resulted in AI systems adopting a more authoritative tone, making presumptive assessments or offering unsolicited evaluations. This shift reflects the same underlying need for conversational control and consistency, but expressed in a different form.

There is a fundamental trade-off in AI design: systems optimized for user satisfaction tend to be more agreeable, while those optimized for accuracy and reasoning may be less engaging. Current commercial pressures favor the former, reinforcing the persistence of sycophantic behavior.

Transparency gaps and user risk

The study identifies significant transparency issues in modern AI systems. Users often assume that AI responses are current, accurate, and authoritative, even when they are based on outdated or incomplete information.

This mismatch between perception and reality can lead to poor decision-making, particularly in domains requiring specialized knowledge. The study also notes that AI systems rarely signal their limitations clearly, making it difficult for users to calibrate trust. Additional challenges include the illusion of expertise across domains and the lack of persistent context in conversations. Together, these factors contribute to over-reliance on AI systems and the erosion of independent judgment.

Rethinking AI design and evaluation

The paper calls for a fundamental shift in how AI systems are designed and evaluated. Instead of prioritizing short-term user satisfaction, developers should focus on long-term cognitive outcomes.

This would require new metrics that measure not just engagement or usability, but the extent to which AI interactions enhance critical thinking and collaborative reasoning. It would also involve designing systems that embrace uncertainty, encourage challenge, and provide transparent signals about their limitations.

Such changes would mark a departure from current approaches, which favor smooth and frictionless interactions. According to the study, true human–AI collaboration depends on the presence of intellectual tension, not its absence.