Generative AI challenges traditional ideas of knowledge and reliability

The growing presence of generative artificial intelligence, genAI, in writing and research is raising new questions about how knowledge is created and validated in the digital age. While AI systems can produce fluent and convincing text, their tendency to generate fabricated or inaccurate information has sparked debate about the reliability of machine-generated knowledge and its role in academic and professional work.

A new study titled "Unreliable Minds, Unreliable Machines: Dyslexic Memory, ChatGPT, and the Epistemic Disobedience of Generative AI," authored by Edward Ademolu and published in AI & Society, investigates this issue by examining how generative AI systems interact with dyslexic cognition and how these encounters expose hidden assumptions about memory, credibility, and intellectual authority.

Dyslexia, Memory, and the History of Assistive Technology

For decades, individuals with dyslexia have relied on digital tools to support writing and communication in environments where fluency and precise recall are essential. Technologies such as speech-to-text programs, predictive spelling systems, grammar correction software, and mind-mapping tools have functioned as cognitive supports that help users translate ideas into structured written form.

These tools were designed primarily to correct or compensate for perceived deficits in reading and writing processes. In many educational and professional contexts, dyslexia has been framed as a learning difficulty that requires remediation in order to meet standardized expectations of written performance.

The study challenges this traditional perspective by drawing on insights from neurodiversity research and disability studies. Rather than treating dyslexic cognition as a disorder, the research frames it as an alternative epistemic orientation characterized by associative reasoning, intuitive pattern recognition, and non-linear thinking. These cognitive patterns may conflict with institutional expectations for linear narrative structure and precise recall, but they can also enable creative problem-solving and innovative forms of insight.

Assistive technologies historically functioned as stabilizing supports within this tension. By correcting spelling errors or facilitating transcription, they allowed dyslexic individuals to participate more fully in institutional writing practices without fundamentally altering the epistemic framework of knowledge production.

The arrival of generative AI changes that dynamic. Instead of simply correcting errors or assisting with transcription, large language models generate entire passages of text. This capability introduces a new dimension of collaboration between human cognition and machine-generated language.

Synthetic Knowing and the Limits of AI Reliability

A key concept introduced in the study is the idea of synthetic knowing, a term used to describe the production of coherent language that appears knowledgeable but lacks reliable grounding in verified information. Generative AI systems operate through probabilistic prediction based on patterns in training data rather than through mechanisms of understanding or truth verification.

Consequently, these systems can produce fluent explanations, plausible references, and authoritative-sounding arguments even when the underlying information is incorrect or fabricated. This phenomenon has been widely discussed in public debates about AI "hallucinations," but the study argues that the problem extends beyond technical inaccuracies.

Synthetic knowing raises deeper epistemological questions about how knowledge is recognized and validated. In many institutional contexts, credibility is assessed through stylistic markers such as clarity, structure, and rhetorical confidence. Generative AI systems are highly effective at reproducing these markers, which can blur the distinction between genuine knowledge and probabilistic language generation.

The research highlights that this ambiguity becomes particularly significant in academic writing environments where citations, logical argumentation, and narrative coherence serve as signals of intellectual authority. When AI systems generate fabricated references or confident but inaccurate statements, they challenge existing assumptions about how knowledge is produced and evaluated.

This issue also intersects with long-standing debates about authorship and intellectual responsibility. When a piece of writing is produced through collaboration between a human writer and a generative AI system, questions arise about who is responsible for verifying accuracy and ensuring reliability.

Neurodivergence and the Politics of Cognitive Reliability

The study further argues that the interaction between dyslexic cognition and generative AI reveals an important asymmetry in how different forms of unreliability are interpreted within institutional systems.

Historically, dyslexic disfluency has been framed as a cognitive deficiency requiring correction. Difficulties with spelling, memory recall, or narrative sequencing are often treated as individual limitations that must be managed through accommodations or assistive technologies.

On the other hand, when generative AI systems produce incorrect or fabricated information, these failures are typically interpreted as technical issues to be optimized through improved training data or algorithmic refinement. In other words, machine unreliability is treated as a design challenge, while human unreliability is often medicalized or stigmatized.

The analysis suggests that this difference reflects deeper cultural assumptions about intelligence and legitimacy. Institutional systems tend to equate fluency with competence and coherence with truth. Dyslexic cognition disrupts these assumptions by demonstrating that insight and creativity can emerge through non-linear reasoning and irregular recall.

Generative AI simultaneously destabilizes these norms from the opposite direction. Machines can produce highly fluent language that mimics the outward form of reliable knowledge while lacking the internal processes associated with understanding or memory.

The encounter between dyslexic cognition and generative AI, therefore, creates what the study describes as an epistemic friction zone. Within this space, conventional criteria for evaluating knowledge become visible and open to critical examination.