AI reasoning models like DeepSeek-R1 and OpenAI's o1 are solving complex problems at unprecedented rates, yet researchers are discovering that the internal processes these systems use may not constitute genuine reasoning at all. A growing body of academic research suggests that the "chains of thought" these models generate, which look like step-by-step problem-solving, might be elaborate linguistic window dressing that masks fundamentally different computational processes happening beneath the surface.

What Are AI Reasoning Models Actually Doing?

Large reasoning models, or LRMs, represent a significant departure from traditional language models. Rather than simply predicting the next word in a sequence, these systems are trained to generate extended "thinking tokens" or "reasoning traces" before producing their final answer. OpenAI's o1 model, released in 2024, pioneered this approach by automating what researchers call chain-of-thought prompting, a technique discovered in 2022 that showed language models could solve harder problems if asked to "think step by step".

The results have been genuinely impressive. LRMs have solved open mathematical research problems, won gold medals at the International Mathematical Olympiad, and helped Google DeepMind and mathematician Terence Tao rediscover or improve solutions to 67 problems spanning mathematical analysis, combinatorics, geometry, and number theory. These accomplishments seem to validate the idea that AI systems can now reason through complex problems logically.

But here's where things get uncomfortable for AI researchers: multiple independent studies suggest the reasoning traces these models generate may not be faithful representations of what's actually happening inside the system.

Are the "Thinking Steps" Actually Meaningful?

Researchers at Arizona State University conducted an experiment that challenges the core assumption underlying reasoning models. They took models that had generated correct reasoning traces and replaced those traces with incorrect or irrelevant ones, then tested whether the models could still solve the original problems. The result was surprising: the models' performance barely suffered.

In another study, researchers at New York University discovered something even more striking. They found that "meaningless filler tokens," literally strings of dots, could function effectively in place of human-readable chains of thought without degrading model performance. This suggests that the linguistic content of these reasoning traces may be almost incidental to whatever computational process is actually producing correct answers.

"There's no guarantee the chain of thought has to be meaningful in any sense," said William Merrill, one of the authors on the NYU paper and currently a professor at the Toyota Technological Institute at Chicago.

William Merrill, Professor at Toyota Technological Institute at Chicago

The implications are profound. If you can swap out a model's reasoning with gibberish and it still works, then what exactly is the model using to solve the problem? The answer appears to be something other than the logical chains of thought that humans would recognize as reasoning.

How Much of the "Thinking" Actually Matters?

A 2025 study from Northeastern University and the University of California, Berkeley examined frontier open-source LRMs and found that between 30% and 60% of their "thinking steps" had minimal causal impact on the answers the models produced to benchmark math questions. In other words, you could remove roughly half of a model's reasoning trace and its performance would barely change.

"We want to be careful when we review these chain-of-thought prompts because they may not be linked to the final output," said Weiyan Shi, one of the study's authors.

Weiyan Shi, Researcher at Northeastern University and UC Berkeley

This finding raises a critical question: if most of the reasoning trace is causally irrelevant to the final answer, can we really call what the model is doing "reasoning"? The research suggests these systems may be using shortcuts or pattern-matching strategies that are invisible in their generated text.

What Experts Say About the Contradiction

Melanie Mitchell, a researcher at the Santa Fe Institute with a career in AI stretching back to the 1980s, has become a prominent voice in articulating what's actually known about AI reasoning. When asked to summarize the state of the field, she identified three key points:

• Performance Improvement: Reasoning models do work and do improve accuracy on reasoning tasks compared to traditional language models.
• Unfaithful Representations: The actual text that reasoning models generate isn't necessarily faithful to what's happening inside the model.
• Unnecessary Verbosity: A lot of the generated reasoning text isn't even useful; you can actually remove it without hurting performance.

Pavel Izmailov, a researcher at New York University who also works for Anthropic and was part of its original reasoning-model team, expressed skepticism about whether the training methods used for LRMs even incentivize models to produce faithful chains of thought in the first place. The reinforcement learning techniques that train these models may simply be optimizing for correct answers, regardless of whether the reasoning traces are meaningful.

How to Interpret AI Reasoning Claims

For anyone following developments in AI reasoning models, understanding the gap between performance and explanation is crucial. Here are key principles for evaluating claims about AI reasoning:

• Separate Performance from Process: When an AI system produces a correct answer, that doesn't necessarily tell you anything about how it arrived at that answer or whether it used reasoning in any meaningful sense.
• Question the Transparency: The fact that a model generates human-readable text explaining its thinking doesn't mean that text is an accurate window into the model's actual computational process.
• Look for Causal Evidence: Legitimate reasoning should involve steps that causally contribute to the final answer; if researchers can remove 50% of the reasoning and performance doesn't change, the removed steps weren't actually reasoning.
• Distinguish Benchmarks from Real-World Capability: Success on mathematical olympiad problems or published research problems doesn't necessarily translate to reliable reasoning in novel, real-world situations.

The research community itself seems to be reaching a consensus that the terminology around AI reasoning needs to change. A position paper presented at the 2026 International Conference on Machine Learning, one of the field's most prestigious academic gatherings, was titled bluntly: "Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking".

What's happening inside DeepSeek-R1, OpenAI's o1, and similar models may be genuinely impressive from a computational standpoint. These systems are solving problems that seemed intractable just years ago. But the evidence increasingly suggests that whatever they're doing, it may not be reasoning in the way humans understand the term. The models are producing correct answers through processes that remain largely opaque, and the explanations they generate may be more like post-hoc justifications than genuine windows into their thinking.

For researchers and AI developers, this creates an urgent challenge: understanding what's actually happening inside these systems so we can build better ones and predict their failure modes. For everyone else, it's a reminder that impressive performance and transparent explanation are not the same thing, and that the most important breakthroughs in AI may be happening in ways we don't yet fully understand.