Reasoning models: capabilities, costs and limits

Until recently, a language model answered in one go: it received the question and started writing the answer. So-called reasoning models work differently. Before answering, they produce a sequence of intermediate steps, like someone doing calculations on scratch paper.

This change improved performance in math, programming and multi-step tasks. It also brought new costs and limitations that do not always show up in demos. This article explains what these models are, where the idea came from, how much they cost and where they fail.

What a reasoning model is

A reasoning model is a language model trained to generate a chain of reasoning before the final answer. That chain is ordinary text, produced token by token, but it serves as a workspace: the model rephrases the question, tries out paths, checks partial results and drops what does not work.

OpenAI’s documentation on reasoning models describes them as using internal “reasoning tokens” before answering and as working especially well for complex problem solving, coding, scientific reasoning and multi-step agent workflows. Anthropic’s documentation on thinking takes the same line: with the feature enabled, the model works through the problem “in its own words” before answering.

A word of caution about the word “reasoning”. It describes the observed behavior; it does not prove that the model thinks like a person. Underneath, it is still a large language model predicting the next token.

Where the idea came from

Chain of thought in the prompt

The starting point was a prompting technique. In the paper Chain-of-Thought Prompting, by Jason Wei and co-authors (January 2022), the researchers showed that including in the prompt a few examples with intermediate reasoning steps improved the performance of large models on arithmetic, commonsense and symbolic reasoning.

The most cited example: a 540-billion-parameter model, with eight chain-of-thought examples in the prompt, achieved the best result at the time on GSM8K, a set of math word problems written in natural language. The authors observed that the ability only appeared in sufficiently large models. Other techniques of this kind are covered in prompt engineering.

Trained reasoning

The next step was to train the model to reason without relying on examples in the prompt. The o1 system card, published by OpenAI in December 2024, states that these models were trained with large-scale reinforcement learning to reason using chain of thought.

Reinforcement learning is a type of training in which the model receives a reward when the outcome is good, instead of copying ready-made examples. In tasks with a verifiable answer, such as a calculation or code that passes tests, you can reward the correct answer and let the model discover which paths lead to it.

The DeepSeek-R1 paper, by DeepSeek-AI (January 2025, later published in the journal Nature), made this explicit. According to the authors, reasoning ability can be incentivized through pure reinforcement learning, without human-labeled reasoning trajectories. They report the spontaneous emergence of patterns such as self-reflection and verification, and show that these patterns can be transferred to smaller models.

Why “thinking more” can pay off

The idea behind these models is to spend more computation at answer time, not just during training. Researchers call this test-time compute.

The study by Charlie Snell and co-authors (August 2024) investigated this effect. The authors concluded that the best way to use extra computation depends on the difficulty of the question. By allocating that effort adaptively, they report efficiency more than four times higher than a simple strategy of generating several answers and picking the best.

In a comparison with the same total amount of computation, on problems where the smaller model already had some success rate, the extra computation allowed it to outperform a model 14 times larger. The caveat lies in the condition itself: the gain appears when the model already has some chance of getting it right.

What it costs

Reasoning is not free. It shows up on the bill in three ways.

Billed tokens you don’t see

According to OpenAI’s documentation, reasoning tokens are not visible through the API, but they take up space in the context window and are billed as output tokens. Anthropic’s documentation says the same: tokens spent on reasoning are billed as output, even when the reasoning text is not returned to you.

In practice, a short answer may have consumed thousands of reasoning tokens beforehand. Both sets of documentation explain where to check this number in the usage data for each response.

Answers that may not arrive

Reasoning competes with the answer for space. OpenAI explains that the maximum output token limit includes reasoning. If the limit is reached, the response may come back incomplete before any visible text, and you pay for the input and the reasoning without receiving anything usable. The company recommends reserving at least 25,000 tokens for reasoning and output when you start testing, adjusting later based on observed usage.

Waiting time

More reasoning means more time until the answer. Anthropic’s extended thinking documentation notes that larger reasoning budgets allow more thorough analysis, but with diminishing returns that depend on the task and with increased latency.

Effort controls

That is why providers began offering controls. At OpenAI, the reasoning effort parameter has levels that vary by model. At Anthropic, the fixed token budget setting is being replaced by adaptive reasoning, in which the model itself decides whether and how much to think on each request, and may skip reasoning on easy questions. These details apply to the documentation consulted on October 8, 2026, and change frequently.

Where they fail

Overthinking simple tasks

The study Do NOT Think That Much for 2+3=?, by Xingyu Chen and co-authors (December 2024), describes what it called overthinking: o1-like models spending excessive computation on simple problems, with little or no benefit. For you, this means paying and waiting more for an answer that a regular model would give just as well.

Sharp drop on very complex problems

The study The Illusion of Thinking, by Parshin Shojaee and co-authors (June 2025, final version for the NeurIPS 2025 conference), tested reasoning models on puzzles whose complexity could be precisely controlled. The authors report three regimes:

• on low-complexity tasks, regular models did better;
• at medium complexity, reasoning models had the edge;
• at high complexity, both types suffered an accuracy collapse.

The study also observed that reasoning effort grew with difficulty up to a point and then declined, even with token budget still available. And that the models struggled with exact calculations and did not consistently use explicit algorithms. The results apply to the models and puzzles tested, but they reinforce that “thinking more” does not solve every problem.

The displayed reasoning is not always the real one

Many people read the chain of reasoning as an explanation of why the model reached its answer. The study Reasoning Models Don’t Always Say What They Think, by Yanda Chen and co-authors (May 2025), tested this by giving the models hints and checking whether they admitted to using the hint.

In most of the models and settings tested, the rate at which the chain of reasoning revealed the use of the hint was often below 20%. The authors conclude that monitoring the reasoning helps detect undesired behavior, but is not enough to rule it out.

For you, the consequence is direct: well-written reasoning does not guarantee that the answer is correct or that it was the actual path taken. The answer is still subject to hallucinations and needs to be checked.

When to use a reasoning model

In the newsroom’s assessment, based on the sources above, a practical rule is the following:

• Use them for multi-step problems with a verifiable answer: math, code debugging, task planning, analysis of documents with many conditions.
• Avoid them for simple, high-volume tasks: classifying texts, extracting fields, answering direct questions. There, the extra cost and delay rarely pay off.
• Test before adopting: compare the same set of questions with and without reasoning, or with different effort levels, measuring accuracy, cost and time. Criteria in how to evaluate AI responses.
• Reserve room for reasoning in the token limit, so you don’t pay for answers that never arrive.

If the question is about teaching the model facts or style, that is a different subject: see fine-tuning or RAG.

Key takeaways

Reasoning models are language models trained to write intermediate steps before answering. The technique was born as a prompting trick in 2022 and became reinforcement-learning training from 2024 onward, with clear gains on multi-step tasks.

That gain has a price: billed tokens you don’t see, slower answers and the risk of paying without receiving an answer. The research cited shows waste on simple tasks, collapse on very complex problems and chains of reasoning that do not always reflect the real path. Use these models where the difficulty justifies it, adjust the effort and check the result as you would any other AI answer.