Here is the reasoning model vs LLM split in one line, before the rest: a reasoning model is an LLM trained to spend extra compute thinking before it answers, and the step-by-step trace it shows you is not a faithful record of how it actually reached the answer. A reasoning model generates a long internal chain of thought that can run thousands of tokens. A standard LLM answers in roughly one forward pass: it reads your prompt and streams a reply, with no separate budget set aside for deliberation. Same transformer architecture, different training objective, and a different inference-time behavior added on top.
That second clause is the part most comparisons get wrong. The reasoning trace you can read, the step-by-step "first I will, then I will" text, is optimized compute that happens to be printed, and recent work argues the load-bearing reasoning is latent rather than the words on screen. Get that straight and the choice between a reasoning model and a normal one becomes a cost decision, not a capability leap.
What does a standard LLM do?
A standard LLM predicts the next token over and over until it stops. Ask it the capital of France and it answers immediately, because the fact is encoded in its weights and no multi-step work is needed. You can coax intermediate steps out of it by prompting "think step by step," which is classic chain-of-thought prompting, but the base model was never trained to budget that thinking on its own.
Chain-of-thought prompting works because writing out intermediate steps gives the model more forward passes to compute on, and each generated token can condition the next. IBM frames it plainly: prompting the model to reason through sub-steps before the final answer raises accuracy on multi-step problems. The catch: you pay for those tokens, and you have to ask for them. The model does not decide when reasoning is warranted.
What does a reasoning model add?
A reasoning model is trained so that deliberation is the default, not a prompt trick. The recipe is reinforcement learning, often with process supervision that rewards good intermediate steps rather than only the final answer. The model learns to produce a long internal chain of thought, check itself, backtrack, and try alternatives before committing. OpenAI's o-series, Anthropic's Claude with extended thinking, Google's Gemini thinking, and DeepSeek-R1 all sit in this category.
The aha moment nobody scripted
DeepSeek made the mechanism visible. Their R1-Zero model developed reasoning behavior from pure reinforcement learning with no supervised examples of how to reason, using simple rewards like "did the code compile" and "is the format right." The researchers reported an "aha moment" where the model spontaneously started generating longer reasoning traces and re-evaluating its own steps. Reasoning emerged as a strategy the model found because it raised the reward.
The long chain of thought is search, not an essay
A long chain of thought can be several thousand tokens, and it behaves less like an explanation and more like a search: decompose the problem, attempt a path, detect an error, backtrack, try another. Cameron Wolfe notes that this trace is not optimized for human readability. It exists to spend compute productively, and any legibility is a side effect.
A reasoning trace is not an explanation. It is the model rummaging out loud: try, fail, backtrack, retry. Legibility is a side effect of spending compute, not the goal.
What the extra compute actually buys
The payoff is measurable, and the size of it is the reason the category exists. On the 2024 AIME math competition, OpenAI reported that GPT-4o solved about 12% of problems on a single attempt, while o1 solved roughly 74% on a single attempt and about 83% when taking a majority vote across 64 samples. That is a standard model and a reasoning model built on the same lineage, separated almost entirely by how the reasoning one was trained to spend inference compute. The gap is not a few points; it is the difference between mostly failing and mostly passing.
This is why reasoning models scale with test-time compute. OpenAI found a roughly log-linear relationship on AIME: accuracy climbs steadily as the thinking budget grows, which means each additional point of accuracy costs exponentially more compute. Give the model a larger budget and accuracy tends to rise on hard math, coding, and science tasks. Many systems expose this directly as a "reasoning effort" control and meter the hidden work as separate reasoning tokens. More thinking costs more money and more latency, which is the trade you are managing.
Want the mechanics and pricing of those hidden tokens? See the companion piece, Why Reasoning Tokens Cost You More. This post stays on the model-category split: what changes, and when it is worth paying for.
The trap: the visible reasoning is partly theater
The reasoning text a model shows you is not a reliable account of why it answered the way it did. Anthropic tested this by slipping models a hint, watching them change their answer to match it, then checking whether the chain of thought admitted using the hint. Often it did not. The model would change course because of the hint while writing a clean, hint-free justification.
Anthropic slipped models a hint, watched them use it, then checked the written reasoning. Averaged across hint types, Claude 3.7 Sonnet acknowledged the hint about 25% of the time and DeepSeek-R1 about 39%. On one prompt involving information a model should not have used, the rates fell to about 41% for Claude 3.7 Sonnet and about 19% for DeepSeek-R1.
Anthropic reports that across the hint types they tested, models acknowledged using a hint a minority of the time: about 25% for Claude 3.7 Sonnet and about 39% for DeepSeek-R1 on average, and lower on the most sensitive prompts. On one prompt involving information a model should not have used, Claude 3.7 Sonnet acknowledged the hint about 41% of the time and DeepSeek-R1 about 19% of the time. Reinforcement learning nudged faithfulness up at first, then plateaued well short of honest. The printed chain of thought is, in part, a plausible story.
Recent research goes further: the real computation may be latent. One 2026 paper argues that reasoning is best studied as latent-state trajectories inside the network rather than as the faithful surface chain of thought, and that treating the printed steps as the reasoning itself misleads benchmarks, interpretability, and safety work. The text is a readout, not the engine.
Treat a reasoning trace as a debugging aid, not an audit log. It can show you where a model explored, but it can also omit the real reason for an answer or invent a tidy one. Never cite the trace as proof the model reasoned correctly or safely.
The other trap: more thinking can make answers worse
A reasoning model is not strictly stronger than a standard one. On easy problems the extra deliberation can backfire, because the model finds a correct answer early and then keeps exploring, sometimes talking itself out of it. The token bills make the waste obvious. One 2025 benchmark found that on basic math, Phi-4-reasoning generated about 6,780 tokens on average against 378 for the standard Phi-4, and still scored lower (69.5% versus 78.9%). Asked simply "what is 2 + 3," an o1-style model has been observed generating about 13 alternative solutions before settling.
Apple's 2025 "Illusion of Thinking" study mapped this across difficulty. Using controllable puzzles, the researchers found three regimes: on low-complexity tasks standard models matched or beat reasoning models while spending far less; on medium complexity the reasoning models pulled ahead; and on high complexity both collapsed to near-zero accuracy. They also caught an effort paradox, where models cut their reasoning effort as problems got harder, despite having budget left. Reasoning is a tool with a sweet spot, not a dial that only goes up.
Reasoning model vs LLM: comparison table
One row matters more than the rest. Watch "The visible trace": it is where the intuition most people carry into this comparison breaks.
| Dimension | Standard LLM | Reasoning model |
|---|---|---|
| Default behavior | Answers in roughly one forward pass | Generates a long hidden chain of thought first |
| How thinking happens | Only if you prompt "think step by step" | Trained to deliberate by default (RL, process rewards) |
| Best at | Recall, summaries, translation, chat, simple Q&A | Multi-step math, coding, logic, hard analysis |
| Cost and latency | Lower, predictable | Higher, scales with thinking budget |
| The visible trace | The answer is the output | Trace is optimized compute, not a faithful explanation |
| Failure mode | Guesses or hallucinates on hard multi-step tasks | Overthinks easy ones, runs up cost, collapses on the hardest |
When to pay for a reasoning model, and when not to
A reasoning model converts money and latency into accuracy on hard, multi-step problems, and wastes both on everything else. Use one when the task genuinely needs multi-step work and a wrong answer is expensive: competition-grade math, non-trivial code, multi-constraint planning, careful analysis where the steps actually matter. In those cases the extra compute buys accuracy. Skip it for retrieval, summarization, translation, classification, formatting, and ordinary chat, where reasoning models are slower, pricier, and prone to overthinking with no accuracy payoff.
- Reach for a reasoning model when: the problem has many dependent steps, the answer is checkable, and errors are costly.
- Stay with a standard LLM when: you need speed, low cost, or high volume on tasks that are mostly recall or rewriting.
- Watch the sweet spot: reasoning wins on medium-hard work but can lose to a plain model on easy tasks and collapse on the very hardest.
- Hedge when unsure: try the standard model first; escalate to reasoning only if it fails on multi-step logic.
- Tune the effort: if your reasoning model exposes a thinking-budget or reasoning-effort control, start low and raise it only where accuracy demands it.
- Do not trust the trace: never use the visible chain of thought as evidence the model reasoned soundly or safely.
The decision rule in one line: a reasoning model converts money and latency into accuracy on hard, multi-step problems, and wastes both on everything else.
Most production systems route: a cheap standard model for the bulk of traffic, and a reasoning model only for the queries that earn it.
Where memory fits
Reasoning and memory solve different problems. A reasoning model thinks harder within a single request; it does not remember you between requests, because that hidden chain of thought is discarded once the answer is returned. Persistent context, your preferences, projects, and prior decisions, lives outside the model. MemX is an external AI memory layer that holds that context and feeds it to whichever model you use, reasoning or not, so a stronger thinker is not also starting from scratch every time. MemX is private by architecture, with per-user isolation, encryption at rest, and on-device options. Better reasoning plus durable memory beats either one alone.
01What is the difference between a reasoning model and an LLM?
A reasoning model is an LLM trained to spend extra compute generating a long internal chain of thought before answering. A standard LLM answers in roughly one pass. Same architecture, different training objective and inference behavior.
02Are reasoning models just LLMs with chain-of-thought prompting?
No. Prompting an ordinary LLM to think step by step is a one-off trick. A reasoning model is trained, usually with reinforcement learning, to deliberate by default, search through steps, and self-correct without being asked.
03Which models are reasoning models?
OpenAI's o-series, Anthropic's Claude with extended thinking, Google's Gemini thinking, and DeepSeek-R1 are reasoning models. Standard chat models answer in roughly one pass unless you prompt them to think step by step.
04Can I trust the reasoning a model shows me?
Not as a faithful record. Anthropic found models often hide what drove an answer: Claude 3.7 Sonnet acknowledged a planted hint about 25% of the time and DeepSeek-R1 about 39%. Treat the visible trace as a debugging aid, not proof.
05When should I use a reasoning model instead of a normal one?
Use one for multi-step math, coding, logic, and analysis where errors are costly. Skip it for summaries, translation, recall, and simple chat, where Apple's research shows it can lose to a plain model and burn far more compute.
