As of May 2026. Gemini 3 Flash launched on December 17, 2025 with a clear pitch from Google: "frontier intelligence built for speed." It is a genuinely strong frontier model and it wins on the workloads it was built for. The MemX classification workload just wasn't one of them. MemX benchmarked it on a real production workload four months later. 103 classification test cases, 5 runs each, real token counts from `usage_metadata`. The result was unexpected. Gemini 3 Flash single-prompt scored 73.4% accuracy on the MemX classification task. The MemX production model, Gemini 2.5 Flash-Lite running a two-stage pipeline, scored 99.4%. On the MemX workload, Gemini 3 cost 10x more per case, ran 76% slower at P50, and flapped on 29 of 103 cases versus 3 on Flash-Lite. These are symptoms of pairing a reasoning model with a non-reasoning job, not symptoms of a weak model.
Editor's note added May 29, 2026: Google released `gemini-3.5-flash` as the GA replacement on May 19, 2026 at Google I/O, with a new pricing tier of $1.50 input and $9.00 output per million tokens. The `gemini-3-flash-preview` ID we benchmarked remains available (Google has not announced a shutdown date). The architectural finding below (thinking-by-default models lose to non-thinking models on production classification) applies equally to 3.5 Flash, which inherits the same thinking-first design and an even higher per-token cost. The recommendation does not change: pick the model by workload fit, not by intelligence index.
This is not an anti-Gemini-3 post. Gemini 3 Flash is a frontier model: 78% on SWE-bench Verified (beating Gemini 3 Pro at 76.2%), 71 on the Artificial Analysis Intelligence Index (a 13-point improvement over Gemini 2.5 Flash's 58), and top score on AA-Omniscience. This is an anti-using-it-for-the-wrong-job post. For production intent classification with sub-second latency budgets and six-figure monthly token volumes, a thinking-by-default reasoning model is the wrong tool. Google's own documentation says so, in plain English.
Quick takeaways. Gemini 3 Flash is a thinking model with `thinking_level` defaulting to high; thinking tokens bill at the output rate. Google's own docs say classification does not require thinking, and the example they give is email classification. Liu et al. (ICML 2025) measured a 36.3% absolute accuracy drop on implicit statistical learning when chain-of-thought was forced (o1-preview vs GPT-4o zero-shot). Frontier models for frontier problems; mid-tier for production classification.
What Gemini 3 Flash is, and what Google promised
Launch date: December 17, 2025. Positioning: Pro-grade reasoning with Flash-level latency, efficiency, and cost. Pricing: $0.50 per million input tokens, $3.00 per million output tokens. Compare Gemini 2.5 Flash-Lite at $0.10 per million input, $0.40 per million output. Gemini 3 Flash costs 5x more on input and 7.5x more on output. The pitch was that better intelligence offsets the higher per-token cost. For some workloads that math works; for classification, it does not.
Architecturally, Gemini 3 Flash is a reasoning model. It exposes a `thinking_level` parameter (minimal, low, medium, high) and defaults to high. The Gemini 3 family always runs dynamic thinking with a maximum allowance of 8,192 thinking tokens per call. Google's pricing page confirms thinking tokens bill at the output rate. You pay for the internal monologue, not just the final answer. If you do not set `thinking_level` explicitly, every call burns maximum reasoning by default.
Our methodology: 103 cases, 5 runs, real token counts
Intent-classification eval. 103 test cases across 15 categories: save_memory, search_memories, general_chat, edge_case, multilingual, mixed_intent, multi-action, follow_up_reminder, language_bias, context_disambiguation, meta_conversation, modify_memory, recurring, ambiguous_save, intent_based. Each case ran 5 times per configuration. Real token counts came from `usage_metadata`, no estimates. We measured P50, P90, and mean latency. A case counts as flapped when the 5 runs produce more than one distinct outcome.
Five configurations were tested: Gemini 2.5 Flash-Lite two-stage (production baseline), Gemini 2.5 Flash single-prompt, Gemini 2.5 Flash two-stage, Gemini 3 Flash single-prompt, and Gemini 3 Flash two-stage. The single-prompt setup uses one giant prompt that classifies and executes. The two-stage setup is the architecture covered in a separate MemX post: Stage 1 classifies intent, Stage 2 runs the action-specific prompt.
The numbers
| Configuration | Accuracy | P50 latency | Cost/case |
|---|---|---|---|
| Gemini 2.5 Flash-Lite two-stage (production) | 99.4% | 1,621ms | $0.000569 |
| Gemini 2.5 Flash two-stage | 98.1% | 1,899ms | ~$0.00194 |
| Gemini 2.5 Flash single-prompt | 96.1% | 1,391ms | ~$0.00155 |
| Gemini 3 Flash two-stage | 95.0% | 2,985ms | $0.002866 |
| Gemini 3 Flash single-prompt | 73.4% | 2,853ms | $0.005774 |
Per-category breakdown on Gemini 3 Flash single-prompt: 12% on search_memories (catastrophic), 50% on multi_action, 57% on follow_up_reminder, 58% on multilingual, 75% on language_bias. The two-stage pipeline rescues classification accuracy from 73.4% to 95.0%. Even with that architectural fix, Gemini 3 Flash never beats Flash-Lite on accuracy, costs 5x more, and runs slower.
Why a thinking model hurt classification
Google's own thinking documentation states it cleanly: "For straightforward requests where complex reasoning isn't required, such as fact retrieval or classification, thinking is not required." The example Google gives is email classification: "Is this email asking for a meeting or just providing information?" Intent classification is structurally identical. Pick the right label from a small set. There is no reasoning chain that needs to unfold. The thinking process is overhead.
Liu et al.'s ICML 2025 paper "Mind Your Step (by Step)" proved this empirically. The headline result: a 36.3% absolute accuracy drop on implicit statistical learning when OpenAI o1-preview used chain-of-thought versus GPT-4o zero-shot. The authors tested six representative tasks where deliberation hurts humans (implicit statistical learning, facial recognition, classification with exceptions) across nine frontier models. The Chegini et al. "Reasoning's Razor" paper (Apple, October 2025) extended the finding to safety and hallucination detection: reasoning improved overall accuracy but underperformed at the low-false-positive-rate thresholds essential for production.
The mechanism is intuitive once you see it. Thinking tokens are stochastic. Each step in the chain is a chance for the model to talk itself into a different answer. For a one-token classification task, the extra steps add no upside and amplify variance. That is why 29 of 103 cases flapped on Gemini 3 Flash and only 3 flapped on Flash-Lite. The same reasoning that earns Gemini 3 Flash a 78% on SWE-bench Verified is exactly what works against you on a yes-or-no intent decision.
Where Gemini 3 Flash actually wins
Honest section, because no model is universally good or bad.
- Coding: 78% on SWE-bench Verified, beating Gemini 3 Pro at 76.2%. If you are building Cursor or Cody competitors, Gemini 3 Flash is strong.
- Math and knowledge: 33.7% on Humanity's Last Exam (no tools), 90.4% on GPQA Diamond. Top score on AA-Omniscience in its price tier at launch.
- Long-horizon agents: thought signatures (encrypted reasoning context passed across API calls) are designed for agentic loops. Right tool for autonomous multi-step work.
- Multimodal reasoning: 81.2% on MMMU-Pro, edging Gemini 3 Pro. If the task is reading a chart and explaining the trend, Gemini 3 Flash is the right call.
What these four wins share: the task benefits from extended reasoning. Code is a reasoning task. Math is a reasoning task. Agents plan. Multimodal interpretation needs synthesis. None of those characteristics describe intent classification.
The cost math, monthly
For 1,000 users sending 10 messages per day:
| Configuration | Monthly cost | vs baseline |
|---|---|---|
| Gemini 2.5 Flash-Lite two-stage (production) | $170.80 | 1.0x |
| Gemini 2.5 Flash single-prompt | ~$465 | 2.7x |
| Gemini 2.5 Flash two-stage | ~$582 | 3.4x |
| Gemini 3 Flash two-stage | $859.81 | 5.0x |
| Gemini 3 Flash single-prompt | $1,732.24 | 10.1x |
Same workload. Same user base. The gap between the MemX production model and Gemini 3 Flash single-prompt is roughly $1,561 per month per 1,000 active users. For a 100K-user product, the same routing decision costs $156K per month. The bill for using the wrong model on the right job is real money.
Counterarguments worth addressing
Three objections worth meeting head-on.
- "You held it wrong; single-prompt is not fair to a reasoning model." Fair. The two-stage Gemini 3 Flash result (95.0%) is the fair comparison. Even at 95.0% it loses to Flash-Lite (99.4%) on accuracy and costs 5x more. The architectural rescue helps, but does not change the verdict.
- "103 cases is too small." 103 cases x 5 runs equals 515 LLM invocations per configuration. The MemX run reported variance ranges and flap counts; the methodology beats most public benchmarks. Anthropic and Google both recommend domain-specific evals for production decisions, which is exactly the MemX approach.
- "Gemini 3 is a leap forward; you are cherry-picking." Conceded. Gemini 3 Flash is the most intelligent model for its cost on the Artificial Analysis Index. It wins SWE-bench. It wins GPQA. The claim of this post is narrower: for production intent classification with sub-second latency budgets and six-figure monthly token volumes, a thinking-by-default model is the wrong tool.
The playbook: right-size, then two-stage
Anthropic's "Choosing a model" doc has the cleanest framing: start with Haiku 4.5, test your use case on your own data, check whether performance meets your requirements, and upgrade only if you hit a specific capability gap. Combine that guidance with a two-stage pipeline (Stage 1 classifies, Stage 2 executes) and most production teams will land on a sub-$200 monthly classification bill at moderate scale.
The MemX two-stage Flash-Lite pipeline beat Gemini 3 Flash single-prompt at 1/10th the cost, because the pipeline architecture matched the workload. Gemini 3 Flash is excellent at what it was built for. Classification was not it. The cost of using a frontier reasoning model on a yes-or-no decision is real money, and Google's own thinking docs flag exactly that mismatch.
Key takeaway. Do not pick LLMs by intelligence index. Pick LLMs by fit-to-workload. Thinking models win at reasoning, coding, math, and agents. They lose to smaller non-thinking models on classification, at higher cost and higher latency. The Anthropic playbook (start with the cheap model) plus a two-stage pipeline gives you the best per-dollar accuracy on classification in 2026.
01Is Gemini 3 Flash better than Gemini 2.5 Flash-Lite?
For reasoning, coding, math, and agentic tasks: yes, materially. For intent classification and similar non-reasoning tasks: no. In the MemX 103-case benchmark (as of May 2026), Gemini 3 Flash single-prompt scored 73.4% vs Flash-Lite two-stage at 99.4%, at 10x the cost and 76% slower P50 latency. Pick by workload fit, not by intelligence index.
02Why does Gemini 3 Flash cost so much more than 2.5 Flash-Lite?
Pricing is 5x higher on input ($0.50 vs $0.10 per million tokens) and 7.5x higher on output ($3.00 vs $0.40 per million). Gemini 3 Flash is a thinking model running dynamic thinking up to 8,192 tokens per call, billed at the output rate. For tasks that do not benefit from extended reasoning (like classification), you pay for the thinking and get no upside.
03What does Google say about using Gemini 3 Flash for classification?
Google's own Gemini Thinking documentation says: "For straightforward requests where complex reasoning isn't required, such as fact retrieval or classification, thinking is not required." The example they cite is email classification: "Is this email asking for a meeting or just providing information?" Intent classification is structurally identical. Use 2.5 Flash-Lite for classification; save Gemini 3 Flash for reasoning-heavy work.
04When SHOULD I use Gemini 3 Flash?
When the task benefits from extended reasoning. Coding (78% on SWE-bench Verified), math and knowledge (GPQA Diamond 90.4%), long-horizon agentic workflows (thought signatures), multimodal reasoning (MMMU-Pro 81.2%). For these workloads the thinking-mode overhead pays for itself in better answers. For classification, search routing, or short structured output, it does not.
05Does the chain-of-thought paper really show that reasoning hurts classification?
Yes, with caveats. Liu et al.'s ICML 2025 paper "Mind Your Step (by Step)" measured a 36.3% absolute accuracy drop on implicit statistical learning when OpenAI o1-preview used chain-of-thought versus GPT-4o zero-shot, plus consistent reductions across nine frontier models on six tasks where deliberation hurts humans. The Chegini et al. Apple "Reasoning's Razor" paper extended the finding to safety and hallucination detection at low-false-positive-rate thresholds. Reasoning helps reasoning tasks; it hurts pattern-recognition classification tasks.
06How do I right-size an LLM for my workload?
Anthropic's official guidance: start with the cheap model (Haiku 4.5, Gemini 2.5 Flash-Lite, GPT-4.1 nano). Test on your own data. Upgrade only if performance does not meet your requirements. Combine with a two-stage pipeline (Stage 1 classifies, Stage 2 executes) for additional accuracy and cost wins. Do not pick by intelligence index; pick by accuracy on your golden set.
07What about the new Gemini 3.1 Flash-Lite?
Google launched Gemini 3.1 Flash-Lite on March 3, 2026 at $0.25 per million input / $1.50 per million output. It is a strong upgrade path from 2.5 Flash-Lite for classification workloads (2.5x faster time-to-first-token, ~94% intent routing accuracy in early production reports). The verdict in this post does not change: pick by fit-to-workload, not by intelligence index, and run the two-stage pipeline.
