Contact us if you want further optimizations to save even more: enterprise@bottlecapai.com

Why do reasoning models overthink?

Reasoning models have changed our expectations around language model performance. Given enough time and enough tokens, they can solve problems that were previously inaccessible to older generations of models.

The downside is that they often think far longer than necessary. Even relatively simple questions can trigger thousands of reasoning tokens:

• revisiting already established assumptions,
• repeatedly reformulating the same argument,
• getting stuck in loops,
• spending more time explaining than solving,
• unnecessary filler words and verbose style for thinking that users do not even read.

This behaviour improves benchmark performance in some settings, but it also introduces costs: higher latency, higher inference spend, lower throughput, increased energy consumption, more compute, and more opportunities for failure.

Long reasoning traces have gradually become associated with intelligence, when in reality they often represent inefficiency. We wanted to cut through all of that, on the machine level. The question we asked was simple: how much of the modern model's reasoning is actually necessary?

The objective

Our goal was intentionally conservative: do not try to make the model smarter or teach it new capabilities. We wanted to preserve knowledge, reasoning ability, answer quality, conversational style, instruction following, and safety behaviour.

The only thing we wanted to change was the amount of computation spent reaching an answer. In other words: keeping the same model, and making sure it overthinks less.

Training approach

Starting from the Qwen3.6-27B (Qwen Team, 2026) base model, we trained on a curated set of problems covering multiple domains and difficulty levels. The training objective rewarded efficient reasoning rather than simply rewarding correctness.

Importantly, the intervention was designed to remain as non-invasive as possible. The resulting model behaves very similarly to the original checkpoint — same style, same capabilities, same knowledge — but with substantially shorter reasoning traces. The model learns to stop once it has enough information to answer confidently.

Evaluation methodology

Evaluating reasoning models is more difficult than evaluating standard language models. At the recommended sampling temperature of 1.0, output quality and length can vary substantially between runs, so single-seed numbers give an incomplete picture. To reduce noise, we evaluated using full benchmark datasets, five independent random seeds, and statistical significance testing across all comparisons.

We evaluated both in-domain tasks (held-out portions of datasets related to training) and out-of-domain tasks (designed to test generalisation). The suite covers scientific reasoning and math, knowledge-based question answering, long-context tasks, system-prompt adherence, coding and agentic tasks, multi-turn conversational behaviour, and safety and model guardrails.

Results

Out-of-domain token efficiency

The chart above shows the mean number of thinking tokens per response on each out-of-domain benchmark. ThinkingCap-Qwen3.6-27B spends far fewer thinking tokens than the base model across the board — on most benchmarks less than half, and on the heaviest reasoning sets (such as GPQA-Diamond) the mean drops by well over 60%.

A natural worry is that simply pushing a model to emit fewer thinking tokens trades away capability — shorter reasoning, worse answers. That is the usual failure mode of token-efficiency work, but it is not what happens here: we deliberately set out to preserve the model's performance, not just to cut its length. Across the same out-of-domain benchmarks, ThinkingCap-Qwen3.6-27B tracks the base model's accuracy almost exactly, despite the large reduction in thinking tokens.

Settings. Base Qwen/Qwen3.6-27B vs bottlecapai/Qwen3.6-27B-Efficient (“Ours”). 5 seeds per condition; thinking on; mean ± 95% CI across seeds. Decoding: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0. Max generation tokens: 100,000 for the general suite (gpqa_diamond, mmlu_pro, longbench_v2, realworldqa) and AA-LCR; 250,000 for HMMT (Nov 2025); 32,768 for supergpqa and livecodebench; 16,384 for ceval and mmlu_redux; 15,000 for llm-system-prompts; 49,152 for Claw-Eval. matched Δ% is the average paired per-question difference in thinking tokens (negative = reduction). looping% is the share of responses stuck repeating the same reasoning chain. macro is the equal-weight mean across benchmarks.

How to read these numbers. The mean thinking-token columns are a plain average over every response, so on their own they can mislead. Picture a model that gives up on the questions it would get wrong — spending almost no tokens there — while thinking for its usual length on the ones it gets right: its mean thinking-token count would look far lower, even though it hasn't reasoned any more efficiently; it has simply stopped trying on the hard questions. A raw average can't separate "thinks more briefly" from "bailed on the tough ones."

The matched Δ% column is built to avoid that: it pairs both models on the same question, measures how much shorter our model's reasoning is there, and averages those per-question reductions — in effect, the expected difference in thinking tokens on a given question. Read together with the accuracy column, it shows the savings come from genuinely more concise reasoning on the same questions — not from answering less.

Across all benchmarks: a -45.8% average reduction in reasoning tokens with an approximately 0.7 percentage-point average accuracy difference. The headline result is straightforward — roughly half the reasoning, with effectively unchanged performance.

In-domain evals

Holdout test splits of datasets whose train splits are part of the finetuning mix — quality retention on in-distribution tasks (in contrast to the out-of-domain benchmarks above).

On these in-domain holdout splits, the token savings are even larger — mean thinking tokens fall by roughly half or more on every benchmark, and accuracy holds or slightly improves across the board — most visibly on GSM8K (93.3% → 96.5%).

Settings. 5 seeds per condition; thinking on; mean ± 95% CI across seeds. Decoding: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0 (bottlecapai/Qwen3.6-27B-Efficient uses the base's sampling). Max generation tokens: 15,000 for GSM8K; 8,192 for the MCQ sets. Data: GSM8K is the full 1,319-row test split; the MCQ sets are capped at 1,000 rows (OpenBookQA = 500 and QASC = 926 are smaller, so full). acc — exact-match on the final answer (GSM8K) / last-letter multiple-choice match (MCQ); matched Δ% and looping% are as defined for the token-efficiency table above.

With focused training on holdout portions of the target datasets, we achieve a -57.7% reduction in reasoning tokens with a 1.0% increase in accuracy.

Production matters more than benchmarks

Unlimited token budgets are not how language models are actually deployed. Real systems operate under constraints: latency targets, context limits, cost ceilings, and throughput requirements. Under those conditions, efficiency matters.

A model that reaches an answer in 2,000 tokens will often outperform one that reaches the same answer in 5,000. Reasoning compression therefore becomes more valuable as deployment scale increases. Our model spends its token budget solving the problem rather than narrating the solution.

A very welcome bonus: more to the point answers

Initially, we aimed to keep the answer style and quality completely unchanged, so that users would never realise they are using a different model without examining the thinking trace.

However, we accidentally applied our shortening also to the text of the answer after the </think> token. We have since fixed it and trained versions that preserve the original answer length.

In internal testing though, we all loved the original model. The style of the answer was still there, but it was much faster and easier to read and generate, and even felt more human — perhaps because humans naturally have a "penalty" for long answers by getting tired writing long texts, something which we introduced into LLMs as well.

We therefore decided to publish this concise model, and save the other checkpoints for future releases.

Download the model, play with it, break it, let us know feedback. And thanks for being here with us. It's not always about the latest model. It's also about making the existing models work better for you. That's what BottleCap's here for.

Get the model on HuggingFace: huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B

Contact us

We are open to feedback, questions, requests, and business partnerships. Please contact us at hey@bottlecapai.com.