Google's Deep-Thinking Ratio: Boosting LLM Accuracy While Slashing Inference Costs by 50%

A New Google AI Research Proposes Deep-Thinking Ratio to Improve LLM Accuracy While Cutting Total Inference Costs by Half

Researchers from the University of Virginia and Google have developed a novel metric called the Deep-Thinking Ratio (DTR) to measure internal model effort. Their findings reveal that raw token length has a negative correlation (r = -0.59) with accuracy, debunking the ‘longer is better’ myth in Chain-of-Thought reasoning. By identifying tokens that stabilize only in the final 15% of transformer layers, they achieved higher accuracy with half the compute.

Why This Matters

The technical reality of current LLMs contradicts the ideal of linear improvement through longer Chain-of-Thought (CoT) traces. In practice, ‘token maxing’ often leads to overthinking, where models get stuck in redundant loops or amplify early mistakes, resulting in a negative correlation between output length and correctness. This inefficiency wastes massive amounts of expensive compute on uninformative tokens that do not contribute to logical resolution.

By shifting the focus from output volume to internal layer-wise stability, the Deep-Thinking Ratio provides a mathematical framework for ‘early halting.’ This allows systems to prune unpromising reasoning paths after just 50 prefix tokens. On the AIME 2025 benchmark, this approach reduced total inference costs from 307.6k to 155.4k tokens while simultaneously raising accuracy from 92.7% to 94.7%.

Key Insights

• Raw token count is a poor predictor of accuracy, exhibiting an average negative correlation of r = -0.59 across reasoning models.
• Deep-thinking tokens are defined as those that undergo significant revision in deeper transformer layers, only stabilizing in the ‘late regime’ (final 15% of layers).
• The Deep-Thinking Ratio (DTR) uses Jensen-Shannon Divergence (JSD) to measure the divergence between intermediate hidden state drafts and final layer distributions.
• Think@n strategy implements early halting by evaluating the DTR of multiple candidates after 50 prefix tokens and terminating unpromising samples.
• Benchmarks on AIME 2025 show that Think@n outperforms standard majority voting (Cons@n) while reducing total inference costs by approximately 49%.

Practical Applications

• Use case: Implementing Think@n in math-heavy reasoning systems like DeepSeek-R1-70B to reduce token consumption during multi-sample majority voting.
• Pitfall: Relying on ‘Token Maxing’ as a proxy for reasoning depth, which frequently results in repetitive loops and decreased overall accuracy.
• Use case: Utilizing layer-wise prediction stability (JSD) to monitor model confidence in real-time for logical symbols and complex mathematical tokens.
• Pitfall: Failing to identify ‘shallow tokens’ that stabilize early (e.g., layer 5 of 36), leading to wasted compute in deep-thinking specialized models.

References:

• https://www.marktechpost.com/2026/02/21/a-new-google-ai-research-proposes-deep-thinking-ratio-to-improve-llm-accuracy-while-cutting-total-inference-costs-by-half/