Reason only where it helps: a local 8B model, from 4.7 to 0.7 seconds when writing
In a real turn, the call that only wrote the answer generated 954 reasoning tokens: 96 of the 141 seconds. Switching reasoning off right there cut that call from 4.7 to 0.7 s, but it had to be switched back on in two specific cases.
- LLM locales (Ollama, Qdrant, RAG)
- Python
The problem
qwen3 can answer «thinking» (it generates internal reasoning before replying) or directly. Thinking made a call six times slower.
A turn with tools has at least two model calls: one deciding which tool to use and with what arguments, and another writing the answer with the result in front of it. The second was spending hundreds of tokens thinking about something already settled.
The rule
- No tool run yet: think. Choosing a tool and working out arguments needs it: without reasoning, the model got relative dates wrong and made up durations.
- Result with data and no warnings: direct. All that's left is writing.
- Empty or failed result: think again. Another step has to be decided.
- A warning was injected next to the result: think. The warning asks for a decision.
- If the direct call asks for another tool, it's repeated with thinking on.
What broke at first
The first version simply switched reasoning off after the first tool. It gained speed and lost two things the suite took a while to show:
- After an empty search, it never tried another source: it said «I found nothing» with the answer one call away.
- It ignored the warnings the code adds to the result: it kept giving an address without a port even when the warning said one was missing.
Result
On the 9 interpretation cases with 5 seeds: 45/45 in both modes, with 31 output tokens instead of 258. Full turns 25 % to 40 % faster, and the whole suite from 10.1 to 8.0 minutes.