How to reduce LLM token usage

Start with retrieved documents, long chat history, tool output, JSON schemas and code that are sent on multiple turns. These sources usually offer more saving than shortening a system prompt by a few lines.

A specific question gives the optimizer a clear relevance boundary. Broad requests such as ‘summarize everything’ legitimately need more of the source and should reduce less.

Keep the sentence that supports a fact together with qualifiers, exceptions, units and dates. A smaller prompt is not useful when it changes the answer.

Return the original input when a candidate cannot establish sufficient coverage. Passthrough is a safe outcome and should be reported separately from an optimization failure.

Measure original tokens minus optimized tokens and any later recovery tokens. Compare the result with answer quality, fallback rate and added optimization latency on representative traffic.