How-to guides
Trim prompt costs
Trim the unnecessary tokens riding along on every call, and confirm quality holds.
AI cost is proportional to token count. So cost per call drops when you leave the model alone and strip what the prompt does not need. Whole conversation histories, verbose instructions, and more examples than necessary are all common.
What rides along on one call
Section titled “What rides along on one call”Open a few of the feature’s calls in Requests and read the breakdown in the detail. Prompt tokens, completion tokens, input cost, and output cost show up separately, so the first thing you learn is whether the cost sits on the input side or the output side.
If it is the input side, open the conversation and see what is actually being sent. There is usually a chunk you can visibly cut.
The trimmed prompt on the same input
Section titled “The trimmed prompt on the same input”Bring the call you just read over with “Open in Testing” and that input becomes the test case itself. Run the current variant and the trimmed-prompt version side by side in Testing: the token breakdown in the results shows the savings immediately, and you compare response quality side by side by eye.
Two numbers to check at 10%
Section titled “Two numbers to check at 10%”Save the trimmed version as a variant and start it at 10% in Routing. Once samples build up, there are two things to read in Compare: does the success metric match the incumbent, and did cost per call drop as expected. If both pass, raise the ratio.
If the output side is the problem
Section titled “If the output side is the problem”Large cost on the output side means responses are running longer than they need to. If a feature that only needs three paragraphs of summary was producing ten, that much was wasted on every call. Make one candidate whose prompt asks for a set length, like “answer in three sentences”, and verify it in the same order.