How-to guides
Respond to a cost spike
Narrow the cause of a sudden cost increase from the feature to the variant to a single call.
It starts with one invoice that is 40% higher than last month. Staring at the total only gets you “let’s all spend less”, so narrow the spike down three times instead.
A step or a steady climb
Section titled “A step or a steady climb”Get the cost ranking by feature from Insights (how to ask is covered in See spend by feature). What matters here is the follow-up “when did it start climbing?”, which pins down the date.
A step up from a specific date points at a deploy or change made that day. A steady climb points at traffic growth or a shift in usage patterns. If outcomes grew alongside, this is growth; cost rising on its own is what needs action.
Cost per call splits the variants
Section titled “Cost per call splits the variants”In Compare, put the feature’s variants side by side and read the cost per call. Each variant is a different model and prompt, so the table shows which combination is driving the cost.
If the feature runs a single variant, there is nothing to compare, so go straight on.
A large prompt or a long response
Section titled “A large prompt or a long response”In Requests, filtering by the model the previous step pointed to leaves only that variant’s calls. Opening one call’s detail shows the token and cost breakdown, which tells you whether the input or the output grew.
| Symptom | Common cause | Action |
|---|---|---|
| Prompt tokens spiking | Conversation history or context stuffed in whole | Trim prompt costs |
| Completion tokens spiking | Responses run longer than they need to | Add a length instruction to the prompt, validate the candidate in Testing, then save it as a variant |
| Call count spiking | Retry loop, crawler, or one abusive user | Spot the pattern in the request list’s session and user columns, then fix the code or policy |
| Expensive model | Model overkill on trivial work | Switch to a cheaper model |
Fix it, then confirm on Overview
Section titled “Fix it, then confirm on Overview”Whether it is a prompt or a model, the fix follows the same order: confirm quality holds on the same inputs in Testing, save it as a variant, and raise its ratio in Routing.
Whether the savings actually landed shows on the cost trend of Overview’s 30-day range. Opening that screen once a month is enough to catch the next spike before the invoice does.