Testing

Testing is the step you take once you have a model or prompt you want to change, before anything reaches real users. Calls do go to the real provider, but the results are not stored anywhere, so nothing mixes into production data and you can run trials as many times as you want.

Success metricsdefine the bar Testingtry candidates Routingroute traffic Comparejudge outcomes

The request body and the slots

Area	What you see
Request body	The request body every slot sends. “Load from history” brings in a real call’s earlier conversation, tool calls and results, and system prompt exactly as recorded, and only the last user message can be edited. Image and PDF attachments are shown as placeholders and are not included in a run. Without loading a call, it starts with one empty user message
Slots	Each slot takes a name, model, system prompt, and parameters. With “Custom” on, the slot’s own system prompt is sent; with it off, the system prompt in the request body goes out unchanged. “Load” fills a slot from a saved Agent Config. On first open, A holds Origin (the setup your code currently sends), and you can add up to five slots
Few-shot examples	In a slot with “Custom” on, “Add few-shot” opens an example dialog. Type an input and answer yourself, or use the dialog’s “Load from request history” to pick a real call and fill in its last user message and response. The example goes into an <examples> block in the slot’s system prompt, and it runs and saves as written
Results	Per slot: the response, why it stopped, tokens, cost, latency, and the prompt-versus-completion token breakdown. The response is split into tabs for reasoning, tool calls, and text in the order the model produced them, opening on the final output. A response that stops to call a tool shows that call as it is

“Run all” runs every slot at once. “Save as Agent Config” saves the slot’s model, system prompt, and parameters as a named Agent Config. You can bring a saved Agent Config back into a slot with “Load”.

Which parameters you can tune depends on the model. Pick one without tunable parameters and the screen says “No tunable parameters for this model.” The request body and slot setups persist until you close the browser.

Using a saved setup on real traffic

A saved setup is added as an Agent Config of the node you are testing. When you create a variant in Routing, pick this Agent Config as that node’s setup, and requests from users who get the variant are answered with it. The save notice’s “Create a variant with it” opens the variant dialog with this Agent Config already picked.

The final verdict happens in Compare

Testing only takes you as far as screening candidates on a few chosen inputs. A candidate that looked great in Testing can behave differently on real traffic. An experiment is only finished once you confirm the outcome in Compare.