Routing

Routing is the step where a variant you saved in Testing starts meeting real users. Move the sliders, save, and it applies from that moment. No deploy, no config file edits.

Success metricsdefine the bar Testingtry candidates Routingroute traffic Comparejudge outcomes

The ratio graph and the sliders

Area	What you see
Ratio graph on top	A picture of how traffic is currently divided across the variants
Variant rows	One slider per variant. Change one row and the others adjust automatically so the total always stays 100. Nothing applies until you press Save
”Results in Compare”, top right	Takes you to the results for the split you are currently running

If someone else saved ratios, or a code change created a new Origin variant, after you opened the screen, saving stops and asks you to reload. The ratios you were editing stay on screen.

Adding variants and rolling back

Action	How
Create a variant	Press “New variant”, name it, then pick an Agent Config saved in Testing with “Use existing” or enter a model and prompt directly. A new variant is added to the list at zero, and is created only when you save routing
Change a variant’s setup	Only a variant’s name can change once created. To use a different model or prompt, create a new variant with that setup and move the old variant’s ratio to it
Turn a variant off	There is no on/off switch. Zero is off
Roll back	Return the new variant to zero and the previous one to 100, and it is done. It takes seconds
How many variants	A policy holds at most five variants. Dropping a ratio to zero keeps the slot, so use “Remove from routing” to free one

Requests routed to a provider whose key is missing simply fail

If you add a model from a provider you have not used before, the code that calls AI has to send that provider’s API key along with the request. Without it, only the requests routed to that variant fail, and those failures never show up on Requests. When you create or route a variant from a provider that is not in routing yet, the dashboard shows this as a confirmation first. Check the key before you raise the ratio.

Archiving variants and Agent Configs

Archive a variant you no longer use from the “All variants” list. Nothing is deleted, so its results stay in Compare.

Item	When you can archive it	What archiving does
Variant	Once it is removed from routing. The current Origin variant cannot be archived	It leaves the Routing list and ratio saves
Agent Config	Any time	It can no longer be picked for a new variant or loaded in Testing. Variants already using it keep responding with it

Archived items collect in a collapsed section at the bottom of the list, where you can restore them. A restored variant only returns to the list, so save a ratio to send it traffic again.

Allowing system prompts

This is a per-node setting, and the default is not allowed. You can change it for each node in the System prompt card on the Routing screen.

• Allowed: Requests assigned to a variant with a system prompt use the variant’s system prompt. Requests assigned to a variant without one use the original request’s system prompt.
• Not allowed: Every request to that node uses the original request’s system prompt. System prompts set on variants are kept but not used, and the variant list marks them as not applied.

A change applies to requests from that point on.

Dynamic system prompts

A dynamic system prompt carries content that changes per request, such as user information. A system prompt set on a variant is the same for every request, so requests assigned to a variant with a system prompt do not pass that per-request content to the model.

System prompts in the Origin variant

Origin variants are created without a system prompt, and requests assigned to an Origin variant use the original request’s system prompt. A request whose system prompt alone changed does not create a new Origin variant.

Assignment is sticky per person

Which variant a user gets is computed from their device, so the same person keeps getting the same variant call after call. Their experience stays steady, and conversion in Compare splits cleanly at the person level. Once you change the ratios, assignment follows the new ones from that point, so some users move to a different variant.

How to raise ratios

Stage	Ratio	What to confirm
Right after saving a new variant	10%	The new variant’s error rate and average latency over a day or two
If nothing looks off	30 to 50%	Whether the new variant is level with or ahead of the incumbent
Once the difference is clear	100%	Drop the variant you are not keeping to zero. After the switch, watch cost and outcome trends on Overview’s 30-day range

You confirm each stage in Compare, and how much sample is enough to judge follows that screen’s rule too. Change one thing at a time on a feature: swap the model and the prompt together and you cannot tell which change made the difference.