Compare

Compare is where you go a few days after splitting traffic in Routing, to read the result. It is also the screen to watch in the first days after a new variant starts taking traffic, in case error rate or latency spikes. Per-variant numbers are read here, side by side.

Success metricsdefine the bar Testingtry candidates Routingroute traffic Comparejudge outcomes

The chart and the variant cards

Area	What you see	How to read it
Basis selector	Choose an event’s conversion rate or a success metric as the basis, and pick the two variants to compare	A Primary metric, if there is one, is selected from the start
Chart	The chosen basis over time, one line overlaid per variant	Read a difference only when the lines separate and hold their gap
Variant cards	Per variant: cost per call, p50 latency, error rate, and the value of the basis you chose	Four numbers, read side by side

Under 100 users per variant, the screen shows a warning

The bar counts people who got that variant, not calls. When that number is small, conversion swings heavily on luck. While the warning is up, do not say which side is better; wait for more traffic to accumulate. The same bar decides whether you are ready to raise a ratio in Routing.

ABTO does not hand down statistical verdicts like p-values. It shows the numbers as they are.

Judge in this order

Order	Metric	The call
1	Error rate	A variant with visibly higher errors is out, regardless of outcomes
2	Success metric or conversion	Is the new candidate level with or ahead of the incumbent
3	Cost per call	At similar outcomes, pick the cheaper one

If conversion is slightly higher at several times the cost, whether that is worth it is a business decision. Once the choice is settled, go back to Routing and move the traffic.