What ABTO solves

Say you built a review summary feature for a shopping site. They say a cheaper model would cut the cost in half, but if quality drops and purchases fall, you lose more than you save. Right now, all you know is last month’s API bill.

ABTO is the tool that turns this dilemma into numbers.

AI callscost and latency recorded User behaviorpurchase, copy, leave Per-feature model performancedata, not gut feel

It joins the call records with what the users who saw each response actually did, showing which model truly performs.

The two questions ABTO answers

Can we cut AI cost?

• Break cost down by feature
• Trace an increase down to the individual call
• Switch to a cheaper model only after confirming performance holds

Are today's model and prompt the best we can do?

• Try new candidates on real customer inputs
• Validate on a slice of traffic and switch to the better variant
• If something goes wrong, roll back instantly from the dashboard

No code deploy anywhere in this process

Switching models and rolling back both finish in the dashboard. You can follow the actual sequence in the how-to guides.

How this differs from public benchmarks

	Public benchmarks	ABTO
Evaluation inputs	Standard problem sets	Inputs your real customers sent
Outcome criteria	Answer-key grading, evaluator scores	Real user behavior like purchases and conversions
Unit of judgment	The model as a whole	Each individual feature in your product

A model that excels at review summaries can be mediocre at support replies. So ABTO re-ranks models per feature on your real traffic, and as traffic accumulates, that ranking becomes a living benchmark unique to your product.

What adoption takes

All it takes is pointing the OpenAI SDK you already use at the gateway and carrying a few headers on each call.

client = OpenAI(
    base_url="https://gateway.abto.app/v1",  # point the base URL at the ABTO gateway
    api_key=os.environ["ABTO_CALLING_KEY"],
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages,
    extra_headers={
        "x-abto-key-openai": os.environ["OPENAI_API_KEY"],  # the provider key you already use
        "x-abto-feature-id": "review.summary",              # which feature this call belongs to
        "x-abto-device-id": device_id,                      # whose call this is
    },
)

• Call models from different model providers (OpenAI, Claude, etc.) the same way.
• Keep your existing provider contracts and API keys.
• To connect user behavior as well, add one small SDK to your web or app. The browser build has no runtime dependencies.

Who ABTO is for

• Your AI cost keeps growing and you cannot break it down by feature
• You want a cheaper model but fear the quality drop too much to switch
• You keep editing prompts and only gut feel tells you whether they improved
• You need to answer “does our AI feature contribute to revenue” with numbers

Next, learn just the six terms in Core concepts and you can read every dashboard screen.