Node / Server JavaScript

Server SDKRuns on your backend (Calling Key)

Published to npm

This install command is checked against the public SDK release tag. The badge reads the latest version directly from npm.

npm install @abto-app/calling openai

The Server SDK does not estimate tokens or cost itself. It carries user and feature context on requests headed to the gateway, and lets the gateway record the actual execution.

import type OpenAI from 'openai';
import { initAbto } from '@abto-app/calling';

const abto = initAbto({
  abtoApiKey: process.env.ABTO_CALLING_KEY,
  gatewayBaseURL: 'https://gateway.abto.app/v1',
  providerKeys: {
    openai: process.env.OPENAI_API_KEY,
  },
  environment: 'production',
});

const { data: completion, response } = await abto.withContext(
  {
    deviceId: 'device-abc',
    featureId: 'review.summary',
  },
  async () => {
    const openai = await abto.openai<OpenAI>();
    return openai.chat.completions
      .create({
        model: 'gpt-4.1-mini',
        messages: [{ role: 'user', content: 'Summarize these 12 product reviews in three lines.' }],
      })
      .withResponse();
  },
);

const requestId = response.headers.get('x-abto-request-id');

• completion: the usual OpenAI response body
• response: where you read the x-abto-request-id the Gateway issued

Gateway headers sent on every request

SDK input	Gateway header	Meaning
abtoApiKey	Authorization: Bearer …	ABTO Calling Key
providerKeys.openai	x-abto-key-openai	Your own OpenAI provider key
featureId	x-abto-feature-id	Feature ID
deviceId	x-abto-device-id	End-user device identifier (optional)

How providerKeys behaves:

• Not a setting that registers keys with the Gateway. It is the input that attaches server-held credentials as per-request headers.
• Passed with request scope only and never stored in call records.
• If the routed provider has no key, the Gateway rejects that request.
• Pass a function instead of a string to re-evaluate it on every request. for key rotation and per-provider credential resolvers.

How caller-supplied headers are handled:

• Do not override x-abto-key-* through extra_headers on a client built by abto.openai().
• The Calling SDK strips the caller’s Authorization, x-abto-key-*, x-abto-feature-id, and x-abto-device-id, then rebuilds them from the trusted initAbto settings and context.
• Using the official OpenAI SDK directly instead? Put the same headers in each request’s extra_headers. see the direct OpenAI SDK example.

Context fields

Field names are the SDK-side spelling; on the wire they become headers.

• featureId: feature ID (e.g. review.summary), sent as x-abto-feature-id
• deviceId: sent as x-abto-device-id. Take the browser-generated device_id (browserAbto.getIdentity().deviceId) from your backend request and pass it through.
  • An arbitrary login id will not match the browser’s device_id, so product behavior and sticky per-user assignment will not join.

Rules for handling identifiers:

• Validate client-supplied deviceId and featureId with your application’s existing request schema.
• Never accept the Calling Key or provider keys from client requests. Read them only from server environment variables or a server-only credential resolver.
• Keep abtoApiKey out of browser bundles, static docs, and URLs.

Joining responses to browser and mobile behavior:

• The Gateway issues x-abto-request-id on the response.
• Read it with .withResponse() as shown above and include it in your backend response to join the Browser or Mobile SDK LLM trace.

OpenAI requests the Gateway supports

• The data path is currently OpenAI Chat Completions.
• Inline base64 images and PDFs in user messages are supported.
• Unsupported fields such as streaming, tool calling, remote image URLs, and audio are rejected with 400 rather than silently ignored.
• See Gateway OpenAI compatibility for the exact field list.

Retries happen at two layers

Because there are two layers, maxRetries alone does not determine how many times a provider is called.

Layer	What it counts	Who decides
Client	Application → Gateway round trips	The official OpenAI SDK’s clientOptions.maxRetries
Gateway	Gateway → provider invocations	Gateway built-in caps and the node retry policy

Client layer: maxRetries

const openai = await abto.openai({
  clientOptions: { maxRetries: 2 },
});

• Keeps the official OpenAI meaning: retries after the initial request. 0 is one round trip, 1 is two.
• The Calling SDK never overwrites it and offers no separate fallback retry count.
• Leave it unset and the official OpenAI SDK default applies.
• baseURL and apiKey come from ABTO for trusted routing; every other official client option is preserved.
• A caller-provided clientOptions.fetch is not discarded. it is composed underneath the ABTO wrapper as the actual transport.

Gateway layer: retries inside a single round trip

Within one round trip the Gateway may re-attempt along the same path.

• Network retries: only when non-delivery is certain (unreachable). Always on, regardless of node policy, up to 2.
• Provider retries: 429 (rate limit), 500, 502, 503, 504, 529. Only when the node retry policy is enabled, up to 2.
• The two budgets add independently, so one round trip can invoke the provider up to 5 times (1 initial + 2 network + 2 provider).
• The x-abto-attempt response header reports which attempt produced the response.

Never retried:

• 429 with credit exhaustion (insufficient_quota). a deterministic failure
• Timeouts, post-send disconnects, body size overruns, and cancellation. risk of double execution and double billing
• Other deterministic statuses such as 501 and 505

Wait times:

• Exponential backoff with full jitter. Network: 50ms base, 250ms cap. Provider: 500ms base, 8s cap.
• A provider-supplied Retry-After is honored as given; beyond the 8s cap the Gateway does not wait and surfaces the error immediately.

OpenAI direct fallback on Gateway failures

This is the escape hatch back to the endpoint this application used before ABTO. Name that destination in fallback.baseURL; there is no default. If you took an API key straight from OpenAI and used the official SDK, that address is https://api.openai.com/v1.

const abto = initAbto({
  abtoApiKey: process.env.ABTO_CALLING_KEY,
  providerKeys: { openai: process.env.OPENAI_API_KEY },
  fallback: {
    // The address this code called before ABTO was put in front of it.
    baseURL: 'https://api.openai.com/v1',
    timeoutMs: 30_000,
    onTimeout: false,
  },
});

Without it there is no fallback during an ABTO outage

baseURL is where fallback sends the request. Leave it unset and fallback stays off, so a Gateway failure has nowhere to fall back to and that request fails outright. Set it if you want your existing path to keep serving traffic after adopting ABTO.

The original Chat Completions body and model go straight to that address; the Gateway’s provider/model policy is not reproduced client-side.

Sends the current request directly

• Failures before the connection is established
• Admission 503 raised before the provider call

Does not fall back the current request

• Timeouts and disconnects with ambiguous delivery. the Gateway may already have run the provider
• Provider, transport, and internal errors; deterministic 4xx and 429; caller abort; anything after streaming has started
• The direct circuit stays closed in these cases, and if the official OpenAI SDK retries it calls the Gateway again

Settings

• baseURL: required. The OpenAI-compatible endpoint used before ABTO. It must accept the OpenAI request path and Authorization: Bearer. Enabling fallback without it makes initAbto throw.
• timeoutMs: end-to-end cap from local dispatcher wait through the Gateway response headers. Direct requests keep the OpenAI client’s own timeout.
• onTimeout: true: also resends the timed-out request. An explicit choice that accepts duplicate execution and billing risk.
• fallback: false: turns the whole feature off.

Boundaries

• Per SDK attempt the transport makes the Gateway judgment and the direct send exactly once each.
• Direct calls bypass Gateway policy, ABTO telemetry, and request_id, and must use a model that endpoint supports.
• Only headers OpenAI needs are forwarded; cookies, proxy credentials, and custom Gateway headers are stripped.
• After switching to direct, OpenAI responses and errors are returned to the official OpenAI SDK unchanged, and that SDK decides whether to retry.
• Anthropic and Gemini keys are Gateway routing candidates only, not targets of the SDK’s native direct fallback today.