Node.js JavaScript

Server SDKRuns on your backend (Calling Key)

Published to npmnpm install @abto-app/calling openai

Connect ABTO to see the cost and response time of your AI calls, and change models or prompts from the dashboard. If you use the OpenAI SDK directly, follow the steps below to send your first request. If you use LangChain, go to Connect LangChain. These examples support OpenAI Chat Completions; streaming and the Responses API are not supported.

1. Install

Install the ABTO and OpenAI SDKs in your server project.

npm install @abto-app/calling openai

2. Set your API keys

Set your ABTO Calling Key and your LLM provider’s API key, such as an OpenAI key, in your server environment. You must use a Calling Key here rather than the browser SDK’s Event Key.

export ABTO_CALLING_KEY=your-abto-calling-key
export OPENAI_API_KEY=your-openai-api-key

3. Send your first request

Use featureId to name the feature making the call. This example uses review.summary.

import type OpenAI from 'openai';
import { initAbto } from '@abto-app/calling';

const abto = initAbto({
  abtoApiKey: process.env.ABTO_CALLING_KEY,
  gatewayBaseURL: 'https://gateway.abto.app/v1',
  providerKeys: {
    openai: process.env.OPENAI_API_KEY,
  },
  environment: 'production',
});

const { data: completion, response } = await abto.withContext(
  {
    featureId: 'review.summary',
  },
  async () => {
    const openai = await abto.openai<OpenAI>();
    return openai.chat.completions
      .create({
        model: 'gpt-4.1-mini',
        messages: [{ role: 'user', content: 'Summarize these 12 product reviews in three lines.' }],
      })
      .withResponse();
  },
);

const requestId = response.headers.get('x-abto-request-id');

• completion: the usual OpenAI response body
• response: where you read the x-abto-request-id the Gateway issued

4. Check the result

Find the review.summary call in Requests in your dashboard to check its response, cost, and response time. If the call fails, first check that both API keys are set in your server environment. If you receive a response but see no record, check the project, date range, and whether direct fallback ran. Once connected, try a different model or prompt to verify routing.

Use CommonJS

Replace the ABTO import in the example with the following line. This requires an SDK release with CommonJS support.

const { initAbto } = require('@abto-app/calling');

Connect LangChain

Connect LangChain to ABTO to record AI calls and change models or prompts from the dashboard. Add ABTO to your ChatOpenAI configuration, then name the feature when you invoke the model.

Keep your existing @langchain/openai installation and add this setting where you create the model. If you already have a configuration object, pass it to abto.openaiOptions(configuration).

configuration: abto.openaiOptions(),

Here is a complete example. Set ABTO_CALLING_KEY and OPENAI_API_KEY in your server environment.

import { ChatOpenAI } from '@langchain/openai';
import { initAbto } from '@abto-app/calling';

const abto = initAbto({
  abtoApiKey: process.env.ABTO_CALLING_KEY,
  gatewayBaseURL: 'https://gateway.abto.app/v1',
  providerKeys: { openai: process.env.OPENAI_API_KEY },
});

const model = new ChatOpenAI({
  model: 'gpt-4.1-mini',
  configuration: abto.openaiOptions(),
});

const reply = await abto.withContext(
  { featureId: 'review.summary' },
  () => model.invoke('Reviews: Delivery was fast. The packaging was careful. Summarize these two reviews in one sentence.'),
);

console.log(reply.content);

Keep your other model options, invoke() calls, chains, parsers, and callbacks. Requests travel from LangChain through the ABTO Gateway to the model provider. After the call, look for the review.summary record in Requests in your dashboard.

Connect calls to user actions

To connect a call to actions such as purchases, page views, and clicks, send the browser SDK’s getIdentity().deviceId to your server. Pass { featureId: 'review.summary', deviceId } to withContext. For background jobs, save and pass this value with the job data.

Keep Langfuse and LangSmith

Keep your existing callbacks and tracing setup. You do not need to replace your tracing tool to connect ABTO.

Your tracing tool records the request sent by your application. If you change the model, prompt, or parameters in the dashboard, check the ABTO request record for what ran. The model name shown by your tracing tool can also depend on its version.

Before you connect

• This example supports OpenAI Chat Completions. Streaming and the Responses API are not supported.
• The connection has been tested with @langchain/openai versions 0.2.7 and 1.5.13. Check the call path and compatibility when using another version.
• With @langchain/[email protected] and [email protected], create a Langfuse handler for each request. Sharing a handler across concurrent requests can combine their traces.
• Keep API keys and ABTO configuration on your server. ABTO keeps its Calling Key out of model settings, but does not automatically hide other secrets you add.
• Do not pass another router’s URL or authentication headers unchanged. openaiOptions() changes the destination to ABTO; it does not chain multiple routers.
• Existing trace headers are preserved up to the Gateway. Tracing through to the provider and forwarding organization or project are not guaranteed.
• You can enable direct fallback to OpenAI during Gateway outages. Those calls bypass ABTO routing and call records.

Gateway headers sent on every request

SDK input	Gateway header	Meaning
abtoApiKey	Authorization: Bearer …	ABTO Calling Key
providerKeys.openai	x-abto-key-openai	Your own OpenAI provider key
providerKeys.gemini	x-abto-key-gemini	Your own Gemini provider key
providerKeys.anthropic	x-abto-key-anthropic	Your own Anthropic provider key
providerKeys.deepseek	x-abto-key-deepseek	Your own DeepSeek provider key
providerKeys.kimi	x-abto-key-kimi	Your own Kimi provider key
featureId	x-abto-feature-id	Feature ID
nodeId	x-abto-node-id	Node inside the feature (optional)
deviceId	x-abto-device-id	End-user device identifier (optional)
traceId	x-abto-trace-id	Run ID (created when omitted)

How providerKeys behaves:

• Not a setting that registers keys with the Gateway. It is the input that attaches server-held credentials as per-request headers.
• Passed with request scope only and never stored in call records.
• If the routed provider has no key, the Gateway rejects that request.
• Pass a function instead of a string to re-evaluate it on every request. for key rotation and per-provider credential resolvers.

How caller-supplied headers are handled:

• Do not override x-abto-key-* through extra_headers on a client built by abto.openai().
• The Calling SDK strips the caller’s Authorization, x-abto-key-*, x-abto-feature-id, x-abto-node-id, x-abto-device-id, and x-abto-trace-id, then rebuilds them from the trusted initAbto settings and context.
• Using the official OpenAI SDK directly instead? Put the same headers in each request’s extra_headers. see the direct OpenAI SDK example.

Context fields

Field names are the SDK-side spelling; on the wire they become headers.

• featureId: feature ID (e.g. review.summary), sent as x-abto-feature-id
• nodeId: the point inside the feature that calls the LLM (e.g. review.draft). Without it, the feature ID becomes the node name. Sent as x-abto-node-id
  • Example: withContext({ featureId: 'review.summary', nodeId: 'review.draft' }, ...)
  • When one feature calls the LLM more than once, give each call site its own value to choose a separate agent config per node in Routing
  • If an inner withContext sets a different featureId without a nodeId, it does not inherit the outer nodeId
• deviceId: sent as x-abto-device-id. Take the browser-generated device_id (browserAbto.getIdentity().deviceId) from your backend request and pass it through.
  • An arbitrary login id will not match the browser’s device_id, so product behavior and sticky per-user assignment will not join.
• traceId: the run ID that groups the calls one agent or workflow run makes. Sent as x-abto-trace-id
  • Without it, the outermost withContext creates a new run ID, and inner withContext calls inherit it
  • Wrap one whole run in a single withContext so every call inside joins the same run. Wrapping each call separately splits the run per call
  • A call made outside any withContext gets its own new run ID
  • When you pass a context directly and leave traceId empty, the enclosing withContext value is used
  • Set it yourself only to continue a run another service passed to you, in the outermost withContext
  • Accepts up to 200 printable ASCII characters (letters, digits, spaces, symbols). The Gateway ignores an empty or non-conforming value without an error
  • The request’s traceparent is not used as the run ID

Rules for handling identifiers:

• Validate client-supplied deviceId and featureId with your application’s existing request schema.
• Never accept the Calling Key or provider keys from client requests. Read them only from server environment variables or a server-only credential resolver.
• Keep abtoApiKey out of browser bundles, static docs, and URLs.

Joining responses to browser and mobile behavior:

• The Gateway issues x-abto-request-id on the response.
• Read it with .withResponse() as shown above and include it in your backend response to join the Browser or Mobile SDK LLM trace.

Preserve existing traces

ABTO keeps the traceparent and tracestate headers supplied by your tracing tool, including during direct fallback. It adds feature and device information separately without replacing your trace ID.

This is supported from 1.1.2. Keep your tracing tool’s request header setup. ABTO does not configure tracing for you or connect traces through to the provider.

The run ID traceId is sent only as x-abto-trace-id, never turned into a traceparent.

OpenAI requests the Gateway supports

• The data path is currently OpenAI Chat Completions.
• Inline base64 images and PDFs in user messages are supported.
• Function tool calling is supported. When you send tool results back, echo each message.tool_calls[].id as tool_call_id (not the top-level response id).
• Unsupported fields such as streaming, remote image URLs, and audio are rejected with 400 rather than silently ignored.
• See Gateway Chat Completions for the exact field list.

Retries

const openai = await abto.openai({
  clientOptions: { maxRetries: 2 },
});

• The official OpenAI SDK’s maxRetries applies as written. ABTO never overwrites it.
• Leave it unset and the official OpenAI SDK default applies.
• The Calling SDK adds no retry count of its own.
• Direct fallback exposes no count either, only the resend switch fallback.onTimeout, which is off by default.
• baseURL and apiKey come from ABTO for trusted routing; every other official client option is preserved.
• A caller-provided clientOptions.fetch is not discarded. it is composed underneath the ABTO wrapper as the actual transport.

OpenAI direct fallback on Gateway failures

This is the escape hatch back to the endpoint this application used before ABTO. Name that destination in fallback.baseURL; there is no default. If you took an API key straight from OpenAI and used the official SDK, that address is https://api.openai.com/v1.

const abto = initAbto({
  abtoApiKey: process.env.ABTO_CALLING_KEY,
  providerKeys: { openai: process.env.OPENAI_API_KEY },
  fallback: {
    // The address this code called before ABTO was put in front of it.
    baseURL: 'https://api.openai.com/v1',
    timeoutMs: 30000,
    onTimeout: false,
  },
});

Without it there is no fallback during an ABTO outage

baseURL is where fallback sends the request. Leave it unset and fallback stays off, so a Gateway failure has nowhere to fall back to and that request fails outright. Set it if you want your existing path to keep serving traffic after adopting ABTO.

The original Chat Completions body and model go straight to that address; the Gateway’s provider/model policy is not reproduced client-side.

Sends the current request directly

• The request never reached ABTO
• It reached ABTO but was rejected before the provider was called

In other words, it switches only when the provider certainly did not run, so it never creates a double execution or a double charge.

Does not fall back the current request

• Timeouts and disconnects with ambiguous delivery. the Gateway may already have run the provider
• Provider and transport errors; deterministic 4xx and 429; caller abort; anything after streaming has started
• The direct circuit stays closed in these cases, and if the official OpenAI SDK retries it calls the Gateway again

Settings

• baseURL: required. The OpenAI-compatible endpoint used before ABTO. It must accept the OpenAI request path and Authorization: Bearer.
  • Naming this destination is how fallback is turned on. There is no separate enable flag.
  • initAbto throws for fallback: true with no destination, and for a destination with no OpenAI key.
• timeoutMs: end-to-end cap from local dispatcher wait through the Gateway response headers. Direct requests keep the OpenAI client’s own timeout.
• onTimeout: true: also resends the timed-out request. An explicit choice that accepts duplicate execution and billing risk.
• fallback: false: turns the whole feature off.

Boundaries

• Per SDK attempt the transport makes the Gateway judgment and the direct send exactly once each.
• Direct calls bypass Gateway policy, ABTO telemetry, and request_id, and must use a model that endpoint supports.
• Only headers OpenAI needs are forwarded; cookies, proxy credentials, and custom Gateway headers are stripped.
• After switching to direct, OpenAI responses and errors are returned to the official OpenAI SDK unchanged, and that SDK decides whether to retry.
• Anthropic and Gemini keys are Gateway routing candidates only, not targets of the SDK’s native direct fallback today.