Java / Kotlin (JVM)

Connect ABTO to see the cost and latency of AI calls and change models and prompts from the dashboard. If you use the official OpenAI Java SDK directly, follow the steps below to send your first request. For Spring AI or LangChain4j, go to framework integration.

These examples use OpenAI Chat Completions and support synchronous calls, asynchronous calls, tool calling, and structured output. Streaming and the Responses API are not supported. Keep your existing provider connection for RAG embeddings.

1. Install the SDK

Choose one package for your server project running Java 17 or later. Java and Kotlin use the same library. The Gradle dependency below is for the official OpenAI client.

Server SDKRuns on your backend (Calling Key)

Published to Maven Centralimplementation("app.abto:abto-calling:1.2.1")

All five packages use the same version. The shared abto-calling-core package is installed as a dependency.

Maven coordinate	Calling path
app.abto:abto-calling	Official OpenAI Java client
app.abto:abto-calling-core	Plain JDK HTTP
app.abto:abto-calling-spring-ai-1	Spring AI 1.1 / Boot 3
app.abto:abto-calling-spring-ai-2	Spring AI 2.0 / Boot 4
app.abto:abto-calling-langchain4j	LangChain4j / LangGraph4j

The base OpenAI package does not pull in Spring or LangChain. Use the Spring AI 1 and 2 adapters in separate applications.

2. Prepare API keys

Set your ABTO Calling Key and OpenAI API key in server environment variables. Use a Calling Key, not a browser Event Key.

export ABTO_CALLING_KEY=your-abto-calling-key
export OPENAI_API_KEY=your-openai-api-key

If you also route to other providers, add their keys with providerKey in the example below.

3. Send your first request

Set up the connection with AbtoOpenAI.create(config) and create an official OpenAI client with openai(). Pass the feature ID used to identify the call to AbtoContext.builder(...). This example uses review.summary. When you send only the feature ID, create it with AbtoContext.of("review.summary").

import app.abto.calling.AbtoConfig;
import app.abto.calling.AbtoContext;
import app.abto.calling.openai.AbtoOpenAI;
import com.openai.models.chat.completions.ChatCompletionCreateParams;

var config = AbtoConfig.builder()
    .apiKey(() -> System.getenv("ABTO_CALLING_KEY"))
    .gatewayBaseUrl("https://gateway.abto.app/v1")
    .providerKey("openai", () -> System.getenv("OPENAI_API_KEY"))
    .build();
try (var abto = AbtoOpenAI.create(config)) {
    var client = abto.forContext(AbtoContext.builder("review.summary").deviceId("device-id").build()).openai();
    var params = ChatCompletionCreateParams.builder()
        .model("gpt-4o-mini").addUserMessage("Summarize these reviews in three lines.")
        .build();
    try (var raw = client.chat().completions().withRawResponse().create(params)) {
        var requestId = raw.headers().values("x-abto-request-id").stream().findFirst().orElse(null);
        var completion = raw.parse();
    }
}

• completion: the usual OpenAI response body
• requestId: the x-abto-request-id issued by the Gateway

In a server application, create abto once, reuse it, and close it at shutdown. Pass feature and user information through forContext(...) for each request. Requests keep separate contexts while sharing the connection pool and circuit state. For deviceId, forward the device identifier received from your browser or mobile app. Pass this context to asynchronous tasks as well.

When one feature calls the LLM more than once, set a node for each call site, such as .nodeId("review.draft"). It is sent as x-abto-node-id, and you can choose a separate agent config per node in Routing. Without it, the feature ID becomes the node name.

Each AbtoContext is one agent or workflow run. Building one without a traceId creates a new run ID, sent as x-abto-trace-id. Keep using the same AbtoContext for the whole run so every call inside joins it. To switch nodes, do not build a new one; use context.withNodeId("review.polish"), which keeps the run ID and every other value and changes only the node. Building a new AbtoContext per call splits the run per call. Set .traceId(...) only to continue a run another service passed to you. It accepts up to 200 printable ASCII characters; the Gateway ignores an empty or non-conforming value without an error. The request’s traceparent is not used as the run ID.

4. Check the request

Find the review.summary call in the dashboard request list to check its response, cost, and latency. If the call fails, first check that the Calling Key and OpenAI API key are set on the server. If you received a response but cannot find a record, check the project, date range, and whether direct fallback ran. Once connected, follow Try a different model or prompt to try routing.

Connect an existing OpenAI client

If you already use an official OpenAI client, pass ABTO’s connection settings to withOptions. This applies the same authentication and fallback policies as creating a client with openai().

var transport = java.net.http.HttpClient.newBuilder()
    .followRedirects(java.net.http.HttpClient.Redirect.NEVER)
    .build();
var scope = abto.forContext(AbtoContext.builder("review.summary").deviceId("device-id").build());
var configured = existingClient.withOptions(scope.openaiOptions(transport));
var completion = configured.chat().completions().create(params);

The supplied JDK client handles HTTP requests and keeps its connection pool, proxy, TLS, and executor settings. The existing OpenAI client’s retry count, total request timeout, and custom headers are preserved. OkHttp-specific interceptors are not copied automatically. The JDK client must use Redirect.NEVER and have no CookieHandler or Authenticator.

Set the connection timeout on the JDK client builder. Read/write timeouts shorter than the total request timeout, or connection timeouts that conflict with the JDK settings, produce configuration errors. ABTO does not close existing clients you supply. Keep your application’s shutdown handling. To reuse existing options when creating an ABTO connection, use AbtoOpenAI.create(config, existingClient, transport).

Call asynchronously from Kotlin

import app.abto.calling.AbtoCancellation
import app.abto.calling.AbtoContext
import kotlinx.coroutines.future.await

val cancellation = AbtoCancellation()
val scope = abto.forContext(AbtoContext.builder("review.summary").deviceId("device-id").build())
    .withCancellation(cancellation)
val completion = cancellation.bind(
    scope.openai().async().chat().completions().create(params)
).await()

val configured = existingClient.withOptions(
    abto.forContext(AbtoContext.builder("review.summary").deviceId("device-id").build())
        .openaiOptions(safeJdkTransport)
)

Create a new AbtoCancellation for each asynchronous request and pass the returned Future to bind. Coroutine cancellation or cancellation.cancel() also cancels the HTTP call and releases any pending Gateway recovery check. Cancelling only the official SDK’s Future does not cancel the HTTP call.

Synchronous code can also pass context through withContext(context, callback). For coroutines and graph execution, use forContext as shown above.

Connect Spring AI and LangChain4j

Apply ABTO’s connection settings to your existing model builder. Keep model options, tools, listeners, and tracing settings in that builder.

Framework	Configuration path	Runnable Java / Kotlin examples
Spring AI 1.1	openAiApiOptions(restBuilder).accept(apiBuilder), chatModelOptions(api).accept(modelBuilder)	Boot 3
Spring AI 2.0	chatModelOptions().accept(modelBuilder)	Boot 4
LangChain4j	openAiOptions().accept(modelBuilder)	Model and graph

Spring AI 1 clones the existing RestClient builder and replaces its HTTP request factory. Apply any proxy or TLS options from your request factory to the JDK client you supply. Spring AI 2 uses official synchronous and asynchronous OpenAI clients configured by ABTO.

The LangChain4j integration uses a JDK HTTP client. Compatibility with other custom HTTP implementations has not been verified. In LangGraph4j nodes, pass the ABTO-configured model and request context. Keep your existing graph state and checkpoints.

Connect over HTTP

To call without the OpenAI SDK, connect ABTO to your JDK HTTP client with AbtoHttpClient.wrap(safeJdkTransport, gateway). Java can use send; Kotlin can use sendAsync(...).await(). When fallback is enabled, use a body that can be sent again, such as BodyPublishers.ofString or ofByteArray.

The Java HTTP example and Kotlin HTTP example also demonstrate factory and existing-client configuration. They build from Maven coordinates without referencing SDK source.

Use direct fallback during a Gateway outage

Direct fallback sends requests to the endpoint you used before ABTO when the Gateway is unavailable. It is off by default. To enable it, pass AbtoFallback.builder().baseUrl(existingEndpoint).build() to AbtoConfig’s fallback setting and configure your OpenAI API key.

A request is sent directly only when it is known not to have reached the Gateway, or the Gateway rejected it before calling the provider. Ambiguous disconnects, caller cancellation, and provider errors do not resend the current request. To resend requests that time out, set onTimeout(true). A request that was already processed may run again and incur duplicate charges.

Direct calls use the original Chat Completions body and remove ABTO credentials, candidate provider keys, and cookies. They do not pass through the Gateway, so dashboard model and prompt policies, request records, and x-abto-request-id do not apply.

Retry settings

Keep the existing retry settings in your official OpenAI SDK and framework. ABTO does not override these values or add a separate retry count.

Preserve Langfuse and existing tracing

Keep your existing Micrometer, OpenTelemetry, LangChain4j listeners, and Langfuse exporter settings. ABTO does not add a tracing exporter or change global settings. Existing traceparent and tracestate are preserved. You can also pass them through AbtoContext.builder(feature).deviceId(device).traceparent(traceparent).tracestate(tracestate).build().

Your tracing tool records the input sent by the app. For the actual execution and cost after dashboard model and prompt changes, check the ABTO request record. ABTO does not configure your tracing tool automatically. Check your Langfuse instance or collector to confirm that trace data arrives.

Verified versions

Java	OpenAI Java	Spring AI 1.1.8 / Boot	Spring AI 2.0.1 / Boot
17	4.49.0	3.4.13	4.0.8
21	4.49.0	3.5.16	4.1.1
25	4.81.0	3.5.16	4.1.1

Alongside the combinations above, we verified Kotlin 2.3.21, coroutines 1.10.2, LangChain4j 1.22.0, and LangGraph4j 1.9.5. Packages were installed and examples run in Maven 3.9.11 and Gradle 9.3.0 projects. Compatibility with other versions is not guaranteed. See the public source and API examples for additional settings.