Production

Reliability & tracing

Retries, fallback chains, timeouts, concurrency limits, and full run tracing — all implemented as composable provider wrappers, not special-cased flags.

Retries and fallback chains

withRetry() and withFallback([...]) wrap any provider — since they implement the same interface, they compose with guardrails, generateObject(), and streaming without special-casing. createResilientProvider() combines both:

resilience.ts

import { createClient, anthropic, openai, createResilientProvider } from "samai-sdk";

// Both retries and fallback: each provider gets its own retries before falling through
const resilient = createResilientProvider(
  [anthropic({ apiKey: "..." }), openai({ apiKey: "..." })],
  {
    retry: { maxRetries: 2, initialDelayMs: 500 },
    fallback: {
      onFallback: ({ failedProvider, nextProvider }) =>
        console.warn(`${failedProvider} failed, falling back to ${nextProvider}`),
    },
  }
);

const client = createClient({ provider: resilient });
const result = await client.generate({ model: "claude-sonnet-4-6", messages: [/* ... */] });

Streaming note

Retries and fallback only apply before the first chunk reaches the caller. Once a stream has started yielding output, a later mid-stream failure surfaces as-is rather than silently restarting or switching providers — restarting would duplicate or drop output the caller already saw.

Timeouts

withTimeout() enforces a real deadline using AbortController — not pattern-matching on error messages after the fact. createResilientProvider() applies a 30s default automatically.

timeouts.ts

import { withTimeout, withRetry, anthropic } from "samai-sdk";

// Put timeout innermost so every retry attempt gets its own fresh window
const provider = withRetry(
  withTimeout(anthropic({ apiKey: "..." }), { timeoutMs: 15_000 }),
  { maxRetries: 2 }
);

Tool execution gets its own independent timeout too — every execute() call is raced against a deadline (default 30s, overridable per-tool or per-run). A hung tool comes back as an isError result instead of hanging the whole run.

Concurrency and rate limiting

concurrency.ts

import { withConcurrencyLimit, withRateLimit, anthropic } from "samai-sdk";

// Caps in-flight calls — a QUEUE, not a rejection
const capped = withConcurrencyLimit(anthropic({ apiKey: "..." }), { maxConcurrent: 5 });

// Caps requests per time window — token-bucket, refills continuously
const throttled = withRateLimit(anthropic({ apiKey: "..." }), { maxRequests: 60, intervalMs: 60_000 });

Tracing & observability

Every run already produces a RunTrace — every model call, tool call, retry, and handoff recorded with real timing. Turn that into something you can actually look at:

tracing.ts

import { writeFileSync } from "node:fs";
import { runAgent, renderTraceHTML, exportRunTraceToOtel } from "samai-sdk";

const result = await runAgent(client, agent, "hi");

// Render as a self-contained, offline-viewable HTML timeline
writeFileSync("trace.json", JSON.stringify(result.trace));
writeFileSync("trace.html", renderTraceHTML(result.trace));

// Or export as real OpenTelemetry spans on your existing tracer
await exportRunTraceToOtel(result.trace); // needs the optional @opentelemetry/api peer dependency

Or skip the intermediate file and serve the same rendered page straight from the CLI:

terminal

npx samai-sdk trace ./trace.json --port 4949
# ✅ Trace viewer running at http://localhost:4949