How to Track Per-Tenant LLM Token Costs in Next.js using OpenTelemetry and PostgreSQL

Introduction

If you're running a B2B SaaS product with LLM features in 2026, you've probably already had this conversation with your CFO: "Why did our OpenAI bill triple last month, and which customer caused it?"

It's an uncomfortable question because, for most teams, the honest answer is: we don't know. Token usage gets buried inside a single provider invoice, request-level cost data lives nowhere durable, and by the time someone notices a tenant is running a background job that fires 40,000 completions a day, the margin on that account has already gone negative.

This is the new cost-of-goods-sold problem for AI-native software. Compute used to be a rounding error next to headcount. Now, for products built on top of GPT, Claude, or open-weight models served through your own inference layer, token spend can be the single largest variable cost on your P&L — and it varies wildly by customer, feature, and even by how verbose a user's prompts happen to be.

The fix isn't a spreadsheet. It's observability infrastructure. In this guide, we'll build a complete pipeline for a Next.js App Router application that:

• Instruments server-side LangChain calls with OpenTelemetry (OTel)
• Captures granular token usage (input, output, cached, reasoning tokens) as span attributes
• Propagates distributed trace context — including tenant identity — across Node.js microservices
• Persists per-request cost data into PostgreSQL for real-time, per-tenant margin analysis

By the end, you'll have a system that can answer "which tenant, which feature, which model call cost us $4.12 at 3:14 AM" in a single SQL query.

Related reading: Full-Stack TypeScript for 2026: Building a Job-Ready App with Next.js, Node, and Postgres and Optimizing Next.js Performance for AI-Powered SaaS Applications build on the same App Router foundations. Browse every Next.js article.

Why OpenTelemetry Is the Right Tool for LLM Cost Tracking

You might be tempted to just wrap your OpenAI/Anthropic SDK calls in a try/catch, log the usage object, and call it a day. That works for a demo. It falls apart in production for three reasons:

1. LLM calls rarely happen in isolation. A single user action — "summarize this contract" — might trigger a retrieval step, a reranking call, two LLM calls (draft + refine), and a moderation check. You need a way to group these into one logical unit of work.
2. Cost attribution requires context that crosses process boundaries. In a microservices setup, the request that knows which tenant is calling often isn't the process that actually talks to the model provider.
3. You need this data to be queryable, not just loggable. Logs are for debugging. A billing and margin system needs structured, relational data.

OpenTelemetry solves all three because it was designed around distributed tracing and structured attributes from day one. As of 2025, the OpenTelemetry community stabilized the Generative AI semantic conventions, which standardize attribute names like:

• gen_ai.system (e.g., openai, anthropic)
• gen_ai.request.model
• gen_ai.usage.input_tokens
• gen_ai.usage.output_tokens
• gen_ai.response.model

This matters because it means your traces are portable. Swap providers, change SDKs, migrate your APM vendor — the underlying data model doesn't change. You're not locked into a proprietary "AI observability" product; you own the pipeline.

Architecture Overview

Before writing code, let's map out the system we're building:

┌───────────────────┐       ┌─────────────────────┐       ┌────────────────────┐
│  Next.js App      │       │  Node.js            │       │  PostgreSQL        │
│  Router (Edge/    │──────▶│  Microservice       │──────▶│  llm_usage_events  │
│  Server Actions)  │ trace │  (LangChain calls)  │ span  │  + tenants         │
└───────────────────┘ ctx   └─────────────────────┘ data  └────────────────────┘
          │                            │                             ▲
          │                            │                             │
          ▼                            ▼                             │
   OTel SDK (Node)              OTel SDK (Node)                      │
          │                            │                             │
          └─────────────┬──────────────┘                             │
                        ▼                                            │
         OTel Collector (batch, process) ────────────────────────────┘
                        │
                        ▼
  Optional: Grafana Tempo / Honeycomb / Jaeger

The core idea: every request that touches an LLM carries a tenant ID in OpenTelemetry baggage, every LLM call emits a span with gen_ai.usage.* attributes, and a custom span processor exports those specific spans into PostgreSQL as structured rows — independent of whatever tracing backend (Tempo, Honeycomb, Datadog) you use for general debugging.

Step 1: Bootstrapping OpenTelemetry in Next.js App Router

Next.js has shipped native OpenTelemetry support since v13 via the instrumentation.ts file, and by 2026 this is the standard, documented way to wire up tracing — no more hacking around _app.tsx.

First, install the dependencies:

npm install @opentelemetry/api @opentelemetry/sdk-node \
  @opentelemetry/auto-instrumentations-node \
  @opentelemetry/exporter-trace-otlp-http \
  @opentelemetry/resources @opentelemetry/semantic-conventions

Enable instrumentation hooks in your Next.js config:

// next.config.js
/** @type {import('next').NextConfig} */
module.exports = {
  experimental: {
    instrumentationHook: true, // stable by default in recent Next.js versions, but explicit is safer
  },
};

Now create the root instrumentation file:

// instrumentation.ts
import { NodeSDK } from '@opentelemetry/sdk-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
import { Resource } from '@opentelemetry/resources';
import { SemanticResourceAttributes } from '@opentelemetry/semantic-conventions';

export async function register() {
  // Only run on the Node.js runtime, not the Edge runtime
  if (process.env.NEXT_RUNTIME !== 'nodejs') return;

  const sdk = new NodeSDK({
    resource: new Resource({
      [SemanticResourceAttributes.SERVICE_NAME]: 'saas-app-web',
      [SemanticResourceAttributes.SERVICE_VERSION]: process.env.APP_VERSION ?? 'dev',
    }),
    traceExporter: new OTLPTraceExporter({
      url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT ?? 'http://localhost:4318/v1/traces',
    }),
    instrumentations: [getNodeAutoInstrumentations()],
  });

  sdk.start();
}

This gives you automatic instrumentation for HTTP, fetch, and popular Node libraries out of the box. But auto-instrumentation won't know anything about tokens or tenants — that's the custom part we build next.

Step 2: Injecting Tenant Context via Baggage

The most common mistake teams make is passing tenantId as a plain function argument and hoping it survives every async call, retry, and queue hop. It doesn't — and even when it does, it's invisible to your observability tooling.

OpenTelemetry has a purpose-built primitive for this: baggage. Baggage is key-value context that propagates alongside the trace, across process and network boundaries, via the baggage HTTP header.

Set it as early as possible — typically in a middleware or the first server action that knows who's calling:

// lib/otel-context.ts
import { propagation, context, trace } from '@opentelemetry/api';

export function withTenantContext<T>(tenantId: string, planTier: string, fn: () => Promise<T>) {
  const baggage = propagation.createBaggage({
    'tenant.id': { value: tenantId },
    'tenant.plan': { value: planTier },
  });

  const ctx = propagation.setBaggage(context.active(), baggage);

  return context.with(ctx, () => {
    const span = trace.getActiveSpan();
    span?.setAttribute('tenant.id', tenantId);
    span?.setAttribute('tenant.plan', planTier);
    return fn();
  });
}

Wrap your route handler or server action:

// app/api/summarize/route.ts
import { NextRequest, NextResponse } from 'next/server';
import { withTenantContext } from '@/lib/otel-context';
import { runSummarizationChain } from '@/lib/chains/summarize';
import { getTenantFromRequest } from '@/lib/auth';

export async function POST(req: NextRequest) {
  const tenant = await getTenantFromRequest(req);
  const { document } = await req.json();

  const result = await withTenantContext(tenant.id, tenant.planTier, () =>
    runSummarizationChain(document)
  );

  return NextResponse.json({ summary: result.text });
}

Because baggage travels with the trace context, any downstream span — even one created inside a completely different Node.js microservice three hops away — can read tenant.id off the active context without you threading it through every function signature.

Step 3: Instrumenting LangChain to Capture Token Usage

LangChain.js exposes a CallbackHandler interface that fires on every LLM call, giving you access to the raw provider response — including the usage object most providers now return by default. This is where we bridge LangChain's internal events into OpenTelemetry spans.

// lib/otel-langchain-handler.ts
import { BaseCallbackHandler } from '@langchain/core/callbacks/base';
import { trace, context, propagation, SpanStatusCode } from '@opentelemetry/api';
import type { LLMResult } from '@langchain/core/outputs';

const tracer = trace.getTracer('langchain-llm-calls');

export class OtelCostTrackingHandler extends BaseCallbackHandler {
  name = 'OtelCostTrackingHandler';
  private spans = new Map<string, any>();

  async handleLLMStart(llm: any, prompts: string[], runId: string) {
    const baggage = propagation.getBaggage(context.active());
    const tenantId = baggage?.getEntry('tenant.id')?.value ?? 'unknown';

    const span = tracer.startSpan(`gen_ai.chat ${llm.id?.at(-1) ?? 'unknown_model'}`, {
      attributes: {
        'gen_ai.system': llm.id?.[0]?.toLowerCase() ?? 'unknown',
        'gen_ai.request.model': llm.lc_kwargs?.model ?? llm.lc_kwargs?.modelName,
        'tenant.id': tenantId,
        'gen_ai.prompt.count': prompts.length,
      },
    });

    this.spans.set(runId, span);
  }

  async handleLLMEnd(output: LLMResult, runId: string) {
    const span = this.spans.get(runId);
    if (!span) return;

    const usage = output.llmOutput?.tokenUsage ?? output.llmOutput?.usage;

    if (usage) {
      span.setAttributes({
        'gen_ai.usage.input_tokens': usage.promptTokens ?? usage.input_tokens ?? 0,
        'gen_ai.usage.output_tokens': usage.completionTokens ?? usage.output_tokens ?? 0,
        'gen_ai.usage.cached_input_tokens': usage.promptTokensDetails?.cachedTokens ?? 0,
        'gen_ai.response.model': output.llmOutput?.model ?? 'unknown',
      });
    }

    span.setStatus({ code: SpanStatusCode.OK });
    span.end();
    this.spans.delete(runId);
  }

  async handleLLMError(err: Error, runId: string) {
    const span = this.spans.get(runId);
    if (!span) return;
    span.recordException(err);
    span.setStatus({ code: SpanStatusCode.ERROR, message: err.message });
    span.end();
    this.spans.delete(runId);
  }
}

Attach the handler when you build your chain:

// lib/chains/summarize.ts
import { ChatOpenAI } from '@langchain/openai';
import { OtelCostTrackingHandler } from '@/lib/otel-langchain-handler';

export async function runSummarizationChain(document: string) {
  const model = new ChatOpenAI({
    model: 'gpt-5.1-mini',
    callbacks: [new OtelCostTrackingHandler()],
  });

  return model.invoke([
    { role: 'system', content: 'Summarize the following document concisely.' },
    { role: 'user', content: document },
  ]);
}

At this point, every LLM invocation produces a span carrying exact token counts and the tenant that triggered it — automatically, without touching your business logic.

Step 4: Propagating Trace Context to Node.js Microservices

If your architecture splits LLM orchestration into a separate microservice (common once you outgrow a monolith), you need trace context — including baggage — to survive the network hop. OpenTelemetry does this via standard traceparent and baggage HTTP headers, but you have to explicitly inject them into outgoing requests.

// lib/http-client.ts
import { propagation, context } from '@opentelemetry/api';

export async function callInferenceService(path: string, body: unknown) {
  const headers: Record<string, string> = { 'Content-Type': 'application/json' };

  // Injects traceparent + baggage headers into the outgoing request
  propagation.inject(context.active(), headers);

  return fetch(`${process.env.INFERENCE_SERVICE_URL}${path}`, {
    method: 'POST',
    headers,
    body: JSON.stringify(body),
  });
}

On the receiving microservice, extract the context before processing the request:

// inference-service/server.ts
import { propagation, context, trace } from '@opentelemetry/api';
import express from 'express';

const app = express();
app.use(express.json());
const tracer = trace.getTracer('inference-service');

app.post('/generate', async (req, res) => {
  const ctx = propagation.extract(context.active(), req.headers);

  await context.with(ctx, async () => {
    const span = tracer.startSpan('inference.generate');
    try {
      const result = await runModel(req.body);
      res.json(result);
    } finally {
      span.end();
    }
  });
});

Because context.with(ctx, ...) restores the propagated baggage, any span created inside runModel — including LLM call spans from Step 3 — automatically inherits tenant.id, even though this code runs in a completely separate process, possibly on a different machine.

Step 5: Designing the PostgreSQL Schema for Cost Attribution

Traces are great for debugging a single request, but for billing and margin analysis you want a durable, queryable, relational store. We'll export the specific gen_ai.* spans into PostgreSQL using a custom OpenTelemetry span processor, decoupled from whatever general-purpose backend (Tempo, Honeycomb) you use for full trace visualization.

Schema:

CREATE TABLE tenants (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  name TEXT NOT NULL,
  plan_tier TEXT NOT NULL DEFAULT 'starter',
  created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE TABLE llm_usage_events (
  id BIGSERIAL PRIMARY KEY,
  trace_id TEXT NOT NULL,
  span_id TEXT NOT NULL,
  tenant_id UUID NOT NULL REFERENCES tenants(id),
  gen_ai_system TEXT NOT NULL,          -- 'openai', 'anthropic', etc.
  request_model TEXT NOT NULL,
  response_model TEXT,
  input_tokens INTEGER NOT NULL DEFAULT 0,
  output_tokens INTEGER NOT NULL DEFAULT 0,
  cached_input_tokens INTEGER NOT NULL DEFAULT 0,
  cost_usd NUMERIC(12, 6) NOT NULL DEFAULT 0,
  feature_name TEXT,                    -- e.g. 'contract-summarizer'
  status TEXT NOT NULL DEFAULT 'ok',
  created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE INDEX idx_llm_usage_tenant_created
  ON llm_usage_events (tenant_id, created_at DESC);

CREATE INDEX idx_llm_usage_trace
  ON llm_usage_events (trace_id);

Keep pricing in a lookup table rather than hardcoding it — model prices change often, and you'll want historical accuracy for past invoices:

CREATE TABLE model_pricing (
  model TEXT PRIMARY KEY,
  input_price_per_1k NUMERIC(10, 6) NOT NULL,
  output_price_per_1k NUMERIC(10, 6) NOT NULL,
  effective_from TIMESTAMPTZ NOT NULL DEFAULT now()
);

Step 6: The Span Processor That Writes to PostgreSQL

A custom SpanProcessor lets you hook into onEnd() for every span, filter for the ones tagged with gen_ai.usage.*, compute cost, and batch-insert into Postgres — without slowing down the request path.

// lib/otel-postgres-exporter.ts
import { SpanProcessor, ReadableSpan } from '@opentelemetry/sdk-trace-node';
import { Pool } from 'pg';

const pool = new Pool({ connectionString: process.env.DATABASE_URL });

// Simple in-memory cache to avoid a DB round trip per span
const priceCache = new Map<string, { input: number; output: number }>();

async function getPricing(model: string) {
  if (priceCache.has(model)) return priceCache.get(model)!;
  const { rows } = await pool.query(
    'SELECT input_price_per_1k, output_price_per_1k FROM model_pricing WHERE model = $1',
    [model]
  );
  const pricing = rows[0]
    ? { input: rows[0].input_price_per_1k, output: rows[0].output_price_per_1k }
    : { input: 0, output: 0 };
  priceCache.set(model, pricing);
  return pricing;
}

export class PostgresCostExporterProcessor implements SpanProcessor {
  onStart() {}

  async onEnd(span: ReadableSpan) {
    const attrs = span.attributes;
    const inputTokens = attrs['gen_ai.usage.input_tokens'] as number | undefined;
    const outputTokens = attrs['gen_ai.usage.output_tokens'] as number | undefined;

    if (inputTokens === undefined && outputTokens === undefined) return; // not an LLM span

    const model = (attrs['gen_ai.request.model'] as string) ?? 'unknown';
    const pricing = await getPricing(model);
    const cost =
      ((inputTokens ?? 0) / 1000) * pricing.input +
      ((outputTokens ?? 0) / 1000) * pricing.output;

    await pool.query(
      `INSERT INTO llm_usage_events
        (trace_id, span_id, tenant_id, gen_ai_system, request_model, response_model,
         input_tokens, output_tokens, cached_input_tokens, cost_usd, feature_name, status)
       VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12)`,
      [
        span.spanContext().traceId,
        span.spanContext().spanId,
        attrs['tenant.id'],
        attrs['gen_ai.system'],
        model,
        attrs['gen_ai.response.model'] ?? null,
        inputTokens ?? 0,
        outputTokens ?? 0,
        attrs['gen_ai.usage.cached_input_tokens'] ?? 0,
        cost,
        attrs['feature.name'] ?? null,
        span.status.code === 2 ? 'error' : 'ok',
      ]
    );
  }

  shutdown() { return pool.end(); }
  forceFlush() { return Promise.resolve(); }
}

Register it alongside your regular exporter in instrumentation.ts:

sdk.addSpanProcessor(new PostgresCostExporterProcessor());

Note: For high-throughput systems, don't write to Postgres synchronously in onEnd(). Buffer spans in memory and flush in batches every few seconds, or route through a lightweight queue (Redis Streams, SQS) to avoid backpressure on your tracing pipeline.

Real-World Example: Querying Tenant Margin

Once events are flowing, cost attribution becomes a SQL problem instead of a mystery. A few queries you'll actually use:

Daily spend per tenant:

SELECT
  t.name,
  date_trunc('day', e.created_at) AS day,
  SUM(e.cost_usd) AS total_cost,
  SUM(e.input_tokens + e.output_tokens) AS total_tokens
FROM llm_usage_events e
JOIN tenants t ON t.id = e.tenant_id
WHERE e.created_at > now() - interval '30 days'
GROUP BY t.name, day
ORDER BY day DESC, total_cost DESC;

Tenants burning more than their plan allows (margin risk):

SELECT
  t.name,
  t.plan_tier,
  SUM(e.cost_usd) AS mtd_cost
FROM llm_usage_events e
JOIN tenants t ON t.id = e.tenant_id
WHERE e.created_at >= date_trunc('month', now())
GROUP BY t.name, t.plan_tier
HAVING SUM(e.cost_usd) > CASE t.plan_tier
  WHEN 'starter' THEN 20
  WHEN 'growth' THEN 100
  ELSE 500
END;

Root-causing a spike using the trace ID:

Since trace_id is stored alongside cost data, you can jump from "this tenant cost us $80 yesterday" straight into your tracing backend (Tempo, Jaeger, Honeycomb) to see the exact request, prompt size, and downstream calls that caused it — closing the loop between cost data and debugging data.

🚀 Pro Tips

• Tag spans with feature.name, not just tenant.id. Knowing a tenant is expensive is useful; knowing it's their "auto-tagging" feature specifically is actionable.
• Capture cached and reasoning tokens separately. Reasoning models bill differently for hidden reasoning tokens — lumping them into output_tokens will quietly distort your margin math.
• Version your pricing table. Model prices drop (and occasionally rise) without warning. An effective_from column lets you recompute historical costs accurately if a provider retroactively adjusts pricing.
• Sample your general traces, but never sample cost spans. It's fine to keep 10% of traces for latency debugging, but every single LLM call must produce a cost event — use a dedicated processor path that bypasses the trace sampler.
• Add a request.id idempotency key to llm_usage_events if you have retry logic anywhere in your stack, so a retried call doesn't get billed to the tenant twice.
• Materialize a daily rollup table (llm_usage_daily_rollup) for dashboards — querying raw event tables for a 90-day chart gets slow past a few million rows.

Best Practices

• Treat tenant.id as a required span attribute, and fail loudly in development if it's missing — a null tenant on a paid LLM call is a silent revenue leak.
• Keep your OTel resource attributes (service.name, service.version) consistent across the Next.js app and every downstream microservice; it makes correlating spans across services trivial.
• Use OpenTelemetry's official Generative AI semantic conventions rather than inventing your own attribute names — it future-proofs your dashboards against tooling changes.
• Separate the tracing export path (for debugging) from the cost export path (for billing) — different consumers, different durability guarantees, different retention needs.
• Run the OTel Collector as a sidecar or dedicated service in production rather than exporting directly from every app instance; it centralizes retries, batching, and backpressure handling.

Common Mistakes

• Passing tenantId as a function parameter instead of context/baggage. It works until an async queue, webhook, or background job breaks the chain — and then you have unattributed spend with no way to trace it back.
• Logging token usage instead of structuring it. A console.log of the usage object is invisible to SQL, dashboards, and alerting. If it's not in a table, it's not tracked.
• Ignoring cached and prompt-caching discounts. Providers increasingly discount cached input tokens; if you bill flat-rate on total input tokens, your internal cost model will overstate real spend.
• Blocking the request path on the Postgres insert. Writing usage rows synchronously inside the request lifecycle adds latency and creates a single point of failure. Batch and flush asynchronously.
• Forgetting the Edge runtime gap. The OpenTelemetry Node SDK doesn't run on Next.js Edge middleware — if part of your request path executes there, propagate context manually or move tenant resolution to a Node.js route handler.

📌 Key Takeaways

• Token usage should be modeled as a structured, per-request observability signal, not scraped after the fact from a provider invoice.
• OpenTelemetry's Generative AI semantic conventions (gen_ai.usage.*) give you a standard, portable data model for LLM cost tracking.
• Tenant identity must travel through OpenTelemetry baggage, not function arguments, to survive async work and microservice boundaries.
• A dedicated SpanProcessor can export cost-relevant spans directly into PostgreSQL, independent of your general tracing backend.
• A normalized schema (tenants, llm_usage_events, model_pricing) is enough to power real-time margin dashboards and usage-based billing.

Conclusion

AI-native SaaS products don't fail because the model was too expensive — they fail because nobody could see the cost breakdown until it was too late to act. The pattern in this guide — OpenTelemetry for capture and propagation, PostgreSQL for durable attribution — isn't exotic infrastructure. It's the same distributed tracing discipline teams have used for latency and error monitoring for years, pointed at a new, more expensive resource: tokens.

Once this pipeline is in place, questions that used to require a support ticket and a week of log-spelunking — "why is this tenant's bill so high," "is this feature actually profitable," "should we gate this model behind a higher plan tier" — become a SQL query away. That's the real ROI: not just cost visibility, but the confidence to price and package your AI features like a business, instead of guessing.

Start small. Instrument one high-traffic LLM call this week, get tenant-tagged spans landing in Postgres, and build your first margin query. The rest of the pipeline gets easier to justify once the first dashboard exists.

References

• OpenTelemetry Generative AI Semantic Conventions — opentelemetry.io/docs/specs/semconv/gen-ai
• Next.js Instrumentation Documentation — nextjs.org/docs/app/building-your-application/optimizing/open-telemetry
• OpenTelemetry JavaScript SDK — github.com/open-telemetry/opentelemetry-js
• LangChain.js Callbacks Documentation — js.langchain.com/docs/concepts/callbacks
• W3C Trace Context and Baggage Specifications — w3.org/TR/trace-context and w3.org/TR/baggage
• PostgreSQL Documentation on Indexing and Partitioning — postgresql.org/docs