How to Kill Zombie LLM Streams: Propagating AbortController from Next.js to Node.js and Ollama

Somewhere right now, a user is staring at a chat window watching an answer stream in. They realise the model misunderstood the question, so they hit Stop generating, or they simply close the tab and go make coffee.

On the server, nothing happens. The GPU keeps chewing through tokens, LangChain keeps iterating its generator, Express keeps writing into a socket that has been dead for twenty seconds. The response is being generated for nobody.

I call these zombie streams. They don't crash anything and they don't show up in your error tracker. They just quietly eat GPU time, block other users behind a full queue, and, if you're on a hosted provider, inflate your bill. In this guide we'll build a complete cancellation pipeline:

• Browser: bind an AbortController to fetch in a React client component
• Next.js: forward request.signal from a Route Handler to the backend
• Express: detect a real client disconnect and convert it into an AbortSignal
• LangChain and Ollama: tear down token generation cleanly
• Verification: prove with actual numbers that compute stops

Everything below is TypeScript and works with current Node.js LTS (22 or newer), the Next.js App Router, and the current @langchain/ollama package.

Why Zombie Streams Are More Expensive Than They Look

Let me put some (illustrative) numbers on this. Say you self-host an 8B-parameter model that produces around 50 tokens per second on your GPU, and your responses can run up to 2,000 tokens.

A user abandons a request at token 500. If nothing cancels the job, that's 1,500 wasted tokens, or 30 seconds of GPU time for a response nobody will ever read.

Now imagine 200 abandoned requests per hour. That's 6,000 seconds of wasted compute, or 100 GPU-minutes every hour. On a single GPU, you've just built a system that can't even keep up with its own garbage.

The damage is not only about money:

• Queue pressure. Ollama serves a limited number of requests in parallel. Every zombie occupies a slot while real users wait behind it.
• Latency spikes. Time-to-first-token for healthy requests balloons during traffic bursts, precisely when zombies pile up.
• Cost on hosted APIs. If you proxy to a paid provider, tokens generated after the user left are pure waste.
• Confusing capacity planning. Your dashboards say the GPU is 100% busy, so you buy more GPUs instead of fixing the leak.

The good news is that the fix is mostly plumbing. The bad news is that the plumbing has five or six joints, and any one of them leaking defeats the whole thing.

How Cancellation Actually Works (and Why It Breaks)

HTTP streaming is fundamentally a one-way conversation. The server pushes bytes, the client reads them. When the client leaves, the operating system tears down the TCP connection (a FIN or RST packet), but your application code is not automatically notified in a useful way. Node.js will emit events, but nobody in your code is subscribed to them, and even if they were, a running LangChain generator has no idea it should stop.

Cancellation has to be propagated hop by hop:

Browser (fetch + AbortController)
   │  abort() closes the connection
   ▼
Next.js Route Handler (request.signal)
   │  forwards signal to upstream fetch
   ▼
Express (res.on('close') → AbortSignal)
   │  passes signal into the chain
   ▼
LangChain ChatOllama (stream options: signal)
   │  stops iterating, closes HTTP request
   ▼
Ollama server (request context cancelled)
   └─ runner stops generating, slot freed

Here's the same thing as a checklist you can keep next to your monitor:

A 60-Second Refresher on AbortController

AbortController is a tiny, standardised primitive available in browsers and Node.js. You create a controller, hand its signal to any cancellable operation, and later call abort().

const controller = new AbortController();

fetch("/api/chat", { signal: controller.signal });

controller.abort(); // rejects the fetch with an AbortError

Three details matter for what we're building:

• signal.reason carries why the abort happened. You can pass your own value to abort(reason), which is how we'll tell a client disconnect from a timeout.
• signal.throwIfAborted() lets you bail out early in the middle of a long function.
• AbortSignal.any([a, b]) and AbortSignal.timeout(ms) let you compose signals, for example "cancel if the client leaves or if 2 minutes pass". Both are available in Node.js 20 and later.

Step 1: Bind the AbortController in the Next.js Client

We'll start where the user starts: the React client component. The hook below handles the three moments a stream must be cancelled:

1. The user clicks Stop
2. The user sends a new message while one is still streaming
3. The component unmounts (navigation, tab close, hot reload)

// app/components/use-chat-stream.ts
"use client";

import { useCallback, useEffect, useRef, useState } from "react";

export type ChatMessage = {
  role: "system" | "user" | "assistant";
  content: string;
};

type Status = "idle" | "streaming" | "stopped" | "error";

export function useChatStream() {
  const controllerRef = useRef<AbortController | null>(null);
  const [text, setText] = useState("");
  const [status, setStatus] = useState<Status>("idle");

  const stop = useCallback(() => {
    controllerRef.current?.abort();
  }, []);

  // Cancel whatever is in flight when the component goes away.
  useEffect(() => {
    return () => controllerRef.current?.abort();
  }, []);

  const send = useCallback(async (messages: ChatMessage[]) => {
    // A new request replaces the old one. Never leave two streams running.
    controllerRef.current?.abort();

    const controller = new AbortController();
    controllerRef.current = controller;
    const isCurrent = () => controllerRef.current === controller;

    setText("");
    setStatus("streaming");

    try {
      const res = await fetch("/api/chat", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ messages }),
        signal: controller.signal,
      });

      if (!res.ok || !res.body) throw new Error(`HTTP ${res.status}`);

      const reader = res.body.getReader();
      const decoder = new TextDecoder();
      let buffer = "";

      while (true) {
        const { done, value } = await reader.read();
        if (done) break;

        buffer += decoder.decode(value, { stream: true });
        const frames = buffer.split("\n\n");
        buffer = frames.pop() ?? ""; // keep the incomplete tail

        for (const frame of frames) {
          const line = frame.split("\n").find((l) => l.startsWith("data: "));
          if (!line) continue; // heartbeat comments like ": ping"

          const data = line.slice(6);
          if (data === "[DONE]") continue;

          const payload = JSON.parse(data);
          if (payload.error) throw new Error(payload.error);
          if (payload.token && isCurrent()) {
            setText((prev) => prev + payload.token);
          }
        }
      }

      if (isCurrent()) setStatus("idle");
    } catch (err) {
      if (!isCurrent()) return; // a newer request owns the UI now
      // An abort is a user action, not an error. Don't show a red banner.
      setStatus(controller.signal.aborted ? "stopped" : "error");
    } finally {
      if (isCurrent()) controllerRef.current = null;
    }
  }, []);

  return { text, status, send, stop };
}

And the component that uses it:

// app/chat/page.tsx
"use client";

import { useState } from "react";
import { useChatStream } from "../components/use-chat-stream";

export default function ChatPage() {
  const [input, setInput] = useState("");
  const { text, status, send, stop } = useChatStream();

  return (
    <main className="mx-auto max-w-2xl p-6">
      <textarea
        value={input}
        onChange={(e) => setInput(e.target.value)}
        placeholder="Ask something..."
      />

      {status === "streaming" ? (
        <button onClick={stop}>Stop generating</button>
      ) : (
        <button
          onClick={() => send([{ role: "user", content: input }])}
          disabled={!input.trim()}
        >
          Send
        </button>
      )}

      <article aria-live="polite">{text}</article>
      {status === "stopped" && <p>Generation stopped.</p>}
    </main>
  );
}

A couple of things worth pointing out. The isCurrent() guard prevents a classic race: request A is aborted, request B starts, and then A's catch block fires late and overwrites B's state with "stopped". Ownership checks like this are cheap and save you from very confusing UI bugs.

Also notice that we don't need a beforeunload handler or navigator.sendBeacon for the "user closed the tab" case. The browser tears down open connections on its own. Our job is to make sure the server notices.

Step 2: Forward the Signal in the Next.js Route Handler

You could let the browser talk to Express directly, but most production setups keep a Next.js Route Handler in the middle. It's where you check the session, apply rate limits, and keep the internal LLM service off the public internet. That extra hop is also the most commonly forgotten link in the chain.

If your handler calls fetch to Express without a signal, then when the browser aborts, Next.js happily keeps its upstream connection open and Express never learns the user left.

// app/api/chat/route.ts
export const runtime = "nodejs";
export const dynamic = "force-dynamic";

const CHAT_API_URL = process.env.CHAT_API_URL ?? "http://127.0.0.1:4000";

export async function POST(request: Request) {
  // Authenticate and rate limit BEFORE touching the GPU service.
  const payload = await request.json();

  let upstream: Response;
  try {
    upstream = await fetch(`${CHAT_API_URL}/v1/chat/stream`, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Accept: "text/event-stream",
        Authorization: `Bearer ${process.env.CHAT_API_TOKEN}`,
      },
      body: JSON.stringify(payload),
      // The important line: the browser leaving now cancels this fetch too.
      signal: request.signal,
    });
  } catch (err) {
    if (request.signal.aborted) {
      // 499 is the de-facto "client closed request" status.
      return new Response(null, { status: 499 });
    }
    return new Response("Upstream unavailable", { status: 502 });
  }

  if (!upstream.ok || !upstream.body) {
    return new Response("Upstream error", { status: 502 });
  }

  // Stream the body straight through. When the client disconnects, the
  // response stream is cancelled, which in turn cancels upstream.body.
  return new Response(upstream.body, {
    headers: {
      "Content-Type": "text/event-stream; charset=utf-8",
      "Cache-Control": "no-cache, no-transform",
      "X-Accel-Buffering": "no",
    },
  });
}

There are two cancellation paths working together here: the explicit signal: request.signal, and the implicit stream cancellation when Next.js stops reading the returned body. I like having both. Belts and suspenders.

Heads up: how quickly request.signal fires depends on your Next.js version, your hosting platform, and any proxies in front of it. Some platforms buffer responses or keep upstream connections alive longer than you'd expect. Don't assume it works, verify it with the tests in the verification section below.

One more gotcha: if you enable Next.js's built-in gzip (compress: true, the default for next start), streamed tokens can get buffered and arrive in lumps. For streaming-heavy apps, many teams set compress: false in next.config and let the reverse proxy handle compression selectively.

Step 3: Catch the Disconnect in Express

Now for the part that most tutorials get subtly wrong. The standard advice is:

// The snippet you'll find in a lot of tutorials
req.on("close", () => controller.abort());

This looks right and worked fine on very old versions of Node.js. But since Node.js 16, the close event on IncomingMessage is emitted when the request has been completed, not when the underlying socket is destroyed. For a POST request, whose body express.json() has already consumed, that can happen almost immediately, which means your handler would abort a perfectly healthy stream a few milliseconds after it started.

The reliable signal is on the response object. res emits close either when the response finished normally or when the connection was terminated prematurely. The trick is to tell those two cases apart with res.writableFinished:

function clientDisconnectSignal(res: Response): AbortSignal {
  const controller = new AbortController();

  res.once("close", () => {
    // If we already finished writing, this is just normal cleanup.
    if (!res.writableFinished) {
      controller.abort(new DOMException("Client disconnected", "AbortError"));
    }
  });

  return controller.signal;
}

If you prefer to think in terms of sockets, req.socket.once("close", ...) works too, but res.on("close") is the version documented as the response lifecycle event, and it plays nicely with HTTP/2 compatibility layers.

The Full Streaming Route

Here's the whole Express server. It composes the disconnect signal with a hard timeout, streams tokens as Server-Sent Events, handles backpressure, and tracks in-flight requests so we can observe cancellation later.

// server.ts
import express, { type Request, type Response } from "express";
import { randomUUID } from "node:crypto";
import { once } from "node:events";
import { ChatOllama } from "@langchain/ollama";
import {
  AIMessage,
  HumanMessage,
  SystemMessage,
} from "@langchain/core/messages";

type ChatMessage = { role: "system" | "user" | "assistant"; content: string };

const OLLAMA_URL = process.env.OLLAMA_URL ?? "http://127.0.0.1:11434";
const MODEL = process.env.OLLAMA_MODEL ?? "llama3.1:8b";
const HARD_TIMEOUT_MS = 120_000;

const app = express();
app.use(express.json({ limit: "1mb" }));

const inFlight = new Map<string, { startedAt: number; tokens: number }>();

function clientDisconnectSignal(res: Response): AbortSignal {
  const controller = new AbortController();
  res.once("close", () => {
    if (!res.writableFinished) {
      controller.abort(new DOMException("Client disconnected", "AbortError"));
    }
  });
  return controller.signal;
}

function toLangChain(messages: ChatMessage[]) {
  return messages.map((m) =>
    m.role === "system"
      ? new SystemMessage(m.content)
      : m.role === "assistant"
        ? new AIMessage(m.content)
        : new HumanMessage(m.content),
  );
}

// Respect backpressure, but never wait for a drain that will never come.
async function write(res: Response, chunk: string, signal: AbortSignal) {
  if (res.write(chunk)) return;
  await once(res, "drain", { signal });
}

app.post("/v1/chat/stream", async (req: Request, res: Response) => {
  const messages = req.body?.messages as ChatMessage[] | undefined;
  if (!Array.isArray(messages) || messages.length === 0) {
    res.status(400).json({ error: "messages is required" });
    return;
  }

  const requestId = randomUUID();
  const signal = AbortSignal.any([
    clientDisconnectSignal(res),
    AbortSignal.timeout(HARD_TIMEOUT_MS),
  ]);

  res.writeHead(200, {
    "Content-Type": "text/event-stream; charset=utf-8",
    "Cache-Control": "no-cache, no-transform",
    Connection: "keep-alive",
    "X-Accel-Buffering": "no",
  });
  res.flushHeaders();

  const state = { startedAt: Date.now(), tokens: 0 };
  inFlight.set(requestId, state);
  let outcome: "completed" | "client_aborted" | "timeout" | "error" =
    "completed";

  // Heartbeats keep proxies from idling us out AND make dead sockets surface faster.
  const heartbeat = setInterval(() => res.write(": ping\n\n"), 15_000);

  try {
    // One model instance per request. It's cheap, and it isolates cancellation.
    const model = new ChatOllama({
      baseUrl: OLLAMA_URL,
      model: MODEL,
      temperature: 0.2,
      numPredict: 1024, // hard cap on output tokens as a safety net
    });

    signal.throwIfAborted();
    const stream = await model.stream(toLangChain(messages), { signal });

    for await (const chunk of stream) {
      if (signal.aborted) break; // leaving the loop closes the iterator
      const token = typeof chunk.content === "string" ? chunk.content : "";
      if (!token) continue;

      state.tokens += 1;
      await write(res, `data: ${JSON.stringify({ token })}\n\n`, signal);
    }

    signal.throwIfAborted();
    res.write("data: [DONE]\n\n");
  } catch (err) {
    if (signal.aborted) {
      outcome =
        (signal.reason as Error | undefined)?.name === "TimeoutError"
          ? "timeout"
          : "client_aborted";
    } else {
      outcome = "error";
      if (!res.writableEnded) {
        res.write(`data: ${JSON.stringify({ error: "generation_failed" })}\n\n`);
      }
    }
  } finally {
    clearInterval(heartbeat);
    inFlight.delete(requestId);
    console.log(
      JSON.stringify({
        event: "chat_stream_finished",
        requestId,
        outcome,
        tokens: state.tokens,
        durationMs: Date.now() - state.startedAt,
      }),
    );
    if (!res.writableEnded) res.end();
  }
});

app.get("/healthz", (_req, res) => {
  res.json({ ok: true, inFlight: inFlight.size });
});

app.listen(4000, () => console.log("chat api listening on :4000"));

Let me walk through the decisions that matter:

• AbortSignal.any merges "client left" and "took too long" into one signal. Downstream code has exactly one thing to check.
• signal.throwIfAborted() before stream() covers the case where the user leaves while the request is still queued or the model is loading.
• break inside for await isn't just control flow. It calls the iterator's return(), which lets LangChain's generator run its cleanup and close the underlying HTTP request.
• once(res, "drain", { signal }) avoids a nasty hang. If the socket buffer is full and the client vanishes, a bare await once(res, "drain") would wait forever because drain never fires.
• The finally block is where the truth is recorded. Whatever happened, we log the outcome, remove the request from the in-flight map, and end the response.

Step 4: Tear Down Generation in LangChain and Ollama

At this point we hand signal to LangChain through the standard runnable options. Every LangChain runnable accepts signal in its config, so the same pattern works for invoke, stream, chains built with the pipe operator, and LangGraph agents.

const stream = await model.stream(messages, { signal });

What happens after that depends on how your version of the Ollama integration wires cancellation. Historically it has done one of two things: passed the signal down to the HTTP client, or checked signal.aborted between chunks and asked the client to abort. Both approaches end the same way, with the HTTP request to Ollama being closed. But they differ in latency. The second one only notices the abort when the next chunk arrives, so during a slow prompt-evaluation phase you might wait a moment.

That's precisely why our route also checks signal.aborted inside the loop and calls throwIfAborted() up front. We don't rely on any one layer to be perfect.

The Shared Client Trap

This is the one that scares me the most, because it only bites in production under concurrency.

In the official ollama-js client, abort() is defined on the client instance and cancels every streaming request running on that client. If your server creates a single shared Ollama client (or a single shared ChatOllama) and wires each user's disconnect to client.abort(), then one user hitting Stop will kill every other user's answer mid-sentence.

The fix is boring and effective: one client per stream.

import { Ollama } from "ollama";

async function* streamWithOllamaJs(
  messages: ChatMessage[],
  signal: AbortSignal,
) {
  const client = new Ollama({ host: OLLAMA_URL }); // scoped to this request
  const onAbort = () => client.abort();
  signal.addEventListener("abort", onAbort, { once: true });

  try {
    const stream = await client.chat({ model: MODEL, messages, stream: true });
    for await (const part of stream) {
      yield part.message.content;
    }
  } finally {
    signal.removeEventListener("abort", onAbort);
  }
}

That's why the Express route above constructs a new ChatOllama inside the handler. Constructing one is just configuration plus an HTTP client, so the cost is negligible.

The Escape Hatch: Talk to Ollama Directly

If you don't need LangChain for a given endpoint, or you want to remove any doubt about how cancellation is wired, call Ollama's HTTP API with plain fetch. Aborting a fetch closes the connection, which is exactly the signal Ollama needs.

async function* streamFromOllama(messages: ChatMessage[], signal: AbortSignal) {
  const upstream = await fetch(`${OLLAMA_URL}/api/chat`, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      model: MODEL,
      messages,
      stream: true,
      options: { num_predict: 1024 },
    }),
    signal,
  });

  if (!upstream.ok || !upstream.body) {
    throw new Error(`Ollama responded with ${upstream.status}`);
  }

  const decoder = new TextDecoder();
  let buffer = "";

  // Ollama streams newline-delimited JSON (NDJSON).
  for await (const bytes of upstream.body) {
    buffer += decoder.decode(bytes, { stream: true });
    const lines = buffer.split("\n");
    buffer = lines.pop() ?? "";

    for (const line of lines) {
      if (!line.trim()) continue;
      const part = JSON.parse(line);
      if (part.message?.content) yield part.message.content as string;
      if (part.done) return;
    }
  }
}

Because it's an async generator, it composes with the same for await loop and the same signal we built for the LangChain version. If the consumer stops iterating, the body stream is cancelled and the connection to Ollama closes.

What Ollama Does on Its Side

When the connection to Ollama closes mid-stream, the server notices through its request context, cancels the in-progress completion, and frees the slot for the next queued request. From your perspective, GPU utilization should drop back to idle within a moment.

Two things people misread here:

• VRAM stays allocated. The model remains loaded until its keep_alive timer expires (five minutes by default). That's expected. Zombie compute means utilization, not resident memory.
• Parallel slots are the real casualty of zombies. Ollama processes a limited number of requests concurrently (see OLLAMA_NUM_PARALLEL and OLLAMA_MAX_QUEUE), so every stream that isn't cancelled blocks a slot that a real user could be using.

Step 5: Don't Let Your Infrastructure Swallow the Abort

You can write perfect application code and still lose to a misconfigured proxy. A buffering layer between the browser and Next.js can hide disconnects, delay tokens, or keep upstream connections alive long after the client is gone.

Here's a sane nginx location block for streaming endpoints:

location /api/chat {
    proxy_pass http://next_app;

    proxy_http_version 1.1;
    proxy_set_header Connection "";

    # Stream tokens as they arrive instead of buffering the whole response.
    proxy_buffering off;
    proxy_cache off;

    # Long generations need long read timeouts.
    proxy_read_timeout 300s;

    # Default is off: nginx closes the upstream when the client leaves.
    # If someone turned this on, you have a zombie factory.
    proxy_ignore_client_abort off;
}

The same principle applies to CDNs and cloud load balancers. Check three things on every layer in front of your app: is response buffering disabled for streaming routes, is the idle timeout longer than your longest generation (or are you sending heartbeats), and does the layer close the upstream connection when the client goes away?

Step 6: Prove It Works

I don't trust cancellation code until I've watched it work, and neither should you. There are three levels of verification.

1. The Manual Smoke Test

Start the stack, kick off a long generation with curl, then kill it after two seconds:

curl -N --max-time 2 \
  -X POST http://localhost:3000/api/chat \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Write a 2000 word essay about the ocean."}]}'

While it runs, watch the GPU in a second terminal:

watch -n 1 nvidia-smi --query-gpu=utilization.gpu,memory.used --format=csv

Utilization should spike while the stream is active and fall back to idle shortly after curl exits. If it stays pinned for another 20 or 30 seconds, you've got a zombie somewhere in the chain. Then bisect: run the same curl directly against Express on port 4000. If that cancels correctly but the Next.js path doesn't, the problem is in the Route Handler or a proxy in front of it.

2. The Health Endpoint

Our Express server exposes /healthz with the number of in-flight streams. During the curl test, poll it:

watch -n 1 'curl -s localhost:4000/healthz'

The count should rise to 1 during generation and return to 0 within a second of the abort. A count that lingers is a leak with a name.

3. An Automated Abort Test

Once you've fixed cancellation, keep it fixed. This script starts a stream, reads a few frames, aborts, and asserts that the server drains its in-flight set quickly:

// scripts/abort-smoke.ts
import assert from "node:assert/strict";

const BASE = process.env.BASE_URL ?? "http://localhost:4000";

async function inFlight(): Promise<number> {
  const res = await fetch(`${BASE}/healthz`);
  return (await res.json()).inFlight;
}

const controller = new AbortController();

const res = await fetch(`${BASE}/v1/chat/stream`, {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    messages: [{ role: "user", content: "Count slowly from 1 to 500." }],
  }),
  signal: controller.signal,
});

const reader = res.body!.getReader();
await reader.read(); // wait for at least one chunk
assert.equal(await inFlight(), 1, "stream should be in flight");

controller.abort();

// Give the server up to 3 seconds to notice.
const deadline = Date.now() + 3000;
while (Date.now() < deadline && (await inFlight()) !== 0) {
  await new Promise((r) => setTimeout(r, 100));
}

assert.equal(await inFlight(), 0, "zombie stream detected");
console.log("OK: abort propagated and in-flight count returned to zero");

Drop this into CI against a staging environment (a small model is fine) and you'll catch regressions the day someone refactors the route handler.

Observability: Make Cancellation a First-Class Metric

Our Express handler already emits a structured log line with an outcome field: completed, client_aborted, timeout, or error. Ship that to your log platform and chart it. It tells you things you couldn't otherwise see:

• The abort rate (what share of streams end early) is a product signal as much as an infrastructure one. A sudden jump might mean your model got slower or your answers got worse.
• Tokens generated per aborted request is the direct measure of waste. After the fixes above, it should hover near the time it takes a disconnect to propagate, not "hundreds of tokens".
• Timeouts should be rare. If they aren't, your HARD_TIMEOUT_MS is either too tight or your prompts are producing runaway output.
• In-flight gauge vs GPU utilization is the alarm I'd wire up first. If the gauge says 0 and the GPU says 100%, something is generating for nobody.

Best Practices

Here are the habits I'd bake into any streaming LLM backend:

• Always accept a signal at the boundary. Every function that does I/O in your generation path (model calls, tool calls, vector-store lookups, database queries) should take an AbortSignal and pass it along. A cancelled request that's still waiting on a slow tool call is only half-cancelled.
• Compose signals rather than checking flags. AbortSignal.any([disconnect, timeout]) gives you one source of truth and preserves the reason.
• Set a hard output cap. numPredict (or max_tokens on hosted APIs) is your last line of defense against runaway generations, even if every cancellation layer fails.
• Set a hard time cap. Two minutes is generous for most chat use cases. Pick a number and enforce it with AbortSignal.timeout.
• Instantiate per-request clients when a client's abort affects more than one stream.
• Treat aborts as outcomes, not errors. Log them at info level, don't page anyone, and never retry a request the user cancelled.
• Persist partial output deliberately. If your product saves conversation history, decide what happens to a half-finished answer. Saving it with an aborted flag is usually a better experience than dropping it.
• Authenticate before you allocate. Check the session and rate limit in the Route Handler so anonymous traffic can't spin up GPU work in the first place.

Common Mistakes

I've made or seen every one of these, some of them more than once.

1. Trusting req.on('close') on modern Node.js. As covered above, it can fire as soon as the request body is read, aborting healthy streams or, if you never wired it up correctly, never firing when you expect it. Use res.on('close') plus res.writableFinished.
2. Aborting in the browser and calling it done. Client-side abort only closes the first connection. If the Route Handler doesn't forward the signal, everything behind it keeps running.
3. Sharing one Ollama client across users. A client-wide abort() turns one person's Stop click into everyone's outage.
4. Forgetting the proxy in the middle. Buffering, aggressive idle timeouts, and proxy_ignore_client_abort on all defeat propagation silently.
5. Awaiting drain without a signal. The client disappears, the buffer never drains, and your handler hangs forever, leaking memory and holding the slot.
6. Only checking the signal between tokens. If the model is still loading or evaluating a huge prompt, no tokens arrive to trigger the check. Pass the signal into the call itself and check throwIfAborted() up front.
7. Reporting aborts as errors. Your error dashboard fills with AbortError noise, on-call gets paged for normal behavior, and real failures get lost in the clutter.
8. Retrying aborted requests. A generic retry wrapper doesn't know the user pressed Stop on purpose. Make sure your retry logic treats an aborted signal as terminal.
9. Skipping timeouts. Without one, a stuck model call or a half-open connection holds a slot indefinitely.
10. Never testing it. Cancellation regressions are invisible in normal functional tests because everything still "works". You only see them on the GPU graph and the invoice.

🚀 Pro Tips

• Send heartbeats. A tiny : ping comment every 15 seconds keeps intermediaries from closing idle connections, and it surfaces half-open sockets (think a phone that dropped off Wi-Fi) much sooner, because a write to a dead connection triggers an error instead of silently succeeding into the void.
• Limit concurrent streams per user. One or two is plenty. When a user sends a new message, abort the old stream on the server as well. It stops a runaway script or a double-clicking user from occupying every slot.
• Pass the signal into your tools. LangChain tools receive the runnable config as their second argument, and it carries the signal. Hand it to any fetch, database call, or subprocess the tool makes so agent runs cancel end to end.
• Batch UI updates. Calling setText on every token can trigger hundreds of renders per second. Accumulate tokens in a ref and flush once per animation frame with requestAnimationFrame.
• Use the same pattern for hosted models. ChatOpenAI, ChatAnthropic, and friends accept the same signal, and closing the connection generally stops further generation and billing. Tokens already produced are still charged, so check your provider's behavior with usage reporting turned on.
• Log the abort reason. signal.reason is free context. Distinguishing Client disconnected from TimeoutError tells you whether you have a UX problem or a latency problem.
• Put a canary in CI. Run the abort smoke test on every deploy of the chat path. It takes seconds and it's the cheapest insurance on your GPU budget.
• Consider a queue-position message. If your instance is saturated, telling users "you're number 3 in line" reduces the frantic reload-and-resubmit behavior that creates zombies in the first place.

📌 Key Takeaways

• Closing a tab or clicking Stop only ends the first connection. Without propagation, the model keeps generating for nobody.
• Bind an AbortController per request in the client, abort on Stop, on new submit, and on unmount, and treat AbortError as a normal outcome.
• In the Next.js Route Handler, pass request.signal to the upstream fetch and stream the upstream body straight through.
• In Express on modern Node.js, use res.on('close') with a writableFinished check instead of relying on req.on('close'), and convert it into an AbortSignal.
• Combine the disconnect signal with AbortSignal.timeout() via AbortSignal.any(), and pass it to LangChain through the stream() options.
• Never share a client whose abort() affects every stream. Use one Ollama or ChatOllama instance per request.
• Audit your proxies, buffering, and idle timeouts, because infrastructure can silently defeat correct application code.
• Measure it: an in-flight gauge, GPU utilization, tokens per aborted request, and an automated abort test in CI.

Conclusion

Zombie streams are one of those problems that hide in plain sight. Your app works, your demo is snappy, and your users are happy, right up until traffic grows and your GPU graph looks like a plateau nobody can explain.

The fix isn't exotic. It's an unbroken chain of small, boring decisions: the browser aborts, the Next.js Route Handler forwards, Express turns a real disconnect into an AbortSignal, LangChain and Ollama stop generating, and your infrastructure gets out of the way. Each link is a handful of lines. Together they turn "we hope the model stops" into "we can prove it stopped."

If you take one thing from this post, make it this: write the abort smoke test first. Watch it fail against your current stack, wire up each hop until it passes, and keep it in CI forever. Your GPUs, your latency numbers, and (if you're paying per token) your finance team will thank you.

References

• MDN: AbortController
• MDN: AbortSignal.any()
• WHATWG DOM Standard: Aborting ongoing activities
• Node.js Docs: HTTP (IncomingMessage and ServerResponse events)
• Node.js Docs: events.once()
• Next.js Docs: Route Handlers
• Express.js Documentation
• LangChain.js: ChatOllama Integration
• Ollama API Documentation
• ollama-js: Official JavaScript Client
• nginx Docs: proxy_ignore_client_abort