How to Implement Hierarchical Context Compaction in LangGraph.js to Prevent Agent Context Drift

Introduction

If you've shipped a long-running LangGraph.js agent into production, you've probably watched it happen in real time: an agent that starts a task with laser focus slowly turns into something else entirely by step 40. It forgets why it opened that third API call. It starts re-fetching data it already has. It confidently hallucinates a conclusion that has nothing to do with the original user request.

This isn't a model quality problem. It's a context management problem, and it's arguably the least glamorous, most under-discussed part of building agentic systems in 2026. Every tool call, every intermediate reasoning step, every verbose JSON payload from a database query gets appended to the conversation history. After a few dozen turns, your context window is 70% noise — old tool outputs nobody will ever reference again — and only 30% signal. The model has to search harder for the original goal, and the further that signal is diluted, the more likely you are to see what practitioners now commonly call context drift: a gradual, cumulative loss of task fidelity that compounds with every additional step.

This guide is deliberately narrow and deliberately technical. We're not going to talk about prompt engineering in the abstract. We're going to build a working hierarchical context compaction layer in Node.js using LangGraph.js, backed by a local Ollama model for summarization and a PostgreSQL checkpointer for persistence. By the end, you'll have a pattern you can drop into any long-horizon agent — coding assistants, research agents, multi-step customer support flows — to keep them anchored to their original goal for hundreds of steps instead of dozens.

What Is Context Drift, Really?

Context drift is often confused with hallucination, but it's more specifically a positional and attentional problem. Modern LLMs, even those with 128K+ token windows, don't attend to every token equally. Research on long-context retrieval consistently shows a "lost in the middle" effect — information at the very start or very end of a context window is retrieved more reliably than information buried in the middle.

In an agentic loop, your original system prompt and goal statement sit at the start of the conversation. As the agent executes more tool calls, that goal gets pushed further and further from the "active" region of attention the model actually weighs heavily. By step 30 or 40, the goal is now buried under thousands of tokens of tool output, and the model starts optimizing for local coherence (what should I do given the last three messages?) instead of global coherence (what am I actually trying to accomplish?).

The symptoms are predictable:

• Goal substitution — the agent starts solving a sub-problem it invented rather than the original task.
• Redundant tool calls — it re-queries data it already fetched because it no longer "sees" the earlier result clearly.
• Hallucination cascades — one slightly wrong inference early in a long chain gets treated as ground truth by every subsequent step, since nothing forces re-verification against the original request.
• Runaway token costs — every additional turn re-sends the entire bloated history, so your API bill grows quadratically with task length even though useful information density is dropping.

Naive fixes like "just truncate old messages" or "just keep the last 10 messages" solve the token-cost problem but make drift worse, because they throw away the original goal statement entirely. You need something smarter: compression that preserves signal while discarding noise, and preserves it hierarchically so that the agent can still reason at multiple time scales.

Why Flat Summarization Isn't Enough

A common first attempt at this problem looks like: "every 10 steps, summarize the whole conversation into one paragraph and replace the history with it." This is a reasonable start, but it has a structural flaw — it treats every summarization pass as independent, with no memory of previous summaries. After three or four compaction cycles, you're re-summarizing a summary of a summary, and detail loss compounds non-linearly. Important early context (the original goal, key constraints, early decisions) gets diluted at the same rate as low-value noise.

Hierarchical compaction solves this by treating memory as layered, similar to how L1/L2/L3 caches work in computer architecture:

• Level 0 — Raw buffer: The last N raw messages (tool calls, tool outputs, reasoning steps), kept in full fidelity.
• Level 1 — Milestones: Every time the raw buffer crosses a threshold (e.g., every 10 steps), it's distilled into a short "milestone" — a 3-5 bullet summary of what was accomplished, decided, or discovered.
• Level 2 — Epochs: Once enough milestones accumulate (e.g., every 5 milestones, or roughly 50 steps), those milestones themselves get compacted into a single higher-level "epoch summary."
• Anchor — Goal statement: The original user request and any hard constraints, which is never compacted or paraphrased away.

This is the same principle behind git's commit history versus a squashed release changelog versus a product roadmap — each layer serves a different granularity of recall, and none of them destroy the layer below until it's genuinely no longer useful.

Architecture Overview

Here's the shape of the system we're building:

┌─────────────────────────────────────────────────────────┐
│                      LangGraph.js Graph                  │
│                                                           │
│   START → [agent] ⇄ [tools]                              │
│              │                                           │
│              ▼ (every 10 steps)                          │
│         [compactionNode] ──► Ollama (Llama 3 8B)         │
│              │                                           │
│              ▼ (every 5 milestones)                      │
│      [epochCompactionNode] ──► Ollama (Llama 3 8B)       │
│              │                                           │
│              ▼                                           │
│      PostgresSaver checkpointer (pruned periodically)    │
└─────────────────────────────────────────────────────────┘

The agent node and tool node behave like any standard LangGraph.js ReAct-style loop. The difference is a conditional edge that routes execution through a compaction node at regular intervals. That node calls a local Ollama model, produces a milestone summary, and rewrites the graph's messages channel to replace bulky raw history with the compressed version — while the original goal anchor stays untouched.

Setting Up the Project

Let's get the dependencies in place.

mkdir langgraph-context-compaction && cd langgraph-context-compaction
npm init -y
npm install @langchain/langgraph @langchain/langgraph-checkpoint-postgres \
  @langchain/ollama @langchain/core pg dotenv
npm install -D typescript tsx @types/node @types/pg

You'll also need Ollama running locally with a small, fast model pulled:

ollama pull llama3:8b
ollama serve

Llama 3 8B is the sweet spot for this job: it's fast enough to run inline in a hot path every 10 steps without noticeably slowing your agent down, and summarization is a task where an 8B model performs nearly as well as much larger ones — you're not asking it to reason, just to condense.

Set up your PostgreSQL connection string in .env:

DATABASE_URL=postgresql://user:password@localhost:5432/agent_memory

Step 1: Defining Graph State with Depth Tracking

The foundation of this whole system is a state schema that tracks where the agent is in its execution, not just what it has said. We use LangGraph.js's Annotation.Root API to define a custom reducer for messages — one that supports both normal appending and a "flush and replace" operation for compaction.

// state.ts
import { Annotation } from "@langchain/langgraph";
import type { BaseMessage } from "@langchain/core/messages";

type FlushUpdate = { flush: true; replacement: BaseMessage[] };
type MessagesUpdate = BaseMessage[] | FlushUpdate;

function isFlush(update: MessagesUpdate): update is FlushUpdate {
  return !Array.isArray(update) && (update as FlushUpdate).flush === true;
}

export const AgentState = Annotation.Root({
  // Raw, high-fidelity message buffer (Level 0)
  messages: Annotation<BaseMessage[], MessagesUpdate>({
    reducer: (current, update) =>
      isFlush(update) ? update.replacement : current.concat(update),
    default: () => [],
  }),

  // Tracks total graph steps executed for this thread
  stepCount: Annotation<number>({
    reducer: (_, update) => update,
    default: () => 0,
  }),

  // Level 1 memory: short summaries produced every N steps
  milestones: Annotation<string[]>({
    reducer: (current, update) => current.concat(update),
    default: () => [],
  }),

  // Level 2 memory: summaries of summaries
  epochSummary: Annotation<string>({
    reducer: (_, update) => update,
    default: () => "",
  }),

  // Never compacted — the immutable original goal
  goalAnchor: Annotation<string>({
    reducer: (_, update) => update,
    default: () => "",
  }),
});

Notice the messages reducer accepts either a plain array (normal append behavior, same as messagesStateReducer) or a { flush: true, replacement } object. This gives the compaction node a clean way to say "throw away the raw buffer and replace it with this compressed version" without fighting the default append-only semantics that LangGraph.js message channels usually assume.

Step 2: Building the Compaction Node

This node is the heart of the system. It fires when stepCount crosses a multiple of 10, pulls the raw message buffer, and asks a local Ollama model to distill it into a milestone.

// compaction.ts
import { ChatOllama } from "@langchain/ollama";
import { SystemMessage } from "@langchain/core/messages";
import type { AgentState } from "./state";

export const COMPACTION_INTERVAL = 10;
export const EPOCH_INTERVAL = 5; // milestones per epoch

const summarizer = new ChatOllama({
  model: "llama3:8b",
  baseUrl: process.env.OLLAMA_BASE_URL ?? "http://127.0.0.1:11434",
  temperature: 0,
});

function serializeForSummary(messages: { _getType(): string; content: unknown }[]) {
  return messages
    .map((m) => `[${m._getType().toUpperCase()}] ${String(m.content).slice(0, 1500)}`)
    .join("\n\n");
}

export async function compactionNode(state: typeof AgentState.State) {
  const transcript = serializeForSummary(state.messages);

  const prompt = `You are a precise engineering note-taker inside an autonomous agent.
Condense the transcript below into at most 5 bullet points covering:
- what was accomplished or discovered
- any decisions made and why
- any unresolved errors, blockers, or pending actions
Do not invent information. Do not restate the original goal — it is tracked separately.
Be terse. Prefer facts over narrative.

TRANSCRIPT:
${transcript}`;

  const response = await summarizer.invoke([new SystemMessage(prompt)]);
  const milestone = String(response.content).trim();

  return {
    milestones: [milestone],
    stepCount: state.stepCount, // unchanged here; agent node increments it
    messages: {
      flush: true,
      replacement: [
        new SystemMessage(
          `[GOAL] ${state.goalAnchor}\n\n[COMPACTED HISTORY — steps ${
            state.stepCount - COMPACTION_INTERVAL + 1
          }-${state.stepCount}]\n${milestone}`
        ),
      ],
    },
  };
}

Two things matter here. First, the summarization prompt explicitly tells the model not to restate the goal — that's handled by the separate, never-compacted goalAnchor field, which we re-inject into every flushed buffer regardless of compaction level. Second, we cap each message at 1,500 characters before summarization; tool outputs (especially raw JSON from APIs or database queries) can be enormous, and there's no reason to spend tokens summarizing the fifteenth field of a payload the agent already acted on.

Step 3: The Epoch-Level (Second Tier) Compaction Node

This is what makes the compaction genuinely hierarchical rather than just periodic. Once five milestones have accumulated (roughly 50 raw steps), we compact the milestones themselves into a single epoch summary — preventing milestone lists from growing unbounded over very long tasks.

// epoch-compaction.ts
import { ChatOllama } from "@langchain/ollama";
import { SystemMessage } from "@langchain/core/messages";
import type { AgentState } from "./state";
import { EPOCH_INTERVAL } from "./compaction";

const summarizer = new ChatOllama({
  model: "llama3:8b",
  baseUrl: process.env.OLLAMA_BASE_URL ?? "http://127.0.0.1:11434",
  temperature: 0,
});

export async function epochCompactionNode(state: typeof AgentState.State) {
  const recentMilestones = state.milestones.slice(-EPOCH_INTERVAL);

  const prompt = `Combine these sequential milestone summaries from an autonomous
agent into a single higher-level progress summary (max 6 bullets). Preserve any
unresolved issues verbatim. Merge duplicate or superseded facts.

MILESTONES:
${recentMilestones.map((m, i) => `Milestone ${i + 1}:\n${m}`).join("\n\n")}`;

  const response = await summarizer.invoke([new SystemMessage(prompt)]);

  return {
    epochSummary: String(response.content).trim(),
  };
}

The epochSummary field becomes part of the agent's system prompt on every turn going forward (shown in Step 5), giving it a persistent, low-token-cost sense of "everything I've done so far" without needing to hold every milestone in the active buffer.

Step 4: Wiring the Graph and Conditional Routing

Now we assemble the graph, with conditional edges that route to the appropriate compaction tier based on stepCount and milestones.length.

// graph.ts
import { StateGraph, START, END } from "@langchain/langgraph";
import { PostgresSaver } from "@langchain/langgraph-checkpoint-postgres";
import { AgentState } from "./state";
import { compactionNode, COMPACTION_INTERVAL, EPOCH_INTERVAL } from "./compaction";
import { epochCompactionNode } from "./epoch-compaction";
import { agentNode } from "./agent-node";
import { toolNode } from "./tool-node";

function routeAfterAgent(state: typeof AgentState.State) {
  const hitCompactionBoundary =
    state.stepCount > 0 && state.stepCount % COMPACTION_INTERVAL === 0;

  if (hitCompactionBoundary) return "compact";
  if (state.messages.at(-1)?.getType?.() === "tool_call") return "tools";
  return END;
}

function routeAfterCompaction(state: typeof AgentState.State) {
  const hitEpochBoundary =
    state.milestones.length > 0 && state.milestones.length % EPOCH_INTERVAL === 0;
  return hitEpochBoundary ? "epochCompact" : "agent";
}

const workflow = new StateGraph(AgentState)
  .addNode("agent", agentNode)
  .addNode("tools", toolNode)
  .addNode("compact", compactionNode)
  .addNode("epochCompact", epochCompactionNode)
  .addEdge(START, "agent")
  .addConditionalEdges("agent", routeAfterAgent, {
    tools: "tools",
    compact: "compact",
    [END]: END,
  })
  .addEdge("tools", "agent")
  .addConditionalEdges("compact", routeAfterCompaction, {
    epochCompact: "epochCompact",
    agent: "agent",
  })
  .addEdge("epochCompact", "agent");

const checkpointer = PostgresSaver.fromConnString(process.env.DATABASE_URL!);
await checkpointer.setup();

export const app = workflow.compile({ checkpointer });

This graph is entirely declarative — the compaction cadence is controlled by two constants (COMPACTION_INTERVAL and EPOCH_INTERVAL), and you can tune them per use case without touching the routing logic itself.

Step 5: Injecting Compressed State Back into the Active Prompt

Compaction is useless if the agent node doesn't actually use the compressed state. The agent node should assemble its prompt from three layers, in order: the goal anchor, the epoch summary (if one exists), and the current raw message buffer.

// agent-node.ts
import { ChatOpenAI } from "@langchain/openai";
import { SystemMessage } from "@langchain/core/messages";
import type { AgentState } from "./state";

const primaryModel = new ChatOpenAI({ model: "gpt-4.1", temperature: 0.2 });

export async function agentNode(state: typeof AgentState.State) {
  const systemContext = [
    `PRIMARY GOAL: ${state.goalAnchor}`,
    state.epochSummary
      ? `LONG-TERM PROGRESS SUMMARY:\n${state.epochSummary}`
      : null,
  ]
    .filter(Boolean)
    .join("\n\n");

  const response = await primaryModel.invoke([
    new SystemMessage(systemContext),
    ...state.messages,
  ]);

  return {
    messages: [response],
    stepCount: state.stepCount + 1,
  };
}

Note the separation of concerns: the primary reasoning model (here, a hosted model like GPT-4.1 or Claude) never has to see the raw, unbounded tool history beyond the current compaction window. It always sees a compact, three-layer context: the immutable goal, the rolled-up long-term progress, and only the most recent raw steps. This is what actually prevents drift — the goal is structurally guaranteed to stay in the highest-attention region of the prompt on every single turn, not just at the start.

Step 6: Flushing the PostgreSQL Checkpointer

Here's a detail that trips up a lot of teams: flushing the in-memory messages channel does not shrink your PostgreSQL checkpoint table. LangGraph.js checkpointers store a new row (or updated blob) on every graph step, and older checkpoints stick around by design — that's what enables time-travel and debugging. If you don't proactively prune them, your checkpoints and checkpoint_writes tables will grow indefinitely even after your active context is nicely compacted.

The safe approach is to prune checkpoint history for a thread after a successful compaction, keeping only the most recent checkpoints needed for potential rollback:

// prune-checkpoints.ts
import { Pool } from "pg";

const pool = new Pool({ connectionString: process.env.DATABASE_URL });

export async function pruneOldCheckpoints(threadId: string, keepLast = 5) {
  await pool.query(
    `
    DELETE FROM checkpoints
    WHERE thread_id = $1
      AND checkpoint_id NOT IN (
        SELECT checkpoint_id FROM checkpoints
        WHERE thread_id = $1
        ORDER BY checkpoint_id DESC
        LIMIT $2
      )
    `,
    [threadId, keepLast]
  );

  await pool.query(
    `DELETE FROM checkpoint_writes WHERE thread_id = $1
     AND checkpoint_id NOT IN (
       SELECT checkpoint_id FROM checkpoints WHERE thread_id = $1
     )`,
    [threadId]
  );
}

Call this from your compaction node (or a scheduled job) right after a successful flush. Keeping the last 3-5 checkpoints, rather than deleting everything, preserves a short rollback window in case a compaction pass drops something important — cheap insurance for a destructive operation.

Real-World Example: A Long-Horizon Coding Agent

Consider an autonomous coding agent tasked with "migrate this Express.js API to Fastify, updating all route handlers and tests." Without compaction, by step 35 the agent's context is dominated by full file contents it read 20 steps ago, verbose linter output, and repeated npm test logs — none of which are relevant anymore, but all of which are still consuming tokens and diluting attention.

With hierarchical compaction running every 10 steps:

• Steps 1-10: Raw buffer contains file reads, initial route handler conversions, and test runs. At step 10, this compacts into Milestone 1: "Converted 4 of 12 route handlers to Fastify syntax. Test suite currently failing on auth middleware due to signature mismatch."
• Steps 11-20: Agent fixes the middleware issue, converts 5 more handlers. Milestone 2 captures this plus the still-unresolved detail that 3 handlers remain.
• Steps 21-50: Three more milestones accumulate. At step 50 (5 milestones), the epoch compaction node rolls them into a single summary: "Migration 90% complete. All route handlers converted except /webhooks. Test suite green except two flaky timeout tests unrelated to migration."

At step 51, the agent's prompt contains: the original goal, that one epoch summary, and only the last 10 raw steps — instead of 50 steps of accumulated noise. Teams running this pattern in production report 60-75% reductions in per-turn token costs on tasks longer than 40 steps, along with a measurable drop in "the agent forgot what it was doing" failure modes, since the goal and rolled-up progress are re-asserted into every single call rather than fading into the middle of an ever-growing transcript.

🚀 Pro Tips

• Never compact the goal anchor. Store it once, at graph initialization, in its own state field — not inside the messages array where it can get swept up in a flush.
• Cap tool output length before it enters state, not just before summarization. A 50KB JSON blob shouldn't even reach the message buffer; truncate or extract only the fields your agent actually needs at the tool-wrapper level.
• Use temperature 0 for the summarizer. Compaction is a compression task, not a creative one — you want deterministic, terse output every time.
• Version your compaction prompts. As you tune the summarization instructions, log which prompt version produced which milestone, so you can debug regressions when an agent starts losing important details.
• Run compaction as a separate graph node, not inline in the agent node. This keeps it independently testable, and lets you swap the summarization model without touching your primary agent logic.
• Add a hard step-count ceiling. Even with perfect compaction, put a maximum total step limit on any thread as a safety net against runaway loops.

Best Practices

• Treat compaction thresholds as tunable configuration, not hardcoded constants — different agents (research vs. coding vs. support) drift at different rates.
• Log every compaction event with before/after token counts so you can measure actual savings, not assumed savings.
• Keep the summarizer model separate from the primary reasoning model to avoid resource contention and to keep compaction costs near-zero.
• Prune your checkpointer on a schedule, not just reactively — a nightly job that trims checkpoints for completed or stale threads keeps your database lean.
• Test compaction against adversarial transcripts (deliberately noisy, contradictory tool outputs) to make sure the summarizer doesn't silently drop error states.

Common Mistakes to Avoid

• Compacting on a fixed step count without accounting for token size. A step that appends a 10-token tool result and a step that appends a 5,000-token document dump are not equivalent — consider hybrid triggers based on both step count and cumulative token estimate.
• Letting the summarizer paraphrase the goal. If your compaction prompt includes the goal in the transcript it summarizes, small wording drifts accumulate across multiple compaction cycles until the "goal" the agent sees is a distorted echo of the original request.
• Forgetting to prune the checkpointer. Assuming that flushing in-memory state also cleans up your database is one of the most common and costly mistakes teams make with this pattern.
• Using the same model for compaction and primary reasoning without rate-limit isolation. This creates contention and can throttle your main agent loop during high-volume periods.
• Over-aggressive summarization. If milestones get compacted down to a single vague sentence, you've traded context bloat for information loss — the failure mode just shows up later instead of sooner.

Conclusion

Context drift isn't a model limitation you can prompt your way out of — it's an architectural problem that requires an architectural solution. Hierarchical context compaction gives your LangGraph.js agents a memory system that mirrors how humans actually track long projects: fine-grained detail for what just happened, condensed summaries for what happened recently, and a stable, unwavering sense of the overarching goal that never gets buried no matter how long the task runs.

The pattern in this guide — a local Ollama model handling summarization, a PostgreSQL checkpointer handling durable state, and a two-tier compaction hierarchy handling memory consolidation — is deliberately modular. You can swap the summarization model, tune the compaction intervals, or add a third hierarchy tier for extremely long-running agents (weeks-long research tasks, for example) without rearchitecting the graph. Start with the 10-step / 5-milestone defaults shown here, measure your actual token savings and drift reduction, and tune from there.

References

• LangChain, LangGraph.js Documentation — Persistence and Checkpointing
• LangChain, @langchain/langgraph-checkpoint-postgres package reference
• Ollama, Official Model Library and API Documentation
• Liu et al., "Lost in the Middle: How Language Models Use Long Contexts"
• LangChain Blog, Memory Management Patterns for Long-Running Agents