If you've shipped an LLM-powered feature in the last two years, there's a good chance you've done something you'd rather not put in a compliance audit: taken a raw customer message — full name, email, maybe a partial card number typed into a support chat — and piped it straight into a prompt sent to a third-party model provider. It works. It ships fast. And it is quietly one of the riskiest patterns in modern application architecture.
This isn't a hypothetical. Support transcripts, CRM notes, and internal tickets are exactly the kind of unstructured text that ends up in retrieval-augmented generation (RAG) pipelines and prompt templates, and they're exactly the kind of text that's saturated with Personally Identifiable Information (PII). Once that data leaves your infrastructure boundary, you've effectively made your LLM vendor a data subprocessor — whether or not you meant to, and whether or not your data processing agreement accounts for it.
This guide walks through building a two-way PII masking and re-hydration middleware in Node.js and Express: a layer that intercepts outbound prompts, detects and replaces sensitive values with deterministic surrogate tokens, forwards the sanitized prompt to the LLM, and then — critically — reverses the process on the way back, re-hydrating those tokens with the original values in real time, even while the response is streaming.
By the end, you'll have a working architecture you can drop into an existing Express API, along with the reasoning behind each design decision, the mistakes teams commonly make, and the production hardening steps that separate a demo from something you'd actually deploy behind a HIPAA-covered workload.
Why PII Leakage Into LLM Prompts Is a Real Problem
Let's be precise about what "sending PII to an LLM" actually means from a risk perspective, because the danger isn't just abstract.
- Subprocessor exposure. Most LLM API providers explicitly state they may retain prompts for abuse monitoring, even when they don't use your data for training. That retention window — often 30 days or more — is a window where sensitive data sits on infrastructure you don't control and didn't disclose to your users.
- Regulatory scope creep. Under GDPR, transmitting personal data to a third party for processing makes that party a data processor, which requires a valid legal basis, a Data Processing Agreement (DPA), and — in many cases — disclosure in your privacy policy. Under HIPAA, sending Protected Health Information (PHI) to a vendor without a signed Business Associate Agreement (BAA) is a direct violation, full stop.
- Breach blast radius. If your LLM vendor experiences a breach, and your prompts contained raw customer names, emails, and financial identifiers, that breach is now your breach too — and you're the one who has to notify affected users.
- Logging sprawl. Prompts get logged. Not just by the vendor — by your own observability stack, your APM tool, your error tracker. Every hop that touches a raw prompt is another place PII can leak, get cached, or get shipped to a third-party logging service you forgot was in the request path.
The fix isn't "don't use LLMs for anything involving customer data." That throws away most of the value. The fix is to build a boundary: a layer that strips sensitive values out before they cross into third-party infrastructure, and restores them after the response comes back, entirely inside your own trust perimeter.
Core Concept: The Two-Way Masking Architecture
The architecture has four moving parts, and understanding how they fit together matters more than any individual line of code.
Client Request
│
▼
┌─────────────────────┐
│ PII Detector │ ← finds emails, names, cards, SSNs, phone numbers
└─────────┬─────────────┘
▼
┌─────────────────────┐
│ Masking Middleware │ ← replaces PII with deterministic tokens
└─────────┬─────────────┘
▼
┌─────────────────────┐
│ Token Vault (Redis) │ ← stores token → encrypted original value
└─────────┬─────────────┘
▼
Sanitized Prompt
│
▼
┌─────────────┐
│ LLM Provider │ ← only ever sees tokens, never raw PII
└──────┬────────┘
▼
Streamed Response (contains tokens)
│
▼
┌─────────────────────┐
│ Rehydration Stream │ ← resolves tokens back to original values,
└─────────┬─────────────┘ buffering across chunk boundaries
▼
Client Response (fully rehydrated)
Four components, each with one job:
- PII Detector — scans outbound text and finds candidate PII spans (structured, via regex, and unstructured, via a Named Entity Recognition engine).
- Masking Middleware — an Express middleware that walks the request body, replaces each detected span with a token, and lets the request continue toward the LLM call.
- Token Vault — a reversible, encrypted, session-scoped key-value store mapping tokens back to their original values.
- Rehydration Stream — a Node.js
Transformstream sitting between the LLM's streamed response and the client, swapping tokens back for real values on the fly.
The subtlety that makes this hard — and the reason a naive implementation breaks in production — is that the LLM response arrives as a stream of arbitrary-sized chunks, and a token can be split right down the middle across two chunks. We'll deal with that head-on later in this guide.
Why Deterministic Tokenization (Not Static Redaction)
The instinctive first approach is to replace PII with a fixed placeholder: [REDACTED], ***, <PII>. Don't do this. It solves the privacy problem and immediately creates a coherence problem.
Consider a support conversation:
"Hi, this is Sarah Connor. Can you also update the shipping address for my other order under sarah.c@personal-email.com? That's a different account than the one I usually use."
If both mentions of Sarah's identity collapse into [REDACTED], the model has no way to know these two redacted spans refer to the same person, or that they're distinct from a third redacted span three messages later that might be someone else entirely. You lose the referential structure the model needs to reason coherently — and for multi-turn conversations, that structure is often the entire point.
Deterministic tokenization solves this by generating the same token for the same underlying value, every time, within a session:
Sarah Connor→[[PERSON_4f9a2b1c3d]]sarah.c@personal-email.com→[[EMAIL_a83e0f21bc]]
Now the model can reason about "the person referenced by [[PERSON_4f9a2b1c3d]]" consistently across the entire conversation, without ever seeing the actual name. The token is generated via an HMAC keyed with a per-tenant secret ("pepper"), so it's both deterministic and non-reversible without access to that secret — an attacker who intercepts the sanitized prompt can't work backward to the original value.
Building the Detection Layer
PII detection splits into two categories that need different techniques.
Structured PII has a predictable shape: emails, phone numbers, SSNs, credit card numbers, IP addresses. Regex handles these well, especially when paired with a validity check like the Luhn algorithm for card numbers (which filters out random 16-digit strings that aren't actually valid card numbers, cutting down false positives).
Unstructured PII — names, physical addresses, dates of birth mentioned in prose — doesn't have a fixed pattern. You need a Named Entity Recognition (NER) model for this. In production, teams typically reach for something like Microsoft Presidio, AWS Comprehend PII detection, Google Cloud DLP, or a self-hosted transformer-based NER model. For this guide, we'll build the detector so an NER engine is pluggable, and keep the example lightweight.
// detectors/piiDetector.js
const PII_PATTERNS = {
EMAIL: /[a-zA-Z0-9.!#$%&'*+/=?^_`{|}~-]+@[a-zA-Z0-9-]+(?:\.[a-zA-Z0-9-]+)+/g,
PHONE: /\b(?:\+?1[-.\s]?)?\(?\d{3}\)?[-.\s]?\d{3}[-.\s]?\d{4}\b/g,
SSN: /\b\d{3}-\d{2}-\d{4}\b/g,
CREDIT_CARD: /\b(?:\d[ -]*?){13,16}\b/g,
IP_ADDRESS: /\b(?:\d{1,3}\.){3}\d{1,3}\b/g,
};
function luhnCheck(rawNumber) {
const digits = rawNumber.replace(/\D/g, '');
let sum = 0;
let shouldDouble = false;
for (let i = digits.length - 1; i >= 0; i--) {
let digit = parseInt(digits[i], 10);
if (shouldDouble) {
digit *= 2;
if (digit > 9) digit -= 9;
}
sum += digit;
shouldDouble = !shouldDouble;
}
return sum % 10 === 0;
}
class PiiDetector {
constructor({ nerEngine } = {}) {
// Pluggable: Presidio, AWS Comprehend, a local transformer model, etc.
this.nerEngine = nerEngine;
}
detect(text) {
const matches = [];
for (const [type, pattern] of Object.entries(PII_PATTERNS)) {
pattern.lastIndex = 0;
let match;
while ((match = pattern.exec(text)) !== null) {
if (type === 'CREDIT_CARD' && !luhnCheck(match[0])) continue;
matches.push({ type, value: match[0], index: match.index });
}
}
if (this.nerEngine) {
const entities = this.nerEngine.extract(text); // [{ type: 'PERSON', value, index }, ...]
matches.push(...entities);
}
return matches.sort((a, b) => a.index - b.index);
}
}
module.exports = { PiiDetector };
Notice the detector returns matches sorted by index — that ordering matters when we splice tokens back into the original string without corrupting offsets.
Building the Token Vault (The Reversible Store)
This is the component that makes the whole system "two-way." It needs to do three things well: generate deterministic tokens, store the mapping securely, and resolve tokens back to their original values on request — scoped to a session, with an expiry.
// vault/tokenVault.js
const crypto = require('crypto');
class TokenVault {
constructor({ redisClient, pepper, ttlSeconds = 1800 }) {
this.redis = redisClient;
this.pepper = pepper; // per-tenant secret, pulled from a KMS or secrets manager
this.ttl = ttlSeconds;
}
// Deterministic: same (type, value) always yields the same token
// within the lifetime of this pepper.
computeToken(type, value) {
const hmac = crypto.createHmac('sha256', this.pepper);
hmac.update(`${type}:${value}`);
const digest = hmac.digest('hex').slice(0, 10);
return `[[${type}_${digest}]]`;
}
_encrypt(plainText) {
const iv = crypto.randomBytes(12);
const key = crypto.createHash('sha256').update(this.pepper).digest();
const cipher = crypto.createCipheriv('aes-256-gcm', key, iv);
const encrypted = Buffer.concat([cipher.update(plainText, 'utf8'), cipher.final()]);
const tag = cipher.getAuthTag();
return Buffer.concat([iv, tag, encrypted]).toString('base64');
}
_decrypt(payload) {
const raw = Buffer.from(payload, 'base64');
const iv = raw.subarray(0, 12);
const tag = raw.subarray(12, 28);
const encrypted = raw.subarray(28);
const key = crypto.createHash('sha256').update(this.pepper).digest();
const decipher = crypto.createDecipheriv('aes-256-gcm', key, iv);
decipher.setAuthTag(tag);
return Buffer.concat([decipher.update(encrypted), decipher.final()]).toString('utf8');
}
async store(sessionId, token, value) {
const key = `vault:${sessionId}:${token}`;
await this.redis.set(key, this._encrypt(value), { EX: this.ttl });
}
async tokenize(sessionId, type, value) {
const token = this.computeToken(type, value);
await this.store(sessionId, token, value);
return token;
}
async resolve(sessionId, token) {
const key = `vault:${sessionId}:${token}`;
const encrypted = await this.redis.get(key);
if (!encrypted) return null;
return this._decrypt(encrypted);
}
}
module.exports = { TokenVault };
A few decisions worth calling out:
- Values are encrypted at rest with AES-256-GCM, not stored in plaintext in Redis. Even if your Redis instance is compromised, the vault entries are useless without the pepper.
- Tokens are scoped by session ID, not global. This limits blast radius: a leaked token from one conversation can't be replayed to extract data from a different user's session.
- TTL matters. Tokens should expire with the conversation. There's no reason a token minted for a support chat needs to be resolvable a week later.
The Express Middleware: Masking Outbound Requests
With detection and storage in place, the middleware itself is mostly plumbing: recursively walk the request body, mask every string field, and let the request proceed once tokenization is complete.
// middleware/maskPii.js
function maskText(text, tokenizeSync) {
const matches = tokenizeSync.detector.detect(text);
if (matches.length === 0) return text;
let result = '';
let cursor = 0;
for (const match of matches) {
result += text.slice(cursor, match.index);
result += tokenizeSync(match.type, match.value);
cursor = match.index + match.value.length;
}
result += text.slice(cursor);
return result;
}
function walkAndMask(node, ctx) {
if (typeof node === 'string') return maskText(node, ctx.tokenizeSync);
if (Array.isArray(node)) return node.map((item) => walkAndMask(item, ctx));
if (node && typeof node === 'object') {
const out = {};
for (const [key, value] of Object.entries(node)) {
out[key] = walkAndMask(value, ctx);
}
return out;
}
return node;
}
function createMaskingMiddleware({ detector, vault }) {
return async function maskingMiddleware(req, res, next) {
try {
const sessionId = req.headers['x-session-id'] || req.ip;
const pendingWrites = [];
const tokenizeSync = (type, value) => {
// Token generation is a pure HMAC computation (synchronous).
// Only the Redis write needs to happen asynchronously, so we
// queue it and await everything before calling next().
const token = vault.computeToken(type, value);
pendingWrites.push(vault.store(sessionId, token, value));
return token;
};
tokenizeSync.detector = detector;
req.body = walkAndMask(req.body, { tokenizeSync });
await Promise.all(pendingWrites);
req.piiSessionId = sessionId;
next();
} catch (err) {
next(err);
}
};
}
module.exports = { createMaskingMiddleware };
The recursive walk means this middleware works whether your payload is a flat { message: "..." } body or a deeply nested object containing conversation history, metadata, and user profile fields — every string leaf gets scanned.
Re-hydrating Streamed Responses: The Hard Part
Here's where most implementations fall apart. It's easy to write a function that does response.replace(token, originalValue) on a complete string. It's much harder to do that correctly when the response is arriving as a stream of chunks from the LLM provider, because a chunk boundary can land in the middle of a token.
Imagine the model outputs ...contact [[PERSON_4f9a2b1 in one chunk and c3d]] for details... in the next. If you run your replace logic on each chunk independently, you'll never find a complete token in either chunk, and the raw, unresolved token fragments will leak straight through to the user.
The fix is a buffering Transform stream that only emits text it's confident is "safe" — meaning it either contains no partial tokens, or any partial token has been held back to be completed by the next chunk.
// stream/rehydrationStream.js
const { Transform } = require('stream');
const TOKEN_PATTERN = /\[\[[A-Z_]+_[a-f0-9]{10}\]\]/g;
class RehydrationStream extends Transform {
constructor({ sessionId, vault }) {
super();
this.sessionId = sessionId;
this.vault = vault;
this.buffer = '';
}
async _transform(chunk, encoding, callback) {
try {
this.buffer += chunk.toString('utf8');
// Find the last "[[" — if it isn't yet followed by a "]]",
// it might be the start of a token still in transit. Hold it back.
const lastOpen = this.buffer.lastIndexOf('[[');
const hasCloseAfter = lastOpen !== -1 && this.buffer.indexOf(']]', lastOpen) !== -1;
let safeText;
if (lastOpen === -1 || hasCloseAfter) {
safeText = this.buffer;
this.buffer = '';
} else {
safeText = this.buffer.slice(0, lastOpen);
this.buffer = this.buffer.slice(lastOpen);
}
this.push(await this._replaceTokens(safeText));
callback();
} catch (err) {
callback(err);
}
}
async _flush(callback) {
try {
if (this.buffer) this.push(await this._replaceTokens(this.buffer));
callback();
} catch (err) {
callback(err);
}
}
async _replaceTokens(text) {
const matches = [...text.matchAll(TOKEN_PATTERN)];
if (matches.length === 0) return text;
let result = '';
let cursor = 0;
for (const match of matches) {
const original = await this.vault.resolve(this.sessionId, match[0]);
result += text.slice(cursor, match.index);
result += original !== null ? original : match[0]; // fail-safe fallback
cursor = match.index + match[0].length;
}
result += text.slice(cursor);
return result;
}
}
module.exports = { RehydrationStream };
Two things worth noting:
- The fail-safe fallback. If a token can't be resolved (expired TTL, vault miss, or a hallucinated token the model invented), the stream leaves the token text as-is rather than throwing. You don't want a vault cache-miss to crash a live chat response — you want it logged and degraded gracefully.
- Token pattern is fixed-format. Because tokens always look like
[[TYPE_<10 hex chars>]], the boundary-detection logic only needs to track unmatched[[sequences, not guess at arbitrary content.
Wiring It All Together
Here's how the pieces connect in a real Express route, streaming a response back to the client over Server-Sent Events.
// routes/chat.js
const express = require('express');
const { createClient } = require('redis');
const { PiiDetector } = require('../detectors/piiDetector');
const { TokenVault } = require('../vault/tokenVault');
const { createMaskingMiddleware } = require('../middleware/maskPii');
const { RehydrationStream } = require('../stream/rehydrationStream');
const router = express.Router();
const redisClient = createClient({ url: process.env.REDIS_URL });
redisClient.connect();
const detector = new PiiDetector();
const vault = new TokenVault({
redisClient,
pepper: process.env.PII_VAULT_PEPPER,
ttlSeconds: 1800,
});
router.post('/chat', createMaskingMiddleware({ detector, vault }), async (req, res) => {
res.setHeader('Content-Type', 'text/event-stream');
res.setHeader('Cache-Control', 'no-cache');
res.setHeader('Connection', 'keep-alive');
const rehydrator = new RehydrationStream({ sessionId: req.piiSessionId, vault });
rehydrator.on('data', (chunk) => res.write(`data: ${chunk}\n\n`));
rehydrator.on('end', () => res.end());
rehydrator.on('error', (err) => {
console.error('Rehydration error:', err.message); // never log err.chunk / raw PII
res.end();
});
const upstream = await fetch('https://api.anthropic.com/v1/messages', {
method: 'POST',
headers: {
'content-type': 'application/json',
'x-api-key': process.env.ANTHROPIC_API_KEY,
'anthropic-version': '2026-01-01',
},
body: JSON.stringify({
model: 'claude-sonnet-4-6',
max_tokens: 1024,
stream: true,
messages: [{ role: 'user', content: req.body.message }], // already masked
}),
});
const reader = upstream.body.getReader();
const decoder = new TextDecoder();
while (true) {
const { done, value } = await reader.read();
if (done) break;
// In production, parse each SSE "data:" line and extract only the
// text delta (e.g. event.delta.text) before writing — don't feed
// raw provider SSE framing into the rehydrator.
rehydrator.write(decoder.decode(value, { stream: true }));
}
rehydrator.end();
});
module.exports = router;
The req.body.message reaching the LLM call has already passed through createMaskingMiddleware, so by the time it hits fetch, every email, phone number, card number, and name has been swapped for a token. The model never sees the original values — and the response streaming back through RehydrationStream restores them before they ever reach res.write.
Real-World Example: A Support Copilot in Action
Say a user types:
"Hi, this is Sarah Connor. My email is sarah.c@personal-email.com and I need help with order #4482 — the card ending in 4444 was charged twice."
After the masking middleware runs, the LLM receives:
"Hi, this is
[[PERSON_4f9a2b1c3d]]. My email is[[EMAIL_a83e0f21bc]]and I need help with order #4482 — the card ending in[[CREDIT_CARD_09b3c7e1aa]]was charged twice."
The model reasons over this fine — it doesn't need the real name or email to draft a helpful reply, it just needs to reference "the customer" consistently. It might respond:
"Hi
[[PERSON_4f9a2b1c3d]], thanks for flagging the duplicate charge on order #4482. I've located the transaction tied to card ending in[[CREDIT_CARD_09b3c7e1aa]]and started a refund — you'll see it reflected within 3-5 business days. We'll also send a confirmation to[[EMAIL_a83e0f21bc]]."
As this streams back through RehydrationStream, each token is resolved from the vault against req.piiSessionId, and the customer sees:
"Hi Sarah Connor, thanks for flagging the duplicate charge on order #4482. I've located the transaction tied to card ending in 4444 and started a refund — you'll see it reflected within 3-5 business days. We'll also send a confirmation to sarah.c@personal-email.com."
At no point did the raw name, email, or card fragment cross your infrastructure boundary into the LLM provider's request path.
Best Practices for Production-Grade PII Middleware
- Encrypt vault entries at rest. Redis persistence (RDB/AOF snapshots) can end up on disk or in backups — encrypting values with AES-256-GCM means a leaked snapshot isn't a PII leak.
- Scope tokens per session, per tenant. Use a distinct pepper per tenant in multi-tenant systems so one tenant's tokens are cryptographically meaningless in another tenant's context.
- Set aggressive TTLs. Tokens should live only as long as the conversation needs them — 30 to 60 minutes of inactivity is a reasonable default for most support and chat use cases.
- Never log raw request or response bodies. Log post-masking payloads only. If you need to debug, log the sanitized version plus the token map's audit trail (which token was issued, when, for which type) — never the resolved value.
- Combine regex with an NER engine. Regex alone will miss names, street addresses, and other unstructured PII entirely. Pair it with Presidio, Comprehend, or a local model for meaningful recall.
- Fail closed on detector errors. If the detection step throws, don't silently forward the raw, unmasked payload — reject the request and alert.
- Test with adversarial and obfuscated PII. Real users type "john dot smith at gmail dot com" or space out card numbers unpredictably. Build a fuzzing test suite around common obfuscation patterns.
- Get your provider's data retention terms in writing. Masking reduces exposure, but a signed DPA or BAA with your LLM vendor is still non-negotiable for regulated data.
Common Mistakes to Avoid
- Using non-deterministic tokens. If
Sarah Connormaps to a different token every time it appears, you destroy the entity coherence the model needs — and you also can't reliably re-hydrate, since a single value might now match multiple tokens. - Ignoring nested JSON. Support tickets, chat history arrays, and metadata objects often bury PII several levels deep. A shallow, top-level-only masking pass misses most of it.
- Not handling chunk boundaries in streaming. Naively running a regex replace on each raw chunk independently will leak partial or entirely unmatched tokens whenever a token spans two chunks — which happens constantly at scale.
- Storing the vault in plaintext. An unencrypted Redis instance with token-to-PII mappings is just a re-identification database waiting to be breached.
- Assuming regex catches everything. Names, addresses, and dates of birth in free text require NER, not pattern matching.
- Blocking the event loop with synchronous NER. Heavy NLP models running synchronously on the main thread will tank your Express server's throughput under load — offload to a worker thread or a dedicated inference service.
- Forgetting multi-turn consistency. If your vault or tokenization logic isn't scoped correctly to the conversation session, the same user's name might tokenize differently turn to turn, breaking context exactly like static redaction does.
🚀 Pro Tips
- Use format-preserving tokens for structured types. A token like
[[PHONE_a83e0f21bc]]tells the model "this is a phone number" via its type prefix, which measurably improves the model's ability to reason about what kind of entity it's referencing, even without the value. - Precompile and cache your regex patterns. Recompiling regex objects on every request adds unnecessary overhead at high throughput — instantiate them once at module load.
- Version your token format. Embed a version marker (
[[v2:PERSON_...]]) so you can evolve your tokenization scheme without breaking in-flight conversations during a deploy. - Plant canary tokens. Periodically inject a fake, uniquely identifiable token into test prompts and monitor whether it ever appears somewhere it shouldn't (logs, analytics, a different tenant's session) — a cheap way to catch exfiltration bugs early.
- Run NER inference in a worker thread pool. Keep your Express event loop free for I/O; hand CPU-bound NER work to
worker_threadsor a sidecar inference service. - Rate-limit vault writes per session. A malicious or malformed payload with thousands of fake PII matches shouldn't be able to hammer Redis with write amplification.
📌 Key Takeaways
- Piping raw customer data into LLM prompts turns your model provider into an undisclosed data processor — a real GDPR and HIPAA exposure, not a theoretical one.
- Deterministic, HMAC-based surrogate tokens keep conversational context intact while ensuring the LLM never sees actual PII values.
- A session-scoped, encrypted token vault (Redis + AES-256-GCM) is what makes the process safely reversible — this is what separates masking from lossy redaction.
- Streaming re-hydration is the trickiest part of the system: buffer aggressively around potential token boundaries so you never leak a half-formed token to the end user.
- This middleware pattern is a strong layer of defense, but it complements — rather than replaces — proper DPAs, BAAs, and a broader data governance program.
Conclusion
PII masking middleware isn't a nice-to-have bolted onto an LLM feature after a security review flags it — it should be part of the request path from day one, the same way you wouldn't ship an API without input validation. The pattern in this guide — detect, tokenize deterministically, store reversibly, and re-hydrate on the way out, even through a live stream — gives you a real boundary between your customers' sensitive data and the third-party infrastructure processing your prompts, without sacrificing the conversational coherence that makes LLM features useful in the first place.
The implementation here is a solid foundation, not a finished compliance program. Pair it with a real NER engine for unstructured PII, a signed DPA or BAA with your model provider, encrypted transport and storage throughout, and an audit trail your legal and security teams can actually read. Treat this as the technical control that makes the rest of that program enforceable — not a substitute for it.
References
- OWASP Top 10 for Large Language Model Applications — owasp.org/www-project-top-10-for-large-language-model-applications
- General Data Protection Regulation (GDPR) — official text and guidance — gdpr.eu
- U.S. Department of Health & Human Services, HIPAA — hhs.gov/hipaa
- Microsoft Presidio (open-source PII detection and anonymization) — microsoft.github.io/presidio
- NIST Privacy Framework — nist.gov/privacy-framework
- Node.js Stream API documentation — nodejs.org/api/stream.html