Avoiding 431 Request Header Too Large Errors

A 431 from oversized baggage is fixed by shrinking the header at its source with a middleware size guard, not by enlarging proxy buffers — trim or reject the baggage before the propagator injects it, and the request fits through every hop.

Context and when it matters

431 Request Header Fields Too Large surfaces when the combined size of a request’s headers exceeds a proxy’s header buffer. In a traced system the growing headers are almost always OpenTelemetry Baggage and tracestate, both of which accumulate a list member at each service hop. A request that stays small on a two-service path breaches the buffer on a seven-service path, so the error reads as intermittent when it is really a function of how deep into the call graph a request travels. It matters because a 431 is fatal to that request: the proxy drops it before your application ever runs, so there is no span, no log line, and no trace — the failing hop simply severs the distributed trace and returns a 4xx the client did not expect. Left unaddressed, the failed requests trigger client retries that resend the same oversized header, turning one bad request into a burst against the ingress tier.

Diagnosing it is quick once you know where to look. Because the application never runs, the evidence lives in the proxy access log, not your service logs: grep the ingress for status=431 and correlate the failing requests by path depth or user segment — they will cluster on the routes that fan out furthest. Confirm the cause by replaying one failing request with curl -sv and watching the response line; then re-send it with the baggage header removed. If the 431 disappears without baggage, the header stack is the culprit and the guard below is the fix. If it persists, the bloat is coming from cookies or an oversized Authorization token instead, and the same size-budget thinking applies — just to a different header.

The size guard: minimal working code first

The fix that actually holds is a guard that caps the serialized baggage size before injection. Here it is in both Node.js and Python — drop it into your outbound request path and an oversized baggage set is trimmed to an allowlist rather than sent to overflow a downstream buffer.

// baggageGuard.js — reject/trim oversized baggage before it is injected
const { propagation, context } = require('@opentelemetry/api');

const MAX_BAGGAGE_BYTES = 4096;                 // stay under the smallest proxy buffer
const KEEP = new Set(['tenant.id', 'region', 'cache.key']); // routing-critical keys

function enforceBaggageBudget(ctx) {
  const carrier = {};
  propagation.inject(ctx, carrier);             // serialize exactly as the wire sees it
  const size = Buffer.byteLength(carrier.baggage || '', 'utf8');
  if (size <= MAX_BAGGAGE_BYTES) return ctx;    // within budget: pass through untouched

  const current = propagation.getBaggage(ctx);
  let trimmed = propagation.createBaggage();    // start empty, keep only allowlisted entries
  for (const [key, entry] of current.getAllEntries()) {
    if (KEEP.has(key)) trimmed = trimmed.setEntry(key, entry);
  }
  return propagation.setBaggage(ctx, trimmed);  // essential members survive, rest dropped
}

module.exports = { enforceBaggageBudget };

# baggage_guard.py — Python equivalent, same allowlist discipline
from opentelemetry import baggage, context
from opentelemetry.baggage.propagation import W3CBaggagePropagator

MAX_BAGGAGE_BYTES = 4096
KEEP = {"tenant.id", "region", "cache.key"}
_propagator = W3CBaggagePropagator()

def enforce_baggage_budget(ctx):
    carrier = {}
    _propagator.inject(carrier, context=ctx)             # build the outbound header
    if len(carrier.get("baggage", "").encode("utf-8")) <= MAX_BAGGAGE_BYTES:
        return ctx                                        # under budget, no change
    trimmed = context.Context()                           # empty context to rebuild into
    for key, value in baggage.get_all(ctx).items():
        if key in KEEP:
            trimmed = baggage.set_baggage(key, value, context=trimmed)
    return trimmed

The guard runs on egress so the service that grew the header is the one that trims it. The allowlist encodes a policy decision: when bytes must be shed, keep the members that downstream routing depends on and drop the rest.

Two details make or break this guard. First, it measures the serialized header, not the member count or the raw value lengths, because percent-encoding inflates the wire size — a value with a comma, space, or non-ASCII character expands under URL encoding, so counting characters underestimates the bytes that actually hit the buffer. Injecting into a throwaway carrier and measuring Buffer.byteLength(..., 'utf8') (or .encode("utf-8") in Python) gives you the true number. Second, the ceiling is deliberately set below the smallest buffer on the path rather than at the spec’s 8192, leaving room for traceparent, tracestate, cookies, and auth headers that share the same buffer. A guard tuned to 8192 still lets the combined header block overflow an 8 KB buffer the moment anything else rides along.

Implementation

Wire the guard into the outbound path and pair it with an offload for values that are large by nature. The comments map each line to the header-size mechanic it controls.

// outboundClient.js — apply the guard on every downstream call
const { context, propagation, trace } = require('@opentelemetry/api');
const { enforceBaggageBudget } = require('./baggageGuard');
const redis = require('./redisClient');

async function callDownstream(url, ctx) {
  // 1. Offload any oversized value: store it once, propagate a 16-char key instead
  const bigValue = propagation.getBaggage(ctx)?.getEntry('permissions')?.value;
  if (bigValue && bigValue.length > 256) {
    const key = require('crypto').randomBytes(8).toString('hex'); // short lookup token
    await redis.setex(`bag:${key}`, 300, bigValue);               // 5-min TTL in shared store
    let b = propagation.getBaggage(ctx).deleteEntry('permissions')
              .setEntry('cache.key', { value: key });             // swap payload for key
    ctx = propagation.setBaggage(ctx, b);
  }

  // 2. Enforce the byte ceiling AFTER offload, so the header cannot exceed budget
  ctx = enforceBaggageBudget(ctx);

  // 3. Inject the now-bounded context into real request headers and send
  const headers = {};
  propagation.inject(ctx, headers);            // writes baggage + traceparent + tracestate
  return fetch(url, { headers });
}

Everything the guard does happens before inject() writes the wire headers, so by the time the request leaves the process the baggage is guaranteed to fit the budget. Services that consume the cache.key fetch the payload from the shared store on demand, keeping the header small across the whole path — the same technique that keeps tenant context propagation cheap when it fans out.

Order matters in that function: the offload runs before the byte-ceiling check, so a genuinely large value is swapped for a 16-byte key first and the guard rarely has to trim anything. If you reverse the order, the guard trims the large value away as a low-priority member and the downstream service that needed it fails a lookup instead of getting a cache key — a silent correctness bug masquerading as a size fix. The TTL on the cached payload should comfortably exceed your longest request’s end-to-end latency, or a slow tail request will find its key expired. And because the guard is on the outbound path, it must sit in whatever spot your framework uses to build downstream requests — an HTTP client interceptor, a gRPC outbound interceptor, or a message-producer wrapper — not in inbound middleware, which never sees the members your own service adds.

Decision criteria and verification

Use the guard as the primary control and the proxy buffer as headroom above it. Set each buffer comfortably larger than your MAX_BAGGAGE_BYTES plus the worst-case size of cookies, auth, and tracestate on the same request. The table below lists the exact knob per proxy and a starting value that sits above a 4 KB baggage budget.

Proxy	Buffer directive	Default	Recommended	Note
NGINX	`large_client_header_buffers`	`4 8k`	`4 16k`	Each buffer holds one header line; raise count and size together.
Envoy	`max_request_headers_kb`	`60`	`96`	Caps the total decoded header list; returns 431 on overflow.
HAProxy	`tune.bufsize` + `tune.http.maxhdr`	`16384` / `101`	`32768` / `200`	`bufsize` bounds the whole block; `maxhdr` bounds header count.

Verify the fix against a realistic worst case rather than a single small request:

Synthesize an oversized header and send it through the full ingress path: curl -sv -H "baggage: $(python3 -c "print(','.join(f'k{i}=' + 'x'*200 for i in range(40)))")" https://gateway/api/checkout and confirm the response is 200, not 431. A pass proves the guard trimmed before overflow.
Log the received baggage header at the leaf service and confirm only allowlisted keys survived — proof the trim policy, not truncation, shaped the header.
Check the trace in Jaeger or Tempo still spans every service; an unbroken end-to-end trace means the header fit through all buffers on the path.
Watch the ingress 431 rate for a full traffic cycle after deploying. A guard that trims correctly drives the rate to zero even during peak fan-out; a residual 431 rate means either a proxy on the path has a buffer below your budget or a service is adding members after the guard runs. Both are configuration errors you can pinpoint by comparing the guard’s ceiling against the buffer table above and by logging the outbound header size at each egress point.

Common pitfalls

Raising buffers instead of shrinking baggage. A bigger buffer stops today’s 431 and hides the real problem: unbounded per-hop growth. The header keeps expanding until it breaches the new ceiling too, and every raised buffer costs per-connection memory across the whole fleet. Cap the header at the source, then raise buffers only for headroom.
Budgeting per-hop instead of accumulated. Baggage and tracestate grow at every service, so a guard sized for one service’s contribution still overflows on a deep path. Measure the accumulated worst case on your longest route, as described in baggage size limits and header constraints, and size the budget for that.
Ignoring retry storms. When a 431 fails a request, default client retries resend the identical oversized header, multiplying load on the already-strained ingress. Do not tune retries to mask the error; fix the header size so the request succeeds on the first attempt and the retries never fire.
Trusting truncation to save you. Some load balancers truncate an over-long header rather than returning a clean 431. That is worse, not better: a baggage list cut off mid-member either fails to parse downstream or, in the pathological case, parses into a corrupted key=val pair that quietly changes a routing decision. Never treat truncation as a soft limit — enforce the ceiling yourself before the header reaches a proxy that might silently mangle it.

FAQ

Which header actually triggers the 431?

The 431 is triggered by the total size of all request headers exceeding the proxy buffer, not by any single named header. Baggage and tracestate are the usual culprits because they grow per hop, but cookies, authorization tokens, and forwarded headers all count toward the same buffer. Measure the whole header block, not just baggage, when you diagnose.

Why do I only get 431 for some requests?

The header stack grows with path depth. Requests that traverse more services accumulate more baggage and tracestate members, so only the longest paths cross the buffer ceiling. The error looks intermittent but it correlates directly with how many hops a given request makes through the call graph.

Is raising the proxy buffer enough to fix 431?

It stops the immediate error but not the underlying growth. Baggage still accumulates per hop, so a larger buffer only delays the next 431, and it raises per-connection memory fleet-wide. Add a middleware size guard so the header cannot grow unbounded, then raise the buffer for headroom above the enforced budget.

Baggage Size Limits and Header Constraints — the full budgeting strategy and the spec, HPACK, and buffer limits behind this error.
Baggage vs Span Attributes: When to Use What — deciding what belongs in the header at all so it never grows unnecessarily.
Propagating Baggage Across Kafka and gRPC — applying the same size discipline to non-HTTP metadata frames.

↑ Back to Baggage Size Limits and Header Constraints