Propagating Baggage Through gRPC Metadata

Inject the active OpenTelemetry context into a metadata carrier in a client interceptor and extract it in a matching server interceptor — gRPC metadata is the transport for both traceparent and the W3C baggage header, but only if Baggage{} (Go) or W3CBaggagePropagator (Python) is registered in the global propagator.

Context & When It Matters

gRPC calls carry metadata: a multimap of string keys to values sent in the HTTP/2 headers frame before the message body. That metadata is where propagated context lives on a gRPC hop, exactly as HTTP headers do on a REST hop. The OpenTelemetry gRPC instrumentation wires traceparent into metadata automatically, which makes traces render across services — and that visible success is precisely what hides the baggage gap. Baggage is a separate header written by a separate propagator, and if it is not in your propagator chain, the routing keys you set never leave the client. This page is the gRPC counterpart to injecting baggage into Kafka message headers; the combined guide covers both transports together.

The reason to invest in an interceptor rather than setting metadata by hand at each call site is uniformity. Baggage is only useful if every RPC in a chain carries it; a single hand-instrumented client that forgets to inject breaks the chain from that point on. An interceptor registered on the channel injects on every outgoing call automatically, including calls made by generated stubs you did not write. The same argument applies on the server: a server interceptor extracts and activates context for every handler before your business code runs, so no handler can accidentally observe an empty context. This is the gRPC analogue of the middleware pattern used for HTTP ingress, and it is why the interceptor — not the individual RPC — is the right place to own propagation.

Minimal Working Example First

The client interceptor injects; the server interceptor extracts. Here is the smallest correct pair in Go, using the standard otel propagator.

package main

import (
	"context"

	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/propagation"
	"google.golang.org/grpc/metadata"
)

// Register once at startup: traceparent AND baggage both propagate only
// because propagation.Baggage{} is included in the composite.
func init() {
	otel.SetTextMapPropagator(propagation.NewCompositeTextMapPropagator(
		propagation.TraceContext{},
		propagation.Baggage{},
	))
}

// metadataCarrier adapts gRPC metadata.MD to the OTel TextMapCarrier interface.
type metadataCarrier struct{ md metadata.MD }

func (c metadataCarrier) Get(k string) string {
	if v := c.md.Get(k); len(v) > 0 {
		return v[0]
	}
	return ""
}
func (c metadataCarrier) Set(k, v string) { c.md.Set(k, v) }
func (c metadataCarrier) Keys() []string {
	keys := make([]string, 0, len(c.md))
	for k := range c.md {
		keys = append(keys, k)
	}
	return keys
}

The metadataCarrier is the adapter that lets the propagator read and write gRPC metadata; both interceptors below use it.

Implementation

Go client interceptor — inject

import (
	"context"
	"go.opentelemetry.io/otel"
	"google.golang.org/grpc"
	"google.golang.org/grpc/metadata"
)

func BaggageUnaryClientInterceptor() grpc.UnaryClientInterceptor {
	return func(ctx context.Context, method string, req, reply any,
		cc *grpc.ClientConn, invoker grpc.UnaryInvoker,
		opts ...grpc.CallOption) error {

		// Copy any existing outgoing metadata so we don't drop it.
		md, ok := metadata.FromOutgoingContext(ctx)
		if !ok {
			md = metadata.MD{}
		} else {
			md = md.Copy()
		}
		// Inject reads baggage + traceparent FROM ctx and writes them INTO md
		// under the plain string keys "baggage" and "traceparent".
		otel.GetTextMapPropagator().Inject(ctx, metadataCarrier{md})
		ctx = metadata.NewOutgoingContext(ctx, md)
		return invoker(ctx, method, req, reply, cc, opts...)
	}
}

Go server interceptor — extract

func BaggageUnaryServerInterceptor() grpc.UnaryServerInterceptor {
	return func(ctx context.Context, req any,
		info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (any, error) {

		md, ok := metadata.FromIncomingContext(ctx)
		if !ok {
			md = metadata.MD{}
		}
		// Extract rebuilds baggage + trace context from incoming metadata and
		// returns a ctx that carries them. Every downstream call inherits it.
		ctx = otel.GetTextMapPropagator().Extract(ctx, metadataCarrier{md})
		return handler(ctx, req) // baggage.FromContext(ctx) now works in the handler
	}
}

Wire both into the channel and server:

conn, _ := grpc.NewClient(target,
	grpc.WithUnaryInterceptor(BaggageUnaryClientInterceptor()))

srv := grpc.NewServer(
	grpc.UnaryInterceptor(BaggageUnaryServerInterceptor()))

For a Python server the shape is identical — read handler_call_details.invocation_metadata into a dict carrier, call extract(), and attach() the context so the handler sees the baggage. This mirrors the service mesh propagation pattern, but the interceptor lives inside the process rather than in a sidecar:

import grpc
from opentelemetry import context as otel_context
from opentelemetry.propagate import extract

class BaggageServerInterceptor(grpc.ServerInterceptor):
    def intercept_service(self, continuation, handler_call_details):
        # invocation_metadata is a tuple of (key, value) string pairs.
        carrier = dict(handler_call_details.invocation_metadata or ())
        ctx = extract(carrier)               # rebuild baggage + trace context
        handler = continuation(handler_call_details)

        def wrap(request, servicer_context):
            token = otel_context.attach(ctx) # baggage live for this handler
            try:
                return handler.unary_unary(request, servicer_context)
            finally:
                otel_context.detach(token)   # avoid leaking across pooled workers

        return grpc.unary_unary_rpc_method_handler(
            wrap,
            request_deserializer=handler.request_deserializer,
            response_serializer=handler.response_serializer,
        )

The attach/detach pair matters because a gRPC server dispatches handlers on a thread pool; without detaching, one request’s baggage can leak into the next task that reuses the worker, exactly as it would across async boundaries. In Go the equivalent hazard does not exist in the same form — context.Context is passed explicitly down the call tree rather than stored in thread-local state — which is why the Go server interceptor simply returns handler(ctx, req) with the extracted context and never has to “detach.” The Python interceptor must manage the active-context slot manually because Python’s OpenTelemetry context is ambient, tied to the executing worker, not threaded through arguments. Recognising which model your runtime uses tells you whether you owe a detach at all: explicit-context runtimes (Go) do not, ambient-context runtimes (Python, Node.js) always do.

For grpc.aio (Python’s asyncio gRPC), the server interceptor extends grpc.aio.ServerInterceptor and the handler is awaited, but the attach/detach discipline is unchanged — the context is scoped around the awaited handler call. Because asyncio multiplexes many handlers on one thread, skipping detach there corrupts context for whatever coroutine the event loop resumes next, which is even harder to debug than the thread-pool case.

Verification & Decision Checklist

Reach for a metadata interceptor whenever a gRPC service downstream needs a tenant, cohort, or region signal to make a routing or authorization decision without re-deriving it. If the value is only wanted for querying later, prefer a span attribute set in the handler — it avoids paying header bytes on every RPC.

A note on trust: metadata arriving on a server is only as trustworthy as the caller. A baggage entry claiming tenant=acme is an assertion, not proof, so a server that makes authorization decisions on baggage must re-validate the claim against the request’s authenticated identity rather than trusting the header verbatim. Use baggage to route and to enrich telemetry freely, but treat any security-relevant value as advisory until it is checked against a token the server verified itself. This keeps a compromised or buggy upstream from escalating privileges simply by setting a metadata key.

Common Pitfalls

  • Reserved grpc- metadata keys. Keys beginning with grpc- belong to the framework and may be silently stripped or rejected. Never rename baggage to a grpc- prefix; keep the propagator-supplied baggage key.
  • The -bin binary suffix. gRPC base64-encodes any key ending in -bin. The W3C baggage value is printable ASCII and must use a string key. Naming it baggage-bin forces encoding and breaks interop with standard OpenTelemetry gRPC instrumentation that expects the string key.
  • Streaming RPC context reuse. Metadata is transmitted once at stream establishment. Baggage captured then is frozen for the whole stream, so per-message changes on a long-lived bidirectional stream do not propagate. Use unary calls, or carry the varying value in the message body, when baggage must change mid-stream.

FAQ

Can I name my gRPC baggage metadata key anything I want?

No. Keys starting with grpc- are reserved by the framework and may be stripped, and keys ending in -bin are treated as binary and base64-encoded. Baggage must ride on the plain string key named baggage that OpenTelemetry’s propagator writes, so that standard instrumentation on the other side recognises it.

Does baggage propagate correctly on gRPC streaming RPCs?

Metadata is sent once when the stream opens, so the baggage captured at that moment is fixed for the life of the stream. Per-message baggage changes on a long-lived stream are not transmitted. If baggage must vary per message, use unary calls or embed the varying value in the message body instead of relying on metadata.

↑ Back to Propagating Baggage Across Kafka and gRPC