Head-based vs tail-based sampling cheatsheet

Q: What sampling rate captures at least one example of a rare error?

With head-based rate r and error frequency f over N requests, the chance of capturing at least one error is 1 minus (1 minus f times r) to the power N. For a 0.05 percent error at a 1 percent head rate over 100000 requests you still capture some, but coverage is thin; tail-based sampling guarantees 100 percent instead.

Reach for head-based sampling to cut volume cheaply and statelessly at the SDK; reach for tail-based sampling when you must guarantee every error and latency outlier survives — and use this page as the quick lookup, not a deep essay.

Context and when it matters

Sampling is where trace cost and trace usefulness collide. The core mechanism of head-based versus tail-based sampling is when the keep/drop decision is made: head-based decides at the first span before the request outcome is known, tail-based decides at the collector after the whole distributed trace has arrived. This cheatsheet compresses the decision, the math, and the config into a single fast reference. For the full narrative on microservice trade-offs, see the tail-based-for-microservices guide.

Decision table

Factor	Head-based	Tail-based
Decision timing	At first span, outcome unknown	At collector, full trace assembled
State required	Stateless	In-memory buffer per trace ID
Latency added	None	`decision_wait` window (30–60 s)
Error capture	Probabilistic — rare errors slip through	Deterministic — 100 % if policy set
Cost driver	CPU-free, pure ratio cut	Collector memory + CPU for policy eval
Config surface	One SDK sampler	Collector `tail_sampling` policy list
Multi-service consistency	Fragile across independent samplers	Robust — one decision per trace
Best for	Bulk noise reduction, edge/serverless	Error-critical, SLO-gated, compliance

Sampling math

Three formulas cover most capacity-planning questions. Keep them close.

Retained trace volume. The steady-state stream you persist is simply the incoming rate scaled by the keep ratio:

retained_traces_per_sec = traffic_traces_per_sec × sampling_rate
retained_spans_per_sec  = retained_traces_per_sec × avg_spans_per_trace

At 8,000 traces/sec, a 5 % head rate, and 12 spans/trace, you persist 8000 × 0.05 × 12 = 4,800 spans/sec. The rate is the only free variable here — traffic and spans-per-trace are properties of your system, so every capacity conversation reduces to “what rate can we afford, and what does it cost us in coverage?”

Storage / cost estimate. Multiply the retained span stream by span size and retention window:

bytes_at_rest = retained_spans_per_sec × avg_span_bytes × retention_seconds

Continuing the example at 1.5 KB/span with a 7-day (604,800 s) window: 4800 × 1536 × 604800 ≈ 4.1 TB before compression. Halving the sampling rate halves this linearly — the single biggest cost lever you control.

Effective error-capture probability. For head-based sampling, the chance of capturing at least one instance of an error that occurs at frequency f, given a keep rate r, over N requests:

P(capture ≥ 1) = 1 − (1 − f·r)^N

A 0.05 % error (f = 0.0005) at a 1 % head rate (r = 0.01) over N = 100,000 requests gives P ≈ 1 − (1 − 5e-6)^100000 ≈ 0.39 — you miss that error roughly 3 times in 5 windows. Tail-based sampling replaces this probability with a guarantee: set a status-code policy and P = 1.

The formula also explains why raising the head rate is a poor substitute for a tail policy. To push a 0.05 % error to a 90 % capture chance over the same window you would need roughly a 40–50 % head rate — an order of magnitude more storage — and you would still miss one incident in ten. Deterministic retention of the small set of traces that matter is almost always cheaper than probabilistically over-sampling everything to catch them by luck.

Config snippets, side by side

Head-based (SDK). Configure the sampler once during OpenTelemetry SDK setup. ParentBased wrapping TraceIdRatioBased keeps a decision consistent for every span in a trace — the root decides, children inherit.

from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.sampling import (
    ParentBased,
    TraceIdRatioBased,
)

# Keep 5% of root traces; children honor the root's decision
# so a single trace is never half-sampled.
sampler = ParentBased(root=TraceIdRatioBased(0.05))

provider = TracerProvider(sampler=sampler)
# register provider globally...

Tail-based (Collector). The decision moves to the tail_sampling processor. Ordered policies evaluate the complete trace; first match wins, so deterministic rules go above the probabilistic fallback.

processors:
  tail_sampling:
    decision_wait: 30s          # hold spans this long before deciding
    num_traces: 60000           # max traces buffered in the decision cache
    policies:
      - name: keep-errors       # deterministic: any ERROR span survives
        type: status_code
        status_code:
          status_codes: [ERROR]
      - name: keep-slow         # capture P99 latency outliers
        type: latency
        latency:
          threshold_ms: 1500
      - name: baseline          # everything else: light probabilistic cut
        type: probabilistic
        probabilistic:
          sampling_percentage: 3

Two structural differences are worth internalizing from these snippets. The head-based sampler runs inside every service process and touches no shared state, which is why it costs nothing and also why it cannot know the request’s outcome. The tail-based processor runs once, at the collector, holding every span for a trace ID until decision_wait elapses — that is what lets it see the error, and also what forces you to size num_traces for peak concurrency. Head-based decisions are consistent only because ParentBased propagates the root’s coin flip; tail-based decisions are consistent by construction because there is exactly one decision per trace.

The two are complementary: a modest head rate at the SDK trims obvious noise before it crosses the network, and the collector makes the final, outcome-aware keep decision. Just keep the head rate above what the tail policy needs, or you will drop errors before the collector ever sees them. A common layered setup is a 30–50 % head rate to halve wire volume, feeding a tail policy that keeps 100 % of errors and slow traces plus a 3–5 % probabilistic baseline — the head stage pays for itself in network and collector CPU while the tail stage guarantees the traces you actually debug with.

Pick this if…

Pick head-based if you want the cheapest possible volume cut, run stateless or memory-constrained collectors (edge, serverless), and can tolerate missing rare failures.
Pick tail-based if you must retain every error and latency outlier, operate async consumers at independent sampling cadences, or have compliance rules mandating capture of specific traces.
Pick both if you have real volume to shed but also hard retention guarantees — light head rate feeding an outcome-aware tail policy.

Common pitfalls

Using a bare TraceIdRatioBased root instead of ParentBased. Without ParentBased, downstream services re-roll the dice and you get half-sampled traces with missing hops. Always wrap the ratio sampler in ParentBased.
Reading the error-capture formula as “good enough.” A 39 % capture chance feels acceptable until the one production incident you needed lands in the missing 61 %. If an error class must be debuggable, only a deterministic tail policy guarantees it.
Forgetting the tail buffer in cost estimates. The storage formula covers bytes at rest; tail-based sampling adds a transient in-memory buffer (traces/sec × decision_wait × span_bytes) that must be provisioned on the collector or spikes trigger silent trace eviction.

FAQ

What sampling rate captures at least one example of a rare error?

With head-based rate r and error frequency f over N requests, the chance of capturing at least one error is 1 − (1 − f·r)^N. For a 0.05 % error at a 1 % head rate over 100,000 requests you still capture some, but coverage is thin; tail-based sampling guarantees 100 % instead.

Can I run head-based and tail-based sampling together?

Yes, and it is common. Apply a light head-based rate at the SDK to shed obvious noise cheaply, then let the collector tail_sampling processor make the final keep decision on errors and latency. Keep the head rate high enough that you do not drop traces the tail policy would have kept.

How do I estimate storage from a sampling rate?

Retained spans per second equals traces per second times sampling rate times average spans per trace. Multiply by average span bytes and by your retention window in seconds to get bytes at rest. Tail-based sampling adds a transient in-memory buffer on top of that at-rest figure.

When to Use Tail-Based Sampling for Microservices — the deep-dive on collector buffering and async edge cases.
Choosing Between Head-Based and Tail-Based Sampling — the full strategy comparison behind this reference.
Trace Storage Backend Comparison: Jaeger vs Tempo — where retained traces land and how retention interacts with sampling.

↑ Back to Choosing Between Head-Based and Tail-Based Sampling