intermediatecontext-propagation

What is baggage in OpenTelemetry and how does it differ from span attributes?

⚡

Quick Answer

Baggage is a mechanism for propagating key-value pairs across service boundaries along with the trace context, making them available to all downstream services. Unlike span attributes, which are local to a single span and not transmitted to downstream services, baggage travels with every outgoing request, allowing upstream context like tenant ID or feature flags to be accessed by any service in the request chain.

Detailed Answer

Think of baggage like a shipping label attached to a package as it moves through a delivery network. Every warehouse and sorting facility along the route can read the label to make decisions: route this package via express, handle with care, deliver to the third floor. Span attributes, by contrast, are like internal notes that each warehouse writes in its own logbook. The next warehouse never sees those notes. Baggage propagates with the request across service boundaries; span attributes stay local to the service that created them.

Baggage in OpenTelemetry is part of the W3C Baggage specification and is propagated via HTTP headers (the baggage header) or gRPC metadata alongside the trace context (traceparent and tracestate headers). When the payments-api sets a baggage entry like tenant.id=acme-corp, every downstream service that handles the same request, including order-processing-service, inventory-sync, and user-auth-service, can read that value without the payments-api needing to explicitly pass it as a parameter. This is particularly powerful for cross-cutting concerns that affect multiple services but originate at the edge, such as tenant identification in multi-tenant systems, A/B test cohort assignment, feature flag values, or request priority levels.

Internally, baggage is stored in the OpenTelemetry context and injected into outgoing requests by the configured propagator. The SDK provides the W3CBaggagePropagator, which serializes baggage entries into the baggage HTTP header as a comma-separated list of key=value pairs. When a downstream service receives the request, the propagator extracts the baggage from the header and makes it available via the baggage API. Importantly, baggage entries are not automatically added to spans. This is a deliberate design decision because baggage can contain sensitive data, and automatically attaching it to every span would export that data to your tracing backend. If you want baggage values to appear as span attributes, you must explicitly copy them, either in application code or using the Collector's baggage processor.

In production, baggage is invaluable for multi-tenant observability. When the checkout-service receives a request, it sets tenant.id in baggage. Every downstream service can then use that tenant ID to add it as a span attribute, include it in log messages, or use it as a metric dimension, all without modifying the API contracts between services. Another common use case is request prioritization: the API gateway sets priority=high in baggage for premium customers, and the order-processing-service reads this to route the request to a dedicated processing queue. Feature flags are another natural fit. If the frontend assigns a user to the new-checkout-flow experiment, it sets experiment.checkout=variant-b in baggage, and every downstream service can record which variant was active, enabling end-to-end A/B test analysis across the entire request path.

The critical gotcha with baggage is that it is propagated to every downstream service, including third-party APIs and external vendors. Sensitive data like user emails, account numbers, or authentication tokens must never be placed in baggage because any service in the chain, including services you do not control, can read and log the values. The baggage header is transmitted in plaintext in HTTP requests, making it visible in network logs and proxy access logs. Another pitfall is baggage size. Each baggage entry adds bytes to every HTTP header on every request. If you add large values or too many entries, you can exceed HTTP header size limits (typically 8 KB in nginx, 16 KB in many load balancers), causing requests to fail with 431 Request Header Fields Too Large errors. Keep baggage entries small, use short keys, and limit the total number of entries to under 10.

# Using OpenTelemetry Baggage in Python for multi-tenant context from opentelemetry import baggage, trace, context # Import the baggage propagator for cross-service propagation from opentelemetry.propagators.composite import CompositePropagator # Import W3C propagators for trace context and baggage from opentelemetry.propagators.textmap import DefaultTextMapPropagator # Import the baggage API for setting and getting entries from opentelemetry.baggage import set_baggage, get_baggage, get_all # Import requests library for HTTP calls import requests # --- Setting baggage at the API gateway / edge service --- # This runs in the checkout-service when a request arrives # Get the current tracer for creating spans tracer = trace.get_tracer("checkout-service") # Set tenant ID in baggage (propagated to all downstream services) ctx = set_baggage("tenant.id", "acme-corp") # Tenant identifier # Set request priority for downstream routing decisions ctx = set_baggage("request.priority", "high", ctx) # Priority level # Set experiment variant for A/B test tracking ctx = set_baggage("experiment.checkout", "variant-b", ctx) # Feature flag # Start a span and make downstream call with baggage context with trace.get_tracer("checkout-service").start_as_current_span( "process-checkout", context=ctx # Use context with baggage ) as span: # Explicitly copy baggage to span attributes (not automatic) tenant_id = get_baggage("tenant.id", ctx) # Read baggage value span.set_attribute("tenant.id", tenant_id) # Add as span attribute # Make HTTP call to order-processing-service # Baggage is automatically injected into HTTP headers by the propagator response = requests.post( # Downstream HTTP call "http://order-processing-service:8080/api/v1/orders", # Target service json={"items": [{"sku": "WIDGET-001", "qty": 2}]}, # Request body ) # --- Reading baggage in a downstream service (order-processing-service) --- # The propagator automatically extracts baggage from incoming HTTP headers # Framework middleware (Flask, FastAPI) handles extraction from flask import Flask, request # Import OpenTelemetry Flask instrumentation from opentelemetry.instrumentation.flask import FlaskInstrumentor app = Flask(__name__) # Create Flask app FlaskInstrumentor().instrument_app(app) # Auto-instrument Flask @app.route("/api/v1/orders", methods=["POST"]) # Order creation endpoint def create_order(): # Read baggage set by the upstream checkout-service tenant_id = get_baggage("tenant.id") # Get tenant from baggage priority = get_baggage("request.priority") # Get priority from baggage experiment = get_baggage("experiment.checkout") # Get experiment variant # Add baggage values as span attributes for this service's span current_span = trace.get_current_span() # Get the active span current_span.set_attribute("tenant.id", tenant_id) # Attach to span current_span.set_attribute("request.priority", priority) # Attach priority # Use priority for routing decisions if priority == "high": # Check priority value # Route to dedicated high-priority processing queue process_high_priority_order(request.json) # Priority queue handler else: process_standard_order(request.json) # Standard queue handler return {"status": "created"}, 201 # Return response # --- HTTP Header format (W3C Baggage) --- # Request from checkout-service to order-processing-service: # baggage: tenant.id=acme-corp,request.priority=high,experiment.checkout=variant-b

What is baggage in OpenTelemetry and how does it differ from span attributes?

More OpenTelemetry questions