What is OpenTelemetry and why was it created?

Think of OpenTelemetry as a universal adapter for observability. Before it existed, every monitoring vendor had its own proprietary agent and data format, so switching from one tool to another meant re-instrumenting your entire codebase. OpenTelemetry removes that lock-in by providing a single, vendor-neutral way to generate and export telemetry data.

OpenTelemetry (often abbreviated as OTel) is a Cloud Native Computing Foundation (CNCF) incubating project that provides a comprehensive observability framework. It defines a standard for how telemetry data (traces, metrics, and logs) is generated, collected, processed, and exported. The project delivers language-specific SDKs for Go, Java, Python, JavaScript, .NET, Ruby, and others, along with the OpenTelemetry Collector, a standalone binary that receives, processes, and forwards telemetry data to any supported backend. In a production environment running services like payments-api, user-auth-service, and order-processing-service, OpenTelemetry instruments each service identically regardless of whether you send data to Jaeger, Prometheus, Datadog, or Grafana Cloud.

The history behind OpenTelemetry is important context for interviews. Two earlier projects tackled distributed tracing independently: OpenTracing (a vendor-neutral tracing API) and OpenCensus (a Google-led project covering tracing and metrics). Both gained significant adoption, but having two competing standards fragmented the ecosystem. Library authors had to choose which one to support, and users sometimes needed both in the same application. In 2019, the two projects merged to form OpenTelemetry, combining the best ideas from each. OpenTracing contributed its flexible span-based tracing model, while OpenCensus brought metrics support and the Collector architecture. This merger gave the community a single project to rally around.

In production environments, OpenTelemetry solves a critical problem for platform teams managing microservices. Consider a company running checkout-service, inventory-sync, and payments-api across Kubernetes clusters. Without a standard, each team might instrument differently: one uses the Datadog agent, another uses Prometheus client libraries, and a third uses custom logging. OpenTelemetry unifies this by providing a single instrumentation layer that every service adopts. The data can then be routed to any combination of backends through the Collector, meaning the observability backend becomes a deployment decision rather than a code decision.

A key architectural principle is the separation of instrumentation from export. Your application code uses OpenTelemetry APIs to create spans, record metrics, and emit logs. The SDK handles sampling, batching, and exporting. If you later switch from Jaeger to Tempo for trace storage, you change a configuration file in the Collector rather than modifying application code. This decoupling is why platform teams at companies running services like user-auth-service and order-processing-service adopt OpenTelemetry as their internal standard, even when individual teams might prefer different analysis tools.