OpenTelemetry Collector — Vendor-Neutral Telemetry Pipeline
The OpenTelemetry Collector is the CNCF-graduated pipeline for receiving, processing, and exporting metrics, logs, and traces across any observability backend, replacing per-vendor agents with one portable binary.
What it is
The OpenTelemetry Collector is the CNCF-graduated pipeline for receiving, processing, and exporting metrics, logs, and traces across any observability backend. It replaces per-vendor agents (Datadog agent, Splunk forwarder, New Relic agent) with one portable binary that routes telemetry data to any destination.
This tool targets platform engineering and SRE teams standardizing their observability stack. Instead of managing multiple vendor agents with different config formats and update schedules, you deploy one Collector that supports all of them.
Why it saves time or tokens
Vendor lock-in in observability means rewriting instrumentation when you switch providers. The Collector decouples collection from export, so switching from Datadog to Grafana Cloud means changing one exporter config, not re-instrumenting your applications. This portability saves engineering time. When generating observability configs with AI, targeting the Collector's single YAML schema is simpler than learning multiple vendor formats.
How to use
- Install the Collector using the official binary, Docker image, or Helm chart
- Define a config.yaml with receivers (data in), processors (transform), and exporters (data out)
- Run:
otelcol-contrib --config config.yaml
Example
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
prometheus:
config:
scrape_configs:
- job_name: 'app'
scrape_interval: 15s
static_configs:
- targets: ['localhost:8080']
processors:
memory_limiter:
check_interval: 1s
limit_mib: 512
batch:
timeout: 5s
exporters:
otlphttp:
endpoint: https://otel.example.com:4318
service:
pipelines:
metrics:
receivers: [otlp, prometheus]
processors: [memory_limiter, batch]
exporters: [otlphttp]
| Deployment Mode | Use Case |
|---|---|
| Agent (DaemonSet) | Per-node log/metric collection |
| Gateway (Deployment) | Centralized processing and routing |
| Sidecar | Per-pod collection in Kubernetes |
| Standalone | Single-server monitoring |
Related on TokRepo
- AI tools for monitoring — observability and monitoring tools on TokRepo
- AI tools for devops — infrastructure and platform engineering tools
Common pitfalls
- Use the contrib distribution, not core; core lacks most vendor-specific exporters that production deployments need
- The memory_limiter processor is essential in production; without it, a burst of telemetry data can OOM the Collector
- Pipeline ordering matters: receivers feed processors in order, and a misconfigured pipeline silently drops data
Frequently Asked Questions
The core distribution includes only the most essential, vendor-neutral components maintained by the OpenTelemetry project. The contrib distribution adds hundreds of additional receivers, processors, and exporters for specific vendors and protocols. Most production deployments use contrib.
The SDKs instrument your application code to generate telemetry. The Collector receives, processes, and routes that telemetry to backends. SDKs can export directly to backends, but the Collector adds buffering, batching, retry logic, and the ability to fan-out to multiple destinations.
Yes. The contrib distribution includes a Datadog exporter that sends metrics and traces in Datadog's native format. You instrument with OpenTelemetry SDKs, send to the Collector, and the Collector exports to Datadog. This lets you switch providers later without changing application code.
Use the official Helm chart or the OpenTelemetry Kubernetes Operator. Deploy as a DaemonSet for per-node collection (logs, node metrics) or as a Deployment for centralized gateway processing. The Kubernetes attributes processor enriches telemetry with pod and namespace metadata automatically.
Yes. The Collector receives logs via OTLP, syslog, filelog (tail files), and other receivers. It can parse, transform, and route logs to backends like Elasticsearch, Loki, or cloud logging services. Log support in OpenTelemetry is stable as of recent releases.
Citations (3)
- OpenTelemetry GitHub— OpenTelemetry is a CNCF graduated project
- OTel Docs— Collector architecture and configuration
- CNCF— CNCF graduated project maturity standards
Related on TokRepo
Discussion
Related Assets
HumHub — Open-Source Enterprise Social Network
A flexible, open-source social networking platform built on Yii2 for creating private communities, intranets, and collaboration spaces within organizations.
Dolibarr — Open-Source ERP & CRM for Business Management
A modular open-source ERP and CRM application written in PHP for managing contacts, invoices, orders, inventory, accounting, and more from a single web interface.
PrestaShop — Open-Source PHP E-Commerce Platform
A widely adopted open-source e-commerce platform written in PHP with a rich module marketplace, multi-language support, and a strong European user base.