OpenTelemetry for Tracing and Metrics — Operational Guide Without Vendor Lock-in • Meteora Web Agency

Your application is in production. Errors increase but logs don't tell you why. Aggregate metrics show a CPU spike but not which request caused it. You need distributed tracing. The problem? Every vendor (Datadog, New Relic, AWS X-Ray) has its own SDK and protocol. Switch providers tomorrow and you rewrite everything. At Meteora Web, we've seen this dozens of times: clients with five tools for logs, metrics, and traces, no coherent data. OpenTelemetry (OTel) is the open standard that unifies them all: one SDK to trace, shared semantics, and freedom to export anywhere. This guide shows you how to use it for real, with working code, avoiding vendor lock-in.

Why did OpenTelemetry become the standard for tracing and metrics?

OpenTelemetry was born from the merger of OpenTracing and OpenCensus, backed by the Cloud Native Computing Foundation (CNCF). It's not a product; it's a framework. It defines how to generate, collect, and transmit telemetry data (traces, metrics, logs) independently of the backend. Why do you need it? In a microservices architecture, a user request crosses dozens of services. Without distributed tracing, you won't know where the millisecond is lost. With OTel, you use one SDK in code and later decide where to send data: Jaeger, Prometheus, Grafana, or even your own custom collector. No lock-in.

Real example: We follow an e-commerce client that used a proprietary APM vendor. The annual renewal cost more than their hosting. With OTel we instrumented microservices in Laravel and Node.js, exported to a self-hosted Prometheus + Jaeger cluster. Costs halved, full control. If they later want to move to Grafana Cloud, they change only one endpoint in the collector.

How does automatic and manual instrumentation work in OpenTelemetry?

There are two approaches: auto-instrumentation and manual. The first is ideal to start: install an agent (e.g., for Python, Java, Node.js) that modifies the runtime and automatically captures HTTP calls, database, cache. You don't touch code. But it won't capture custom business logic. That's where manual instrumentation comes in: add spans in code to trace specific operations (e.g., a discount calculation, a complex query).

Practical example: instrument a Flask endpoint in Python

pip install opentelemetry-distro opentelemetry-exporter-otlp

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.instrumentation.flask import FlaskInstrumentor

# Setup provider
trace.set_tracer_provider(TracerProvider())

# OTLP exporter to collector or backend
exporter = OTLPSpanExporter(endpoint="http://localhost:4317", insecure=True)
span_processor = BatchSpanProcessor(exporter)
trace.get_tracer_provider().add_span_processor(span_processor)

# Auto-instrumentation Flask
app = Flask(__name__)
FlaskInstrumentor().instrument_app(app)

# Manual instrumentation
tracer = trace.get_tracer(__name__)

@app.route('/calculate-discount')
def calculate_discount():
    with tracer.start_as_current_span('calculate_discount') as span:
        price = request.args.get('price', 0, type=float)
        discount = price * 0.1
        span.set_attribute('original_price', price)
        span.set_attribute('applied_discount', discount)
        return {'total': price - discount}

if __name__ == '__main__':
    app.run()

With a few lines you have a trace for every request, including custom details. The same pattern works for Node.js, Java, Go. We use it in Laravel with the open-telemetry/opentelemetry-laravel package.

How to send traces and metrics to a backend with OpenTelemetry Collector?

The OTel Collector is a key component: it receives data from multiple sources (SDKs, agents, other apps), processes it (filters, sampling, attribute enrichment), and forwards it to one or more backends. Run it as a sidecar or daemon. Example minimal config to send to Jaeger and Prometheus:

config.yaml for OpenTelemetry Collector

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

exporters:
  jaeger:
    endpoint: jaeger:14250
    tls:
      insecure: true
  prometheus:
    endpoint: 0.0.0.0:8889

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [jaeger]
    metrics:
      receivers: [otlp]
      exporters: [prometheus]

Start the collector with otelcol --config config.yaml. Now every application sending OTLP data to the collector's address (e.g., http://collector:4318) will automatically end up in both Jaeger (traces) and Prometheus (metrics). Serious stuff, not toys: we used this architecture to monitor a Kubernetes cluster with 20 microservices. Transport latency averages under 5ms.

What are the hidden costs of an observability infrastructure with OpenTelemetry?

OpenTelemetry is free and open source, but resources aren't free. Every span, every metric consumes CPU, RAM, storage, and network bandwidth. We've seen clients send millions of spans per minute without sampling, saturating the collector. Sampling is mandatory. Use tail-based sampling to decide which traces to keep whole. For example, keep all traces with errors or latency >500ms, and sample the rest at 10%. The OTel collector natively supports the tail_sampling processor.

processors:
  tail_sampling:
    decision_match:
      - name: error
        policies:
          - type: status_code
            status_code: ERROR
      - name: slow
        policies:
          - type: latency
            threshold_ms: 500
    sampling_percentage: 10

Watch out: if you use a SaaS backend (Datadog, Grafana Cloud), cost is based on data volume. A local collector with intelligent sampling can cut your bill by 70%.

How to manage security and privacy of data in traces?

Traces often contain sensitive data: HTTP headers, query parameters, payloads. OpenTelemetry provides processors to mask or delete attributes before export. Examples: remove Authorization header, obscure email addresses, hash SELECT * FROM users queries. Configure in the collector with the attributes or filter processor.

processors:
  attributes:
    actions:
      - key: http.request.header.authorization
        action: delete
      - key: db.statement
        action: hash

We make this filter automatic in every project: protecting customer data is a GDPR requirement not left to chance. If you use an external backend, ensure compliance (e.g., Grafana Cloud with DPA).

Does OpenTelemetry integrate with other monitoring tools?

Absolutely. The beauty of OTel is neutrality. You can collect metrics with the OTel SDK and send them to Prometheus, while keeping traces on Jaeger and logs on Loki. Or unify everything on Grafana. We built a unified dashboard for a client using Redis (for job queues) and MongoDB (for geospatial data). With OTel we traced every Redis and MongoDB query with specific spans, without changing libraries. Dive deeper with our articles on Redis as a job queue and MongoDB compound indexes.

What to do now

1. Install the OpenTelemetry SDK in your main project. Start with auto-instrumentation for a single service.
2. Set up an OpenTelemetry Collector on a server or container. Use the sample config above and point to a free backend like Jaeger (local) or Grafana Cloud (free tier).
3. Enable sampling from day one. Costs explode without sampling.
4. Add manual instrumentation for critical operations. Don't stop at auto-instrumentation: trace checkout, cart calculation, email sending.
5. Verify the security of exported data. Apply filters to remove PII.
6. Read the official documentation: OpenTelemetry Documentation.
7. Check our mother guide on monitoring: Monitoring e Observability – pillar.