opentelemetry data model

understanding how opentelemetry handles data

OpenTelemetry

OpenTelemetry, the new standard in telemetry (observability data) collection, has a spec (that I think is obtuse) and protobufs defining its data model.

Data Model

Keywords:

• Resource: an app/service
• InstrumentationLibrary: instance of sdk used to generate data in app

For the apps (resources) that generate data, ResourceSpans will typically contain a single element.

Traces

ref: trace.proto

Traces

• ResourceSpans[]
  • InstrumentationLibrarySpans[]
    • Spans[]

Metrics

ref: metrics.proto

Metrics

• ResourceMetrics[]
  • InstrumentationLibraryMetrics[]
    • Metrics[]
      • metric type
        • data points[]

Logs

ref: logs.proto

Logs

• ResourceLogs[]
  • InstrumentationLibraryLogs[]
    • LogRecord[]

Collector

Where is the data model useful? In the collector, where the internal representation is a thin wrapper over the protobufs.

Ignoring the fact that there are layers of factories and indirection in the code, processors are constructed with config and the next processor (eg typed consumer.Traces).

This means each processor is simply a method ConsumeXXX, handed an instance of one the top level resources shown above, and calls the next processor with the remaining/mutated data when its done.

The last processor is given a fanout processor which connects to all the exporters.