Schema-first Application Telemetry

Event: SRECon22 Europe/Middle East/Africa

Date: October 27, 2022

Speakers: Yuri Shkuro

Video: YouTube (or below). Slides: SpeakerDeck

This talk is based on Positional Paper: Schema-First Application Telemetry.

The journey to cloud-native observability starts with basic telemetry gathering: metrics, logs, events, traces, etc. Teams soon realize that they are collecting a lot of fragmented telemetry data that is difficult to interpret without additional metadata. Instead, users often need to rely on tribal knowledge about telemetry datasets. The missing metadata can range from simple things like descriptions, types, and units of measure, to machine-readable semantic data identifying joinable dimensions, privacy policies, etc. In this talk we present a schema-first approach to application telemetry, including an improved developer experience that minimizes the initial overhead of authoring telemetry signals, and the schema definition language based on Thrift IDL with annotations and rich types that capture semantic meaning and other metadata suitable for automated processing. We contrast this approach with the existing techniques popular in the industry, including the OpenTelemetry Semantic Conventions, to demonstrate the benefits and the trade-offs.