OpenTelemetry.jl: Collecting Logs, Traces, and Metrics Together

OpenTelemetry.jl is a pure Julia implementation of the OpenTelemetry specification. It enables developers to collect logs, traces, and metrics in a unified approach to improve the observability of complex systems. With OpenTelemetry.jl, users can not only analyze the telemetry data in Julia but also across many other different languages or services.

Introduction

Logs, traces, and metrics are used independently to help diagnose systems for a long time. As applications are becoming more complex and more heterogeneous, pinpointing a problem is more difficult than before. Formed through a merger of the OpenTracing and OpenCensus projects, OpenTelemetry is aimed to improve the situation. In addition to the concrete specification, OpenTelemetry comes with a collection of APIs and SDKs in many different programming languages. As for now, Julia is not included yet. And that's why we created OpenTelemetry.jl. By fully respecting the OpenTelemetry specification, data collected in Julia with OpenTelemetry.jl can be analyzed together with those from other languages uniformly.

Highlights of OpenTelemetry.jl

• Simple API

  The APIs are elaborately designed by balancing conventions in Julia and specifications stated in OpenTelemetry.

• Fully Configurable

  Most components in OpenTelemetrySDK are configurable (either explicitly through keyword arguments or with environment variables). Users can decide how to collect the data, when to send the data, and where to store them. With the architecture of OpenTelemetry, users can choose any APM which supports the OpenTelemetry collector.

• Pluggable auto instrumentation

  Several commonly used packages in Julia are already auto-instrumented (Downloads.jl, Genie.jl, HTTP.jl, CUDA.jl, etc). Users can simply enable them by importing corresponding instrumentation packages.

• It's FAST

  The API layer and SDK layer are separated by design. The API layer is lightweight enough so that package developers can safely add it as a dependency with only very little overhead introduced. Our benchmark results show that the implementation in Julia is much faster than many SDKs in other languages.

Agenda

This talk contains the following three parts:

1. A general introduction to OpenTelemetry and its benefits.
2. An in-depth explanation of how OpenTelemetry.jl is implemented plus some personal experiences on how to implement and organize a mega Julia package in a more Julian way.
3. Demonstrations of how we apply OpenTelemetry.jl in realworld products.

Target Audience

Both package developers and application developers who would like to improve the observability of their services in production will benefit from this talk. General Julia users can also learn how to better collect, manage, and analyze telemetry data in a unified approach after this talk.