OpenTelemetryDecorator

Build status badgeHex version badgeHex downloads badge

A function decorator for OpenTelemetry traces.

Installation

Add open_telemetry_decorator to your list of dependencies in mix.exs. We include the opentelemetry_api package, but you'll need to add opentelemetry yourself in order to report spans and traces.

def deps do
  [
    {:opentelemetry, "~> 1.2"},
    {:opentelemetry_exporter, "~> 1.4"},
    {:open_telemetry_decorator, "~> 1.4"}
  ]
end

Then follow the directions for the exporter of your choice to send traces to to zipkin, honeycomb, etc. https://github.com/open-telemetry/opentelemetry-erlang/tree/main/apps/opentelemetry_zipkin

Honeycomb Example

config/runtime.exs

api_key = Map.fetch!(System.get_env(), "HONEYCOMB_KEY")

config :opentelemetry, :processors,
  otel_batch_processor: %{
    exporter:
      {:opentelemetry_exporter,
       %{
         protocol: :grpc,
         headers: [
           {'x-honeycomb-team', api_key},
           {'x-honeycomb-dataset', 'YOUR_APP_NAME'}
         ],
         endpoints: [{:https, 'api.honeycomb.io', 443, []}]
       }}
  }

Usage

Add use OpenTelemetryDecorator to the module, and decorate any methods you want to trace with @decorate with_span("span name").

The with_span decorator will automatically wrap the decorated function in an opentelemetry span with the provided name.

defmodule MyApp.Worker do
  use OpenTelemetryDecorator

  @decorate with_span("worker.do_work")
  def do_work(arg1, arg2) do
    ...doing work
  end
end

Span Attributes

The with_span decorator allows you to specify an include option which gives you more flexibility with what you can include in the span attributes. Omitting the include option with with_span means no attributes will be added to the span by the decorator.

defmodule MyApp.Worker do
  use OpenTelemetryDecorator

  @decorate with_span("worker.do_work", include: [:arg1, :arg2])
  def do_work(arg1, arg2) do
    # ...doing work
  end
end

The Attributes module includes a helper for setting additional attributes outside of the include option. Attributes added in either a set call or in the include that are not primitive OTLP values will be converted to strings with Kernel.inspect/1.

defmodule MyApp.Worker do
  use OpenTelemetryDecorator
  alias OpenTelemetryDecorator.Attributes

  @decorate with_span("worker.do_work")
  def do_work(arg1, arg2) do
    Attributes.set(arg1: arg1, arg2: arg2)
    # ...doing work
    Attributes.set(:output, "something")
  end
end

The decorator uses a macro to insert code into your function at compile time to wrap the body in a new span and link it to the currently active span. In the example above, the do_work method would become something like this:

defmodule MyApp.Worker do
  require OpenTelemetry.Tracer, as: Tracer

  def do_work(arg1, arg2) do
    Tracer.with_span "my_app.worker.do_work" do
      # ...doing work
      Tracer.set_attributes(arg1: arg1, arg2: arg2)
    end
  end
end

Configuration

Prefixing Span Attributes

Honeycomb suggests that you namespace custom fields, specifically putting manual instrumentation under app.

In order to do this, you'll configure the attr_prefix option in config/config.exs

config :open_telemetry_decorator, attr_prefix: "app."

Changing the join character for nested attributes

By default, nested attributes are joined with an underscore. However, when you have an object with underscores and a property with underscores, this can be hard to visually parse. For example, my_struct.other_struct.field, would be exported as my_struct_other_struct_field.

To override this, you'll configure the attr_joiner option in config/config.exs. The default value will likely change from _ to . in a future version.

config :open_telemetry_decorator, attr_joiner: "."

Thanks to @benregn for the examples and inspiration for these two options!

Switching to the v2 Attributes module

I'm dropping support for specifying nested attributes in the include option, but will be adding the ability to implement a protocol that will let you define which keys to include as span attributes and what they should be called. In the meantime, I will default to the v1 version of the Attributes module, and expose a configuration option to opt in to the v2 version.

config :open_telemetry_decorator, attrs_version: "v2"

Additional Examples

You can provide span attributes by specifying a list of variable names as atoms.

This list can include...

Any variables (in the top level closure) available when the function exits. Note that variables declared as part of a with block are in a separate scope so NOT available for include attributes

defmodule MyApp.Math do
  use OpenTelemetryDecorator

  @decorate with_span("my_app.math.add", include: [:a, :b, :sum])
  def add(a, b) do
    sum = a + b
    {:ok, sum}
  end
end

The result of the function by including the atom :result:

defmodule MyApp.Math do
  use OpenTelemetryDecorator

  @decorate with_span("my_app.math.add", include: [:result])
  def add(a, b) do
    {:ok, a + b}
  end
end

Map/struct properties using nested lists of atoms:

defmodule MyApp.Worker do
  use OpenTelemetryDecorator

  @decorate with_span("my_app.worker.do_work", include: [[:arg1, :count], [:arg2, :count], :total])
  def do_work(arg1, arg2) do
    total = some_calculation(arg1.count, arg2.count)
    {:ok, total}
  end
end
defmodule MyApp.Worker do
  use OpenTelemetryDecorator

  @decorate with_span("my_app.worker.do_work", include: [[:calc, "sum"], [:calc, "product"]])
  def do_work(obj) do
    calc = %{"sum" => 10, "product" => 25}
    {:ok, calc}
  end
end

The map/struct properties of the result of the function:

defmodule MyApp.Math do
  use OpenTelemetryDecorator

  @decorate with_span("my_app.math.add", include: [[:result, :sum]])
  def add(a, b) do
    %{sum: a + b}
  end
end

Development

make check before you commit! If you'd prefer to do it manually: