Greptimex

Elixir client for GreptimeDB ingestion and PromQL over gRPC.

def deps do
  [{:greptimex, "~> 0.4.0"}]
end

Requires Elixir 1.18 or newer. For the breaking API changes in 0.4, see MIGRATION.md.

Start a client

Add a named client to your application's supervision tree:

children = [
  {Greptimex,
   name: MyApp.Greptime,
   address: "localhost:4001",
   pool_size: 2,
   header: [catalog: "greptime", dbname: "public"]}
]

Supervisor.start_link(children, strategy: :one_for_one)

You can also call Greptimex.start_link/1. Pass the registered atom name to operations. Each named client has its own configuration and pool. The client reads configuration at startup, so restart it to apply changes.

Headers accept catalog, dbname, schema, timezone and auth. Authentication:

header: [auth: {:basic, {"username", "password"}}]
# or
header: [auth: {:token, "token"}]

TLS uses the underlying gRPC credential configuration:

channel_options: [
  cred: GRPC.Credential.new(ssl: [
    verify: :verify_peer,
    cacerts: :public_key.cacerts_get(),
    server_name_indication: ~c"your-host.example"
  ])
]

Insert points and batches

point = %{
  tags: %{host: "server1", region: "us-west"},
  fields: %{cpu: 0.8},
  timestamp: DateTime.utc_now()
}

Greptimex.insert(MyApp.Greptime, "metrics", point)
# {:ok, 1}

Greptimex.insert(MyApp.Greptime, "metrics", [point, point], timeout: 5_000)
# {:ok, 2}

Greptimex.insert_batch(MyApp.Greptime, [
  {"metrics", [point]},
  {"other_metrics", [point], timestamp_unit: :microsecond}
])
# {:ok, 2}

fields and a non-null timestamp are required; tags defaults to %{}. Greptimex converts atom names to strings. It rejects duplicate names, including collisions between tags, fields and the timestamp.

Greptimex validates the whole batch before sending one RPC. Empty lists return {:ok, 0} without sending data. It does not split, buffer or retry batches.

Success returns the server's affected row count. A batch is not a transaction. A timeout can leave you unsure whether the server wrote the data, so the application must decide whether to retry.

Inference and explicit types

Inference examines each column across all points in a table entry:

Greptimex converts a mix of integers and floats to :float64 only when it can represent the integers exactly. It rejects other incompatible mixtures. Inference uses the points in each table entry, without checking the server or caching schemas. Use a reusable schema to keep column types stable across batches.

Use {value, type} for explicit types, for example {255, :uint8}, {<<0, 255>>, :binary} or {nil, :float64}. Explicit declarations must agree with each other and with a supplied schema. Missing fields become null. A column containing only nulls requires an explicit type.

Supported types are :int8/16/32/64, :uint8/16/32/64, :float32, :float64, :boolean, :string, :binary, :date, :datetime, and :time_* / :timestamp_* with suffix second, millisecond, microsecond or nanosecond. These slash abbreviations describe separate atoms, such as :int8 and :int16. Greptimex checks integer bounds. Strings must be valid UTF-8. Binary fields can contain arbitrary bytes. Float32 rounds floats to IEEE single precision but requires exact conversion for integers.

Ingestion timestamp integers use the configured unit. Greptimex converts DateTime values to that unit. For precision finer than microseconds, use integer nanosecond timestamps.

Date integers count days since the Unix epoch. Integers typed as :datetime count milliseconds. Greptimex encodes these fields as TIMESTAMP_MILLISECOND because GreptimeDB rejects the legacy DATETIME column type. It treats NaiveDateTime fields as UTC. Time values count units since midnight.

Version 0.4 returns validation errors for JSON, decimal, interval, list, vector, struct and dictionary values.

Reusable schema

schema = %Greptimex.Schema{columns: [
  %{name: "ts", type: :timestamp_millisecond, semantic: :timestamp},
  %{name: "host", type: :string, semantic: :tag},
  %{name: "cpu", type: :float64, semantic: :field}
]}

Greptimex.insert(MyApp.Greptime, "metrics", %{
  timestamp: DateTime.utc_now(),
  tags: %{host: "server1"},
  fields: %{cpu: 0.8}
}, schema: schema)

Use Greptimex.Schema.new(columns) to validate a schema before inserting data. It returns {:ok, schema} or {:error, error}. A schema needs exactly one timestamp column with a timestamp type. Its column order determines wire order, and its timestamp name and type override the inference defaults. Greptimex rejects unknown columns, semantic changes and incompatible values.

A batch entry can override schema, timestamp_name and timestamp_unit for that entry. The request's header and timeout apply to the whole batch. Request headers merge with client headers.

PromQL

Greptimex.query_instant(MyApp.Greptime, "metrics", time: DateTime.utc_now())
# {:ok, %Greptimex.PromQL.Result{
#   type: :vector,
#   data: [%{metric: %{"host" => "server1"}, value: {datetime, 0.8}}],
#   warnings: [], infos: []
# }}

Greptimex.query_range(MyApp.Greptime, "metrics",
  ~U[2025-01-01 00:00:00Z], ~U[2025-01-01 01:00:00Z], "5m",
  lookback: "10m", timeout: 5_000)

Query times accept DateTime or Unix seconds, including fractional seconds. Instant queries default to the current time; lookback defaults to "5m". Range bounds are inclusive and must be ordered. Step and lookback accept positive seconds or duration strings such as "250ms", "5m" or "1h30m". Duration strings use integer components in descending unit order, without repeated units. Use numeric seconds for fractional durations, for example 0.25.

Normalized results retain warnings and infos:

• :vector: list of %{metric: labels, value: {datetime, number}}.
• :matrix: list of %{metric: labels, values: [{datetime, number}, ...]}.
• :scalar: {datetime, number}.
• :string: {datetime, string}.

Labels keep string keys. Numeric samples are floats. NaN, +Inf and -Inf become :nan, :infinity and :neg_infinity. Greptimex rounds JSON response timestamps to microseconds.

Use decode: :raw to get the decoded JSON envelope. This also gives you access to formats the normalizer does not support, such as native histograms. Server error envelopes still return an error.

Optional module facade

defmodule MyApp.Greptime do
  use Greptimex, otp_app: :my_app
end

# config/runtime.exs
config :my_app, MyApp.Greptime,
  address: System.fetch_env!("GREPTIME_ADDRESS"),
  header: [dbname: "public"]

# supervision children
children = [MyApp.Greptime]

MyApp.Greptime.insert("metrics", point)
MyApp.Greptime.query_instant("metrics", time: DateTime.utc_now())

The facade calls the same Greptimex functions and uses its module as the client name. At startup, it merges defaults, macro options, application configuration and child-spec options in that order. Later values win. Operation options override the startup configuration for that call.

The facade reads application configuration on every start, including supervisor restarts.

Errors and telemetry

Invalid data and expected connection, transport, server or decoding failures return {:error, %Greptimex.Error{kind: kind, code: code, message: message, context: context}}. Kinds are :validation, :connection, :timeout, :transport, :server and :decode. Validation context identifies the table, column and one-based row when available. Greptimex keeps server error codes and redacts configured credentials from server error messages.

Invalid configuration and keyword options raise ArgumentError. A missing client is a programming error. Unexpected exceptions, throws and process exits propagate to the caller, including races with shutdown. Greptimex does not wrap them in error tuples.

Each operation emits [:greptimex, operation, :start | :stop | :exception], where operation is :insert, :query_instant or :query_range. Batch writes use :insert. The span includes configuration lookup, validation, RPC and decoding. Returned errors emit :stop with status: :error; unexpected exceptions emit :exception.

Start measurements contain system and monotonic time; stop measurements include duration in native units, plus affected_rows or normalized result_count on success. Metadata includes client, status and error kind/code when applicable. Greptimex does not add queries, row contents or credentials to metadata. Exception events include the original reason and stacktrace. Review what your telemetry handlers log, since an exception reason can contain application data. The gRPC library emits its own connection events.

Development

mix deps.get
mix test
mix format --check-formatted
mix docs --warnings-as-errors

The default suite starts a local gRPC test server. Its TLS test uses openssl to generate certificates. Run the integration tests against a disposable GreptimeDB instance. They create uniquely named tables:

docker run -d --name greptimex-test \
  -p 127.0.0.1:14000:4000 -p 127.0.0.1:14001:4001 \
  greptime/greptimedb:v1.2.0 standalone start \
  --http-addr 0.0.0.0:4000 --grpc-bind-addr 0.0.0.0:4001

# Wait until this returns HTTP 200 before running the tests.
curl --fail http://127.0.0.1:14000/health

GREPTIMEX_TEST_ADDRESS=127.0.0.1:14001 mix test --include integration

# Remove the disposable database and its test data when finished.
docker rm -f greptimex-test

Integration tests cover scalar ingestion, explicit schemas, multi-table batches, timestamp units, concurrent writes, instant/range PromQL, raw results, and server errors. The dashboard is available at http://localhost:14000/dashboard while the container is running.

Protocol sources are pinned to greptime-proto commit 549ff0aa8e866c6042d2e755fa35d919b42fde30. With protoc 36.1 and protoc-gen-elixir 0.16.0 installed:

elixir scripts/sync_pb.exs --check
elixir scripts/sync_pb.exs

This repository-only script is excluded from the Hex package. It downloads and generates files in a temporary directory. If either step fails, it leaves the project files untouched. --check compares both the proto sources and generated modules with the pinned versions.

Connection lifecycle

Greptimex supervises GRPC.Client.Connection workers using gRPC 1.0.5 or newer with Mint. The client supervisor stores configuration in ETS. Each caller reads that configuration and executes its RPC, so requests can run concurrently on one HTTP/2 channel.

Startup does not wait for a connection. Until a channel is ready, operations return a connection error. The supervisor restarts connection workers when they exit. On gRPC UNKNOWN, INTERNAL or UNAVAILABLE, the executor disconnects the channel and lets supervision start a new worker. Disconnecting can interrupt other calls on that channel. Greptimex never resends a failed RPC.