nebula_graph_ex
An Elixir client for NebulaGraph, a distributed open-source graph database built for massive-scale graph data.
nebula_graph_ex handles the full lifecycle of communicating with NebulaGraph from Elixir: opening and pooling TCP connections, authenticating sessions, encoding and executing nGQL statements, and decoding the response into idiomatic Elixir values. Your application code works entirely with native Elixir types — integers, binaries, structs — and never touches the wire protocol directly.
Key capabilities:
- Connection pooling with automatic reconnection and back-off
- Complete value decoding — vertices, edges, paths, temporal types, geography, lists, maps, sets, and all primitive types
- Parameterised queries that encode values as typed Thrift structs, keeping nGQL strings free of interpolation
- TLS/SSL support for encrypted connections
:telemetryevents on every query for metrics and tracing- Per-query option overrides for space switching, timeouts, and row transformation
Prerequisites
- Elixir 1.15 or later
- A running NebulaGraph 3.x instance (see Running locally for a Docker setup)
Installation
Add nebula_graph_ex to your dependencies:
# mix.exs
defp deps do
[
{:nebula_graph_ex, "~> 0.2"}
]
end
Setup
There are two ways to set up the library: using the generator (recommended) or manually.
Option 1: Generator (recommended)
Run the Mix task to wire up the library in one step:
mix ngex.gen.graph MyApp.Graph
The generator does six things:
- Creates
lib/my_app/graph.exwith the graph module - Inserts a default config block, and the
nebula_graphs: [MyApp.Graph]list used by the migration tasks, intoconfig/config.exs - Creates
priv/graph/migrations/with a.formatter.exsfor the migration DSL - Injects the module as the first child in
lib/my_app/application.ex - Adds NebulaGraphEx query metrics to
lib/my_app_web/telemetry.exwhen that file exists - Adds
metrics: MyAppWeb.Telemetryto the LiveDashboard route when a dashboard is present and metrics are not already configured
It then prints a config/runtime.exs snippet for secrets, which you add manually.
The generator is idempotent — re-running it on an already-configured app skips each step that is already in place without overwriting anything.
Option 2: Manual setup
The generator performs these steps. Do them by hand if you prefer not to use it.
1. Create the graph module
Create a dedicated module for your graph connection. The module name doubles as the registered pool name — no PIDs to pass around.
# lib/my_app/graph.ex
defmodule MyApp.Graph do
use NebulaGraphEx.Graph, otp_app: :my_app
end
2. Configure the connection
Static options (hostname, pool size, space) belong in config/config.exs:
# config/config.exs
config :my_app, MyApp.Graph,
hostname: "localhost",
port: 9669,
username: "root",
password: "my_password",
space: "my_graph",
pool_size: 10
Secrets and environment-specific values belong in config/runtime.exs, where they are resolved at boot rather than embedded at compile time:
# config/runtime.exs
import Config
config :my_app, MyApp.Graph,
hostname: System.get_env("NEBULA_HOST", "localhost"),
password: fn -> System.fetch_env!("NEBULA_PASS") end
Passing :password as a zero-arity function ensures the value is never captured in a compiled module or logged in a crash report.
3. Start the pool
Add the module to your application's supervision tree:
# lib/my_app/application.ex
def start(_type, _args) do
children = [MyApp.Graph]
Supervisor.start_link(children, strategy: :one_for_one)
end
The pool starts, authenticates, and is ready to accept queries before start/2 returns.
4. Add telemetry metrics
If your Phoenix app has lib/my_app_web/telemetry.ex, add NebulaGraphEx metrics
to the metrics/0 list:
summary("nebula_graph_ex.query.duration",
event_name: [:nebula_graph_ex, :query, :stop],
measurement: :duration,
unit: {:native, :millisecond}
),
counter("nebula_graph_ex.query.count",
event_name: [:nebula_graph_ex, :query, :stop]
),
counter("nebula_graph_ex.query.error.count",
event_name: [:nebula_graph_ex, :query, :exception]
)
If your app exposes Phoenix LiveDashboard, make sure the route uses the telemetry module for metrics:
live_dashboard "/dashboard", metrics: MyAppWeb.Telemetry
Status checks
To verify that the pool process is running and can execute a simple query:
{:ok, status} = MyApp.Graph.status()
status.connected?
#=> true
status.queryable?
#=> true
Under the hood, status/0 performs an active health check by running
RETURN 1 AS status.
If you want to call it through the generic API:
{:ok, status} = NebulaGraphEx.Graph.status(MyApp.Graph)
If the pool is not running, you get:
{:error,
%{
error: :not_running,
error_details: %{reason: :not_running, message: "pool process is not running"},
connected?: false,
queryable?: false
}}
If the pool is running but the probe query fails, status/0 also returns the
underlying error and normalized details so callers can surface a useful message:
{:error, status} = MyApp.Graph.status(probe_statement: "THIS IS NOT VALID NGQL")
status.error_details
#=> %{code: :e_syntax_error, message: "...", category: :query, statement: "THIS IS NOT VALID NGQL"}
If you only want to know whether the pool process exists without probing the database, disable the query probe:
{:ok, status} = MyApp.Graph.status(probe_query: false)
Querying
alias NebulaGraphEx.{Record, ResultSet}
{:ok, rs} = MyApp.Graph.query("MATCH (v:Player) RETURN v.name, v.age LIMIT 5")
rs |> ResultSet.count() #=> 5
rs |> ResultSet.columns() #=> ["v.name", "v.age"]
rs |> ResultSet.to_maps() #=> [%{"v.name" => "Tim Duncan", "v.age" => 42}, ...]
Use query!/3 when you want the result set directly and prefer an exception over a pattern-match on error:
MyApp.Graph.query!("RETURN 1+1 AS n")
|> ResultSet.first!()
|> Record.get!("n")
#=> 2
Parameterised queries
Pass user-supplied values as parameters, never by interpolating them into the statement string. Parameters are encoded as typed Thrift values — there is no string substitution, so injection is not possible.
MyApp.Graph.query(
"MATCH (v:Player{name: $name}) RETURN v.age AS age",
%{"name" => "Tim Duncan"}
)
Working with graph types
Vertices, edges, and paths are decoded into typed Elixir structs automatically:
{:ok, rs} = MyApp.Graph.query("MATCH (v:Player) RETURN v LIMIT 1")
rs
|> ResultSet.first!()
|> Record.get!("v")
#=> %NebulaGraphEx.Types.Vertex{
# vid: "player100",
# tags: [%NebulaGraphEx.Types.Tag{name: "Player", props: %{"name" => "Tim Duncan", "age" => 42}}]
# }
rs
|> ResultSet.first!()
|> Record.get!("v")
|> NebulaGraphEx.Types.Vertex.prop("Player", "name")
#=> "Tim Duncan"
See the Type mapping table for a full list of how NebulaGraph types map to Elixir terms.
Per-query options
Any pool-level option can be overridden for a single call:
MyApp.Graph.query(stmt, %{},
space: "other_space", # switch graph space for this query only
timeout: 60_000, # override execution timeout (ms)
decode_mapper: &Record.to_map/1 # transform each row at decode time
)
Query logging
Enable structured query logging to see exactly what is being sent to NebulaGraph, where it was called from, and how long it took.
Pool-level logging
Turn on logging for all queries from a pool via config:
# config/config.exs
config :my_app, MyApp.Graph,
log: true
Per-query logging
Enable or suppress logging for a single call:
MyApp.Graph.query("MATCH (v:Player) RETURN v.name LIMIT 5", %{}, log: true)
Log levels
The log level is inferred automatically from the nGQL verb so that read traffic stays at :debug while mutations and schema changes stand out:
| Verb | Level |
|---|---|
MATCH, GO, LOOKUP, FETCH, RETURN, FIND, SHOW, DESCRIBE, EXPLAIN, USE |
:debug |
INSERT, UPSERT, UPDATE |
:info |
DELETE, DROP, CREATE, ALTER, ADD, REMOVE, REBUILD |
:warning |
Override the inferred level with :log_level:
config :my_app, MyApp.Graph,
log: true,
log_level: :info # force all queries to :info regardless of verb
Log output
Each logged query emits a muted gray ↳ caller line followed by a [NGE]-prefixed status line and the statement:
↳ MyApp.IntelModule.recent_snapshots/1, at: lib/my_app/intel_module.ex:42
[NGE] QUERY OK 3 rows db=0.6ms queue=0.1ms
MATCH (v:Player) RETURN v.name LIMIT 3 []
For a query with parameters:
↳ MyApp.Players.find/1, at: lib/my_app/players.ex:17
[NGE] QUERY OK 1 rows db=1.2ms queue=0.1ms
MATCH (v:Player{name: $name}) RETURN v %{"name" => "Tim Duncan"}
On error:
↳ MyApp.Players.find/1, at: lib/my_app/players.ex:17
[NGE] QUERY ERROR (e_syntax_error) db=0.3ms queue=0.1ms
THIS IS NOT VALID NGQL []
Slow query logging
Log only queries that exceed a time threshold, without enabling full logging:
config :my_app, MyApp.Graph,
slow_query_threshold: 100 # log queries that take longer than 100ms
:slow_query_threshold and :log are independent — both can be set together, and the threshold fires regardless of the :log setting.
Multiple pools
When your application needs connections to more than one NebulaGraph cluster or space, create a module per connection — either with the generator or manually, following the same steps as the single-pool setup.
Using the generator:
mix ngex.gen.graph MyApp.PrimaryGraph
mix ngex.gen.graph MyApp.AnalyticsGraph
Each module reads its own config key independently:
# config/runtime.exs
config :my_app, MyApp.PrimaryGraph,
hostname: System.fetch_env!("NEBULA_PRIMARY_HOST"),
space: "primary"
config :my_app, MyApp.AnalyticsGraph,
hostname: System.fetch_env!("NEBULA_ANALYTICS_HOST"),
space: "analytics"
Add all modules to the supervision tree:
children = [MyApp.PrimaryGraph, MyApp.AnalyticsGraph]
Each graph has its own migrations. See Multiple graphs under Migrations.
Migrations
Graph schemas (tags, edge types and indexes) are managed with versioned migrations that work like Ecto's. If you know mix ecto.gen.migration and mix ecto.migrate, you already know how to use them.
mix ngex.gen.migration create_people
# priv/graph/migrations/20261109120000_create_people.exs
defmodule MyApp.Graph.Migrations.CreatePeople do
use NebulaGraphEx.Migration
def change do
create tag(:person) do
add :id, :string, null: false
add :name, :string
add :age, :int, default: 0
add :created_at, :datetime, default: fragment("datetime()")
end
create edge(:knows) do
add :since, :date
end
# string properties need an index prefix length
create tag_index(:person, id: 64)
end
end
mix ngex.migrate # all pending
mix ngex.rollback # the latest one
mix ngex.migrations # status table
Every task also has a long name (mix nebula_graph_ex.migrate, and so on). The ngex.* names are short aliases that take the same arguments.
The tasks read graphs from config :my_app, nebula_graphs: [MyApp.Graph] (or -g MyApp.Graph) and accept --step, --to and --all, as in Ecto.
The DSL
| Ecto | nebula_graph_ex |
|---|---|
create table(:t) do add ... end |
create tag(:t) do add ... end / create edge(:t) do ... end |
alter table(:t) do ... end |
alter tag(:t) do ... end / alter edge(:t) do ... end |
add, modify, remove |
add, modify, remove |
create index(:t, [:a]) |
create tag_index(:t, [:a]) / create edge_index(:t, [:a]) |
drop table(:t) / drop index(...) |
drop tag(:t) / drop tag_index(...) |
execute "SQL" / execute up, down |
execute "nGQL" / execute up, down |
fragment("now()") |
fragment("datetime()") |
change/0, up/0, down/0 |
same |
Types: :string, :fixed_string (size: n), :int/:int64, :int32, :int16, :int8, :float, :double, :bool, :date, :time, :datetime, :timestamp, :duration, :geography (geo_shape: :point). Field options: null: false, default:, comment:. Tag/edge options: ttl_duration:, ttl_col:, comment:.
create always emits IF NOT EXISTS, and drop always emits IF EXISTS. NebulaGraph DDL isn't transactional, so a migration that fails halfway can simply be re-run.
Waiting for NebulaGraph
NebulaGraph applies schema changes asynchronously. A successful CREATE TAG doesn't mean graphd can plan writes against the tag yet. For about one heartbeat, INSERT fails with TagNotFound, and REBUILD TAG INDEX reports "index not found" even after SHOW TAG INDEXES lists the index. The migrator handles this, so you don't need Process.sleep:
- after each operation it polls
DESC/SHOWand aFETCH PROP ONprobe (which graphd plans against its cached schema) until the change is usable; - new indexes are rebuilt by default (
rebuild: falseto opt out), so data that already exists gets indexed, and the rebuild job is polled withSHOW JOBuntil it reportsFINISHED. AFAILEDjob fails the migration; - transient errors (leader changes, RPC failures) are retried with capped exponential backoff, while syntax and permission errors fail immediately.
A migration is recorded as applied only once all of its operations are complete. When mix ngex.migrate finishes, the schema is ready to use.
Migration state and locking
By default, applied versions are stored in the migrated space itself, in a nebula_graph_ex_migrations tag, the way Ecto uses schema_migrations. No setup is needed. A compare-and-set lock (UPSERT ... WHEN) with a heartbeat stops two deploys from migrating at the same time. A crashed runner's lock expires after migration_lock_ttl.
To keep migration state somewhere else, such as Postgres with an advisory lock, implement NebulaGraphEx.Migration.Store and set migration_store: in the graph config. The behaviour's docs include a complete Ecto example.
Configuration
config :my_app, MyApp.Graph,
space: "my_graph", # required: the space to migrate
migrations_path: "priv/graph/migrations", # default: priv/<graph>/migrations
migration_store: NebulaGraphEx.Migration.GraphStore,
migration_timeout: :timer.minutes(2), # readiness deadline per operation
rebuild_timeout: :timer.minutes(15), # per index rebuild job
migration_lock_ttl: :timer.minutes(2),
migration_confirmations: 1 # default: number of :hosts
With several graphd :hosts, each one refreshes its schema cache on its own, so readiness probes must pass migration_confirmations times in a row.
Migrations don't create graph spaces. Spaces are infrastructure (partitions, replicas, VID type), so create them before you migrate.
Releases
Mix isn't available in a release. Call NebulaGraphEx.Migrator, which mirrors Ecto.Migrator:
defmodule MyApp.Release do
@app :my_app
def migrate do
Application.ensure_all_started(:ssl)
for graph <- Application.fetch_env!(@app, :nebula_graphs) do
{:ok, _, _} =
NebulaGraphEx.Migrator.with_graph(graph, &NebulaGraphEx.Migrator.run(&1, :up, all: true))
end
end
end
NebulaGraphEx.Migrator.run/3 returns the versions it ran and raises NebulaGraphEx.MigrationError (with :version, :operation, :statement and :reason) when a migration can't complete.
Multiple graphs
Each graph module has its own migrations directory, priv/<graph>/migrations (for example priv/primary_graph/migrations for MyApp.PrimaryGraph), and its own migration state in its own space. List every graph in the config, and the tasks run them in order:
config :my_app, nebula_graphs: [MyApp.PrimaryGraph, MyApp.AnalyticsGraph]
mix ngex.gen.migration add_people -g MyApp.PrimaryGraph
mix ngex.migrate # every graph in :nebula_graphs
mix ngex.migrate -g MyApp.AnalyticsGraph # just one
Formatting
The generator creates priv/graph/migrations/.formatter.exs, which imports the DSL's formatter rules so that add :name, :string keeps its no-parentheses style. Make sure your root .formatter.exs includes it:
[
import_deps: [:nebula_graph_ex],
subdirectories: ["priv/*/migrations"],
inputs: ["*.{ex,exs}", "{config,lib,test}/**/*.{ex,exs}"]
]
Running migrations before tests
As with Ecto, add an alias so the test space is migrated before the suite runs. Because migrations are idempotent and already-applied versions are skipped, this is quick after the first run:
# mix.exs
defp aliases do
[
test: ["ngex.migrate --quiet", "test"]
]
end
The test environment needs its own graph config (config/test.exs) pointing at a space that already exists.
Testing a migration without a database
NebulaGraphEx.Migration.operations/2 returns the operations a migration would run, as plain structs, without connecting. This makes it easy to check that a change/0 can be rolled back:
test "create people migration is reversible" do
migration = MyApp.Graph.Migrations.CreatePeople
assert [{:create, %NebulaGraphEx.Migration.Schema{name: "person"}} | _] =
NebulaGraphEx.Migration.operations(migration, :up)
# raises NebulaGraphEx.MigrationError if change/0 can't be reversed
assert NebulaGraphEx.Migration.operations(migration, :down) != []
end
Troubleshooting
could not describe space: the configured:spacedoesn't exist. Create it withCREATE SPACEfirst.- Timed out waiting for the change to propagate: the schema didn't become visible within
migration_timeout. That usually points to an unhealthy storaged or metad, or to a type that NebulaGraph silently normalises. The error's:reasonholds the last probe result. rebuild job reported FAILED (...): the message lists each storage task's error code fromSHOW JOB. Fix the cause and re-run. The migration wasn't recorded, and its statements are idempotent.waiting for the graph migration lock: another runner holds the lock. If that runner crashed, the lock expires aftermigration_lock_ttl.
Migration progress is also emitted as telemetry: [:nebula_graph_ex, :migration, :start | :stop | :exception], plus [:nebula_graph_ex, :migration, :operation, :start | :stop | :retry].
Configuration reference
| Option | Default | Description |
|---|---|---|
:hostname |
"localhost" |
NebulaGraph graphd host |
:port |
9669 |
NebulaGraph graphd port |
:hosts |
nil |
Multi-host list [{"h1", 9669}, {"h2", 9669}] — overrides :hostname/:port |
:load_balancing |
:round_robin |
Host selection when :hosts is set: :round_robin or :random |
:username |
"root" |
Authentication username |
:password |
"nebula" |
Authentication password — accepts a string or a zero-arity function |
:space |
nil |
Default graph space; nil means no space is selected on connect |
:pool_size |
10 |
Number of connections to maintain |
:max_overflow |
0 |
Extra connections spawned under burst load |
:connect_timeout |
5_000 |
TCP connect timeout (ms) |
:recv_timeout |
15_000 |
Socket read timeout per frame (ms) |
:send_timeout |
15_000 |
Socket write timeout (ms) |
:timeout |
15_000 |
Per-query execution timeout (ms) |
:idle_interval |
1_000 |
How often idle connections are pinged (ms) |
:ssl |
false |
Enable TLS |
:ssl_opts |
[] |
Options forwarded to :ssl.connect/3 (:verify, :cacertfile, etc.) |
:backoff_type |
:rand_exp |
Reconnect strategy: :stop, :exp, :rand, or :rand_exp |
:backoff_min |
1_000 |
Minimum reconnect back-off (ms) |
:backoff_max |
30_000 |
Maximum reconnect back-off (ms) |
:decode_mapper |
nil |
Function applied to each %Record{} row; nil returns records as-is |
:prefer_json |
false |
Use executeJson instead of execute; returns raw JSON from the server |
:telemetry_prefix |
[:nebula_graph_ex, :query] |
Prefix for :telemetry events |
:log |
false |
Emit a structured [NGE] Logger message for each query |
:log_level |
inferred | Override the auto-inferred log level (reads → :debug, writes → :info, deletes/DDL → :warning) |
:slow_query_threshold |
nil |
Log queries whose total time exceeds this threshold in ms, regardless of :log |
:show_sensitive_data_on_connection_error |
false |
Include the password in connection error messages |
Full option documentation: NebulaGraphEx.Options.
Telemetry
Every query emits three :telemetry events:
| Event | When |
|---|---|
[:nebula_graph_ex, :query, :start] |
Before the query is sent |
[:nebula_graph_ex, :query, :stop] |
After a successful response |
[:nebula_graph_ex, :query, :exception] |
If the query raises |
The :stop and :exception events include a :duration measurement in native time units, and a metadata map containing :statement, :params, and :opts.
Attach the built-in logger handler during development:
NebulaGraphEx.Telemetry.attach_default_handler()
Attach a custom handler for production metrics:
:telemetry.attach(
"my-app-nebula-metrics",
[:nebula_graph_ex, :query, :stop],
fn _event, %{duration: duration}, %{statement: statement}, _config ->
ms = System.convert_time_unit(duration, :native, :millisecond)
MyMetrics.histogram("nebula.query_ms", ms, tags: [statement: statement])
end,
nil
)
The prefix can be changed per-pool or per-query with the :telemetry_prefix option.
Type mapping
All NebulaGraph value types are decoded to native Elixir terms. The mapping is:
| NebulaGraph type | Elixir term |
|---|---|
null |
nil |
bool |
true / false |
int |
integer() |
float |
float() |
string |
binary() |
date |
%NebulaGraphEx.Types.NebDate{} |
time |
%NebulaGraphEx.Types.NebTime{} |
datetime |
%NebulaGraphEx.Types.NebDateTime{} |
vertex |
%NebulaGraphEx.Types.Vertex{} |
edge |
%NebulaGraphEx.Types.Edge{} |
path |
%NebulaGraphEx.Types.Path{} |
list |
list() |
map |
map() |
set |
MapSet.t() |
geography |
%NebulaGraphEx.Types.Geography{} |
duration |
%NebulaGraphEx.Types.Duration{} |
Temporal structs provide conversion helpers to standard Elixir types where a direct mapping exists — see NebulaGraphEx.Types.NebDate, NebulaGraphEx.Types.NebTime, and NebulaGraphEx.Types.NebDateTime.
Running locally
A docker-compose.yml is included at the project root. It starts a single-node NebulaGraph 3.x instance on the default port 9669 with the default credentials (root / nebula):
docker compose up -d
Running integration tests
Integration tests require a live NebulaGraph instance and are excluded from the default test run. Start the Docker cluster, then:
mix test --include integration
Contributing
Contributions should keep the public API, docs, and release metadata aligned.
CHANGELOG.md is updated for you at release time (see Releasing).
The only thing to get right is the commit type, because changelog entries are
generated from commit messages:
feat: ...goes under Added,fix: ...under Fixed, andrefactor:,perf:and untyped messages under Changed. Add!(feat!: ...) for breaking changes.chore:,docs:,test:,ci:,build:andstyle:are left out. Don't use them for changes users will notice.
To word an entry yourself, write it under "Unreleased" in CHANGELOG.md.
Hand-written notes replace the generated ones for that release.
Before opening a release PR or tagging a version:
mix precommit
mix test --include integration
Integration tests require a running NebulaGraph instance, so start the local Docker setup first if you want full coverage.
Versioning
This project should follow Semantic Versioning. The version in mix.exs is the
source of truth, and git tags should always match it as v<version>.
Industry-standard SemVer guidance:
patch: backwards-compatible bug fixes, documentation-only fixes that affect the shipped version, and internal changes that do not change public behaviourminor: backwards-compatible new features, new public APIs, and additive functionalitymajor: breaking changes to the public API, behaviour, configuration, or supported upgrade path
For a library published to Hex, the safest default is:
- use
patchfor fixes - use
minorfor additive improvements - use
majoronly when users must change their code or upgrade process
The version and tag tasks below live in dev/. They exist only when you work
in this repository, and are never compiled into or shipped with the Hex package.
To bump the version in mix.exs:
mix nebula_graph_ex.version
Running the task without an argument prompts for the bump type interactively. You can also specify it directly:
mix nebula_graph_ex.version patch
mix nebula_graph_ex.version minor
mix nebula_graph_ex.version major
Releasing
The nebula_graph_ex.version task also releases the changelog. It moves the
"Unreleased" notes under a new ## <version> - <date> heading and opens an
empty "Unreleased" above it. If there are no notes, it generates entries from
the commits since the last tag. Review CHANGELOG.md before committing, since
generated entries are only as clear as the commit messages they come from.
To create the matching annotated git tag after updating mix.exs:
mix nebula_graph_ex.tag
The tag task reads @version from mix.exs, creates v<version>, and refuses
to run if the git worktree is dirty, the tag already exists, or CHANGELOG.md
has no entry for the version.
Typical release flow:
mix precommit
mix docs
mix nebula_graph_ex.version minor # also releases CHANGELOG.md; review it
git add mix.exs README.md CHANGELOG.md lib test
git commit -m "Prepare 0.2.0 release"
mix nebula_graph_ex.tag
git push origin main
git push origin v0.2.0
mix hex.publish
License
MIT — see LICENSE.