ExRocket

About

ExRocket is an Elixir NIF backed by RocksDB through Rust. It provides a safe, fast API while keeping keys and values as binaries, so applications can choose their own storage format.

ExRocket is a continuation of Rocker, a RocksDB NIF for Erlang.

Looking for the complete API reference? See the ExRocket Cheatsheet for examples covering key/value operations, column families, batches, iterators, snapshots, backups, and merge operators.

Installation

Add ex_rocket to the dependencies in mix.exs:

def deps do
  [
    {:ex_rocket, "~> 0.4"}
  ]
end

Quick start

Open a database, write binary keys and values, and read them back:

db_path = Path.join(System.tmp_dir!(), "my_ex_rocket_db")

{:ok, db} = ExRocket.open(db_path, %{create_if_missing: true})

:ok = ExRocket.put(db, "greeting", "hello")
{:ok, "hello"} = ExRocket.get(db, "greeting")
:undefined = ExRocket.get(db, "missing")

:ok = ExRocket.delete(db, "greeting")

Write several changes atomically:

{:ok, 3} =
  ExRocket.write_batch(db, [
    {:put, "user:1", "alice"},
    {:put, "user:2", "bob"},
    {:delete, "obsolete"}
  ])

{:ok, [{:ok, "alice"}, {:ok, "bob"}, :undefined]} =
  ExRocket.multi_get(db, ["user:1", "user:2", "missing"])

Atomic visibility is not the same as machine-crash durability. The default batch path keeps the WAL enabled but does not request a synchronous filesystem flush. ExRocket 0.5.0 adds write_batch/3 for explicit durability boundaries:

{:ok, 1} =
  ExRocket.write_batch(
    db,
    [{:put, "materialization/checkpoint", "clean:42"}],
    %{sync: true}
  )

The write-option defaults are %{sync: false, disable_wal: false}. Do not combine sync: true with disable_wal: true; a synchronous WAL guarantee cannot be provided while the WAL is disabled. WAL-disabled writes are intended only for rebuildable bulk-load data.

Applications that deliberately group non-synchronous writes can establish a separate WAL boundary with ExRocket.flush_wal(db, true). A synchronous clean checkpoint written with write_batch/3 is usually easier to reason about.

Use a column family to keep related data separate:

:ok = ExRocket.create_cf(db, "sessions")
:ok = ExRocket.put_cf(db, "sessions", "token", "active")
{:ok, "active"} = ExRocket.get_cf(db, "sessions", "token")

Page through one stable iterator view with bounded native transfers:

{:ok, iterator} = ExRocket.iterator(db, {:start})

{:ok, rows, status} =
  ExRocket.iterator_take(iterator, %{
    max_entries: 1_000,
    max_bytes: 4 * 1024 * 1024
  })

max_entries is required and capped at 100,000. max_bytes counts key/value payload, is capped at 64 MiB, and excludes Erlang term overhead. If the first row alone exceeds the byte limit, it is returned to guarantee progress. A bound-first page returns :more; only observed exhaustion returns :end_of_iterator.

Release the database path deterministically when ownership ends:

:ok = ExRocket.close(db)
:ok = ExRocket.close(db) # idempotent
{:error, :closed} = ExRocket.get(db, "key")

A live iterator or snapshot keeps its native owner safe, so close returns {:error, :resource_busy} until that child resource is released. Successful close drops RocksDB and releases its filesystem lock before returning; normal resource garbage collection remains a fallback.

See the full cheatsheet for iterators, merge operators, snapshots, checkpoints, backups, and error-handling patterns. Configuration keys and types are defined in the canonical option reference; existing applications should follow the 0.5.0 upgrade guide.

Technology

Supported OS

Precompiled NIFs are published for:

• Apple Silicon and Intel macOS
• ARM64 and x86-64 Linux with glibc
• ARM64 and x86-64 Linux with musl, including Alpine Linux
• x86-64 Windows

Other targets can build from source with FORCE_BUILD=1 and the build requirements below.

Features

• Key/value operations
• Column families
• Atomic batch writes with explicit sync/WAL options
• Exact, validated database options
• Range, prefix, and bounded bulk iterators
• Range deletion
• Multi-get
• Snapshots
• Deterministic safe database close
• Checkpoints
• Backup API
• Counter, Erlang term, and bitset merge operators

Performance

Storage-backed NIFs run on Rustler dirty I/O schedulers so cache misses, WAL flushes, iterator walks, and native lock waits do not block normal BEAM schedulers. Dirty dispatch adds overhead to very small hot-cache calls; use iterator_take/2 to amortize NIF transitions during scans.

ExRocket's historical normal-scheduler hot-key microbenchmark measured approximately 2.46 million reads/second and 470,000 writes/second on an Apple M3 Ultra. ADR002 benchmark reports include both storage throughput and independent BEAM heartbeat latency. Real workloads vary with key/value size, cache state, durability settings, compaction, concurrency, and storage hardware.

Source build requirements

• Erlang 24 or newer
• Rust 1.91 or newer; the version is pinned in rust-toolchain.toml
• Clang 15 or newer

Validation

Run scripts/qa_check.sh for the staged Elixir, Rust, NIF smoke, and package quality gate. Successful dependency setup stays quiet; failures retain their complete diagnostic output.

CI uses source-aware Mix/Cargo caches with lockfile restore keys, per-target release caches, and no-Rust consumer jobs that compile the unpacked Hex package and load each published native artifact through the public API.

Release

1. Bump the version in mix.exs and native/rocker/Cargo.toml; verify parity with scripts/project_version.sh.
2. Run scripts/qa_check.sh and push the green version commit.
3. Dispatch Build precompiled NIFs with publish: false; verify all seven target builds and functional smoke tests.
4. Dispatch the same workflow with publish: true. It validates the exact artifact set and digests in a draft before atomically publishing the tag and GitHub release.
5. Run mix clean && mix compile, followed by mix rustler_precompiled.download ExRocket --all locally.
6. Commit the generated checksum manifest and require all source and no-Rust precompiled-consumer CI jobs to pass.
7. Inspect the unpacked package before publishing Hex.

Status

Passed all functional and performance tests.

License

ExRocket is licensed under the Apache License 2.0.