ExScylla

An Elixir wrapper around the scylla-rust-driver using Rustler.

Documentation and API examples are available at: https://hexdocs.pm/ex_scylla Technical architecture and design docs can be found in the docs/ folder. Agent instructions and guidelines are located in AGENTS.md.

Prerequisites

Architecture & Performance

NIF/Rustler & Tokio Runtime

ExScylla uses Rustler to interface with the scylla-rust-driver. Because the Rust driver is heavily asynchronous and relies on the Tokio runtime, ExScylla spins up a dedicated Tokio runtime in the background to handle the driver's futures.

Resource Reference Model & Cloning

SessionBuilder, Session, and statements (Batch, Query, Prepared) use Arc references to the underlying Rust objects. These references are passed to Elixir as NIF resources.

Performance Note: Statement setters (Query, Prepared, Batch) return a new reference to an updated copy (cloning the underlying Rust struct). As old instances are not immediately removed (waiting for Elixir GC), these setters should be avoided in the hot path. It is better to create and configure a statement once, then reuse it multiple times. Builders (SessionBuilder, ExecutionProfileBuilder, DefaultPolicyBuilder, LatencyAwarenessBuilder) are different: their setters mutate the builder in place and return the same reference.

Async API & Timeouts

The library uses asynchronous NIFs for non-blocking operations to avoid starving the Erlang schedulers. An async NIF returns immediately and later sends {opaque, result} to the calling process. The async_* functions have no timeout. The synchronous wrappers (for example Session.query/3) wait for the reply with a default timeout of 5 seconds, set by an optional trailing timeout_ms argument.

Many requests from one process: replies are sent from several Tokio threads at once. A process that keeps many async_* requests in flight should call Process.flag(:message_queue_data, :off_heap), which avoids mailbox lock contention (about 15–20% faster for 1000 concurrent inserts in our benchmarks).

Dual API (Decoded vs Raw)

Both APIs call the same NIFs. Rust always returns the result rows as a raw binary, and the two APIs differ only in how Elixir decodes it:

Example

This example assumes a running ScyllaDB node and an existing test.s_doc table with columns a text, b int, c double.

alias ExScylla.SessionBuilder
alias ExScylla.Session
alias ExScylla.Types.QueryResult

{:ok, session} = SessionBuilder.new()
                 |> SessionBuilder.known_node("127.0.0.1:9042")
                 |> SessionBuilder.build()
{:ok, ps} = Session.prepare(session, "INSERT INTO test.s_doc (a, b, c) VALUES (?, ?, ?)")
values = [{:text, "test"}, {:int, 2}, {:double, 1.0}]
{:ok, %QueryResult{}} = Session.execute(session, ps, values)

# Iterative queries (streams)
Session.query_stream(session, "SELECT * FROM test.s_doc", [])
|> Enum.each(fn row -> IO.inspect(row) end)

Installation

https://hex.pm/packages/ex_scylla

def deps do
  [
    {:ex_scylla, "~> 0.11.0"}
  ]
end

Repository Blueprint

System Overview

ExScylla has two layers that must stay in sync:

  1. Elixir API (lib/ex_scylla): SessionBuilder, Session, statements, execution profiles, load-balancing builders, result types, and CqlTypes row decoding. Wrappers are generated by ExScylla.Macros.Native and call stubs in ExScylla.Native.
  2. Rust NIF crate (native/ex_scylla): a Rustler cdylib loaded as Elixir.ExScylla.Native. It wraps the scylla 1.10.0 driver objects as ResourceArc resources and runs all network I/O on one shared Tokio runtime.

Request flow for an I/O call such as Session.query/3:

  1. The Elixir wrapper calls ExScylla.Native.s_query(opaque, ...). The default opaque is {function_name, make_ref()}.
  2. The NIF decodes the arguments synchronously, spawns a future through async_elixir!, and returns :ok right away.
  3. When the future finishes, the caller receives {opaque, result}. The synchronous wrapper waits up to timeout_ms (default 5000), then returns {:error, :timeout}.
  4. Result rows reach Elixir as a raw, resource-backed binary (ScyllaRawRowsResource). ExScylla.Types.QueryResult.decode/1 (decoded API) or decode_raw/1 (raw API) parses it in Elixir. Both APIs use the same NIFs.

Local Interface & Invariants

Subdirectories & System Boundaries

Global Invariants

Global Side Effects

System Verification

Contributing

We welcome contributions! Please see the CONTRIBUTING.md file for guidelines on how to contribute to this project.

Changelog

For a detailed list of changes and updates, please refer to the CHANGELOG.md file.