Hex.pm

Note: This library is under active development and the API may change.

AshScylla

An Ash Framework data layer for ScyllaDB/Apache Cassandra

Quick StartFeaturesDocumentationContributingLicense


Overview

AshScylla enables you to use ScyllaDB or Apache Cassandra as a persistence layer for your Ash Framework resources. It implements the Ash.DataLayer behaviour using Xandra (a native Elixir CQL driver) to communicate via CQL (Cassandra Query Language).

Current version: 0.13.1

Key Benefits


Quick Start

Prerequisites

Installation

Add ash_scylla to your dependencies in mix.exs:

def deps do
[
{:ash_scylla, "~> 0.13"}
]
end

Minimal Setup

# 1. Configure a Repo
defmodule MyApp.Repo do
use AshScylla.Repo, otp_app: :my_app
end
# 2. Configure in config/config.exs
config :my_app, MyApp.Repo,
nodes: ["127.0.0.1:9042"],
keyspace: "my_app_dev",
pool_size: 10
# 3. Add to your supervision tree
# lib/my_app/application.ex
children = [MyApp.Repo, ...]
# 4. Create a resource
defmodule MyApp.User do
use Ash.Resource,
data_layer: AshScylla.DataLayer,
domain: MyApp.Domain
scylla do
table "users"
consistency :quorum
end
attributes do
uuid_primary_key :id
attribute :name, :string
attribute :email, :string
end
actions do
defaults [:create, :read, :update, :destroy]
end
end
# 5. Create a Domain
defmodule MyApp.Domain do
use Ash.Domain
resources do
resource MyApp.User
end
end
# 6. Create keyspace and run migrations
# mix ash_scylla.setup
# mix ash_scylla.migrate
# 7. Use it
{:ok, user} = Ash.create(MyApp.User, %{name: "John", email: "john@example.com"})
users = MyApp.User |> Ash.read!()

For a complete step-by-step guide, see the Usage Guide.


Features

Core Ash Features

FeatureStatusDescription
CreateInsert records with TTL support
ReadQuery with filtering and sorting
UpdateUpdate existing records
DestroyDelete records
FilterPowerful filter syntax with CQL WHERE conversion
SortORDER BY on clustering columns (within partition)
Keyset paginationToken-based pagination via paging_state (default mode)
LimitLIMIT is natively supported

| Select | ✅ | Select specific fields | | Multitenancy | ✅ | Keyspace-based multitenancy | | Bulk Create | ✅ | Batch INSERT operations | | Upsert | ✅ | INSERT with lightweight transactions (LWT) | | Update Query | ✅ | Bulk update via filtered queries | | Destroy Query | ✅ | Bulk delete via filtered queries | | Distinct | ✅ | DISTINCT on partition key columns | | Calculate | ✅ | In-memory calculations | | Aggregate (count) | ✅ | Per-partition COUNT |

ScyllaDB-Specific Features

FeatureDescription
TTLAuto-expire data after a specified time
Consistency LevelsPer-resource or per-action consistency (:one, :quorum, :all, etc.)
Secondary IndexesQuery non-primary key columns efficiently
Materialized ViewsAlternative query patterns with automatic view maintenance
Batch OperationsBATCH INSERT/UPDATE/DELETE, including async partition-aware batching
Token-Based PaginationEfficient pagination via Xandra's native paging_state
Lightweight TransactionsIF NOT EXISTS on create, IF clauses on update
CompressionApplication-level compression (LZ4, Snappy, Deflate, Zstd)
User Defined TypesFull UDT encoding/decoding and CQL generation
Collection TypesLIST, SET, MAP with CONTAINS filter support
Prepared Statement CachingETS-based cache for high-throughput workloads

Configuration

Resource Configuration

defmodule MyApp.User do
use Ash.Resource,
data_layer: AshScylla.DataLayer,
domain: MyApp.Domain
scylla do
table "users"
consistency :quorum
ttl 3600
lwt true
secondary_index :email
secondary_index [:name, :age]
materialized_view :users_by_email,
primary_key: [:email, :id],
include_columns: [:name, :age]
per_action_consistency read: :one, create: :quorum
end
end

Repo Configuration

# Single-node
config :my_app, MyApp.Repo,
nodes: ["127.0.0.1:9042"],
keyspace: "my_app_dev"
# Multi-node cluster (all nodes must use the same port)
config :my_app, MyApp.Repo,
nodes: ["scylla-1:9042", "scylla-2:9042", "scylla-3:9042"],
keyspace: "my_app_prod",
pool_size: 50

Pool Size Formula:pool_size = num_nodes * num_cores_per_node


Limitations

LimitationWorkaround
No JOINsDenormalize or application-side joins
No complex aggregationsMaterialized views or custom aggregation
No ACID transactionsUse LWT for single-partition operations
Limited WHERE without indexesCreate secondary indexes or materialized views
No OFFSETUse keyset pagination via paging_state (default mode)
Cluster requires same portConfigure all nodes on the same port, or use single-node connection

Observability

Telemetry

AshScylla emits standard :telemetry events for all query and batch operations:

:telemetry.attach(
"ash_scylla-logger",
[:ash_scylla, :query, :stop],
&MyApp.Telemetry.handle_event/4,
nil
)

Events:[:ash_scylla, :query, :start|stop|exception], [:ash_scylla, :batch, :start|stop]

Prepared Statement Caching

children = [
AshScylla.PreparedStatementCache,
# ...
]

Documentation

DocumentDescription
Usage GuideComprehensive guide: setup, CRUD, querying, data modeling, migrations
Development GuideDev container setup, testing, type mapping, CQL query building
Production GuideMulti-node cluster deployment, monitoring, backup, rolling upgrades
Implementation SummaryTechnical architecture and module reference
Error HandlingError types, common scenarios
ChangelogVersion history and release notes
API DocumentationModule documentation (when published)

Common Commands

# ── Testing ──────────────────────────────────────────────────────────────────
mix test --exclude integration # Unit tests only (no database)
mix test --only integration # Integration tests (needs ScyllaDB)
SCYLLA_DIRECT=1 mix test --only integration # Integration tests against local DB
mix test test/integration/cluster_integration_test.exs --only integration # Cluster tests
mix test --exclude integration --cover # Unit tests + coverage report
# ── Code Quality ─────────────────────────────────────────────────────────────
mix format --check-formatted # Check formatting
mix credo --strict # Static analysis
mix dialyzer # Type checking
mix quality # All three above
# ── Benchmarks ───────────────────────────────────────────────────────────────
mix run benchmarks/run_benchmarks.exs
# ── Database ─────────────────────────────────────────────────────────────────
mix ash_scylla.setup # Create keyspace
mix ash_scylla.migrate # Run all migrations
mix ash_scylla.migrate --schemas-only # Run only schema files
mix ash_scylla.migrate --resource MyApp.User # Run migrations for one resource
mix ash_scylla.gen --dev # Generate schema migration from DSL
# ── Schema Generation ────────────────────────────────────────────────────────
mix ash_scylla.generate_migrations # Generate CQL from Ash resource DSL
# ── Ash Extension Callbacks ──────────────────────────────────────────────────
mix ash.install AshScylla --resource MyApp.User # Install for a resource
mix ash.reset AshScylla # Reset database
mix ash.rollback AshScylla --version 20240101 # Rollback (logs warning)
mix ash.tear_down AshScylla # Drop keyspace

Contributing

Contributions are welcome!

  1. Fork the repository
  2. Clone your fork
  3. Create a feature branch: git checkout -b feature/my-feature
  4. Make your changes
  5. Run tests: mix test --exclude integration
  6. Check quality: mix quality
  7. Commit and push
  8. Open a Pull Request

Development Setup

mix deps.get
podman-compose -f podman-compose.yml up -d
mix test

A .devcontainer/devcontainer.json is provided for VS Code Dev Containers.


License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.


Acknowledgments


Made with ❤️ for the Elixir and Ash communities