Note: This library is under active development and the API may change.
AshScylla
An Ash Framework data layer for ScyllaDB/Apache Cassandra
Quick Start • Features • Documentation • Contributing • License
Overview
AshScylla enables you to use ScyllaDB or Apache Cassandra as a persistence layer for your Ash Framework resources. It implements the Ash.DataLayer behaviour using Xandra (a native Elixir CQL driver) to communicate via CQL (Cassandra Query Language).
Current version: 0.12.0
Key Benefits
- Seamless Ash Integration: Use familiar Ash resources, actions, and queries
- ScyllaDB Performance: Leverage ScyllaDB's high-performance, low-latency architecture
- Cassandra Compatibility: Works with Apache Cassandra and ScyllaDB
- Rich Feature Set: TTL, consistency levels, secondary indexes, materialized views, batch operations, lightweight transactions
Quick Start
Prerequisites
- Elixir 1.17+
- Running ScyllaDB or Cassandra instance
- Basic knowledge of Ash Framework
Installation
Add ash_scylla to your dependencies in mix.exs:
def deps do
[
{:ash_scylla, "~> 0.12"}
]
end
Minimal Setup
# 1. Configure a Repo
defmodule MyApp.Repo do
use AshScylla.Repo, otp_app: :my_app
end
# 2. Configure in config/config.exs
config :my_app, MyApp.Repo,
nodes: ["127.0.0.1:9042"],
keyspace: "my_app_dev",
pool_size: 10
# 3. Add to your supervision tree
# lib/my_app/application.ex
children = [MyApp.Repo, ...]
# 4. Create a resource
defmodule MyApp.User do
use Ash.Resource,
data_layer: AshScylla.DataLayer,
domain: MyApp.Domain
ash_scylla do
table "users"
consistency :quorum
end
attributes do
uuid_primary_key :id
attribute :name, :string
attribute :email, :string
end
actions do
defaults [:create, :read, :update, :destroy]
end
end
# 5. Create a Domain
defmodule MyApp.Domain do
use Ash.Domain
resources do
resource MyApp.User
end
end
# 6. Create keyspace and run migrations
# mix ash_scylla.setup
# mix ash_scylla.migrate
# 7. Use it
{:ok, user} = Ash.create(MyApp.User, %{name: "John", email: "john@example.com"})
users = MyApp.User |> Ash.read!()
For a complete step-by-step guide, see the Usage Guide.
Features
Core Ash Features
| Feature | Status | Description |
|---|---|---|
| Create | ✅ | Insert records with TTL support |
| Read | ✅ | Query with filtering and sorting |
| Update | ✅ | Update existing records |
| Destroy | ✅ | Delete records |
| Filter | ✅ | Powerful filter syntax with CQL WHERE conversion |
| Sort | ✅ | ORDER BY on clustering columns (within partition) |
| Keyset pagination | ✅ | Token-based pagination via paging_state (default mode) |
| Limit | ✅ | LIMIT is natively supported |
| Offset | ❌ | Raises error — use keyset pagination instead |
| Select | ✅ | Select specific fields |
| Multitenancy | ✅ | Keyspace-based multitenancy |
| Bulk Create | ✅ | Batch INSERT operations |
| Upsert | ✅ | INSERT with lightweight transactions (LWT) |
| Update Query | ✅ | Bulk update via filtered queries |
| Destroy Query | ✅ | Bulk delete via filtered queries |
| Distinct | ✅ | DISTINCT on partition key columns |
| Calculate | ✅ | In-memory calculations |
| Aggregate (count) | ✅ | Per-partition COUNT |
ScyllaDB-Specific Features
| Feature | Description |
|---|---|
| TTL | Auto-expire data after a specified time |
| Consistency Levels | Per-resource or per-action consistency (:one, :quorum, :all, etc.) |
| Secondary Indexes | Query non-primary key columns efficiently |
| Materialized Views | Alternative query patterns with automatic view maintenance |
| Batch Operations | BATCH INSERT/UPDATE/DELETE, including async partition-aware batching |
| Token-Based Pagination | Efficient pagination via Xandra's native paging_state |
| Lightweight Transactions | IF NOT EXISTS on create, IF clauses on update |
| Compression | Application-level compression (LZ4, Snappy, Deflate, Zstd) |
| User Defined Types | Full UDT encoding/decoding and CQL generation |
| Collection Types | LIST, SET, MAP with CONTAINS filter support |
| Prepared Statement Caching | ETS-based cache for high-throughput workloads |
Configuration
Resource Configuration
defmodule MyApp.User do
use Ash.Resource,
data_layer: AshScylla.DataLayer,
domain: MyApp.Domain
ash_scylla do
table "users"
consistency :quorum
ttl 3600
lwt true
secondary_index :email
secondary_index [:name, :age]
materialized_view :users_by_email,
primary_key: [:email, :id],
include_columns: [:name, :age]
per_action_consistency read: :one, create: :quorum
end
end
Repo Configuration
# Single-node
config :my_app, MyApp.Repo,
nodes: ["127.0.0.1:9042"],
keyspace: "my_app_dev"
# Multi-node cluster (all nodes must use the same port)
config :my_app, MyApp.Repo,
nodes: ["scylla-1:9042", "scylla-2:9042", "scylla-3:9042"],
keyspace: "my_app_prod",
pool_size: 50
Pool Size Formula:pool_size = num_nodes * num_cores_per_node
Limitations
| Limitation | Workaround |
|---|---|
| No JOINs | Denormalize or application-side joins |
| No complex aggregations | Materialized views or custom aggregation |
| No ACID transactions | Use LWT for single-partition operations |
| Limited WHERE without indexes | Create secondary indexes or materialized views |
| No OFFSET | Use keyset pagination (data_layer_keyset_by_default?/0 returns true) |
| Cluster requires same port | Configure all nodes on the same port, or use single-node connection |
Observability
Telemetry
AshScylla emits standard :telemetry events for all query and batch operations:
:telemetry.attach(
"ash_scylla-logger",
[:ash_scylla, :query, :stop],
&MyApp.Telemetry.handle_event/4,
nil
)
Events:[:ash_scylla, :query, :start|stop|exception], [:ash_scylla, :batch, :start|stop]
Prepared Statement Caching
children = [
AshScylla.PreparedStatementCache,
# ...
]
Documentation
| Document | Description |
|---|---|
| Usage Guide | Comprehensive guide: setup, CRUD, querying, data modeling, migrations |
| Development Guide | Dev container setup, testing, type mapping, CQL query building |
| Production Guide | Multi-node cluster deployment, monitoring, backup, rolling upgrades |
| Implementation Summary | Technical architecture and module reference |
| Error Handling | Error types, retry logic, common scenarios |
| Changelog | Version history and release notes |
| API Documentation | Module documentation (when published) |
Common Commands
# ── Testing ──────────────────────────────────────────────────────────────────
mix test --exclude integration # Unit tests only (no database)
mix test --only integration # Integration tests (needs ScyllaDB)
SCYLLA_DIRECT=1 mix test --only integration # Integration tests against local DB
mix test test/integration/cluster_integration_test.exs --only integration # Cluster tests
mix test --exclude integration --cover # Unit tests + coverage report
# ── Code Quality ─────────────────────────────────────────────────────────────
mix format --check-formatted # Check formatting
mix credo --strict # Static analysis
mix dialyzer # Type checking
mix quality # All three above
# ── Benchmarks ───────────────────────────────────────────────────────────────
mix run benchmarks/run_benchmarks.exs
# ── Database ─────────────────────────────────────────────────────────────────
mix ash_scylla.setup # Create keyspace
mix ash_scylla.migrate # Run all migrations
mix ash_scylla.migrate --schemas-only # Run only schema files
mix ash_scylla.migrate --resource MyApp.User # Run migrations for one resource
mix ash_scylla.gen --dev # Generate schema migration from DSL
Contributing
Contributions are welcome!
- Fork the repository
- Clone your fork
- Create a feature branch:
git checkout -b feature/my-feature - Make your changes
- Run tests:
mix test --exclude integration - Check quality:
mix quality - Commit and push
- Open a Pull Request
Development Setup
mix deps.get
podman-compose -f podman-compose.yml up -d
mix test
A .devcontainer/devcontainer.json is provided for VS Code Dev Containers.
License
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
Acknowledgments
- Ash Framework - The Elixir framework this data layer integrates with
- Xandra - Native Elixir CQL driver for ScyllaDB/Cassandra
- ScyllaDB - High-performance NoSQL database
Made with ❤️ for the Elixir and Ash communities