Ecto.Adapters.Recall

An Ecto 3 adapter for Mnesia — the distributed, transactional database that ships with the BEAM.

Memory, recollected.

Unlike an ETS-backed adapter, Mnesia gives you real ACID transactions and optional disk persistence (disc_copies) for free, mapping almost directly onto Ecto's Transaction and Storage behaviours. Queries are translated to Erlang match specifications and run through :mnesia.select, so where/select are pushed down to Mnesia rather than evaluated row-by-row in Elixir.

Two existing adapters cover nearby ground; Recall differs from both:

etso — an ETS-backed Ecto 3 adapter. ETS is fast and in-memory, but it has no transactions, no disk persistence, and no joins, aggregates, locking, or migrations. etso is excellent for caching rarely-changing data (and ships with assocs/preloads). Recall targets the same in-process, BEAM-native niche but is Mnesia-backed, so it adds real ACID transactions, optional disc persistence, joins, and aggregates.
ecto_mnesia — the other Mnesia adapter, but for Ecto 2.x and no longer actively maintained. It emulates select/order_by in Elixir (O(n log n)), ignores field types entirely (Mnesia stores any term), generates :id keys via a sequence table, and supports migrations and secondary indexes — but has no joins, no aggregates, and no type casting. Recall is for Ecto 3, pushes where/select down into match specs instead of emulating them, casts types through Ecto's loaders/dumpers, implements joins and aggregates, and supports declarative secondary indexes — at the cost of (currently) no migrations.

Installation

Add recall to your dependencies:

def deps do
  [
    {:recall, "~> 0.1.0"}
  ]
end

Usage

Define a repo as usual, pointing it at the adapter:

defmodule MyApp.Repo do
  use Ecto.Repo, otp_app: :my_app, adapter: Ecto.Adapters.Recall
end

Define schemas with Recall.Schema (a drop-in replacement for use Ecto.Schema), naming the table with an atom — the form a Mnesia table actually takes:

defmodule MyApp.User do
  use Recall.Schema

  schema :users do
    field :name, :string
    field :age, :integer
  end
end

Everything Ecto.Schema provides (the struct, __schema__/1, changesets, associations) keeps working untouched — this only adds the compiled accessors the adapter uses (see Why a dedicated schema macro). The schema is purely logical: a table's type and any secondary indexes are set in migrations, not on the schema.

import Ecto.Query

{:ok, user} = MyApp.Repo.insert(%MyApp.User{name: "Ada", age: 36})

MyApp.Repo.all(from u in MyApp.User, where: u.age > 30, select: u.name)
#=> ["Ada"]

MyApp.Repo.transaction(fn ->
  MyApp.Repo.insert!(%MyApp.User{name: "Grace"})
  # ...real ACID transaction; raise or Repo.rollback/1 to abort
end)

Tables are created lazily on first use with the configured storage type, so you can get going without migrations. Real apps (especially disc-backed ones) manage their schema with migrations.

Configuration

Set on the repo config (e.g. in config/config.exs):

config :my_app, MyApp.Repo,
  storage: :ram_copies,   # or :disc_copies, :disc_only_copies
  type: :ordered_set,     # or :set
  nodes: [node()],        # nodes that hold copies
  dir: "priv/mnesia"      # Mnesia data directory (disc storage only)

Option	Default	Description
`:storage`	`:ram_copies`	Where auto-created tables live: `:ram_copies`, `:disc_copies`, or `:disc_only_copies`.
`:type`	`:ordered_set`	`:ordered_set` keeps records in primary-key order, so reads come back sorted. Falls back to `:set` for `:disc_only_copies` (Mnesia disallows `:ordered_set` there).
`:nodes`	`[node()]`	Nodes that hold table copies.
`:dir`	—	Mnesia data directory (only relevant for disc storage).

For disc-backed storage, mix ecto.create / mix ecto.drop create and delete the Mnesia schema on the configured nodes.

Migrations

Migrations are written like any Ecto app — create / alter / drop in a migration module — with one difference: run them with the Recall tasks rather than mix ecto.migrate.

mix recall.migrate            # run pending migrations
mix recall.rollback --step 1  # roll back

defmodule MyApp.Repo.Migrations.CreateUsers do
  use Ecto.Migration

  def change do
    create table(:users) do
      add :name, :string
      add :age, :integer
    end

    create index(:users, [:age])   # a secondary index the planners can use
  end
end

Supported: create / create_if_not_exists / drop / drop_if_exists for tables and indexes; alter table with add / remove / modify; and column and table renames. Mnesia is typeless, so column types are ignored — only the attribute names, which one is the primary key (Mnesia's key is the first attribute), and which carry an index matter — and modify is a no-op. An alter rewrites every existing record in place via :mnesia.transform_table, defaulting any added column.

Raised loudly rather than silently mishandled: raw SQL (execute "..."), composite primary keys, changing or removing the primary key, unique indexes (Mnesia has no unique secondary constraint — use the primary key for uniqueness), and Postgres-style prefixes. A composite index(:t, [:a, :b]) becomes one single-attribute index per column (with a warning), since Mnesia indexes are single-attribute.

Why a dedicated task? Stock mix ecto.migrate tracks applied versions by querying schema_migrations with a string source (from m in "schema_migrations"), and this adapter only serves schema-backed tables (see Limitations). mix recall.migrate runs the very same migrations through Ecto's migration engine, but does version bookkeeping through a real schema (Recall.SchemaMigration) so it stays within that rule. Both tasks accept --step, --to, --all, and -r MyApp.Repo.

Why a dedicated schema macro

Recall.Schema is required — a plain use Ecto.Schema schema will not work with this adapter.

Ecto.Schema is built for SQL, where a query carries field names and the database resolves them. Recall stores every field as a positional Mnesia attribute, so the macro bakes the schema's logical shape — the table tag, field names in storage order, the struct, and a load recipe — into compiled __recall__/1 accessors, and the adapter reads field metadata only through them (a plain use Ecto.Schema has no fallback path). A field's physical position is resolved at runtime from the table's live layout in Mnesia's catalog (cached in :persistent_term), so a migration that reshapes a table is honored on the next read, and reordering fields in source can't silently misalign storage.

It also requires the table name to be an atom (schema :users, not schema "users") — the form a Mnesia table actually takes. __schema__(:source) still returns the equivalent string "users" for the rest of Ecto, so changesets, associations, and the struct are unaffected.

Secondary indexes

A table's built-in index is its primary key. Additional secondary indexes are physical — created by a migration (create index(:users, [:age])), never declared on the schema or built on the fly. The read and join planners serve an equality (or in) on the primary key, or on any attribute that is physically indexed, with :mnesia.read / :mnesia.index_read instead of a full-table scan; an equality on any other field still runs correctly — it just takes the scan path. The full where is re-checked against the fetched rows either way, so results are identical to a scan.

What works

Insert / get / update / delete, insert_all (with placeholders and :returning)
Upserts / on_conflict (:raise, :nothing, :replace_all, :replace_all_except, {:replace, fields})
where (pushed into match specs), order_by, limit / offset, in, is_nil
Key/index reads on where — an equality or in that pins a field to a known value (Repo.get/3, u.id == ^id, u.email == ^e, u.id in ^ids) skips the full-table scan: a primary-key match becomes an O(1) :mnesia.read, and a match on an indexed field uses :mnesia.index_read. The full where is still re-checked against the fetched rows, so results are identical to a scan — just fewer rows touched. Applies to all, update_all, and delete_all (stream stays on the cursor-based scan path). (Disable with config :recall, force_where_scan: true.)
Joins — inner / left / right / cross, across N tables, with a tiered right-side fetch (primary-key read, index_read on an indexed field, or full scan) and where pushdown
Joined update_all / delete_all
Single aggregates: count, sum, avg, min, max (incl. count(field, :distinct))
Computed select expressions: date/time arithmetic (date_add / datetime_add / from_now / ago), type/2 casts, and the operators + - * == != < > <= >= and or not is_nil (evaluated in Elixir during projection)
Real ACID transactions and Repo.rollback/1
Optimistic locking, idempotent delete, field ..., source: mapping
Repo.stream/2 (lazily within a transaction; eagerly otherwise)
Guard fragment/1s as an escape hatch to native Erlang guards (see below)

Match-spec guard fragments

A match spec runs Erlang guards against each record, and guards reach things SQL has no notion of: type tests, structural access into a term, term ordering across heterogeneous types. To expose those, a fragment/1 whose body is a guard expression is translated into the match-spec conditions and pushed down with the rest of the where:

import Ecto.Query

# type test — filter on the *shape* of a term, not its value
from u in User, where: fragment("is_map(?)", u.metadata)

# structural access into a map-typed column
from u in User, where: fragment("map_get(?, ?)", ^"role", u.metadata) == ^"admin"

# arithmetic / bit guards
from u in User, where: fragment("rem(?, ?)", u.id, ^2) == 0

# anything expressible as an Erlang guard
from u in User, where: fragment("length(?) > ?", u.tags, ^min_tags)

This is not the SQL-fragment escape hatch. A SQL adapter passes the fragment string to the database verbatim — there, the string is the query. Mnesia has no query string: :mnesia.select/2 takes an Erlang term ({head, conditions, body}), so the fragment is parsed and translated into guard tuples (e.g. is_map(?) → {:is_map, :"$5"}), once per query build. The ? placeholders bind to fields or interpolated ^values exactly as in any fragment.

Only the Erlang guard BIFs (is_*, element, map_get, map_size, byte_size, length, rem, abs, the bit operators, node, self, …) and comparison/boolean operators are allowed. Anything else — an unknown name, or a real BIF that isn't a valid match-spec guard — raises at query time rather than building a spec Mnesia would reject or smuggling an arbitrary call into the match. Fragments are supported in where, not in select (the match-spec body has a narrower operation set, and select is projected in Elixir regardless).

Demo app

A small, runnable example lives in examples/blog/ — a blog (authors → posts → comments) backed entirely by this adapter, no external database. From the repo root:

cd examples/blog
mix deps.get
mix blog.demo

mix blog.demo migrates, seeds sample data, and prints a guided tour of an indexed-field index_read, where/order_by/limit pushdown, a posts ⨝ authors join, count/sum/avg aggregates, a committed transaction, a Repo.rollback/1, and — to make the storage model concrete — the raw Mnesia tuples next to the same rows loaded back as Ecto structs. See examples/blog/README.md for the breakdown.

Limitations

Unsupported query clauses raise at query-prepare time rather than silently dropping (so you never get wrong rows back):

group_by / having / distinct / window functions
set operations (union / except / intersect) and CTEs (with_cte)

Other gaps, fundamental to Mnesia or not yet implemented:

Non-primary-key unique constraints are not enforced — only primary-key uniqueness is checked. (Mnesia has no unique secondary index, so unique_index/2 in a migration raises rather than silently allowing duplicates.)
No composite primary keys — a Mnesia key is a single attribute.
Single-node ID generation.autogenerate(:id) uses :erlang.unique_integer/1, which is unique per node; use binary_id for multi-node clusters.
Schemaless queries are not supported — from("posts", ...) with a string source (no schema) has no compiled field layout to resolve against; use a schema module.
Not supported in select: aggregate expressions wrapped in type/2 (e.g. type(sum(x), :integer)), fragments, selected_as, and VALUES lists. (The arithmetic/comparison/date-time expressions listed under What worksare supported.)
Not supported: subqueries (in where / select / as a source), insert_all from a query source, database-level constraints / foreign keys, Postgres prefixes. Fragments are supported only as guard expressions in where (see Match-spec guard fragments) — not as raw SQL and not in select.

License

MIT.