slate

Type-safe Gleam wrapper for Erlang DETS (Disk Erlang Term Storage).

DETS provides persistent key-value storage backed by files on disk. Tables survive process crashes and node restarts. DETS is built into OTP — no external database or dependency is needed.

Important

Erlang target only — DETS is a BEAM feature with no JavaScript target support.

When to use DETS

Approach	Complexity	Persistence	Query capability
JSON file	Low	Yes	None
DETS	Low	Yes	Key lookup, fold
SQLite/Postgres	High	Yes	Full SQL
Mnesia	High	Yes	Transactions, distribution

DETS fills the gap between "serialize to a file" and "add a database dependency."

Installation

gleam add slate

Usage

Set tables (one value per key)

import gleam/dynamic/decode
import slate/set

pub fn main() {
  // Open or create a table
  let assert Ok(users) = set.open("data/users.dets",
    key_decoder: decode.string, value_decoder: decode.int)

  // Insert key-value pairs
  let assert Ok(Nil) = set.insert(users, "alice", 42)
  let assert Ok(Nil) = set.insert(users, "bob", 37)

  // Look up values
  let assert Ok(age) = set.lookup(users, key: "alice")
  // age == 42

  // Check membership
  let assert Ok(True) = set.member(users, key: "alice")
  let assert Ok(False) = set.member(users, key: "charlie")

  // Always close when done
  let assert Ok(Nil) = set.close(users)
}

Safe table lifecycle with `with_table`

import gleam/dynamic/decode
import slate
import slate/set

pub fn main() {
  // Table is closed after the callback returns
  let assert Ok(Nil) = set.with_table("data/config.dets",
    repair: slate.AutoRepair, access: slate.ReadWrite,
    key_decoder: decode.string, value_decoder: decode.string,
    callback: fn(table) {
      set.insert(table, "theme", "dark")
    })
}

Use with_table for short-lived operations. Select the repair policy and access mode for each call. The helper closes the table when the callback returns and attempts cleanup if the callback raises. If the owning process is terminated before cleanup runs, DETS may still need repair on the next open.

For read-only access without automatic repair:

import gleam/dynamic/decode
import slate
import slate/set

use table <- set.with_table(
  path: "data/config.dets", repair: slate.NoRepair, access: slate.ReadOnly,
  key_decoder: decode.string, value_decoder: decode.string)
set.lookup(table, key: "theme")

ReadOnly requires an existing file. This helper is available in all three table modules.

Migrating from 1.x:with_table now requires repair and access. Add repair: slate.AutoRepair and access: slate.ReadWrite to keep the previous behavior. Cleanup and error handling are unchanged.

Bag tables (multiple values per key)

import gleam/dynamic/decode
import slate/bag

pub fn main() {
  let assert Ok(tags) = bag.open("data/tags.dets",
    key_decoder: decode.string, value_decoder: decode.string)

  let assert Ok(Nil) = bag.insert(tags, "color", "red")
  let assert Ok(Nil) = bag.insert(tags, "color", "blue")

  let assert Ok(colors) = bag.lookup(tags, key: "color")
  // colors == ["red", "blue"]

  let assert Ok(Nil) = bag.close(tags)
}

Duplicate bag tables

import gleam/dynamic/decode
import slate/duplicate_bag

pub fn main() {
  let assert Ok(events) = duplicate_bag.open("data/events.dets",
    key_decoder: decode.string, value_decoder: decode.string)

  let assert Ok(Nil) = duplicate_bag.insert(events, "click", "button_a")
  let assert Ok(Nil) = duplicate_bag.insert(events, "click", "button_a")

  let assert Ok(clicks) = duplicate_bag.lookup(events, key: "click")
  // clicks == ["button_a", "button_a"]

  let assert Ok(Nil) = duplicate_bag.close(events)
}

Data persists across restarts

import gleam/dynamic/decode
import slate/set

pub fn write() {
  let assert Ok(table) = set.open("data/state.dets",
    key_decoder: decode.string, value_decoder: decode.int)
  let assert Ok(Nil) = set.insert(table, "counter", 42)
  let assert Ok(Nil) = set.close(table)
}

pub fn read() {
  let assert Ok(table) = set.open("data/state.dets",
    key_decoder: decode.string, value_decoder: decode.int)
  let assert Ok(42) = set.lookup(table, key: "counter")
  let assert Ok(Nil) = set.close(table)
}

Error handling

Most public operations return Result(_, slate.DetsError).

slate/set.update_counter returns Result(_, set.UpdateCounterError) so it can add the operation-specific set.CounterValueNotInteger case without widening the shared slate.DetsError contract for unrelated APIs.

Match on the specific variants you expect in normal flows, and use the helper functions when you want a stable code or a user-facing message:

import slate
import slate/set

case set.lookup(table, key: "missing") {
  Ok(value) -> Ok(value)
  Error(slate.NotFound) -> Ok(default_value)
  Error(error) -> {
    let code = slate.error_code(error)
    let message = slate.error_message(error)
    // log code/message here
    Error(error)
  }
}

UnexpectedError(detail) is intended for diagnostics only; the detail string is not a stable API contract, and error_message intentionally returns a generic message for that variant.

When opening existing files, Error(slate.NotADetsFile(context)) means the path is readable but not a DETS file, and Error(slate.NeedsRepair(context)) means the file was not closed cleanly and you opened it with NoRepair.

File-related errors carry FileErrorContext(path: Option(String), reason: String). The path comes from OTP; None means OTP supplied no single filename. The reason retains lower-level diagnostics, including nested OTP reasons. Both fields are for trusted diagnostics only and can contain sensitive data. error_code and error_message do not include these details and their outputs are unchanged.

Breaking migration:FileNotFound, AlreadyOpen, AccessDenied, TypeMismatch, NeedsRepair, NotADetsFile, and FileSizeLimitExceeded now require context. Change patterns such as Error(slate.AccessDenied) to Error(slate.AccessDenied(_)), or bind context to inspect it. See the migration guide for before/after examples, including code that constructs errors. No DETS file migration is needed.

For set.update_counter, match Error(set.CounterValueNotInteger) directly and unwrap shared table failures as Error(set.TableError(error)).

API Overview

The three table types (set, bag, duplicate_bag) share a common core API:

Function	Description
`open(path, key_decoder, value_decoder)`	Open or create a table
`open_with(path, repair, key_decoder, value_decoder)`	Open with repair policy
`open_with_access(path, repair, access, key_decoder, value_decoder)`	Open with repair and access mode
`close(table)`	Close and flush to disk
`sync(table)`	Flush without closing
`with_table(path, repair, access, key_decoder, value_decoder, fn)`	Auto-closing callback with repair and access options
`insert(table, key, value)`	Insert a key-value pair
`insert_list(table, entries)`	Batch insert
`lookup(table, key)`	Get value(s) for key
`member(table, key)`	Check if key exists
`delete_key(table, key)`	Remove by key
`delete_object(table, key, value)`	Remove a specific key-value pair (`duplicate_bag` removes all exact duplicates)
`delete_all(table)`	Clear all entries
`to_list(table)`	Get all entries
`fold(table, acc, fn)`	Fold over entries
`size(table)`	Count entries
`info(table)`	Get the file size, object count, and absolute file path

Use info(table) to retrieve the file path from an open handle:

let assert Ok(info) = set.info(table)
info.file_path

Migrating from 1.x:TableInfo now has a file_path: String field. Supply it when constructing the record. Include the third field, or use .., when matching the constructor. Existing info.file_size and info.object_count access remains valid.

slate/set also provides:

Function	Description
`insert_new(table, key, value)`	Insert if key is absent
`update_counter(table, key, amount)`	Atomic counter increment

slate/bag also provides:

Function	Description
`insert_new(table, key, value)`	Reject an exact duplicate key-value pair (best-effort under concurrent shared access)

The top-level slate module also provides:

Function	Description
`is_dets_file(path)`	Check if a file is a valid DETS file
`error_code(error)`	Stable machine-readable error code
`error_message(error)`	User-facing error message

Limitations

2 GB maximum file size per table — a hard limit in DETS
No ordered_set — DETS only supports set, bag, and duplicate_bag
Disk I/O on every operation — for high-frequency reads, load into ETS at startup
Must close properly — with_table closes on callback return and attempts cleanup on callback failure, otherwise ensure close is called
Bounded table name pool — slate uses an internal bounded set of DETS table names to avoid unbounded atom growth. Opening too many distinct tables at once can fail with TableNamePoolExhausted; close tables when no longer needed
Erlang only — DETS is a BEAM feature, no JavaScript target support

Stability

slate follows Semantic Versioning. The public API covered by semver guarantees consists of four modules:

slate — shared types (DetsError, FileErrorContext, AccessMode, RepairPolicy, TableInfo) and helpers
slate/set — set tables
slate/bag — bag tables
slate/duplicate_bag — duplicate bag tables

The Erlang FFI files (slate_dets_ffi.erl, slate_with_table_ffi.erl) are internal implementation details and are not part of the public API. They may change in any release without notice.

Versioning policy: patch releases contain bug fixes only, minor releases add backward-compatible features, and major releases may include breaking changes. The error_code() strings returned by slate.error_code are stable across minor and patch releases and are safe for programmatic matching (e.g., in error-handling logic or logging). The error_message() strings are human-readable and may change in any release.

See CHANGELOG.md for release history and upgrade notes, and the GitHub Releases page for tagged versions.

bravo — Comprehensive ETS (in-memory) bindings for Gleam
shelf — Persistent ETS tables backed by DETS, combining fast in-memory reads with durable storage

For details on the underlying storage engine, see the Erlang DETS documentation.

Development

See DEV.md for setup instructions, build tasks, and contribution guidelines.

License

MIT