DoubleEntryLedger

DoubleEntryLedger is an event sourced, multi-tenant double entry accounting engine for Elixir and PostgreSQL. It provides typed accounts, signed amount APIs, pending/posting flows, an optimistic-concurrency command queue, and a fully auditable journal so you can embed reliable ledgering without rebuilding the fundamentals.

Highlights

• Multi tenant ledger instances with typed accounts (asset/liability/equity/revenue/expense) and Money backed multi currency support.
• Signed amount API converts intent into the correct debit or credit entry and enforces balanced transactions per currency.
• Immutable Command, JournalEvent, and BalanceHistoryEntry records plus idempotency keys give a complete audit trail.
• Background command queue with OCC, exponential retries, per-instance processors, and idempotency controls makes command processing safe to retry.
• Pending vs. posted projections with automatic available balances support holds, authorizations, and delayed settlements.
• Rich stores and APIs (InstanceStore, AccountStore, TransactionStore, CommandStore, CommandApi, JournalEventStore) keep ledger interactions safe and consistent.
• Everything lives inside the configurable double_entry_ledger schema so it coexists peacefully with your application tables.

System Overview

Instances & Accounts

DoubleEntryLedger.Stores.InstanceStore (lib/double_entry_ledger/stores/instance_store.ex) defines isolation boundaries. Each instance owns its own configuration, accounts, and transactions. DoubleEntryLedger.Stores.AccountStore validates account type, currency, addressing format, and maintains embedded posted, pending, and available balance structs. Helpers in DoubleEntryLedger.Types and DoubleEntryLedger.Utils.Currency encapsulate allowed values.

Commands, Journal Events & Transactions

External requests enter through DoubleEntryLedger.Apis.CommandApi (lib/double_entry_ledger/apis/command_api.ex). Requests are normalized into TransactionCommandMap or AccountCommandMap structs, hashed for idempotency, and saved as immutable Command records (lib/double_entry_ledger/schemas/command.ex). Successful processing creates JournalEvent records plus Transaction + Entry rows. Each journal event stores direct foreign keys to its command and related transaction or account. Query stores such as DoubleEntryLedger.Stores.TransactionStore and DoubleEntryLedger.Stores.JournalEventStore expose read models by instance, account, or transaction.

Queues, Workers & OCC

The command queue (lib/double_entry_ledger/command_queue) polls for pending commands via InstanceMonitor, spins up InstanceProcessor processes per instance, and uses CommandQueue.Scheduling to claim, retry, or dead-letter work. The transaction-related workers under lib/double_entry_ledger/workers/command_worker implement DoubleEntryLedger.Occ.Processor, translating event maps into Ecto.Multi workflows that retry on Ecto.StaleEntryError. Journal-event relationships are written synchronously with the journal event; there is no internal linking job in 0.5.0.

Balances & Audit Trails

Each transaction updates Account projections plus immutable BalanceHistoryEntry snapshots, enabling temporal queries and reconciliation. Instances can be validated with Instance.validate_account_balances/1, ensuring posted and pending debits/credits remain equal. Direct command_id, transaction_id, and account_id foreign keys on journal events provide traceability from the original request to the final projection.

Idempotency & Isolation

Every command requires a source and source_idempk (plus update_idempk for updates). These keys are hashed via DoubleEntryLedger.Command.IdempotencyKey to prevent duplicates, while PendingTransactionLookup enforces a single open update chain for each pending transaction. All tables live inside a dedicated Postgres schema (double_entry_ledger by default, overridable via config :double_entry_ledger, schema_prefix: …), so migrations never clash with your application schema.

Requirements

• Elixir ~> 1.15 and OTP 26.
• PostgreSQL 14+ with permission to create the double_entry_ledger schema.
• Access to run Mix tasks (mix ecto.create, mix ecto.migrate, mix test, etc.).
• Runtime dependencies are installed automatically through Hex. Credo, Dialyzer, and other development tools are only needed when working from a source checkout.

Installation

1. Add the dependency

def deps do
  [
    {:double_entry_ledger, "~> 0.5.0"}
  ]
end

Run mix deps.get after updating mix.exs.

2. Configure the application

Most consumers point DoubleEntryLedger at their own Ecto repo so the library shares one connection pool (and one Ecto sandbox in tests):

# config/config.exs
import Config

config :double_entry_ledger,
  repo: MyApp.Repo,
  idempotency_secret: System.fetch_env!("LEDGER_IDEMPOTENCY_SECRET"),
  start_command_queue: true,
  insert_path: :legacy,
  batch_enabled: false,
  batch_size: 8,
  max_batch_retries: 3,
  max_retries: 5,
  retry_interval: 200

config :double_entry_ledger, :command_queue,
  poll_interval: 5_000,
  pending_fetch_limit: 64,
  max_retries: 5,
  base_retry_delay: 30,
  max_retry_delay: 3_600,
  stale_processing_after: 300,  # seconds a command may sit in :processing before recovery
  processor_name: "command_queue"

In this "BYO-repo" mode the library does not start its own repo. The command queue needs the consumer's repo to be running, so add DoubleEntryLedger.children/0 to your supervision tree after your repo.

Set a strong idempotency_secret — it hashes incoming keys. Set start_command_queue: false to disable background processing (useful in tests or when embedding the ledger without the queue). retry_interval is read at runtime; max_retries is captured into each OCC worker at compile time (Occ.Processor.__using__/1), so changing it needs a recompile. The :command_queue list is read with Application.compile_env/3 and is likewise compile-time.

insert_path: :legacy and batch_enabled: false are the conservative defaults. To opt into the new paths, set insert_path: :insert_all and/or batch_enabled: true in the consuming application's configuration. Set batch_size to control how many compatible transaction commands are processed together and max_batch_retries to control retries after a stale account write. Account commands continue through the single-command path. pending_fetch_limit controls how many queue IDs an instance processor fetches per database read; it is independent of batch_size. When batching is enabled, set it to at least batch_size and preferably to a multiple of batch_size so each database fetch can be divided into full batches. Dependency configuration files are not loaded by a host application, so the INSERT_PATH, BATCH, and BATCH_SIZE environment-variable helpers in this repository's config/runtime.exs only apply when running this repository directly. A consuming release must translate any environment variables into :double_entry_ledger configuration in its own config/runtime.exs.

Standalone mode (omit :repo): the library ships its own DoubleEntryLedger.Repo and supervises it automatically. Configure it as a normal Ecto repo per env:

config :double_entry_ledger, ecto_repos: [DoubleEntryLedger.Repo]

config :double_entry_ledger, DoubleEntryLedger.Repo,
  database: "double_entry_ledger_repo",
  username: "postgres",
  password: "postgres",
  hostname: "localhost",
  pool_size: 10

This is useful for running DEL on its own (tests, demos), but for production apps prefer the BYO-repo form above.

3. Run the migrations

Fresh install (recommended)

mix double_entry_ledger.install
mix ecto.migrate

This generates a migration file for the core ledger tables.

Upgrading from v0.1.0

If you previously copied migration files from v0.1.0, generate an upgrade migration instead:

mix double_entry_ledger.install --from 1
mix ecto.migrate

This applies schema versions 2–5, including the FK fixes, negative_limit, trace context, query indexes, direct journal-event foreign keys, queue instance IDs, and widened balance columns.

Historical background-job migration: v0.1.0 included a migration for the then-used job runner. An already-applied migration and its tables may remain; 0.5.0 does not drop them because the host application may use that job runner independently. If you still need to execute or roll back that migration, declare the original dependency in your application rather than relying on DEL to provide it.

Manual migration

Create a migration and call the migration module directly:

defmodule MyApp.Repo.Migrations.SetupDoubleEntryLedger do
  use Ecto.Migration

  def up, do: DoubleEntryLedger.Migration.up()
  def down, do: DoubleEntryLedger.Migration.down()
end

See DoubleEntryLedger.Migration docs for all options (:version, :from, :prefix).

4. Add the command queue to your supervision tree

In BYO-repo mode, add DoubleEntryLedger.children/0 to your supervision tree so DEL's command queue starts after your repo:

# lib/my_app/application.ex
children =
  [
    MyApp.Repo,
    # …other children…
  ] ++ DoubleEntryLedger.children()

In standalone mode the library supervises the command queue itself and consumers do not need to call DoubleEntryLedger.children/0.

Quickstart

Create a ledger instance and accounts

alias DoubleEntryLedger.Stores.{InstanceStore, AccountStore}

{:ok, instance} =
  InstanceStore.create(%{
    address: "Acme:Ledger",
    description: "Internal ledger for ACME Corp"
  })

{:ok, cash} =
  AccountStore.create(instance.address, %{
    address: "cash:operating",
    type: :asset,
    currency: :USD,
    name: "Operating Cash",
    negative_limit: 0             # default; rejects any negative available balance
  })

{:ok, equity} =
  AccountStore.create(instance.address, %{
    address: "equity:capital",
    type: :equity,
    currency: :USD,
    name: "Owners' Equity",
    negative_limit: 1_000_00      # allow available to go as low as -1_000_00
  })

Process a transaction synchronously

alias DoubleEntryLedger.Apis.CommandApi

command = %{
  "instance_address" => instance.address,
  "action" => "create_transaction",
  "source" => "back-office",
  "source_idempk" => "initial-capital-1",
  "payload" => %{
    status: :posted,
    entries: [
      %{"account_address" => cash.address, "amount" => 1_000_00, "currency" => :USD},
      %{"account_address" => equity.address, "amount" => 1_000_00, "currency" => :USD}
    ]
  }
}

{:ok, transaction, processed_command} = CommandApi.process_from_params(command)

Provide positive amounts to add value and negative amounts to subtract it—the ledger will derive the correct debit or credit per account type and reject unbalanced transactions.

Queue a command for asynchronous processing

async_command = Map.put(command, "source_idempk", "initial-capital-async")
{:ok, queued_command} = CommandApi.create_from_params(async_command)
# InstanceMonitor will claim it, process it, and update the command_queue_item status.

Inspect queued work with DoubleEntryLedger.Stores.CommandStore.list_for_instance/2 or check command.command_queue_item.status.

Reserve funds with pending transactions

hold_event = %{
  "instance_address" => instance.address,
  "action" => "create_transaction",
  "source" => "checkout",
  "source_idempk" => "order-123",
  "payload" => %{
    status: :pending,
    entries: [
      %{"account_address" => cash.address, "amount" => -200_00, "currency" => :USD},
      %{"account_address" => equity.address, "amount" => -200_00, "currency" => :USD}
    ]
  }
}

{:ok, pending_tx, _command} = CommandApi.process_from_params(hold_event)

# Later, finalize the hold
CommandApi.process_from_params(%{
  "instance_address" => instance.address,
  "action" => "update_transaction",
  "source" => "checkout",
  "source_idempk" => "order-123",
  "update_idempk" => "order-123-post",
  "payload" => %{status: :posted}
})

source + source_idempk uniquely identify the original event, and update_idempk must be unique per update. Only pending transactions can be updated.

Query ledger state

alias DoubleEntryLedger.Instance
alias DoubleEntryLedger.Stores.{AccountStore, TransactionStore, JournalEventStore, CommandStore}

AccountStore.get_by_id(cash.id).available

{:ok, {history, _meta}} = AccountStore.list_balance_history(cash.id)
{:ok, {transactions, _meta}} = TransactionStore.list_for_instance(instance.id)
{:ok, {events, _meta}} = JournalEventStore.list_for_account(cash.id)

CommandStore.get_by_id(command.id)

Each list function accepts an optional second-argument map of Flop params (cursor, filters, ordering) — see the store moduledocs for the allow-listed filter fields.

Use Instance.validate_account_balances(instance) to assert the ledger still balances, or PendingTransactionLookup to inspect open holds.

Background Processing

• DoubleEntryLedger.CommandQueue.InstanceMonitor polls for commands in :pending, :occ_timeout, or :failed status and ensures each instance has an InstanceProcessor.
• InstanceProcessor claims work through an atomic scheduling claim — CommandQueue.Scheduling.claim_command_for_processing/3 for a single command, claim_batch_for_processing/3 in batch mode — runs the appropriate worker, and marks the CommandQueueItem as :processed. Each worker task is monitored via Process.monitor/1; if the task crashes, the processor schedules a retry automatically.
• OCC is handled inside the workers (see lib/double_entry_ledger/occ). Retries use exponential backoff until max_retries is reached, after which commands are marked as :dead_letter.
• Errors and retry metadata live on the command_queue_item, so you can inspect processing attempts via CommandStore or SQL views.
• Journal-event relationships are persisted synchronously through direct foreign keys; the command path no longer enqueues an internal linking job.
• Commands left in :processing by a node that died are recovered by InstanceMonitor once they are older than stale_processing_after and are sent back through the normal retry or dead-letter path. A successful enqueue also wakes the monitor when the queue is supervised locally, so an idle queue starts draining immediately instead of waiting a full poll_interval; otherwise the command waits for the next poll on a node that runs the queue.

Deployment scope: one node per ledger

The command queue is designed and tested for a single node processing a given ledger. Run it that way in production.

Every write that completes, retries, or dead-letters a command is fenced by command_queue_items.processor_version, so a stale processor can never overwrite work after ownership has moved, and commands stranded in :processing by a node that died are recovered automatically. Those guarantees hold under concurrency and are what make a crash or restart safe.

What is not yet guaranteed across several nodes:

• CommandQueue.Registry is a node-local Registry. It prevents duplicate InstanceProcessors for a ledger on one node only, so each node can run its own processor for the same instance.
• Atomic claims protect individual commands, not a ledger. Processors on different nodes can claim different commands of the same instance, so those commands may execute concurrently and out of order.
• Every node runs its own InstanceMonitor, producing redundant polling and competing processor starts.
• batch_enabled, batch_size, pending_fetch_limit, and stale_processing_after are read from each node's application environment, so inconsistent deployments process the same queue with different behaviour.
• Enqueueing wakes the local monitor only; other nodes still wait for their next poll.

Cluster-wide safety needs an ownership lease per ledger, held in PostgreSQL with a fencing token and a database-clock expiry, so that only the current owner may claim work for that ledger. That is not implemented.

Documentation & Further Reading

• Ledger internals & synchronous walkthrough
• Asynchronous processing details
• Handling pending transactions and available balances
• Event sourcing architecture notes
• Telemetry events and metrics

Generate fresh API docs locally with:

mix docs

Extras are bundled in pages/ when you run mix docs.

Development

• mix deps.get – install dependencies.
• mix ecto.create && mix ecto.migrate – prepare the database.
• mix test – run the test suite (aliases automatically create/migrate the test DB).
• mix credo --strict and mix dialyzer – static analysis.
• mix docs – regenerate documentation, or mix tidewave to preview docs via the built-in dev server.

Migrating from 0.4.x to 0.5.0

⚠️ 0.5.0 contains breaking schema and API changes. Migration 5 is not compatible with a mixed 0.4.x/0.5.0 rolling deployment. Stop 0.4.x command processing, apply the upgrade migration, deploy 0.5.0, and then resume processing.

Generate and run an upgrade migration from schema version 4:

mix double_entry_ledger.install --from 4
mix ecto.migrate

The migration performs these changes:

1. Replaces journal_event_*_links tables with direct command_id, transaction_id, and account_id columns on journal_events. Update custom queries and preloads to use the direct associations.
2. Adds and backfills command_queue_items.instance_id, then makes it required. Custom code building queue-item changesets must provide instance_id.
3. Widens account and balance-history integer balance/limit columns to bigint. A later downgrade can fail if stored values exceed the old integer range.
4. Moves commands.inserted_at plus command-queue insertion, update, processing-start, and processing-completion timestamp generation to PostgreSQL so processing times do not depend on application-node clocks.
5. Adds and backfills command_queue_items.queue_position from a PostgreSQL sequence, then uses it as the stable queue-processing order. Existing rows receive dense positions in (inserted_at, command_id) order; PostgreSQL uses the sequence for rows inserted after the backfill.
6. Adds the transient command_queue_items.retry_delay_seconds column and the queue-trigger rule that turns it into next_retry_after on the PostgreSQL clock, so retry timing no longer depends on application-node clocks.

The queue-position backfill rewrites every command_queue_items row, including processed and dead-letter history, while holding an ACCESS EXCLUSIVE table lock. Its runtime therefore scales with total queue history, not only currently processable work. Plan an appropriate maintenance window before applying it to a large table.

Command.processing_start_changeset/3 and CommandQueueItem.processing_start_changeset/3 were removed; callers must claim through CommandQueue.Scheduling.claim_command_for_processing/3, which enforces queue status and the retry deadline in one atomic UPDATE.

The JournalEventAccountLink, JournalEventCommandLink, JournalEventTransactionLink, and legacy journal-event link worker have been removed. DEL no longer depends on or supervises a third-party job runner, and journal-event relationships are now written synchronously. Remove the obsolete DEL job-runner configuration; applications using a job runner for their own work must declare, migrate, configure, and supervise it independently. Existing job-runner tables are intentionally left untouched.

Batching remains opt-in. Configure insert_path, batch_enabled, and batch_size in the consuming application; this dependency's config/runtime.exs is not loaded by the host release.

Migrating from 0.3.x to 0.4.0

⚠️ 0.4.0 contains breaking changes. Read this whole section before bumping the dependency — at minimum you'll update store call sites and supervision. See CHANGELOG.md for the canonical migration notes per item.

Breaking changes at a glance

1. Store list functions renamed and re-shaped. list_all_* and get_all_accounts_* are gone; replacements are list_for_* under Flop. Returns are now {:ok, {[item], %Flop.Meta{}}}. Update every call site — see the Function rename map and the Before/After example below.

2. Pagination is cursor-based via Flop. (id, page, per_page) → (id, flop_params_map). No consumer config :flop, repo: … required — DEL ships its own backend.

3. Supervision shift in BYO-repo mode. If you opt into BYO-repo via config :double_entry_ledger, repo: MyApp.Repo, DEL no longer supervises the command queue from its own tree. Add DoubleEntryLedger.children/0 to your app's supervisor. Standalone consumers (no :repo set) are unaffected.

New: bring your own repo

0.4.0 adds config :double_entry_ledger, repo: MyApp.Repo so the library shares the host's connection pool (and one Ecto sandbox in tests) instead of shipping its own DoubleEntryLedger.Repo. When :repo is omitted, the library runs in standalone mode as before. See Configuration for the full setup.

Before (0.3.x)

transactions = TransactionStore.list_all_for_instance_id(instance.id, 1, 40)

After (0.4.x)

{:ok, {transactions, meta}} = TransactionStore.list_for_instance(instance)

# Next page — cursor pagination
{:ok, {next, _meta}} =
  TransactionStore.list_for_instance(instance, %{first: 40, after: meta.end_cursor})

# With a filter (allow-listed field)
{:ok, {pending, _meta}} =
  TransactionStore.list_for_instance(instance, %{
    filters: [%{field: :status, op: :==, value: :pending}]
  })

Function rename map

All list_for_* functions that take a parent scope accept either the parent struct (Instance.t(), Account.t(), Transaction.t()) or its UUID string.

License

DoubleEntryLedger is released under the MIT License.