AuditTrail
Structured audit logging, database change tracking, and crash reporting for Elixir/Phoenix applications. Supports multiple storage backends — logs can be shipped to Grafana Loki, stored in your existing PostgreSQL database, or stored in TimescaleDB for automatic time partitioning and compression.
Why you need it
Most apps bolt on audit logging as an afterthought — scattered Logger.info calls, a logs database table that grows unbounded, or nothing at all. AuditTrail gives you:
- Who did what, when, and from where — every event carries actor identity, timestamp, IP, and user agent
- Field-level database change tracking — see exactly which fields changed, before and after, on any Ecto schema
- Controller-level auditing without clutter — one
plugdeclaration at the top of a controller covers all its actions with no per-function code - Crash alerting — OTP process crashes are intercepted, deduplicated, and emailed to the right team
- Queryable history — read logs back from Loki filtered by user, event type, date range, or free text
- Zero-risk emit — every emit is wrapped in a try/rescue; a logging failure never breaks your request
Table of contents
- Quick start — see a real log row in 5 minutes, no new infrastructure
- Core concepts — the mental model, and "which piece do I actually need?"
- Infrastructure requirements — only relevant if you pick the Loki adapter
- Installation
- Configuration
- Storage adapters — Loki, Postgres, TimescaleDB, custom
- Usage — the five instrumentation levels
- 1. Schema level — automatic DB change tracking
- 2. Controller level — declarative plug
- 3. API / service level — manual
emit/monitor - 4. Pipeline level — log every request
- 5. LiveView level — declarative
handle_event
- Reading logs back
- Testing — asserting on audit events in your test suite
- Loki stream labels
- Crash reporting
- Limitations — what this library deliberately doesn't do
- Troubleshooting
Quick start (5 minutes, no Loki required)
The fastest way to see a real audit log end to end: store it in the Postgres you already have. No Loki, no Grafana, no docker-compose.
# 1. mix.exs
{:audit_trail, path: "../audit_trail"} # or git: — see Installation below
# 2. config/config.exs (or runtime.exs)
config :audit_trail,
app_name: "my_app",
storage_adapter: {AuditTrail.Adapters.Postgres, repo: MyApp.Repo}
# 3. Fetch, generate the migration, migrate
mix deps.get
mix audit_trail.gen.migration
mix ecto.migrate
# 4. Log something from anywhere — a controller, a LiveView, an Oban job, iex
AuditTrail.emit("user:signed_up", %{resource_id: user.id, plan: "free"})
# 5. Read it back
{:ok, logs} = AuditTrail.get_logs(%{type: "user:signed_up"})
That's it — a real, queryable audit log with five lines of config. Once this feels good, keep reading:
- Want every change to a DB row tracked automatically, with no code in your business logic? → Schema level
- Want controller actions logged declaratively instead of hand-writing
emitcalls everywhere? → Controller level - Want Grafana dashboards and LogQL instead of SQL? → switch
storage_adapter:to Loki, see Storage adapters
Core concepts
Three ideas cover almost everything below — read this before the deep-dive sections if you're new to the library.
- Event — one audit log entry. An
event_typestring ("item:approved","schema:update","payment:processed"), an actor, a timestamp, and a free-form payload. Every function in this library —emit,monitor, schema tracking, both plugs — just builds one of these and hands it to the buffer. You will mostly interact with events, not with any of the machinery that ships them. - Actor — who did it. Set once per request or job with
AuditTrail.set_actor/1; every event logged in that process afterward picks it up automatically, no need to pass it around. Defaults to"anonymous"if you never set it. (There's alsoAuditTrail.set_tenant/1for multi-tenant apps — same idea, for an org/tenant id instead of a user.) - Adapter — where events end up. Loki, Postgres, TimescaleDB, or an in-memory Test adapter — swap with a single config line, and
AuditTrail.get_logs/1works identically regardless of which one you're on.
Everything else in this README is either "how do I produce an event" (the five instrumentation levels) or "how do I configure/query where events go" (adapters, filters).
Which instrumentation level do I need?
| You want to... | Use | Effort |
|---|---|---|
| Track every change to a DB row automatically | Schema level — use AuditTrail.Schema | Add one line to the schema, one to the Repo |
| Log specific controller actions ("item approved", "user logged in") | Controller level — plug AuditTrail.ControllerAudit | One plug declaration, zero code in the action |
Same, but for LiveView handle_event | LiveView level — on_mount {AuditTrail.LiveViewAudit, ...} | One on_mount, zero code in the callback |
| Log something with no schema and no controller — a background job, a webhook, a third-party API call | API / service level — AuditTrail.emit/2 | One explicit call at the point it happens |
| Log literally every HTTP request, unconditionally | Pipeline level — plug AuditTrail.Plug | One plug in your router pipeline |
These aren't mutually exclusive — most real apps use two or three together (e.g. schema tracking on a couple of sensitive tables, the controller plug for actions that matter, emit/2 for one-off business events like a payment webhook). Pick based on where the event naturally happens in your code, not on trying to standardize on one mechanism.
Which storage adapter do I need?
| If you... | Use |
|---|---|
| Don't want to stand up new infrastructure | Postgres — logs live in your existing DB, see Quick start |
| Expect high volume, or want old logs auto-compressed | TimescaleDB — same as Postgres, plus time partitioning |
| Already run Grafana/Loki, or want LogQL + dashboards | Loki (the default) — see Infrastructure requirements |
| Are writing tests and want to assert what got logged | Test — in-memory, zero setup, see Testing |
Full setup for each is under Storage adapters.
Infrastructure requirements
Skip this section entirely if you're using the Postgres, TimescaleDB, or Test adapter — it only applies to the Loki adapter (the default, but not required).
AuditTrail defaults to Grafana Loki as its storage backend, but this is optional. If you configure the Postgres or TimescaleDB adapter, no Loki instance is needed. See Storage adapters for how to switch.
| Component | Purpose |
|---|---|
| Loki | Log storage and query engine |
| Grafana (optional) | Visual log explorer over Loki |
| MinIO / S3 (optional) | Object storage backend for Loki in production |
With S3 / MinIO (production)
common:
storage:
s3:
endpoint: http://minio:9000
bucketnames: loki-data # ← "bucketnames" not "buckets" (Loki 3.x)
access_key_id: YOUR_KEY
secret_access_key: YOUR_SECRET
s3forcepathstyle: true
insecure: true # set false in real production
Loki 3.x gotcha: the S3 field is
bucketnames, notbuckets. The wrong name causes Loki to crash-loop on startup withfield buckets not found in type aws.S3Config.
Installation
1. Add to mix.exs
# Monorepo / local path
{:audit_trail, path: "../audit_trail"}
# Git
{:audit_trail, git: "https://github.com/your-org/audit_trail.git", tag: "v0.1.0"}
mix deps.get
mix deps.compile audit_trail
mix compile
Recompile tip: when you add new source files to the library, run
mix deps.compile audit_trail --force && mix compile --force. Theplugmacro callsinit/1at compile time, so the library must be fully compiled before the host app compiles its controllers.
2. Do NOT add to your supervision tree
AuditTrail is an OTP application — it starts itself when listed as a mix dependency. Adding it to application.ex causes {:already_started, pid}:
# ✗ Wrong
children = [
{AuditTrail, []}, # remove this
MyApp.Repo,
]
# ✓ Correct — just the mix dep, nothing in children
Configuration
All config lives under :audit_trail in your config/*.exs files.
Required
config :audit_trail,
app_name: "my-app",
loki_push_url: System.get_env("LOKI_PUSH_URL"), # http://localhost:3100/loki/api/v1/push
loki_read_url: System.get_env("LOKI_READ_URL") # http://localhost:3100/loki/api/v1/query_range
Buffer tuning (optional)
config :audit_trail,
enabled: true, # set false to disable all shipping (dev/test)
partition_count: 8, # GenServer buffer partitions (default 8)
batch_size: 100, # flush after this many entries (default 100)
flush_interval: 2_000, # flush every N ms even if batch not reached (default 2000)
max_buffer: 5_000, # drop logs when a partition's buffer exceeds this (default 5000, PER PARTITION — real cap is partition_count * max_buffer)
max_dead_letter: 10_000 # in-memory dead-letter capacity (default 10000)
Shipper retries (optional)
config :audit_trail,
shipper_timeout: 5_000, # HTTP timeout per Loki push attempt in ms (default 5000)
shipper_retries: 3 # retries with exponential backoff before dead-letter (default 3)
Crash reporting (optional)
config :audit_trail,
crash_reporting: true,
dedup_window_minutes: 30, # suppress duplicate crash emails within this window
dedup_max_entries: 1_000,
mailer_adapter: {AuditTrail.Adapters.Swoosh, []},
swoosh_mailer: MyApp.Mailer,
from_email: {"MyApp Alerts", "alerts@myapp.com"},
crash_routing: [
%{match: {:module, ~r/MyApp\.Items/}, to: ["items-team@myapp.com"]},
%{match: {:module, ~r/MyApp\.Auth/}, to: ["security@myapp.com"]},
%{match: :default, to: ["admin@myapp.com"]}
]
Recommended per-environment setup
# config/dev.exs
config :audit_trail,
enabled: true, # or false to silence all Loki calls during local dev
loki_push_url: "http://localhost:3100/loki/api/v1/push",
loki_read_url: "http://localhost:3100/loki/api/v1/query_range"
# config/test.exs
config :audit_trail,
enabled: false # never ship logs during tests
# config/runtime.exs (production)
config :audit_trail,
enabled: true,
loki_push_url: System.fetch_env!("LOKI_PUSH_URL"),
loki_read_url: System.fetch_env!("LOKI_READ_URL")
When enabled: false the buffer still accepts log calls (no errors, no warnings) but the shipper never fires. Safe to leave logging calls in your code in all environments.
Storage adapters
AuditTrail ships with four storage adapters. All four implement the same interface, so AuditTrail.get_logs/1 works identically regardless of which backend you use.
| Adapter | Requires | Best for |
|---|---|---|
AuditTrail.Adapters.Loki | Grafana Loki | Production observability stacks, long-term log retention, Grafana dashboards |
AuditTrail.Adapters.Postgres | Your existing Ecto Repo | Apps already on Postgres that don't want to run Loki |
AuditTrail.Adapters.TimescaleDB | TimescaleDB extension | High-volume logs, automatic compression, time-range query performance |
AuditTrail.Adapters.Test | Nothing | Asserting on audit events in your test suite — see Testing |
The default is Loki. No adapter config is needed if you set loki_push_url.
Loki adapter (default)
# config/runtime.exs — Loki is the default, this line is optional
config :audit_trail,
storage_adapter: AuditTrail.Adapters.Loki, # can omit this line
loki_push_url: System.fetch_env!("LOKI_PUSH_URL"),
loki_read_url: System.fetch_env!("LOKI_READ_URL")
No migration needed. Logs are stored as labelled JSON streams in Loki and are queryable with LogQL via Grafana or the built-in reader.
Postgres adapter
Stores logs in an audit_logs table in your own PostgreSQL database. No external services required.
1. Configure
config :audit_trail,
storage_adapter: {AuditTrail.Adapters.Postgres, repo: MyApp.Repo}
The repo: option is required — pass your app's Ecto.Repo module.
2. Generate and run the migration
mix audit_trail.gen.migration
mix ecto.migrate
This creates a timestamped migration file in priv/repo/migrations/. The migration adds the audit_logs table with these columns:
| Column | Type | Description |
|---|---|---|
id | bigserial | Auto-incrementing primary key |
event_id | text | UUID assigned at emit time |
event_type | text | e.g. "auth:login" |
actor_id | text | User ID or "anonymous" |
actor_name | text | Display name of the actor |
actor_email | text | Email of the actor |
payload | jsonb | Full structured event payload |
inserted_at | timestamptz | When the event was recorded |
A GIN index on payload is added automatically so JSON field searches are fast — this is also what makes the optional resource/operation filters (see Reading logs back) work without a schema change.
3. Options for the migration task
# Target a specific repo
mix audit_trail.gen.migration --repo MyApp.Repo
# Write to a custom path
mix audit_trail.gen.migration --migrations-path priv/audit_trail/migrations
TimescaleDB adapter
TimescaleDB is PostgreSQL with automatic time-based partitioning. The adapter is functionally identical to the Postgres adapter — it differs only in the migration, which also calls create_hypertable.
1. Configure
config :audit_trail,
storage_adapter: {AuditTrail.Adapters.TimescaleDB, repo: MyApp.Repo}
2. Generate and run the migration
mix audit_trail.gen.migration
mix ecto.migrate
The generated migration includes the standard table plus:
SELECT create_hypertable('audit_logs', 'inserted_at', chunk_time_interval => INTERVAL '1 day')
3. Enable compression (optional, after migrating)
Run these directly in your database to compress chunks older than 7 days:
ALTER TABLE audit_logs SET (
timescaledb.compress,
timescaledb.compress_segmentby = 'event_type, actor_id'
);
SELECT add_compression_policy('audit_logs', INTERVAL '7 days');
The compress_segmentby columns match the fields you filter on most — queries on those fields stay fast even against compressed chunks.
Why TimescaleDB over plain Postgres?
| Feature | Postgres | TimescaleDB |
|---|---|---|
| Time-range queries | Table scan (unless indexed) | Chunk pruning — only relevant chunks scanned |
| Storage for old data | Full row size always | Compressed chunks (4–20× compression typical) |
| Partition management | Manual | Automatic by time interval |
TimescaleDB is the better choice when you expect more than a few million log rows or run frequent date-range queries (the default pattern for audit logs). It is standard Postgres-compatible — your existing mix ecto.migrate and all Ecto queries work without modification.
Switching adapters
You can switch adapters at any time by changing the config. Logs already written are not migrated — the new adapter starts fresh from the moment the config change takes effect. For a zero-downtime switch, deploy the new config and run the migration before cutting over traffic.
Writing a custom adapter
Implement the AuditTrail.Adapter behaviour:
defmodule MyApp.AuditAdapter do
@behaviour AuditTrail.Adapter
@impl true
def push(logs) do
# logs is a list of sanitized map()s with at minimum:
# :event_type, :actor_id, :actor_name, :actor_email, :timestamp
:ok
end
@impl true
def get_logs(filters) do
# filters: %{actor_id:, type:, status:, search:, start_date:, end_date:, limit:}
# Must return {:ok, [%{timestamp:, labels: map(), payload: map()}]}
# or {:error, reason}
{:ok, []}
end
@impl true
def child_spec(_opts), do: nil # return nil if no supervised process needed
end
Then configure it:
config :audit_trail, storage_adapter: MyApp.AuditAdapter
# or with opts:
config :audit_trail, storage_adapter: {MyApp.AuditAdapter, some_option: true}
The output shape of get_logs/1 must match the structure the rest of the system expects:
%{
timestamp: ~U[2026-06-12 10:34:21Z], # DateTime
labels: %{"app" => "...", "type" => "..."}, # string keys
payload: %{"event_type" => "...", ...} # string keys, full event data
}
Usage
There are five levels at which you can instrument your app. They are independent — use as many or as few as you need.
1. Schema level — automatic field-level change tracking
The zero-effort layer. Mark a schema, and every insert, update, and delete through your Repo is recorded automatically with a before/after diff.
Instrument your Repo
defmodule MyApp.Repo do
use Ecto.Repo, otp_app: :my_app, adapter: Ecto.Adapters.Postgres
use AuditTrail.RepoWatcher # add this line
end
RepoWatcher overrides insert/2, update/2, delete/2 (and their bang variants). Operations on non-tracked schemas pass through completely unchanged.
Mark schemas to track
defmodule MyApp.Items.Item do
use Ecto.Schema
use AuditTrail.Schema, track: [:status, :condition, :approved_by]
schema "items" do
field :name, :string
field :status, :string # tracked ✓
field :condition, :string # tracked ✓
field :approved_by, :string # tracked ✓
field :internal_notes, :string # NOT tracked
end
end
track: :all tracks every field. A list of atoms tracks only those fields — useful to exclude high-frequency or irrelevant fields like updated_at.
Embeds and associations are diffed too
embeds_one/embeds_many and has_one/has_many/belongs_to/many_to_many (via cast_assoc/put_assoc/cast_embed) are diffed recursively, not dumped as an opaque struct:
defmodule MyApp.Items.Item do
use Ecto.Schema
use AuditTrail.Schema, track: :all
schema "items" do
field :status, :string
embeds_one :shipping_address, MyApp.Items.Address
has_many :line_items, MyApp.Items.LineItem
end
end
Updating shipping_address.city and adding a new line_items row produces:
%{
"shipping_address" => %{
"action" => "update",
"record_id" => "addr-1",
"changes" => %{"city" => %{"from" => "Nairobi", "to" => "Mombasa"}}
},
"line_items" => [
%{"action" => "insert", "changes" => %{"sku" => %{"from" => nil, "to" => "ABC-123"}}}
]
}
action per nested entry is "insert", "update", "delete", or "replace" (Ecto's own terms for what happened to that embed/association). The parent's track: field list only filters which top-level fields are tracked — once a top-level field like shipping_address is in scope, all of its nested fields are included unless the nested schema opts into its own filtering:
defmodule MyApp.Items.Address do
use Ecto.Schema
use AuditTrail.Schema, track: [:city, :country] # filters this embed's own fields
embedded_schema do
field :city, :string
field :country, :string
field :contact_phone, :string # NOT tracked, even though shipping_address is
end
end
Embeds/associations don't go through the Repo directly, so this doesn't give them independent tracking (no standalone schema:update event for just the embed) — it only scopes which of their fields show up in the parent's diff.
What gets logged
| Field | Example value |
|---|---|
event_type | "schema:update" |
actor_id | "550e8400-e29b-41d4-a716-446655440000" |
schema | "Elixir.MyApp.Items.Item" |
resource | nil unless resource: is set (see below) |
resource_id | "abc-123" — same value as record_id, kept under both names so filters can use either |
operation | "insert" / "update" / "delete" |
record_id | "abc-123" |
changes | {"status": {"from": "pending", "to": "approved"}} |
The actor is whatever was last set via AuditTrail.set_actor/1 in the current process. Defaults to "anonymous".
Naming the resource (optional)
schema already identifies what was changed (the full module name), but that's noisy to filter on. Pass resource: for a short, business-facing label instead:
defmodule MyApp.Payments.Payment do
use Ecto.Schema
use AuditTrail.Schema, track: [:status, :amount], resource: "payment"
# ...
end
This is purely additive — omit resource: and tracking behaves exactly as before, with resource simply absent (nil) from the payload.
Repo.transaction and Ecto.Multi are commit-safe
RepoWatcher also wraps transact/2 — the function both Repo.transaction/2 and Ecto.Multi funnel through. Any audit log produced while a transaction is still open (an Ecto.Multi.insert/update/delete step, or a manual AuditTrail.emit/monitor call made inside the transaction function) is held, not shipped immediately:
Ecto.Multi.new()
|> Ecto.Multi.update(:item, item_changeset)
|> Ecto.Multi.insert(:approval, approval_changeset)
|> MyApp.Repo.transaction()
- If the transaction commits, the held logs ship in the order they were emitted.
- If the transaction rolls back (a later step fails, or you call
Repo.rollback/1), the held logs are dropped — the audit trail never claims a change happened that the database then undid.
This applies regardless of nesting depth — only the outcome of the outermost transaction decides whether held logs ship or are dropped.
Limitation
Schema tracking intercepts Repo.insert, Repo.update, Repo.delete (single-record), including when called through Ecto.Multi.insert/update/delete. It does not intercept:
Repo.update_all,Repo.delete_all,Ecto.Multi.insert_all/update_all/delete_all, or rawEcto.Queryoperations (no changeset, so no diff is possible)Repo.insert_or_update/Ecto.Multi.insert_or_update— Ecto resolves this internally without calling back through the repo's publicinsert/2/update/2, so it bypasses the override. Use an explicitMulti.insert/Multi.update(or checkchangeset.data.idyourself) if you need this tracked.
2. Controller level — declarative plug
For business events that span multiple schemas, or events with no schema change. Declare once at the top of a controller — no per-function code.
defmodule MyAppWeb.Items.ItemApprovalController do
use MyAppWeb, :controller
plug AuditTrail.ControllerAudit,
write: [
api_approve: "item:approved",
api_decline: "item:declined",
api_request_revision: "item:revision_requested",
api_acknowledge: "item:revision_acknowledged"
]
def api_approve(conn, %{"id" => id} = params) do
# No AuditTrail call needed here — the plug handles it
case ItemApproval.approve(id, params) do
{:ok, updated} -> json(conn, %{data: updated})
{:error, _} = e -> respond_error(conn, e)
end
end
end
The plug hooks into Plug.Conn.register_before_send and fires after your action returns, capturing the final HTTP status code.
write vs read
| Key | Logs when |
|---|---|
write | Always — success and failure. A rejected approval is still an auditable event. |
read | Only on 2xx — failed reads (404, 403) are noise and are dropped. |
What gets logged per event
| Field | Value |
|---|---|
event_type | the string you provided, e.g. "item:approved" |
action | the Phoenix action atom, e.g. api_approve |
operation | "write" or "read" by default — "create"/"update"/"delete" if you use those keys (see below) |
resource | nil unless resource: is set on the plug (see below) |
resource_id | nil unless found in conn.params["id"]/conn.params["uuid"], or id_param: (see below) |
http_status | 200, 422, 403, etc. |
success | true / false |
params | sanitized request params (sensitive fields stripped automatically) |
Tagging the resource, record, and CRUD operation (optional)
write/read control when an event is logged, but don't say what was changed or which one. If you want logs to carry a resource name, the specific record's id, and a specific CRUD operation (useful for filtering audit history down to e.g. "every update on paymentc9b031c6-..."), pass resource: and split actions across create:, update:, delete: instead of (or alongside) write::
plug AuditTrail.ControllerAudit,
resource: "payment",
create: [api_charge: "payment:charged"],
update: [api_refund: "payment:refunded"],
delete: [api_void: "payment:voided"],
read: [api_show: "payment:viewed"]
create, update, and delete behave exactly like write (logged on success and failure) — they just set a more specific operation value. This is entirely optional: existing write:/read: declarations keep working unchanged, and resource is simply nil if you don't set it.
resource_id — the id of the specific record touched — is captured automatically from conn.params["id"] or conn.params["uuid"], whichever is present. If your route uses a different param name, point at it with id_param::
plug AuditTrail.ControllerAudit,
resource: "idf",
id_param: "uuid", # matches conn.params["uuid"]; also accepts an atom or a list of names
update: [api_update_idf: "idf:updated"]
With this in place you can pull the full history of one record:
AuditTrail.get_logs(%{resource: "idf", resource_id: "c9b031c6-6d9e-494b-8399-236940ca9d10"})
If conn.params has neither the default keys nor your id_param:, resource_id is just nil — no error, no change to existing behavior.
Auto-stripped sensitive fields
The following param keys are dropped entirely before logging, regardless of what you pass:
password new_password current_password confirm_password
code token access_token refresh_token methods_token
secret client_secret api_key private_key
This list is AuditTrail.Config.sensitive_fields/0 — shared by ControllerAudit, AuditTrail.LiveViewAudit, and AuditTrail.Sanitizer (which redacts any matching key, at any nesting depth, in every payload — schema-level diffs, monitor/emit's original_record: diffing, everything — right before it ships). One list, one place to extend it:
config :audit_trail, sensitive_fields: ~w(
password token secret
kra_pin id_number tax_payer_pin
)
This replaces the defaults rather than adding to them — include what you still want redacted alongside your own app-specific fields (national ID numbers, tax PINs, anything else that shouldn't end up in Loki/Postgres). Matching is exact and case-insensitive against the key name (password_hash does not match password — list the exact field name your schema uses).
ControllerAudit/LiveViewAuditdrop the key entirely from params; Sanitizer (the backstop covering diffs, changes, and anything else) replaces the value with "[REDACTED]" instead of dropping the key — so a diff still shows that a sensitive field changed, just not to what.
Oversized values are truncated
File uploads and other large payloads (base64 blobs, big lists) are truncated before being logged, so a single upload action doesn't dump megabytes into the audit trail:
- Any string/binary value over 2,000 bytes becomes
"...truncated (N bytes)" - Any list with more than 20 items becomes
"...truncated (N items)"
This applies recursively through nested maps/lists in params.
Login flows — set the actor before returning
Because the authentication plug runs before the login action, there is no current_user in conn.assigns at login time. Call AuditTrail.set_actor/1 inside the success branch so the plug captures the real identity when before_send fires:
plug AuditTrail.ControllerAudit,
write: [api_login: "auth:login", api_logout: "auth:logout"]
def api_login(conn, params) do
case Auth.login(params) do
{:ok, user, token} ->
AuditTrail.set_actor(user) # ← must be called before returning
json(conn, %{token: token})
{:error, reason} ->
# actor stays anonymous — failed logins are still logged
respond_error(conn, reason)
end
end
Plain atom shorthand
If you don't care about the event name and just want the action name (minus the api_ prefix), pass a plain atom:
plug AuditTrail.ControllerAudit,
write: [:api_approve, :api_decline]
# logs as "approve" and "decline"
3. API / service level — manual emit
For background jobs, external API calls, or any event that doesn't go through a controller.
AuditTrail.emit/2
AuditTrail.emit("payment:processed", %{
amount: order.total,
currency: "KES",
gateway: "mpesa",
reference: txn.ref
})
Tagging the resource, record, and CRUD operation (optional)
Pass resource:, resource_id:, and operation: in the data map to record what kind of thing this event concerns, which specific record, and which CRUD operation it represents. All three are optional — omit them and they're simply absent (nil) from the payload, exactly like before this was added:
AuditTrail.emit("payment:processed", %{
resource: "payment",
resource_id: txn.id,
operation: "update",
amount: order.total,
currency: "KES",
gateway: "mpesa",
reference: txn.ref
})
There's no fixed list of operations — use whatever fits, e.g. "create", "read", "update", "delete". If you omit resource_id: but the data map has an :id or :uuid key, that value is used automatically.
Diffing against a prior record (optional) — for data outside your own Repo
monitor/4's original_record:/params: diffing only helps when you have a conn and an Ecto result tuple. For something with no Ecto changeset at all — a row that lives in a library's own database, a third-party API's own resource — pass original_record: to emit/2 instead:
# New payment: no original_record, meta is exactly what you passed
AuditTrail.emit("payment:processed", %{
resource_id: txn.ref,
operation: "create",
amount: order.total,
currency: "KES",
gateway: "mpesa",
reference: txn.ref
})
# Existing payment moving to a new state: original_record triggers a diff
AuditTrail.emit("payment:processed", %{
resource_id: txn.ref,
original_record: previous_txn, # a plain map or struct — whatever you have
status: "success",
amount: txn.amount
})
When original_record: is present, it's popped out of the logged payload (never dumped whole) and replaced with meta.changes — a field-level diff of the rest of the data map against it, reusing the exact same comparison monitor/4 uses. operation defaults to "update" in that case (still overridable). Omit original_record: and behavior is identical to before this existed.
This has no conn/socket dependency — unlike monitor/4, emit/2 never needed one, so this works unchanged from a controller, a LiveView handle_event, an Oban job, or a webhook handler. If you also want request context (IP, user agent, request id) attached, pass ip_address:/user_agent:/request_id: explicitly — extract them however your caller has them (conn in a controller, socket.assigns if you captured them at mount/3, nothing at all in a background job).
Limitations worth knowing:
- This is a self-reported diff — there's no changeset validating it against what the external system actually persisted, unlike
RepoWatcher's changeset-based diffing. It's only as honest as theoriginal_recordyou hand it. - Comparison is shallow/top-level per field, same as
monitor/4— a nested map that changed is shown as one wholefrom/toblob, not deep-diffed. - Field names must match between
original_recordand the rest of the data map for a field to be compared at all. :statusis deliberately excluded from the "reserved, never diffed" keys, since it doubles as a very common business field name (payment/item status) as well asemit/2's own log-severity hint ("info"/"success"/"failure") — know that overlap exists if you rely on both meanings in the same call.
The actor is picked up from the process dictionary automatically. Override it per-event when needed:
AuditTrail.emit("nightly:sync", %{
actor_id: "system:scheduler",
actor_name: "Nightly Sync",
records: synced_count
})
AuditTrail.monitor/4 — wrap a result tuple inline
def approve(conn, %{"id" => id}) do
original = Items.get!(id)
Items.approve(id)
|> AuditTrail.monitor(conn, "item:approved", original_record: original)
|> case do
{:ok, item} -> json(conn, %{data: item})
{:error, _} = e -> respond_error(conn, e)
end
end
monitor/4 returns the result unchanged so it can be piped. Available options:
| Option | Purpose |
|---|---|
original_record: | record before the change — used to compute a before/after diff |
params: | the submitted params |
extra_meta: | any extra map merged into the log payload |
resource: | optional resource label, e.g. "payment" — nil if omitted |
operation is set for you automatically ("create" when original_record: is absent, "update" when it's present) — no need to pass it. resource_id is also automatic: it's the id of the record returned in result (on success) or of original_record: (on failure) — no option needed.
The second argument only ever gets pattern-matched by shape (%{remote_ip: ...}, %{req_headers: ...}) to pull IP/user-agent/request-id, never by struct name — so passing something that isn't a conn (a LiveView socket, a plain map, nil) doesn't crash, it just yields "unknown" for those three fields while the original_record:/params: diffing still works exactly the same. If you're not in a Plug pipeline at all, AuditTrail.emit/2 (below) is usually the better fit anyway — it never needed a conn in the first place.
AuditTrail.log_external_api/3 — wrap an external HTTP call
# Wrap a function — timing and success/failure captured automatically
AuditTrail.log_external_api("mpesa:stk_push", "M-Pesa", fn ->
Mpesa.initiate_stk_push(phone, amount)
end)
# Log a result you already have
AuditTrail.log_external_api("twilio:sms", "Twilio", %{
status: "success",
duration_ms: 340,
to: phone_number
})
AuditTrail.set_actor/1 — set who is acting
# From a user struct — picks up :id, :email, :name / :username automatically
AuditTrail.set_actor(user)
# For background/system actors
AuditTrail.set_actor(:system, "nightly-cleanup")
Stored in the process dictionary. Every subsequent emit, monitor, and log_external_api in the same process uses this identity.
Background jobs: new processes (Oban jobs,
Task.async) do not inherit the process dictionary. CallAuditTrail.set_actor/1at the top of your job'sperform/1. Spawning viaAuditTrail.Task.async/start/start_linkinstead ofTask's own propagates both the actor and tenant (see below) automatically.
AuditTrail.set_tenant/1 — multi-tenant scoping
For multi-tenant apps, tag every subsequent event in the current process with an org/tenant id:
AuditTrail.set_tenant(org.id)
Like set_actor/1, this is stored in the process dictionary — set it once per request (e.g. in a plug, alongside set_actor/1) and every emit/monitor/schema-tracked Repo call in that process picks it up automatically as a tenant field. On the Loki adapter it's also added as a stream label (only when set, so single-tenant apps see no change), so AuditTrail.get_logs(%{tenant: org.id}) is a fast label-scoped query rather than a full JSON body scan. On Postgres/TimescaleDB it's a JSONB field filter, same as resource/operation.
4. Pipeline level — blanket HTTP logging
AuditTrail.Plug logs every HTTP request. This is high volume for an API-only app (one entry per request, including health checks). Only use it if you need a full HTTP access log separate from your business events.
# router.ex
pipeline :api do
plug :accepts, ["json"]
plug AuditTrail.Plug # logs every request
end
For most apps, AuditTrail.ControllerAudit on specific controllers is the better choice — it logs only the actions you explicitly care about.
Custom actor extractor
If your conn.assigns key isn't :current_user:
config :audit_trail, actor_extractor: MyApp.AuditActorExtractor
defmodule MyApp.AuditActorExtractor do
def extract(conn) do
user = conn.assigns[:my_user]
%{id: to_string(user.id), name: user.display_name, email: user.email}
end
end
5. LiveView level — declarative handle_event auditing
AuditTrail.ControllerAudit only covers Phoenix controllers. AuditTrail.LiveViewAudit is the LiveView equivalent, hooking into handle_event/3 via Phoenix.LiveView.attach_hook/4:
defmodule MyAppWeb.ItemLive do
use MyAppWeb, :live_view
on_mount {AuditTrail.LiveViewAudit,
resource: "item",
write: ["approve", "decline"],
read: ["view_details"]}
def handle_event("approve", %{"id" => id}, socket) do
# No AuditTrail call needed here — the hook handles it
...
end
end
Or attach directly (e.g. inside mount/3) if you don't use on_mount:
def mount(_params, _session, socket) do
{:ok, AuditTrail.LiveViewAudit.attach(socket, resource: "item", write: ["approve"])}
end
This isn't identical to ControllerAudit.attach_hook intercepts handle_event/3before your callback runs — there's no LiveView equivalent of register_before_send to check an outcome afterward. So every configured event is logged the moment it's dispatched, regardless of whether your handle_event/3 clause then succeeds, raises, or no-ops. write:/read: behave identically today — the split exists so the config shape matches ControllerAudit and operation gets tagged for filtering. resource_id is picked up from the event params' "id"/"uuid" key (or id_param:), and sensitive keys are stripped, same conventions as ControllerAudit.
Reading logs back
# All logs (last 24 hours by default)
{:ok, logs} = AuditTrail.get_logs()
# Only a specific user's logs
{:ok, logs} = AuditTrail.get_logs(%{actor_id: "user-uuid"})
# Filter by event type
{:ok, logs} = AuditTrail.get_logs(%{type: "auth:login"})
# Filter by resource and CRUD operation (only matches logs that set these — see below)
{:ok, logs} = AuditTrail.get_logs(%{resource: "payment", operation: "update"})
# Full history of one specific record
{:ok, logs} = AuditTrail.get_logs(%{resource: "idf", resource_id: "c9b031c6-6d9e-494b-8399-236940ca9d10"})
# Date range + search term
{:ok, logs} = AuditTrail.get_logs(%{
start_date: "2026-06-01",
end_date: "2026-06-12",
search: "alice@example.com",
limit: 200
})
Available filters
| Filter | Type | Description |
|---|---|---|
actor_id | string | Returns only this user's logs |
type | string | Match a specific event type e.g. "auth:login" |
status | string | "success" or "failure" |
resource | string | (optional field) Match logs tagged with this resource, e.g. "payment" |
resource_id | string | (optional field) Match logs for this one record's id, e.g. a uuid — pairs well with resource to get a single record's full history |
operation | string | (optional field) Match logs tagged with this CRUD operation, e.g. "update" |
tenant | string | (optional field) Match logs tagged with this tenant/org id — see AuditTrail.set_tenant/1 |
search | string | Substring match on the log line body |
start_date | "YYYY-MM-DD" | Start of window (default: 24 hours ago — logged via Logger.info when defaulted) |
end_date | "YYYY-MM-DD" | End of window (default: now) |
limit | integer | Max results returned (default 100, max 500) |
before | string | Cursor for the next page — see AuditTrail.get_logs_page/1 |
String-keyed filter maps (e.g. straight from conn.params) are normalized automatically — %{"resource_id" => "abc"} and %{resource_id: "abc"} both work.
resource,resource_id, andoperationare optional. They only appear in the payload for events where you explicitly set them, or whereControllerAuditcould auto-detectresource_idfromid/uuidparams. See "Tagging the resource, record, and CRUD operation" under Controller level and API / service level. Logs written without these fields simply won't match aresource/resource_id/operationfilter — existing code and queries are unaffected.
Log entry structure
%{
timestamp: ~U[2026-06-12 10:34:21Z],
labels: %{
"app" => "heart-ke",
"type" => "auth:login",
"user" => "550e8400-e29b-41d4-a716-446655440000"
},
payload: %{
"event_id" => "d4e5f6...",
"event_type" => "auth:login",
"actor_id" => "550e8400-...",
"actor_name" => "Alice Kamau",
"actor_email" => "alice@example.com",
"http_status" => 200,
"success" => true,
"action" => "api_login",
"resource" => nil,
"resource_id" => nil,
"operation" => "write",
"params" => %{"email" => "alice@example.com"},
"timestamp" => "2026-06-12T10:34:21Z"
}
}
resource and resource_id are nil, and operation is "write"/"read", by default. They only carry a more specific value (e.g. "payment" / "c9b031c6-..." / "update") when you opt in — see the "Tagging the resource, record, and CRUD operation" subsections above.
Access control pattern
The reader does no access control — enforce it in your controller:
defmodule MyAppWeb.AuditLogController do
def index(conn, params) do
user = conn.assigns.current_user
is_admin = user_is_admin?(user)
# Non-admins see only their own logs
filters = if is_admin, do: params, else: Map.put(params, :actor_id, to_string(user.id))
case AuditTrail.get_logs(filters) do
{:ok, logs} -> json(conn, %{data: logs})
{:error, _} -> json(conn, %{data: []}) # degrade gracefully if Loki is down
end
end
end
Testing
enabled: false (the recommended config/test.exs setting) silences AuditTrail entirely — good for not spamming Loki in CI, bad for asserting your code actually logs what it should. Use AuditTrail.Adapters.Test instead when you want that coverage:
# config/test.exs
config :audit_trail,
enabled: true,
storage_adapter: AuditTrail.Adapters.Test,
batch_size: 1 # flush every log immediately (default flush_interval is 2000ms)
defmodule MyApp.Items.ApprovalTest do
use MyApp.DataCase, async: false # shared in-memory table — don't run async
setup do
AuditTrail.Test.clear()
:ok
end
test "approving an item logs an audit event" do
{:ok, item} = Items.approve(item.id)
AuditTrail.Test.assert_logged("item:approved", %{resource_id: item.id})
end
test "rejecting an item does not log an approval" do
Items.reject(item.id)
AuditTrail.Test.assert_not_logged("item:approved")
end
end
assert_logged/2andassert_not_logged/2match onevent_typeplus an optional map of expected fields, checked recursively — so you can assert intometa:AuditTrail.Test.assert_logged("schema:update", %{meta: %{changes: %{status: %{"to" => "approved"}}}}).- Matching happens against the raw, atom-keyed log map built by
AuditTrail.Logger, not the JSON-round-tripped payload you get back fromget_logs/1. - Logging is inherently asynchronous (buffered, flushed on a timer/batch size), so
assert_logged/2polls for up to 200ms before failing. - This is a single shared ETS table, not scoped per test process — run these tests with
async: false.
Loki stream labels
Every log entry is stored in Loki as a stream with three labels:
| Label | Source | Example |
|---|---|---|
app | Config.app_name() | "heart-ke" |
type | event type | "auth:login" |
user | actor ID | "550e8400-..." |
All other fields live in the JSON log line body. Loki stream labels are what make queries fast — the actor_id filter works because the user label is indexed.
Cardinality note: the
userlabel creates one stream per(app, type, user)combination. For apps with tens of thousands of users this is high cardinality. If Loki performance degrades, switch to a JSON line filter in the reader:| json | actor_id="<uuid>"and removeuserfrom the stream labels inShipper.build_loki_body/1.
Crash reporting
When crash_reporting: true (the default), AuditTrail registers an OTP :logger handler that intercepts :error, :critical, :emergency, and :alert level events.
- Crash details (exception, module, stacktrace, PID, actor context) are extracted
- Crashes are deduplicated — the same exception within
dedup_window_minutesis emailed only once - An email is routed to the matching team via
crash_routing - The crash is also emitted to Loki as
event_type: "crash:error"so it appears in Grafana
config :audit_trail,
crash_routing: [
%{match: {:module, ~r/MyApp\.Payments/}, to: ["payments@company.com"]},
%{match: :default, to: ["engineering@company.com"]}
]
To disable:
config :audit_trail, crash_reporting: false
Limitations
| Limitation | Detail |
|---|---|
| Loki dependency (Loki adapter only) | If Loki is unreachable, logs are retried up to shipper_retries times then dropped to in-memory dead-letter. Dead-letter does not survive a node restart. Use the Postgres or TimescaleDB adapter to avoid this entirely. |
| No bulk operation tracking | Repo.update_all, Repo.delete_all, Ecto.Multi.insert_all/update_all/delete_all, and raw Ecto.Query calls bypass RepoWatcher — there's no changeset to diff. Repo.insert_or_update/Ecto.Multi.insert_or_update also bypass it (Ecto resolves these without calling back through the repo's public insert/2/update/2). |
| Nested-field filtering requires opting the embed/assoc in separately | The parent's track: [...] only filters top-level fields. To filter within an embed/association, add use AuditTrail.Schema, track: [...] to that nested schema module too — see Embeds and associations are diffed too. Embeds/associations still don't get an independent schema:* event of their own, since they don't go through the Repo directly. |
| Process-local actor context | set_actor/1 uses the process dictionary and is not inherited by child processes. Background jobs must call it themselves. |
| Log immutability (Loki adapter) | Loki logs cannot be updated or deleted once written. Postgres/TimescaleDB logs can be deleted via SQL but there is no built-in API for it. |
| Label changes not retroactive (Loki adapter) | Adding or removing a stream label (e.g. user) only affects new logs. Old logs remain unfiltered by that label. |
| Default 24h read window (Loki adapter) | get_logs/1 with the Loki adapter defaults to the last 24 hours. Pass start_date for a longer range. Postgres/TimescaleDB adapters query all rows up to limit if no date filters are given. |
| Adapter migration is one-way | Switching adapters does not migrate historical logs. Both backends can run in parallel temporarily by writing a thin adapter that fans out to both. |
| In-memory deduplication | Crash dedup state is in ETS and is lost on restart. |
resource/resource_id/operation are opt-in, not retroactive | They're plain payload fields (no migration needed — payload is already jsonb/free-form). Logs written before you start setting them have resource: nil and won't match a resource/resource_id/operation filter. resource_id auto-detection in ControllerAudit only checks id/uuid params unless you set id_param: — a different param name won't be picked up without it. |
Troubleshooting
UndefinedFunctionError: AuditTrail.ControllerAudit.init/1 is undefined
The plug macro calls init/1 at compile time. Recompile with force:
mix deps.compile audit_trail --force && mix compile --force
Warnings about undefined AuditTrail.ControllerAudit during mix compile
Same root cause as above. The library beam files must exist before the host app compiles its controllers.
Regular users see no logs (empty list)
The user Loki stream label must match what was written. Logs written before the user label was added to the shipper won't be matched by {user="<id>"}. Only newly generated logs filter correctly.
{:already_started, #PID<...>} on startup
AuditTrail is listed in application.ex children. Remove it — it starts itself as an OTP application.
Connection refused spam in dev logs
Set enabled: false in config/dev.exs. The buffer accepts calls silently but the shipper never fires.
Loki crash-loops on startup
Check for field buckets not found — the S3 config key must be bucketnames not buckets. Also ensure schema_config is present (required in Loki 3.x even with filesystem storage).