Kathikon
Kathikon (Greek: καθήκον - duty, obligation) is a BEAM-native durable job queue and task execution platform for Elixir.
Jobs are treated as durable obligations that must eventually be fulfilled: completed, retried, cancelled, or discarded - but never silently lost.
Kathikon uses Mnesia as its coordination store and OTP as its execution substrate. No PostgreSQL, Redis, RabbitMQ, or external brokers are required.
Status
v0.2.1 - Operations tooling
Kathikon.Dashboard- inspect and control facade for UIs and RPCmix kathikon.ops- terminal CLI (summary,jobs, pause/resume, retry, purge, …)- Remote ops over Erlang distribution (
Kathikon.Dashboard.RPC)
v0.2.0 - Control, scheduling, batches, and correctness
- Storage behaviour with atomic lifecycle operations
- Formal job state machine and durable history
- Dead-letter queue with rerun semantics
Kathikon.schedule/3and scheduler adapters (built-in + optional Quantum)- Management and reporting APIs
- Parent/child batch workflows
Previous: v0.1.0 - Phase 1: Durable Job Queue
Installation
def deps do
[
{:kathikon, "~> 0.2.1"}
]
end
Quick start
Define a worker:
defmodule MyApp.EmailWorker do
use Kathikon.Worker
@impl true
def perform(%Kathikon.Job{args: %{"email" => email}}) do
MyApp.Mailer.deliver(email)
:ok
end
end
Configure queues:
# config/config.exs
config :kathikon,
queues: [
default: [concurrency: 10],
emails: [concurrency: 5]
]
Enqueue jobs:
{:ok, job} = Kathikon.insert(MyApp.EmailWorker, %{"email" => "user@example.com"})
{:ok, job} =
Kathikon.insert(MyApp.EmailWorker, %{"email" => "user@example.com"},
queue: :emails,
priority: 5,
schedule_in: 60,
max_attempts: 10
)
Cancel a pending job:
{:ok, job} = Kathikon.cancel(job_id)
Architecture
Kathikon.Supervisor
├── Registry
├── Kathikon.Queue (DynamicSupervisor)
│ └── Kathikon.Dispatcher (one per queue)
├── Kathikon.Scheduler.Promoter
└── Kathikon.Pruner
| Module | Role |
|---|---|
Kathikon.Job | Job struct and state machine |
Kathikon.Worker | Worker behaviour (perform/1) |
Kathikon.Storage | Storage behaviour (default: Kathikon.Storage.Mnesia) |
Kathikon.Dashboard | Ops facade - queue summary, job lists, bulk control |
Kathikon.Report | Queue/job reporting helpers |
Kathikon.Batch | Fan-out/fan-in batch workflows |
Kathikon.Scheduler.Promoter | Promotes scheduled jobs to available |
Kathikon.Pruner | Enforces retention on terminal jobs |
Kathikon.Telemetry | [:kathikon, ...] telemetry events |
Job states
scheduled → available → claimed → running → completed
↘ ↘ retryable → available
cancelled failed → dead
Telemetry
Kathikon emits standard telemetry events. See Telemetry guide for the full list. Highlights:
- Job:
[:kathikon, :job, :inserted],:claimed,:started,:completed,:failed,:sleep,:retry,:discard,:cancel,:dead,:prune - Runtime:
[:kathikon, :scheduler, :tick],[:kathikon, :scheduler, :fired],[:kathikon, :pruner, :tick],[:kathikon, :dispatcher, :poll]
Attach the default logger in development:
Kathikon.Telemetry.attach_default_logger()
Configuration
| Key | Default | Description |
|---|---|---|
:queues | [default: [concurrency: 10]] | Queue names and concurrency |
:poll_interval | 200 | Dispatcher poll interval (ms) |
:scheduler_interval | 1000 | Scheduler tick interval (ms) |
:prune_interval | 60000 | Pruner tick interval (ms) |
:retention_period | 7 days | How long to keep terminal jobs (ms) |
:max_attempts | 20 | Default retry limit |
:storage_backend | Kathikon.Storage.Mnesia | Storage implementation |
:mnesia_copies | :auto | Mnesia storage: :ram, :disc, or :auto (ram on nonode@nohost and Livebook nodes) |
Examples
Runnable scripts under examples/ (use mix run examples/<name>.exs):
| Script | Demonstrates |
|---|---|
| basic_worker.exs | Insert and poll status |
| scheduled_job.exs | schedule with :at / :in |
| dead_letter_retry.exs | Failures, dead letter, rerun |
| batch_fanout_fanin.exs | Parent/child batches |
| reporting.exs | Kathikon.Report summaries |
| quantum_scheduler_adapter.exs | Optional Quantum scheduler |
Operations CLI
Inspect and control a running node from the terminal:
mix kathikon.ops summary
mix kathikon.ops jobs --queue default --tab completed --limit 20
mix kathikon.ops pause --all
Remote (named node + matching cookie on both sides):
elixir --name ops@127.0.0.1 --cookie SECRET -S mix kathikon.ops --node kathikon@127.0.0.1 summary
See Management API and Kathikon.Dashboard docs.
Roadmap
| Phase | Focus |
|---|---|
| 1 | Durable job queue (done) |
| 2 | Distributed coordination, leases, lifeline |
| 3 | Uniqueness, dynamic queues (cron done in v0.2) |
| 4 | Rate limits (pause/resume done in v0.2) |
| 5 | Batches (done in v0.2) |
| 6 | Observability APIs (reporting done in v0.2) |
| 7 | Workflows and DAGs |
| 8 | LiveView dashboard UI (ops API done in v0.2.1) |
Documentation
Generate HTML docs with ExDoc:
mix docs
open doc/index.html
- Documentation index - guides and module reference (source)
- CHANGELOG - release history
- Quick start
- Module reference
- Configuration
- Interactive demo (Livebook)
License
MIT. See LICENSE.