Kathikon

CICoverage StatusHex versionHex Docs

Kathikon (Greek: καθήκον - duty, obligation) is a BEAM-native durable job queue and task execution platform for Elixir.

Jobs are treated as durable obligations that must eventually be fulfilled: completed, retried, cancelled, or discarded - but never silently lost.

Kathikon uses Mnesia as its coordination store and OTP as its execution substrate. No PostgreSQL, Redis, RabbitMQ, or external brokers are required.

Status

v0.2.1 - Operations tooling

v0.2.0 - Control, scheduling, batches, and correctness

Previous: v0.1.0 - Phase 1: Durable Job Queue

Installation

def deps do
[
{:kathikon, "~> 0.2.1"}
]
end

Quick start

Define a worker:

defmodule MyApp.EmailWorker do
use Kathikon.Worker
@impl true
def perform(%Kathikon.Job{args: %{"email" => email}}) do
MyApp.Mailer.deliver(email)
:ok
end
end

Configure queues:

# config/config.exs
config :kathikon,
queues: [
default: [concurrency: 10],
emails: [concurrency: 5]
]

Enqueue jobs:

{:ok, job} = Kathikon.insert(MyApp.EmailWorker, %{"email" => "user@example.com"})
{:ok, job} =
Kathikon.insert(MyApp.EmailWorker, %{"email" => "user@example.com"},
queue: :emails,
priority: 5,
schedule_in: 60,
max_attempts: 10
)

Cancel a pending job:

{:ok, job} = Kathikon.cancel(job_id)

Architecture

Kathikon.Supervisor
├── Registry
├── Kathikon.Queue (DynamicSupervisor)
│ └── Kathikon.Dispatcher (one per queue)
├── Kathikon.Scheduler.Promoter
└── Kathikon.Pruner
ModuleRole
Kathikon.JobJob struct and state machine
Kathikon.WorkerWorker behaviour (perform/1)
Kathikon.StorageStorage behaviour (default: Kathikon.Storage.Mnesia)
Kathikon.DashboardOps facade - queue summary, job lists, bulk control
Kathikon.ReportQueue/job reporting helpers
Kathikon.BatchFan-out/fan-in batch workflows
Kathikon.Scheduler.PromoterPromotes scheduled jobs to available
Kathikon.PrunerEnforces retention on terminal jobs
Kathikon.Telemetry[:kathikon, ...] telemetry events

Job states

scheduledavailableclaimedrunningcompleted
↘ ↘ retryableavailable
cancelled faileddead

Telemetry

Kathikon emits standard telemetry events. See Telemetry guide for the full list. Highlights:

Attach the default logger in development:

Kathikon.Telemetry.attach_default_logger()

Configuration

KeyDefaultDescription
:queues[default: [concurrency: 10]]Queue names and concurrency
:poll_interval200Dispatcher poll interval (ms)
:scheduler_interval1000Scheduler tick interval (ms)
:prune_interval60000Pruner tick interval (ms)
:retention_period7 daysHow long to keep terminal jobs (ms)
:max_attempts20Default retry limit
:storage_backendKathikon.Storage.MnesiaStorage implementation
:mnesia_copies:autoMnesia storage: :ram, :disc, or :auto (ram on nonode@nohost and Livebook nodes)

Examples

Runnable scripts under examples/ (use mix run examples/<name>.exs):

ScriptDemonstrates
basic_worker.exsInsert and poll status
scheduled_job.exsschedule with :at / :in
dead_letter_retry.exsFailures, dead letter, rerun
batch_fanout_fanin.exsParent/child batches
reporting.exsKathikon.Report summaries
quantum_scheduler_adapter.exsOptional Quantum scheduler

Operations CLI

Inspect and control a running node from the terminal:

mix kathikon.ops summary
mix kathikon.ops jobs --queue default --tab completed --limit 20
mix kathikon.ops pause --all

Remote (named node + matching cookie on both sides):

elixir --name ops@127.0.0.1 --cookie SECRET -S mix kathikon.ops --node kathikon@127.0.0.1 summary

See Management API and Kathikon.Dashboard docs.

Roadmap

PhaseFocus
1Durable job queue (done)
2Distributed coordination, leases, lifeline
3Uniqueness, dynamic queues (cron done in v0.2)
4Rate limits (pause/resume done in v0.2)
5Batches (done in v0.2)
6Observability APIs (reporting done in v0.2)
7Workflows and DAGs
8LiveView dashboard UI (ops API done in v0.2.1)

Documentation

Generate HTML docs with ExDoc:

mix docs
open doc/index.html

License

MIT. See LICENSE.