Kathikon

CICoverage StatusHex versionHex Docs

Kathikon (Greek: καθήκον — duty, obligation) is a BEAM-native durable job queue and task execution platform for Elixir.

Jobs are treated as durable obligations that must eventually be fulfilled: completed, retried, cancelled, or discarded — but never silently lost.

Kathikon uses Mnesia as its coordination store and OTP as its execution substrate. No PostgreSQL, Redis, RabbitMQ, or external brokers are required.

Status

v0.2.0 — Control, scheduling, batches, and correctness

Previous: v0.1.0 — Phase 1: Durable Job Queue

Installation

def deps do
[
{:kathikon, "~> 0.2.0"}
]
end

Quick start

Define a worker:

defmodule MyApp.EmailWorker do
use Kathikon.Worker
@impl true
def perform(%Kathikon.Job{args: %{"email" => email}}) do
MyApp.Mailer.deliver(email)
:ok
end
end

Configure queues:

# config/config.exs
config :kathikon,
queues: [
default: [concurrency: 10],
emails: [concurrency: 5]
]

Enqueue jobs:

{:ok, job} = Kathikon.insert(MyApp.EmailWorker, %{"email" => "user@example.com"})
{:ok, job} =
Kathikon.insert(MyApp.EmailWorker, %{"email" => "user@example.com"},
queue: :emails,
priority: 5,
schedule_in: 60,
max_attempts: 10
)

Cancel a pending job:

{:ok, job} = Kathikon.cancel(job_id)

Architecture

Kathikon.Supervisor
├── Registry
├── Kathikon.Queue (DynamicSupervisor)
│ └── Kathikon.Dispatcher (one per queue)
├── Kathikon.Scheduler.Promoter
└── Kathikon.Pruner
ModuleRole
Kathikon.JobJob struct and state machine
Kathikon.WorkerWorker behaviour (perform/1)
Kathikon.StorageStorage behaviour (default: Kathikon.Storage.Mnesia)
Kathikon.ReportQueue/job reporting helpers
Kathikon.BatchFan-out/fan-in batch workflows
Kathikon.Scheduler.PromoterPromotes scheduled jobs to available
Kathikon.PrunerEnforces retention on terminal jobs
Kathikon.Telemetry[:kathikon, ...] telemetry events

Job states

scheduledavailableclaimedrunningcompleted
↘ ↘ retryableavailable
cancelled faileddead

Telemetry

Kathikon emits standard telemetry events. See Telemetry guide for the full list. Highlights:

Attach the default logger in development:

Kathikon.Telemetry.attach_default_logger()

Configuration

KeyDefaultDescription
:queues[default: [concurrency: 10]]Queue names and concurrency
:poll_interval200Dispatcher poll interval (ms)
:scheduler_interval1000Scheduler tick interval (ms)
:prune_interval60000Pruner tick interval (ms)
:retention_period7 daysHow long to keep terminal jobs (ms)
:max_attempts20Default retry limit
:storage_backendKathikon.Storage.MnesiaStorage implementation
:mnesia_copies:autoMnesia storage: :ram, :disc, or :auto (ram on nonode@nohost and Livebook nodes)

Examples

Runnable scripts under examples/ (use mix run examples/<name>.exs):

ScriptDemonstrates
basic_worker.exsInsert and poll status
scheduled_job.exsschedule with :at / :in
dead_letter_retry.exsFailures, dead letter, rerun
batch_fanout_fanin.exsParent/child batches
reporting.exsKathikon.Report summaries
quantum_scheduler_adapter.exsOptional Quantum scheduler

Roadmap

PhaseFocus
1Durable job queue (done)
2Distributed coordination, leases, lifeline
3Uniqueness, dynamic queues (cron done in v0.2)
4Rate limits (pause/resume done in v0.2)
5Batches (done in v0.2)
6Observability APIs (reporting done in v0.2)
7Workflows and DAGs
8Optional LiveView dashboard

Documentation

Generate HTML docs with ExDoc:

mix docs
open doc/index.html

License

MIT. See LICENSE.