Ankusa

Don't fight the traffic. Steer it.

Ankusa is a self-hosted webhook receiver. Point Stripe, GitHub, or any provider at it: every hook is written to a durable log before Ankusa answers 2xx, every accepted POST is stored under a fresh id, and each hook is delivered to your own worker over HTTP, RabbitMQ, Kafka, or NATS JetStream, with retries, a dead-letter queue, and replay. Start with one container. Grow into a fleet by changing config, not code.

Quickstart

1. Run it

docker run -d --name ankusa \
  -p 4000:4000 -p 127.0.0.1:4002:4002 \
  -v ankusa-data:/var/lib/ankusa \
  jamescarr/ankusa:edge

# the image ships a healthcheck: wait for it rather than racing the listener
until [ "$(docker inspect --format '{{.State.Health.Status}}' ankusa)" = healthy ]; do sleep 1; done

curl -XPOST localhost:4000/webhooks/demo -H 'content-type: application/json' -d '{"id":"evt_1"}'
# => {"id":"01a0...","status":"accepted","seq":1}   (returned only after the WAL fsync)

Done? docker rm -f ankusa

That is a complete, durable webhook receiver. The image ships a demo source that accepts anything, so it works before you have any provider credentials: 4000 is ingest, 4002 is the operator API (/health, /metrics), and /var/lib/ankusa holds the only state worth keeping.

2. Deliver to your own worker

Ankusa POSTs each hook to your endpoint with the raw body intact and x-ankusa-id for idempotency; short outages are retried, long ones land in a dead-letter queue you can replay with one call.

git clone https://github.com/jamescarr/ankusa
cd ankusa/examples/quickstart
docker compose up -d --wait
curl -XPOST localhost:4000/webhooks/demo -H 'content-type: application/json' -d '{"id":"evt_1","type":"invoice.paid"}'
sleep 1 && docker compose logs worker
# received id=01a0... source=demo seq=1 bytes=36 body={"id":"evt_1","type":"invoice.paid"}

The wiring is one sink in ankusa.yml:

sources:
  demo:
    verify: {type: none}
    on_verify_failure: accept_flag
    sinks:
      - type: http
        url: http://worker:8080/hooks
        timeout_ms: 5000

The worker is 40 lines of standard-library Python; replace it with your own service. The quickstart guide walks through outages, dead letters, replay, and pointing a real provider at it.

How it works

Ankusa writes every hook to a durable log before it answers 2xx: the fsync lands first, and 201 accepted is the only committed response. There is no 200. If a node dies before the write, the provider never got an ack and retries; that retry is a new hook with a new id, stored and delivered again, because ingest does no deduplication and delivery is at-least-once. When storage slows down, Ankusa answers 503 with Retry-After, so providers back off and try again instead of losing events. You get that guarantee on day one, on one machine, and you keep it when you run a hundred.

docs/architecture.md walks the full pipeline and shows why each guarantee holds.

flowchart LR
    P[Provider] -->|POST catch URL| E[Edge]
    E --> B[Group-commit Batcher]
    B -->|one fsync| W[(WAL)]
    W -->|ack| P
    W --> C[Compactor]
    W --> D[Dispatch]
    C --> S[(Object store)]
    D --> SK[Sinks]

What you get

• Signature verification via a configurable HMAC engine, with named schemes for Stripe, GitHub, Standard Webhooks, Shopify, and Slack, plus any body-HMAC scheme you describe in config
• Multi-tenant catch URLs, with a pluggable resolver for custom schemes
• Delivery over HTTP, RabbitMQ, Kafka, and NATS JetStream, with retries, backoff, a dead letter queue, and replay
• Quarantine for hooks that fail verification, so nothing is silently dropped
• Archiving to S3, GCS, Azure Blob Storage, OCI Object Storage, Cloudflare R2, or MinIO
• Telemetry on every stage of the pipeline
• A clean handoff to job frameworks like Oban and Celery

Every piece is a swappable module, so when your needs outgrow a default you replace that piece and keep the rest.

Grow into a fleet

When one box is not enough, run N independent all-role nodes behind a load balancer. Each node has its own data volume, its own WAL, and its own DLQ/admin API. Archive to S3 or GCS, and fan out to Kafka, NATS, or RabbitMQ. WAL.DiskLog keeps edge, dispatch, and storage in one BEAM node, so a node is the unit of scale: add nodes, not roles. Give each node its own bucket (or LocalFS) for segments. Segment keys are seg/<first_seq>-<last_seq>.seg, and remote blob stores ignore the instance, so nodes sharing a bucket overwrite each other's segments.

flowchart LR
    P[Stripe, GitHub, ...] --> LB[Load balancer]
    LB --> E1[Ankusa node 1]
    LB --> E2[Ankusa node 2]
    LB --> E3[Ankusa node N]
    E1 --> WAL1[("own WAL\n+ volume")]
    E2 --> WAL2[("own WAL\n+ volume")]
    E3 --> WAL3[("own WAL\n+ volume")]
    WAL1 & WAL2 & WAL3 --> Q[Kafka / NATS / RabbitMQ / HTTP]
    WAL1 & WAL2 & WAL3 --> S[(S3 / GCS)]
    Q --> W[Your workers]

Your workers can be written in anything. A TypeScript, Python, or Go consumer reads from the queue and fetches large payloads over HTTP without ever holding storage credentials.

Roles, the container, and fleets: docs/deployment.md.

Examples

How to pick one: examples/README.md

Documentation

• Quickstart
• Configuration
• Deployment and scaling
• Delivery, retries, and dead letters
• Integrations: Oban, Celery, queues
• Architecture and guarantees
• Storage
• Multi-tenancy
• Claim check

Ankusa is also an Elixir library you can embed in your own app: docs/elixir.md · HexDocs.

The name

अंकुश (aṅkuśa) is Sanskrit for "hook" or "goad": the curved tool a mahout uses to steer an elephant. Its root, aṅka, means "to bend" or "curve".

Webhook traffic behaves like the elephant. It is large, it arrives on its own schedule, and it will not wait for you. So the framework takes the same shape: absorb the traffic durably and fast through the WAL and batcher, then steer where it goes next through routing and dispatch.

License

Apache 2.0. See LICENSE.