FerricStore Elixir SDK

Elixir SDK for FerricStore and FerricFlow over native TCP and stateless HTTP.

Status: public beta. SDK 0.11.14 requires FerricStore ~> 0.11.4, negotiates compact Stream mode 34 and compact Pub/Sub mode 35 with FerricStore 0.11.8 and later, and is validated against FerricStore 0.11.11. Native wire framing and the generic compatibility paths remain protocol v1. APIs may change before 1.0, but the SDK is covered by command-construction tests, architecture tests, Docker-backed integration tests, and local benchmark scripts.

FerricFlow keeps each workflow or job's state and history in one durable place. It is an explicit durable state pipeline, not a hidden deterministic replay engine:

create -> claim -> handler -> transition/complete/retry/fail

Handlers should be idempotent because work can be retried after lease expiry, worker crash, or explicit retry.

Durability is the default contract. A workflow command returns success only after the state change is accepted by FerricStore and written through its durable path.

First 10 minutes

1. Install

def deps do
  [
    {:ferricstore_sdk, "~> 0.11.14"}
  ]
end

For local SDK development:

mix deps.get
mix test

2. Start FerricStore

For local development, run the same immutable FerricStore 0.11.11 image used by the SDK integration workflow:

docker run --rm \
  -e FERRICSTORE_PROTECTED_MODE=false \
  -e FERRICSTORE_NATIVE_ADVERTISE_HOST=127.0.0.1 \
  -e FERRICSTORE_NATIVE_ADVERTISE_PORT=6388 \
  -p 6388:6388 \
  quay.io/ferricstore/ferricstore:0.11.11@sha256:d9f488539f0d6c1a513d2315e7a9c2947cc795b393f3774c9de8ba5e5b5c21b5

The SDK examples assume:

ferric://127.0.0.1:6388

3. Connect

{:ok, client} = FerricStore.start_link(url: "ferric://127.0.0.1:6388")

:ok = FerricStore.set(client, "hello", "world")
"world" = FerricStore.get(client, "hello")

The same command API can use a FerricStore HTTP server. HTTP/1.1 keeps connections alive; set http2: true for a multiplexed HTTP/2 connection:

{:ok, client} =
  FerricStore.SDK.from_url("https://ferricstore-http.example.com",
    username: "default",
    password: password,
    http2: true
  )

{:ok, "PONG"} = FerricStore.SDK.ping(client)

Use bearer_token: for Bearer authentication. Basic username/password authentication is accepted only with https://; omitting the username uses default. An SDK pipeline becomes one ordered HTTP request. Limits include timeout:, max_request_bytes:, max_response_bytes:, max_batch_items:, max_connections:, and max_concurrent_requests:. For HTTP/1.1, max_connections: is the default per-client admission width; the shared Finch pool remains capped at 100 connections per origin. HTTP/2 uses one multiplexed connection and max_concurrent_requests: bounds active streams.

For private certificate authorities, configure the shared Finch pools before the application starts:

config :ferricstore_sdk,
  http_pool_transport_options: [verify: :verify_peer, cacertfile: "/etc/ssl/ferricstore-ca.pem"]

HTTP requests are stateless. AUTH, CLIENT, transactions, Pub/Sub subscriptions, and WATCH require native TCP and fail locally when used through HTTP. BLPOP, BRPOP, BLMOVE, BLMPOP, XREAD BLOCK, and XREADGROUP BLOCK run as long-lived HTTP requests and may appear with ordinary commands in one ordered SDK pipeline. Finite blocking waits extend the implicit SDK deadline; BLOCK 0 removes that implicit deadline. An explicit request timeout: or call_timeout: remains authoritative, so use an explicit finite deadline when an unbounded wait is not intended. Redirects retain authentication and custom headers across origins; configure only endpoints and redirect targets you trust. The SDK owns HTTP framing headers such as host, content-length, and transfer-encoding; custom headers cannot override them.

Run the complete HTTP-compatible integration surface through a real TLS listener with ACL authentication using:

FERRICSTORE_TEST_IMAGE=quay.io/ferricstore/ferricstore:0.11.11@sha256:d9f488539f0d6c1a513d2315e7a9c2947cc795b393f3774c9de8ba5e5b5c21b5 \
  scripts/test_http_integration.sh

The runner creates a private CA, verifies that unauthenticated access and a restricted user's forbidden SET are rejected, and supplies FERRICSTORE_USERNAME, FERRICSTORE_PASSWORD, and FERRICSTORE_CA_FILE. Connection-affine tests remain in the native integration job.

4. Query durable runs

Use parameterized FQL for bounded, partition-scoped reads. Cursors are opaque and must be reused with the same query and parameters.

query = """
FROM runs
WHERE partition_key = @partition AND type = @type AND state = @state
ORDER BY updated_at_ms DESC
LIMIT 25
RETURN RECORDS
"""

params = %{"partition" => "partition-a", "type" => "invoice", "state" => "queued"}
%FerricStore.Flow.QueryResult{records: records, page: page} =
  FerricStore.Flow.query(client, query, params)

%FerricStore.Flow.QueryExplainResult{} = FerricStore.Flow.explain(client, query, params)
%FerricStore.Flow.QueryIndexStatus{} = FerricStore.Flow.query_indexes(client)

Each %FerricStore.Flow.QueryIndex{} reports covering_fields, including covered dynamic attribute.* and state_meta.* paths, and an opaque format describing its derived-storage generation. Use format changes to identify a rebuild requirement; do not decode server storage from these values. The counter format is nil when an index has no exact count prefix.

Select a sparse result map by adding up to 32 source-specific fields after RETURN RECORD or RETURN RECORDS, for example RETURN RECORDS (run_id, state, attribute['customer']). A bare return keeps the complete public record. Projection runs after authorization, authoritative recheck, ordering, and cursor calculation: it reduces retained result data, encoding, network, and client decoding work, but not index scans or hydration.

Build the return clause with validated source-aware selectors instead of hand-quoting metadata names:

{:ok, projected} =
  FerricStore.Flow.QueryProjection.project(
    "FROM runs WHERE partition_key = @partition AND run_id = @run",
    :record,
    [:run_id, :state, {:attribute, "customer"}]
  )

%FerricStore.Flow.QueryResult{} = FerricStore.Flow.query(client, projected, params)

Collection helpers accept the same run selectors through fields: and compile the projection into FQL before transport:

records =
  FerricStore.Flow.list(client,
    type: "invoice",
    state: "queued",
    partition_key: "tenant-a",
    fields: [:run_id, :state, {:attribute, "customer"}]
  )

fields: is supported by list, search, terminals, failures, stuck, by_parent, by_root, and by_correlation. Omitting it returns complete public records. An explicit projection must contain 1 to 32 unique, source-valid selectors.

5. Create a durable queue item

queue = FerricStore.Queue.new(client, "email", worker: "email-worker")

FerricStore.Queue.enqueue(queue, "email-1",
  payload: "welcome:user-1",
  attributes: %{tenant: "acme", campaign: "summer"}
)

Attributes are small indexed metadata. They are useful for search, filtering, and debugging. They are not payload bytes.

5. Process one queue batch

FerricStore.Queue.run_once(queue, fn job ->
  send_email(job["payload"])
  "sent"
end)

run_once/3 claims due work and completes or fails the job based on the handler result. For a long-running worker, call it from a supervised process with your own shutdown and concurrency policy.

6. Create a workflow/state machine

Use workflows when one durable flow moves through named states.

workflow = FerricStore.Workflow.new(client, "order", initial_state: "created")

FerricStore.Workflow.start(workflow, "order-1",
  payload: "order payload",
  attributes: %{tenant: "acme"},
  values: %{order: :erlang.term_to_binary(%{total: 120})}
)

Claim, transition, and complete explicitly:

[job | _] = FerricStore.Workflow.claim(workflow, "created", limit: 1)

FerricStore.Workflow.transition(workflow, job["id"], "running", "charged",
  partition_key: job["partition_key"],
  lease_token: job["lease_token"],
  fencing_token: job["fencing_token"],
  payload: "charged"
)

[job | _] = FerricStore.Workflow.claim(workflow, "charged", limit: 1)

FerricStore.Workflow.complete(workflow, job["id"],
  partition_key: job["partition_key"],
  lease_token: job["lease_token"],
  fencing_token: job["fencing_token"],
  result: "ok"
)

After claim_due, the current durable state is running; the original claimed state is tracked as run state. Pass from_state: "running" when transitioning a claimed job.

7. Store and fetch named values

Use named values/value refs when different states need different pieces of data. Values are only hydrated when requested.

meta = FerricStore.Flow.value_put(client, "large invoice bytes",
  owner_flow_id: "order-1",
  name: "invoice_pdf",
  override: false
)

ref = meta["ref"]
["large invoice bytes"] = FerricStore.Flow.value_mget(client, [ref])

Keep override: false for normal first-write values. Use override: true only when replacing a value is intentional.

8. Inspect state and history

record = FerricStore.Flow.get(client, "order-1", payload: true)
history = FerricStore.Flow.history(client, "order-1")

History is for debugging and audit. Handlers should use claimed job data and requested values, not history replay.

9. Index one state metadata key

State metadata is stored per flow state. A flow type may choose one state metadata key for server-side indexing:

%FerricStore.Flow.PolicySnapshot{generation: generation} =
  FerricStore.Flow.policy_set(client, "order", indexed_state_meta: "version")

FerricStore.Flow.create(client, "order-2",
  type: "order",
  state: "accept",
  state_meta: %{version: 1, owner: "risk"}
)

FerricStore.Flow.search(client,
  type: "order",
  state: "accept",
  state_meta: %{version: 1},
  count: 10
)

FIFO state policy is opt-in per state. Use an explicit partition key for records that enter FIFO states; priority ordering is a parallel-state feature and the server rejects priority on FIFO entries:

FerricStore.Flow.policy_set(client, "order",
  states: %{"created" => [mode: :fifo]}
)

FerricStore.Flow.create(client, "order-3",
  type: "order",
  state: "created",
  partition_key: "tenant-a:order-3",
  payload: "payload"
)

Direct policy updates deep-patch by default. Use replace: true to reset omitted fields, or compare-and-swap against a snapshot generation:

FerricStore.Flow.policy_set(client, "order",
  expected_generation: generation,
  states: %{"created" => [mode: :fifo]}
)

Stale generations return FerricStore.Flow.StalePolicyGenerationError and are never retried automatically. Workflow.install_policy/2 uses full replacement by default because a workflow declaration is a complete policy snapshot.

Use FerricStore.SDK when you want explicit {:ok, value} results and the complete topology-aware API surface:

{:ok, sdk} = FerricStore.SDK.start_link(url: "ferric://127.0.0.1:6388")
{:ok, :ok} = FerricStore.SDK.set(sdk, "{tenant:1}:hello", "world")
{:ok, "world"} = FerricStore.SDK.get(sdk, "{tenant:1}:hello")

FerricStore.start_link/1 and FerricStore.SDK.start_link/1 return the same topology-aware client type. It can be shared across FerricStore, FerricStore.Flow, Queue, Workflow, and every FerricStore.SDK namespace.

10. Probe management capabilities

Control-plane callers should probe capabilities before enabling management UI or automation:

{:ok, caps} = FerricStore.SDK.capabilities(sdk)

if caps["acl_management"] do
  FerricStore.SDK.acl_set_user(sdk, "platform_worker_abcd", [
    "on",
    ">secret",
    "+PING",
    "+@read",
    "+@write",
    "-@dangerous",
    "-@admin",
    "~tenant:namespace:*"
  ])
end

The SDK also exposes narrow namespace, quota, and safe telemetry helpers through FerricStore.SDK.Management and top-level FerricStore.SDK delegates.

11. Enterprise invocation helpers

FerricStore Enterprise exposes invocation definitions and invocation creation through the same native SDK client:

{:ok, sdk} = FerricStore.SDK.start_link(url: "ferric://127.0.0.1:6388")

{:ok, definition} =
  FerricStore.SDK.invocation_definition_put(sdk, %{
    name: "send-email",
    acl: %{scope_required: true},
    partition: %{key: "tenant:{tenant}:invocation:send-email"}
  })

{:ok, created} =
  FerricStore.SDK.invocation_create(sdk, "send-email", %{tenant: "acme"},
    context: %{subject: "user-1"}
  )

{:ok, invocation} = FerricStore.SDK.invocation_get(sdk, created["invocation_id"])
{:ok, partitions} = FerricStore.SDK.invocation_partition_list(sdk, "send-email")

Trusted proxy deployments can pass request_context: %{...}. The context is sent out-of-band through the native command envelope, so untrusted callers cannot spoof it by only editing the invocation payload.

What you use

• FerricStore for native protocol connection and KV/data-structure helpers.
• FerricStore.SDK for topology-aware routing and native command wrappers.
• FerricStore.SDK.Flow, FerricStore.SDK.Admin, and FerricStore.SDK.Management for typed-map Flow, admin, and control-plane command surfaces.
• FerricStore.Flow for exact FerricFlow command-level control.
• FerricStore.Queue for simple durable queue helpers.
• FerricStore.Workflow for explicit state-machine helpers.
• FerricStore.Codec.Raw by default.
• FerricStore.Codec.Term for Elixir-only term payloads.
• FerricStore.command/4 as the low-level command escape hatch.

Production shape

Use one process/service to create work and a separate long-lived worker service to claim and complete work.

Phoenix/API/serverless producer -> FerricStore -> supervised worker service

Before production, configure timeouts, lease duration, backpressure behavior, graceful shutdown, and value hydration caps. The ferric:// transport uses one multiplexed native socket per SDK client process. HTTP/1.1 uses a bounded shared keep-alive pool and HTTP/2 multiplexes requests over a shared connection; create more clients only after profiling shows client-side saturation.

Docs

• Documentation index
• Quickstart
• Client API
• Workflow and queue APIs
• Data, attributes, and value refs
• Configuration
• Production readiness
• Use cases
• Web and serverless usage
• Testing
• Troubleshooting
• Benchmark notes
• Development checks

Integration tests

Integration tests are explicit ExUnit integration tests. They run against the same Docker image used by CI:

scripts/test_integration.sh