gleamunison

Content-addressed language runtime on the BEAM, built in Gleam.

A working implementation of a Unison-style content-addressed programming language: a definition is identified by its canonical structure and inferred type, compiled on demand to BEAM bytecode, and loaded into a running VM by hash. Not a filename, not a module path, not a package — the hash is the identity.

• Version: 3.26.0
• Verified local suite: 822 tests, no failures (factory acceptance)
• CI: Erlang/OTP 27, 28, 29 × Ubuntu, macOS — matrix configured; local receipts are not a substitute for every hosted leg
• Classification: authenticated-single-node-service (production witness)

Status: what this is, and what it is not

The honest framing matters more than the marketing. This runtime is a serious content-addressed execution platform with a strong identity model. It is not a safe place to run arbitrary untrusted code, and it does not claim to be.

The untrusted-execution deployment profile is deliberately disabled in the codebase: it refuses to admit work until a real OS-level sandbox exists.

Rationale & Gap Analysis

Why Gleamunison?

Gleamunison combines BEAM concurrency with Unison-style content identity and algebraic effects. Its verified factory loop turns bounded model output into admitted definitions, durable application facts, and HTTP results. Hot code loading and effect handlers do not by themselves provide zero-downtime application upgrades or an operating-system sandbox.

Feature Set Differences (Gleamunison vs Unison)

Complexity vs. Utility

How Hashes Act as Executables

A program is not a monolithic binary. It is a Merkle DAG of content-addressed AST definitions stored in a codebase.

The Merkle DAG code structure

Each definition is parsed into an AST and hashed on its structure, stripping variable names to ensure α-equivalence. References to other definitions are stored as hashes (RefTo nodes), so the graph is content-addressed all the way down.

                  ┌───────────────────────┐
                  │   start-server Hash   │  ◄─── This single Root Hash IS the executable
                  │       [#ff45e8]       │
                  └───────────┬───────────┘
                              │
             ┌────────────────┴────────────────┐
             ▼                                 ▼
 ┌───────────────────────┐         ┌───────────────────────┐
 │  page-template Hash   │         │    query-db Hash      │
 │       [#ac88b2]       │         │       [#3b12ef]       │
 └───────────────────────┘         └───────────┬───────────┘
                                               │
                                               ▼
                                   ┌───────────────────────┐
                                   │  Datalog-Transact Jet │
                                   │  [#00000064] (Builtin)│
                                   └───────────────────────┘

Step-by-step execution

1. DAG resolution — the runtime looks up the root hash and walks the dependency graph in the selected local store and branch namespace. Missing content must be transferred explicitly with sync or pkg --peer before run; execution does not fetch from peers implicitly.
2. On-the-fly compilation — resolved AST nodes compile to BEAM bytecode in modules named after their hashes (e.g. m_ff45e8).
3. Dynamic loading — the VM loads the binary with code:load_binary/3 and calls the entrypoint 'm_ff45e8':'$eval'().

Hash-named modules allow old and new definitions to coexist without filename collisions. Branch namespaces select names; content refs select immutable definitions. Application upgrade semantics remain the caller's responsibility.

What works today

These are implemented and exercised by the suite or the playbook — not aspirations.

• Hash-named code loading. Compile new logic to a new hash-named module; old and new definitions can coexist while callers choose which root or branch to invoke.
• Content-addressed sync. Peers exchange refs and pull missing definitions; production deployment remains single-node unless an external design supplies the missing trust and transport boundaries.
• Effect handlers and execution profiles. In-process ability frames and profile admission constrain evaluation. They are not OS/container isolation; the runtime and its host FFI remain inside the trusted computing base.
• Bounded evaluation. Every evaluation runs in a scope process with a wall clock deadline, a reduction ceiling, and a memory ceiling. Descendant process trees are owned and drained on timeout rather than leaked.
• Supervised services. HTTP and MCP run under supervisors with graceful drain; readiness reports storage, supervisors, and queues, and returns 503 when any of them is unhealthy.
• Reproducible evidence. Attestations bind results to definition hashes, so a claim of "this passed" names exactly which code passed.
• Self-hosted Datalog. The database engine is written in Gleamunison's own language (13 .lisp modules under gleamunison_datalog/src/), not in Gleam — the runtime dogfoods itself.
• Agent interfaces and admitted writes. MCP, LSP, and HTTP share the selected durable branch/CAS read model with explicit scratch overlays. agent_generate composes bounded generation with the existing publication gate; it does not create a second write authority.
• Durable applications. The typed Db ability journals application facts, and admitted definitions execute through /app/:name or a bound content ref.

Designed, not implemented

Named here so nobody plans around them by accident:

• Live process migration (serialize a continuation, ship it, resume remotely).
• OS-level sandbox for third-party code (untrusted-execution profile).
• Signed package distribution with identity and revocation.
• Multi-tenant isolation across trust domains.

Production boundaries

The verified release classification is authenticated-single-node-service. Each boundary has one policy owner and executable evidence.

Everything above is verified by scripts/verify_production_boundaries.clj, which runs the full battery and writes a machine-readable witness to docs/verification/.

Explicitly not claimed: transport encryption (no TLS), OS-level isolation, package signing, and multi-node trust. Deployments that need those must supply them at the infrastructure layer.

Project state

At v3.25.0, the local factory acceptance proof covers generation, signed admission, durable Db recovery, and /app execution. The full battery is recorded in the factory readiness witness; the suite has since grown to 822 tests, no failures. The hosted CI matrix remains a separate check, not a local claim.

Dated witnesses own counts and measurements. This guide owns durable contracts.

Modules

The source tree is grouped by concern; src/ is the authoritative module inventory rather than a second, manually maintained count:

Library and CLI boundaries

sap, patch, prompt, and schema_printer now compose in the bounded agent_generate loop (ADR 0072); they are not standalone CLI commands. The ordinary llm CLI still expects its own well-formed completion shape. trust is different: production publish, pkg install, and run call its policy gate; its multi-key overlap constructor remains a library/test surface.

sandbox_policy is wired: /eval, /define, /mcp, and /app/* admit through it, and the untrusted-execution profile refuses with 403 missing_external_executor.

Toolchain and CI

Kept in lockstep with the CI matrix:

Pushes to main/master and pull requests run the OTP/OS matrix. Separate jobs cover release gates and the remaining boundary batteries; see .github/workflows/test.yml for the authoritative job list.

gleam test                                    # unit and integration suite
gleam format --check src test                 # formatting gate
./build_escript.sh                            # standalone artifact
bb scripts/run_playbook_tests.clj             # default playbook gate
bb scripts/verify_production_boundaries.clj   # full boundary gate + witness

Quick start

git clone https://github.com/moneyacademyKE/gleamunison.git
cd gleamunison
gleam deps download
./build_escript.sh
./gleamunison_escript repl        # interactive REPL
./gleamunison_escript server 8080 # HTTP API
gleam test                        # run the suite

CLI

gleamunison <command>

  server [port]          start the web server (default 8080)
  repl                   interactive REPL (optionally preload a .lisp file)
  verify [file]          verify a scratch file
  publish <file.lisp>    publish definitions behind the evidence gate
  unpublish <name>       retract a branch name; preserve bytes still in use
  purge <hash> [--yes]   remove a definition and the deps only it kept alive
  log [content|exec|all]  what the store has seen (--tail N)
  test <file.lisp>       run the test forms in a file
  run <content-hash>     run a definition from the store, by hash
                         [args...] [--json] [--inspect] [--store path]
                         [--branch name] (default: main)
  llm <provider> <prompt> call an LLM (openai | anthropic)
  mcp [port] [--load f]  MCP server for agents (default 8099)
  all                    run all available levels
  schema-version         print the storage/codec schema version
  levelN                 run playbook level N

Definitions are the program

publish admits a definition only after its evidence gate passes, and admission puts the definition itself in the content-addressed store: the bytes are inserted under their content hash and the name is bound to that ref on the branch namespace. From then on the definition is fetchable and runnable by identity:

gleam run -- test lib.lisp         # evidence attaches to content hashes
gleam run -- publish lib.lisp      # admitted: bytes stored, names bound
gleam run -- run <hash>            # executes it, dependencies first
gleam run -- run <hash> 40 2       # a function takes its declared arity
gleam run -- run <hash> 40 2 --json # {"ref":"…","value":42}
gleam run -- run <hash> --inspect  # module, arity, exports: does NOT call
gleam run -- unpublish <name>      # retract the name (refuses if in use)
gleam run -- purge <hash>          # dry run: what would go, and why
gleam run -- purge <hash> --yes    # remove it and its exclusive deps
gleam run -- log content           # what entered the store, oldest first
gleam run -- log exec --tail 50    # execution attempts and their outcomes
gleam run -- log all               # both, in one chronological view

--json and --inspect are different questions. --json always executes and changes only how the result is rendered; --inspect is the only mode that does not execute — it loads the module to read its export table and never applies '$eval'. One flag cannot mean two things depending on arity.

run resolves the dependency closure before executing, so a stored definition that calls another stored definition works with no source on disk. Dependencies are referenced by name ref, which is why the namespace binding matters; builtin names (+, eq?) come from the baseline environment, the same one the REPL establishes. A ref the store does not hold fails with the missing hash instead of pretending to run.

Retraction is the reverse of admission, and it removes the facts a name stands on: the index line, the namespace binding, and the bytes. unpublish refuses rather than half-finishing — if another stored definition depends on the content, or another name on the same branch still binds it, nothing is removed and the message names what stands in the way.

Branches complicate exactly one of those facts, because the name is branch-scoped and the bytes are not. unpublish on main removes the binding there and keeps the bytes when another branch still resolves to them, naming that branch: two branches binding the same content can never both retract first, so a refusal would deadlock. The last branch to let go removes the bytes. The branch namespace is the authority for its own bindings, so a branch can still retract a name whose store-wide index line is already gone.

purge works by content hash instead of by name, and it is a dry run until --yes: it prints the removal set (the definition plus every stored dependency that nothing outside the set uses) and changes nothing. Both commands refuse when something outside the set depends on the ref, because a store that loses the thing its definitions call is a store that lies about being runnable.

The store keeps its own logs

Two append-only logs sit beside the store file, both durable across restarts:

log all merges them into one chronological view. pkg install adds no content — it installs what the store already holds — so it writes no content entry.

They are separate from the store on purpose. The store is content-addressed and immutable; a log is time-ordered. Writing a changing fact into the store would put it behind a hash that cannot change. The execution log records refusals too, because the attempt you need to see is the one that did not happen.

What admits content

Three admission mechanisms write code to the store. MCP agent_publish and successful agent_generate calls use the publication path, not a separate storage authority:

sync is not an unguarded door: accept_chunk refuses a chunk whose bytes do not hash to its ref, and refuses a conflicting chunk for a ref already held (HashConflict). A peer cannot name its bytes something they are not, because the name is the hash. What sync does not re-check is evidence — it moves content, not claims — which is why run re-checks the evidence gate at the moment of execution rather than trusting that admission happened. The gate you can rely on is the one at the point of use.

pkg install shares the contract: an absent manifest is 2 (could not resolve), a manifest the evidence gate declines is 1, and usage errors stay 1. Existence is settled before policy, so a fresh store names the missing manifest rather than asking you to configure a signing key for content it does not hold.

--branch names the namespace a run resolves dependency names through. The target is content, so its hash is the same on every branch: an explicit branch that does not exist is 2, and a branch that exists but does not bind this content runs it and says so on stderr rather than leaving you to assume --branch picked a version of the target.

Errors go to stderr, so stdout carries only the result: run … | jq works. run executes verified content by default; GLEAMUNISON_STORE_ALLOW_UNVERIFIED=1 is the explicit hatch for local work and warns on stderr. GLEAMUNISON_STORE_DIR points every command at another store directory (the index lives beside it), which is how a test or CI run keeps its own.

Configuration

Gleam-side settings are resolved CLI → TOML → environment. The optional TOML file is gleamunison.toml in the working directory, or the path named by GLEAMUNISON_CONFIG. A missing file is normal; malformed TOML or a wrong value type fails the command rather than being silently ignored.

GLEAMUNISON_STORE_DIR = "/absolute/store"
GLEAMUNISON_BRANCH = "factory"
GLEAMUNISON_PROFILE = "local"
GLEAMUNISON_STORE_ALLOW_UNVERIFIED = false
GLEAMUNISON_EVIDENCE_KEY_SINCE = 0
GLEAMUNISON_EVIDENCE_KEY_UNTIL = 9223372036854775807
GLEAMUNISON_EVIDENCE_MAX_AGE_SECS = 0
GLEAMUNISON_EVIDENCE_KEY_REVOKED = false

GLEAMUNISON_EVIDENCE_SIGNING_KEY remains environment-only because the signer identity recorded in evidence names that source. Erlang-owned runtime limits are still configured through their existing environment variables.

HTTP API

Data routes are gated by GLEAMUNISON_HTTP_TOKEN when set. Readiness and liveness are always available for orchestrators.

Application routes accept exactly {"args":[...]} (at most eight values) and return {"ref":"…","branch":"factory","value":…}. An empty body means no arguments. GLEAMUNISON_BRANCH selects the namespace, defaulting to main; the request cannot override it. Named routes resolve that branch's binding, and content-ref routes additionally require the ref to be bound there. Scratch and arbitrary unbound CAS content return 404. The normal evidence, request authorization, execution-profile, and evaluation gates still apply; see ADR 0071.

Escript runtime

The standalone escript contains the runtime and its BEAM dependencies, but not the Erlang VM. Run it on a supported Erlang/OTP release; no Gleam installation is needed at runtime. Its size is an artifact measurement, not an API contract. See Build Escript for the build and reproducibility gates.

Documentation

• Documentation index — current references, evidence, and history
• Reference Manual — language and CLI guide
• Architecture — runtime design and boundaries
• Operations Runbook — configure, run, monitor, upgrade
• Playbook — development and conformance workflow
• Roadmap — remaining work and explicit non-goals
• ADRs — architecture decision history
• Factory acceptance — dated end-to-end proof
• Production witness — machine-readable gate results
• Archive — superseded analyses and walkthroughs

License

MIT