gleamunison

Content-addressed language runtime on the BEAM, built in Gleam.

A running prototype of a Unison-style content-addressed programming language that compiles to BEAM bytecode and loads dynamically into the Erlang VM.

Rationale & Gap Analysis

Why Gleamunison?

Gleamunison combines the type-safe concurrency of the BEAM (via Gleam) with Unison's content-addressed codebase and algebraic effects, enabling zero-downtime hot upgrades and dynamic sandboxing.

Feature Set Differences (Gleamunison vs Unison)

Complexity vs. Utility

How Hashes Act as Executables

In gleamunison, a program is not a monolithic binary file. It is a Merkle Directed Acyclic Graph (Merkle DAG) of content-addressed AST definitions stored in a database.

The Merkle DAG Code Structure

Each function definition is parsed into an AST and hashed based on its structure, stripping variable names to ensure α-equivalence. If a function references other functions, it references them strictly by their hashes (RefTo AST node).

                  ┌───────────────────────┐
                  │   start-server Hash   │  ◄─── This single Root Hash IS the executable
                  │       [#ff45e8]       │
                  └───────────┬───────────┘
                              │
             ┌────────────────┴────────────────┐
             ▼                                 ▼
 ┌───────────────────────┐         ┌───────────────────────┐
 │  page-template Hash   │         │    query-db Hash      │
 │       [#ac88b2]       │         │       [#3b12ef]       │
 └───────────────────────┘         └───────────┬───────────┘
                                               │
                                               ▼
                                   ┌───────────────────────┐
                                   │  Datalog-Transact Jet │
                                   │  [#00000064] (Builtin)│
                                   └───────────────────────┘

The entire application (CMS, database, or GUI) is represented by a single root hash (e.g. #ff45e8).

Step-by-Step Execution

When running a program by its hash:

1. DAG Resolution: The runtime looks up the root hash in its database, transitively walks the dependency graph, and fetches any missing dependency hashes from peers via the Pull Sync Protocol.
2. On-the-Fly Compilation: The compiler translates the resolved AST nodes into Erlang BEAM bytecode binary arrays named after their hashes (e.g. m_ff45e8.beam).
3. Dynamic Loading & Run: The VM dynamically loads the bytecode (code:load_binary/3) and calls the module's entrypoint function 'm_ff45e8':'$eval'().

This enables zero-downtime hot upgrades (running new logic by swapping to a new root hash) and conflict-free dependency resolution.

Unique Usecases (Impossible on Gleam or Unison Alone)

Gleamunison combines the scheduling, distribution, and runtime efficiency of the BEAM with the content-addressing and algebraic effects constraints of Unison:

Hot-Upgrades & Evolution

1. Zero-Downtime Stateful Actor Upgrades: Hot-swapping active actor code on-the-fly without state loss. Standard Gleam module updates clash on name collisions; Gleamunison addresses this by compiling into hash-named modules.
2. Stateful Chatbot Hot-Upgrades: Preserve active user conversation states in Erlang actors. Swaps the actor message-handling loop to a new hash definition on the fly without state or connection loss.
3. IoT Firmware Hot-Patching: IoT devices download modular function hashes instead of heavy firmware images, updating local logic dynamically without device resets.
4. Dynamic API Gateways: Route HTTP requests based on endpoint definition hashes, dynamically compiling and loading handlers on demand.

Secure Sandboxing & Multitenancy

1. Decoupled Multi-tenant Sandboxing: Run untrusted plugins concurrently. Process boundaries isolate resource usage, while algebraic effects intercept and sandbox system actions (file, network).
2. Zero-Trust Serverless Executions: Execute third-party thunks securely. The host caps execution time via process CPU schedulers and restricts access using custom effect handlers.
3. Capabilities-as-Code: Database and network handles are represented strictly as abilities. User code only typechecks if the required abilities match their group access privileges.
4. Sandbox Game Modding: Run game modding scripts in isolated BEAM processes. Mod APIs are exposed as abilities, preventing malicious access to the host filesystem.

Distributed Compute & Edge topographies

1. Resilient P2P Job Stealing: Edge nodes dynamically pull, structurally verify, locally compile, and run job definitions by hash.
2. Live Process Migration: Serialize a running actor's continuation closure, ship it to a remote node, sync missing code dependencies via pull protocol, and resume execution.
3. Zero-Config Clustered Map-Reduce: Parallelize map-reduce workflows. Code dependencies are automatically resolved and shipped by the runtime on target locations using Merkle sync.
4. Edge-Cloud Compute Offloading: IoT devices offload heavy compute thunks to BEAM cloud nodes, verifying code integrity by hash to prevent remote exploits.
5. Content-Addressable CDN Handlers: CDNs compile custom request handlers, push them to edge nodes by hash, and process CDN requests concurrently on edge processes.
6. P2P Software Distribution: Sync codebases incrementally. Nodes exchange root hashes and request only missing modules, reducing patching bandwidth.

Determinism, Auditing & Tools

1. Time-Traveling Replay Debugging: Capture trace logs of execution. Replay the exact execution path deterministically using mock clock/random effect handlers.
2. Distributed Event Sourcing with Code Auditing: Event stores record event payloads alongside the handler's hash, allowing historic events to be replayed with the exact code version.
3. Smart Contract Workflows: Execute decentralized workflows. All state mutations and payments are modeled as abilities, sandboxed by host-defined contract handlers.
4. Multi-Tenant Concurrent Parsers: Compile user-provided parser grammar thunks. Preemptive BEAM scheduling prevents a single bad parsing loop from blocking others.
5. Self-Documenting Code Registries: Code definitions are hashed and immutable. Documentation and tests are linked directly to hashes; renaming never breaks documentation.
6. Immutable Cloud Shell: Run interactive REPL sessions where every expression is compiled and stored. Keeps old module versions in memory for historic comparisons.
7. Content-Addressable Microservices: Services call others by passing function hashes over RPC. The runtime resolves, syncs, and loads the code dynamically.
8. Decentralized Knowledge Graph: A wiki-like graph where nodes are content-addressed definitions and links are type-safe references forming a Merkle DAG.
9. Reproducible Monte Carlo Simulations: Replay complex stochastic simulations by mock-handling random generator and timer abilities using fixed seeds.

Project State

Production-grade runtime (Phases 0–14 complete). All components are implemented and verified. The runtime is fully playbook-certified, passing all 5696 playbook conformance levels and 54 unit test suites (all passed). v3.11.0 includes native content-addressed Datalog database engine with temporal indexing, graph algorithms, BM25, and vector search, alongside safe FFI deserialization, localhost endpoint restrictions, Ranch port monitoring safeguards, modular genesis builtin extraction, and a data-driven test runner framework.

Conformance Tests

To execute the unit and integration test suites:

gleam test                          # Runs all 54 unit test suites including native Datalog
bb scripts/run_playbook_tests.clj   # Runs playbook conformance suite

Why the escript is only 1.2 MB

The standalone binary (gleamunison_escript) contains the full content-addressed runtime — parser, elaborator, typechecker, compiler, loader, codebase, effects system, web server, REPL, 52 genesis modules, and all stdlib dependencies. At ~1.2 MB, it's compact because:

BEAM bytecode is dense. The compiled .beam files are ~2.4 MB uncompressed; zip compression brings that to ~1.2 MB.

No VM bundled. Unlike Go or Rust binaries that statically link a runtime, the escript relies on the system's Erlang/OTP installation (~150 MB, installed once). The escript itself is just a zip archive with a 50-byte launcher header.

If you already have Erlang installed, this is as close to a zero-install language runtime as it gets.

Modules (44 Gleam source modules, Erlang FFI, 52 genesis modules, 13 Datalog engine modules)

Quick start

cd ~/Desktop/gleamunison_dogfood/gleamunison_repo
gleam run -- all             # Run all dogfooding levels via dynamic VM runner
gleam test                  # Run unit tests (54 test suites)
./gleamunison_escript repl   # Start interactive REPL via standalone escript (after ./build_escript.sh)

Documentation

• Playbook — how to work on this project
• Architecture — deep dive into the design
• Roadmap — 14-phase plan from spec to production
• Operations Runbook — deploy, configure, monitor, upgrade, troubleshoot
• Learnings — architectural insights discovered
• Patterns — design patterns used
• Reference Manual — complete reference user guide
• ADRs — Architecture Decision Records (62 records)
• Standard Library — module reference

Runtime output

=== Gleamunison ===
Int(42)           ✓ Hash → Compile → Load
Lambda(id)        ✓ Hash → Compile → Load
Apply(id, 99)     ✓ Hash → Compile → Load
Let(V0=42, V0)    ✓ Hash → Compile → Load
Text(hello)       ✓ Hash → Compile → Load
List([1,2,3])     ✓ Hash → Compile → Load
Match(42, cases)  ✓ Hash → Compile → Load
Type Inference    ✓ Int/Float/Text/List
Elaboration       ✓ Surface → typed
Effects           ✓ RuntimeConfig
Sync              ✓ PeerId/SyncState

License

MIT