🚀 zaml

A fast YAML 1.2 parser for Elixir, powered by a Zig NIF that wraps libyaml. Heavily inspired by kubkon/zig-yaml and the original Python prototype that lives in this same repository (see Python prototype below).

iex> Zaml.load("foo: 1\nbar: [a, b, 3.14]\n")
%{"bar" => ["a", "b", 3.14], "foo" => 1}

iex> Zaml.load("""
...> name: zaml
...> tags: [yaml, elixir, nif, zig]
...> ratio: 3.14
...> flag: true
...> empty: ~
...> """)
%{"name" => "zaml", "tags" => ["yaml", "elixir", "nif", "zig"],
  "ratio" => 3.14, "flag" => true, "empty" => nil}

Why?

The most popular Elixir YAML libraries — yaml_elixir and its underlying yamerl — are pure-Erlang ports. They're correct but slow. The existing native option on Hex is fast_yaml (processone), which is the obvious thing to compare against because it also wraps libyaml.

Zaml is fast_yaml but with a thinner event-to-term bridge: the libyaml event stream is consumed in Zig and turned into Erlang terms in a single pass, building maps directly instead of going through intermediate proplists. That gives a ~3–3.5× speedup over fast_yaml across flat and nested mappings, and 36–46× over the pure-Erlang parsers. See benchmarks below.

The newer glazer NIF is the one library that can beat Zaml — it's faster on nested data (~1.3×) and slower on a single flat 1M-key map (~2.1×). It gets there with a hand-rolled C++ YAML parser built with profile-guided optimisation, whereas Zaml deliberately stays on top of battle-tested libyaml.

The toolchain is also notably lighter: Zaml is built with the single zig CLI invocation you see in the Makefile, with no ./configure, no rebar3 port-compiler, and no autoconf.

Benchmarks

Two fixtures, both parsed to native Erlang terms, on an Apple M3 Pro. Full report at benchmark/RESULTS.md.

1M-line flat mapping (16 MB)

Parser Avg (s) Relative
Zaml (NIF, libyaml + Zig) 0.84 1.00×
glazer 1.1.6 (C++ NIF, PGO) 1.73 2.06×
fast_yaml 1.0.40 (rebar3 NIF) 2.54 3.03×
yamerl 0.10.0 (pure Erlang) 30.27 36.1×
yaml_elixir 2.12.2 (yamerl) 30.95 36.9×

100k-key nested mapping (15 MB)

Parser Avg (s) Relative
Zaml (NIF, libyaml + Zig) 0.53 1.00×
glazer 1.1.6 (C++ NIF, PGO) 0.41 0.77×
fast_yaml 1.0.40 (rebar3 NIF) 1.86 3.50×
yamerl 0.10.0 (pure Erlang) 23.88 45.0×
yaml_elixir 2.12.2 (yamerl) 24.40 45.9×

Both Zaml and fast_yaml wrap the same libyaml C parser, so the ~3× gap there is purely the event→term bridge. glazer is a different design (a hand-rolled C++ parser, not libyaml): it wins on nested data and loses on the wide flat map, so there's no single "fastest" library across shapes.

Installation

Add zaml to your mix.exs:

def deps do
  [
    {:zaml, "~> 0.2.0"}
  ]
end

Build prerequisites

The NIF is built at compile time from source. You need:

The compiler auto-detects libyaml via pkg-config first, then falls back to well-known locations. Override with YAML_INCLUDE_DIR and YAML_LIB_DIR env vars or make variables if it lives somewhere unusual.

mix deps.get
mix compile   # builds the NIF via elixir_make / Makefile

Usage

Zaml.load(yaml_string) :: term

Returns the corresponding Elixir term for the first YAML document in the input. Scalar type detection follows the YAML 1.2 core schema:

YAML scalar Elixir term
42, -3, +1_000 integer (42, -3, 1000)
3.14, 1.0e-3 float (3.14, 0.001)
true, yes, on atom true (or :true quoted)
false, no, off atom false
null, ~, empty atom nil
everything else binary (string)
!!str 42 binary "42" (explicit tag respected)
!!int "42" integer 42 (explicit tag respected)

Sequences become lists, mappings become maps with binary keys.

Returns :error for non-binary input, :parse_error for malformed YAML, and :nil for an empty document.

Anchor / alias example

iex> Zaml.load("""
...> defaults: &d {a: 1, b: 2}
...> prod:    *d
...> """)
%{"defaults" => %{"a" => 1, "b" => 2}, "prod" => %{"a" => 1, "b" => 2}}

Note: YAML merge keys (<<: *alias) are not yet supported.

Using the alternatives

For reference, here's how the same parse is done with each library the benchmarks compare against:

# zaml — returns the first document as a term
Zaml.load(yaml)

# fast_yaml — returns {:ok, [doc]}; :maps gives maps instead of proplists
:fast_yaml.decode(yaml, [:sane_scalars, :maps])

# glazer — hand-rolled C++ NIF; use_nil maps null -> nil (default is :null)
:glazer_yaml.decode(yaml, [:use_nil])
# ...or the Elixir wrapper:
Glazer.YAML.decode!(yaml)

# yaml_elixir — the most widely used wrapper, over pure-Erlang yamerl
YamlElixir.read_from_string(yaml)   # => {:ok, term}

# yamerl directly (raw records; yaml_elixir adds the Elixir mapper)
:yamerl_constr.string(yaml, detailed_constr: true, str_node_as_binary: true)

Two other libraries in the space aren't in the parse benchmarks: ymlr is an encoder (Ymlr.document!/1), not a parser, and yamleam is a pure-Gleam parser, so it isn't reachable as a plain Mix dependency.

Running tests

mix test

The test suite covers basic scalars, nested maps and lists, explicit tags, anchors, empty input, and parse-error handling.

Running benchmarks

The repo ships a Mix task that compares the Elixir NIF against fast_yaml, glazer, yaml_elixir, and yamerl on the same fixture (any parser whose dependency isn't installed is skipped).

# 1. Build the Elixir NIF + the benchmark deps
#    (glazer needs a C++23 compiler and rebar3; `brew install rebar3`)
mix deps.get
mix compile

# 2. Generate the 100k-key nested fixture
python3 benchmark/gen_big_yaml.py 100000 benchmark/big.yml

# 3. Generate the 1M-line flat fixture
python3 -c "
import yaml
yaml.dump({f'a{i}': f'b{i+1}' for i in range(1_000_000)},
          Dumper=yaml.CSafeDumper, default_flow_style=False,
          open('benchmark/big_1m.yml', 'w'))
"

# 4. Run the head-to-head
mix zaml_nif.bench benchmark/big.yml        # Zaml parse ~0.5 s (all parsers run)
mix zaml_nif.bench benchmark/big_1m.yml     # Zaml parse ~0.8 s (all parsers run)

A separate cross-language script (benchmark/run_benchmark.py) also compares against PyYAML and ruamel.yaml if you want the Python numbers.

Project layout

.
├── lib/
│   ├── zaml.ex                      # public Elixir API
│   └── mix/tasks/
│       ├── zaml_nif.build.ex        # manual NIF build (`mix zaml_nif.build`)
│       └── zaml_nif.bench.ex        # `mix zaml_nif.bench` benchmark task
├── native/
│   └── zaml_nif.zig                 # the NIF (uses libyaml via @cImport)
├── priv/                            # compiled zaml.so lands here
├── test/                            # ExUnit tests
├── benchmark/
│   ├── benchmark.exs                # Elixir side
│   ├── run_benchmark.py             # Python driver
│   ├── gen_big_yaml.py              # fixture generator
│   ├── big.yml                      # generated 100k-key fixture
│   └── RESULTS.md                   # last benchmark run
├── Makefile                         # `make` builds the NIF directly
└── mix.exs

Limitations / status

Credits

Python prototype

The original proof-of-concept — a pip install-able Python C extension built in pure Zig by importing Python.h directly — still lives in this repository. It demonstrates that you can build a CPython extension module using only the Zig toolchain, with no clang or setuptools workarounds. It was the subject of a PyCon DE 2022 talk on Speeding Up Python with Zig.

Building the Python prototype

python -m venv .venv
source .venv/bin/activate
pip install -e .
python test.py

Note: the Python prototype pins Zig 0.10.0. Newer Zig versions have dropped fields from struct _object that the original zamlmodule.zig references, so the prototype needs the older toolchain to build. Use the Elixir NIF (above) for current Zig versions.

Python benchmark

cd benchmark
python benchmark.py

The original prototype was benchmarked on a 2.3 GHz Quad-Core Intel Core i7 as follows:

zaml took 0.89 seconds
PyYAML CSafeLoader took 13.36 seconds
ruamel took 38.86 seconds
PyYAML SafeLoader took 81.78 seconds

The prototype is intentionally minimal (top-level dict[str, str] only); it was meant to validate the toolchain story, not to compete with full YAML libraries.

Cross-platform notes (Python prototype)