Torque

High-performance JSON library for Elixir via Rustler NIFs, powered by sonic-rs (SIMD-accelerated).

Torque provides the fastest JSON encoding and decoding available in the BEAM ecosystem, with a selective field extraction API for workloads that only need a subset of fields from each document.

Features

• SIMD-accelerated decoding (AVX2/SSE4.2 on x86, NEON on ARM)
• Ultra-low memory encoder (64 B per encode vs ~4 KB for OTP json/jason)
• Parse-then-get API for selective field extraction via JSON Pointer (RFC 6901)
• Batch field extraction (get_many/2) with single NIF call
• Pre-compiled pointers with fused parse + extract (parse_get_many_nil/2)
• Automatic dirty CPU scheduler dispatch for inputs larger than 20 KB
• jiffy-compatible {proplist} encoding

Installation

Add to your mix.exs:

def deps do
  [
    {:torque, "~> 0.2.3"}
  ]
end

Precompiled binaries are available for common targets. To compile from source, install a stable Rust toolchain and set TORQUE_BUILD=true.

CPU-optimized variants

On x86_64, precompiled binaries are available for three CPU feature levels:

At compile time, Torque auto-detects the host CPU and downloads the best matching variant. To override detection (e.g., when cross-compiling for a different target):

TORQUE_CPU_VARIANT=v2 mix compile  # force SSE4.2 variant
TORQUE_CPU_VARIANT=v3 mix compile  # force AVX2 variant
TORQUE_CPU_VARIANT=base mix compile  # force baseline

Usage

Decoding

{:ok, data} = Torque.decode(~s({"name":"Alice","age":30}))
# %{"name" => "Alice", "age" => 30}

data = Torque.decode!(json)

Selective Field Extraction

Parse once, extract many fields without building the full Elixir term tree:

{:ok, doc} = Torque.parse(json)

{:ok, "example.com"} = Torque.get(doc, "/site/domain")
nil = Torque.get(doc, "/missing/field", nil)

# Batch extraction (single NIF call, fastest path)
results = Torque.get_many(doc, ["/id", "/site/domain", "/device/ip"])
# [{:ok, "req-1"}, {:ok, "example.com"}, {:ok, "1.2.3.4"}]

When your JSON is known to have no duplicate object keys, pass unique_keys: true for faster field lookups (uses sonic-rs internal indexing instead of linear scan):

{:ok, doc} = Torque.parse(json, unique_keys: true)

Compiled Pointers

When the same fixed set of paths is extracted from every document, compile the pointers once and reuse the handle. parse_get_many_nil/2 then fuses the parse and extraction into a single NIF call, skipping all per-request path parsing — roughly 1.5× faster end-to-end than parse/2 + get_many_nil/2.

# Once, at startup (e.g. a module attribute or :persistent_term):
pointers = Torque.compile_pointers(["/id", "/site/domain", "/imp/0/banner/w"], unique_keys: true)

# Per document — parse + extract in one call:
{:ok, ["req-1", "example.com", 300]} = Torque.parse_get_many_nil(json, pointers)

Missing fields and JSON null both become nil. The handle also works with an already-parsed document via Torque.get_many_nil(doc, pointers).

Encoding

# Maps with atom or binary keys
{:ok, json} = Torque.encode(%{id: "abc", price: 1.5})
# "{\"id\":\"abc\",\"price\":1.5}"

# Bang variant
json = Torque.encode!(%{id: "abc"})

# iodata variant (fastest, no {:ok, ...} tuple wrapping)
json = Torque.encode_to_iodata(%{id: "abc"})

# jiffy-compatible proplist format
{:ok, json} = Torque.encode({[{:id, "abc"}, {:price, 1.5}]})

API

Type Conversion

JSON to Elixir

For objects with duplicate keys, the last value wins (unless unique_keys: true is passed to parse/2).

Integers outside the signed/unsigned 64-bit range decode as exact arbitrary-precision integers (Erlang bignums) via decode/1, rather than degrading to lossy floats. The parse/2 + get/2 path returns them as floats, since the parsed document cannot hold a bignum.

Elixir to JSON

Errors

Functions return {:error, reason} tuples (or raise ArgumentError for bang/iodata variants). Possible reason atoms:

Decode / Parse

parse/1, decode/1, and parse_get_many_nil/2 also return {:error, binary} with a message from sonic-rs for malformed JSON.

Encode

Benchmarks

Apple M2 Pro, OTP 29, Elixir 1.20. Both libraries are profile-guided optimised (PGO) builds: Torque PGO (via scripts/pgo-build.sh) and Glazer PGO (via OPTIMIZE=1).

Decode (1.2 KB OpenRTB)

Decode (750 KB Twitter)

Encode (1.2 KB OpenRTB)

Encode (750 KB Twitter)

Parse (1.2 KB OpenRTB)

Extract 5 fields from raw JSON (1.2 KB OpenRTB)

End-to-end cost of pulling 5 fields out of a JSON blob: parse + get (torque, simdjsone) vs decode + find (glazer has no lazy handle, so it must fully decode first). This is the apples-to-apples version of "get" — torque's selective extraction skips materializing the whole document.

Run benchmarks locally:

MIX_ENV=bench mix run bench/torque_bench.exs

Limitations

• Nesting depth: JSON documents nested deeper than 128 levels return {:error, :nesting_too_deep} from decode/1, parse/1, get/2, get_many/2, and encode/1 rather than crashing the VM. Real-world documents are never this deep; the limit exists to prevent stack overflow in the NIF (the dirty CPU scheduler, used for inputs over 20 KB, has a small stack).

License

MIT