OmniBuf for Elixir
Elixir implementation of the OmniBuf v0.1 binary wire format.
OmniBuf uses numbered Protobuf fields, 1/2/4/9-byte Prefix-Varints, distinct wire codes for all 16 scalar and container types, single-pass iodata encoding, lazy zero-decode Opaque views with in-place iodata wire splicing, and optional Field #0 (SchemaMeta) self-describing metadata.
Installation
Add omnibuf to your mix.exs dependencies:
def deps do
[
{:omnibuf, "~> 0.1.0"}
]
end
Requires Elixir 1.17+ and Erlang/OTP 27+.
Four Implementation Tiers
Tier 1: protoc-gen-omnibuf-elixir Plugin & Lazy Opaque Views
Build the escript plugin and run buf generate:
mix escript.build
buf generate
Generated modules provide fast struct codecs and lazy <Message>.Opaque views that read nested fields in O(fields) time and splice mutations using zero-copy BEAM sub-binaries:
alias Omnibuf.Bench.{Location, PeerUpdate}
# Zero-decode nested field read
city =
packet
|> PeerUpdate.Opaque.new()
|> PeerUpdate.Opaque.get_location()
|> Location.Opaque.get_city()
# Selective in-place wire splice (untouched fields stay raw sub-binaries)
updated =
packet
|> PeerUpdate.Opaque.new()
|> PeerUpdate.Opaque.set_sequence(9_876_543_211)
|> PeerUpdate.Opaque.marshal!()
Tier 2: elixir-protobuf Integration
Encode and decode standard use Protobuf structs directly to and from OmniBuf wire bytes:
wire = OmniBuf.encode_proto(pb_struct)
{:ok, decoded} = OmniBuf.decode_proto(wire, MyApp.PB.PeerUpdate)
Tier 3: Declarative Struct DSL (OmniBuf.Schema)
Define OmniBuf messages directly in Elixir without .proto files. Compile-time macros generate specialized iodata encoders and binary pattern-matching decoders:
defmodule MyApp.Location do
use OmniBuf.Schema, name: "Location"
field 1, :region, :string
field 2, :datacenter, :string
field 3, :city, :string
field 4, :rack, :uint32
field 5, :latitude, :float
field 6, :longitude, :float
end
wire = MyApp.Location.encode(%MyApp.Location{city: "Frankfurt", rack: 42})
{:ok, loc} = MyApp.Location.decode(wire)
Pass self_describing: true to prepend Field #0 SchemaMeta:
wire = MyApp.Location.encode(loc, self_describing: true)
Tier 4: Schema-less Blind Decode (OmniBuf.Blind)
Inspect any OmniBuf packet with no compiled schema. Declared Field #0 names propagate through nested messages, arrays, and maps; unnamed fields fall back to "field_<N>".
{:ok, map} = OmniBuf.decode_blind(wire)
{:ok, json} = OmniBuf.decode_blind_json(wire)
Conformance & Benchmarks
just fmt-check
just lint
just test
just conformance
just bench
The conformance suite passes 992 / 992 test cases across all 4 tiers (TIER_PLUGIN, TIER_PROTOBUF_TYPES, TIER_ANNOTATED_STRUCT, TIER_BLIND) with 0 failures.
License
Apache-2.0. See LICENSE.