Protox

Protox is an Elixir library to work with Google's Protocol Buffers. It supports both versions 2 and 3.

Usage

From files:

defmodule Foo do
  use Protox, files: [
    "./defs/foo.proto",
    "./defs/bar.proto",
    "./defs/baz/fiz.proto",
  ]
end

From a textual description:

defmodule Bar do
  use Protox, schema: """
  syntax = "proto3";

  package fiz;

  message Baz {
  }

  message Foo {
    int32 a = 1;
    map<int32, Baz> b = 2;
  }
  """
end

The previous example will generate two modules: Fiz.Baz and Fiz.Foo.

It's possible to prepend a namespace to all generated modules:

defmodule Bar do
  use Protox, schema: """
    syntax = "proto3";

    enum Enum {
        FOO = 0;
        BAR = 1;
      }
    """,
    namespace: Namespace
end

In this case, the module Namespace.Enum will be generated.

Here's how to create a new message:

iex> %Fiz.Foo{a: 3, b: %{1 => %Fiz.Baz{}}} |> Protox.Encode.encode()
[[[], "\b", <<3>>], <<18>>, <<4>>, "\b", <<1>>, <<18>>, <<0>>]

Note that Protox.Encode.encode/1 creates an iolist, not a binary. Such iolists can be used directly with file or sockets read/write operations. However, you can use :binary.list_to_bin() to get a binary:

iex> %Fiz.Foo{a: 3, b: %{1 => %Fiz.Baz{}}} |> Protox.Encode.encode() |> :binary.list_to_bin()
<<8, 3, 18, 4, 8, 1, 18, 0>>

Finally, here's how to decode:

iex> <<8, 3, 18, 4, 8, 1, 18, 0>> |> Fiz.Foo.decode()
{:ok, %Fiz.Foo{a: 3, b: %{1 => %Fiz.Baz{}}}}

Prerequisites

Protox uses Google's protoc (>= 3.0) to parse .proto files. It must be available in $PATH. You can get it here.

Unsupported features

protobuf 3 JSON mapping
groups
rpc

Furthermore, all options other than packed and default are ignored.

Implementation choices

When decoding enum aliases, the last encountered constant will be used. For instance, in the following example, :BAR will always be used if the value 1 is read on the wire.
```
enum E {
  option allow_alias = true;
  FOO = 0;
  BAZ = 1;
  BAR = 1;
}
```
When decoding, fields for which tags are unknown are discarded.
Unset optionals
- For protobuf 2, unset optional fields are mapped to nil
- For protobuf 3, unset optional fields are mapped to their default values, as mandated by the protobuf spec

Types mapping

Protobuf	Elixir
int32	integer()
int64	integer()
uint32	integer()
uint64	integer()
sint32	integer()
sint64	integer()
fixed32	integer()
fixed64	integer()
sfixed32	integer()
sfixed64	integer()
float	float()
double	float()
bool	boolean()
string	String.t
bytes	binary()
map	%{}
oneof	{:field, value}
enum	atom()
message	struct()

Performance

TODO. Do some benchmarks.

Conformance

This library has been tested using the conformance checker provided by Google. Note that only the protobuf part is tested: as protox doesn't support JSON output, the corresponding tests are skipped.

Here's how to launch the conformance test:

Get conformance-test-runner (https://github.com/google/protobuf/tree/master/conformance)
mix protox.conformance --runner=/path/to/conformance-test-runner A report will be generated in a file named conformance_report.txt. If everything's fine, something like the following should be displayed:
```
CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 149 successes, 384 skipped, 0 expected failures, 0 unexpected failures.
```

Credits

Both gpb and exprotobuf were very useful in understanding how to implement Protocol Buffers.