GoogleapisProtoEx

Google API protocol buffer definitions for Elixir - including Datastore, Bigtable, Pub/Sub, Storage, KMS, Tasks, Secret Manager, and AI Generative Language APIs.

This package provides pre-compiled Elixir modules from Google's Protocol Buffer definitions, allowing you to easily interact with Google Cloud services using gRPC in your Elixir applications.

Included APIs

Shared modules inherited from dependencies

To avoid duplicate-module collisions in your application, this package does not generate the common Google types that its dependencies already ship — it reuses them:

Installation

Add googleapis_proto_ex to your list of dependencies in mix.exs:

def deps do

  [

    {:googleapis_proto_ex, "~> 0.4"},

    {:grpc, "~> 1.0"}

  ]

end

Usage

After installation, you can use the generated modules in your application:

# Example: Working with Pub/Sub

alias Google.Pubsub.V1.{PublisherClient, PubsubMessage}



# Example: Working with Datastore  

alias Google.Datastore.V1.{DatastoreClient, Entity, Key}



# Example: Working with Cloud Storage

alias Google.Storage.V1.StorageClient

Development Setup

If you want to update the protobuf files or contribute to this project, you'll need to set up the development environment.

Prerequisites

  1. Elixir (>= 1.15)
  2. Mix
  3. Git
  4. protoc (Protocol Buffer Compiler) - uses your system protoc if present, otherwise downloads a pinned version
  5. protoc-gen-elixir plugin - automatically installed by the script (resolved even behind asdf/rtx shims)

Updating Protobuf Files

The project includes an automated script to update all protobuf definitions from Google's official repositories:

# Make the script executable (if needed)

chmod +x update-protos.sh



# Run the update script

./update-protos.sh

What the Update Script Does

  1. Installs dependencies: Automatically installs the protoc-gen-elixir plugin if not present
  2. Resolves protoc: Prefers a system protoc; falls back to downloading a pinned version (override with PROTOC_VERSION=xx.y)
  3. Clones googleapis: Shallow-clones (or updates) the latest Google API definitions
  4. Generates Elixir code: Compiles .proto files into Elixir modules with gRPC support, skipping types provided by the protobuf/googleapis deps
  5. Flattens & cleans: Collapses the duplicated package directories that protobuf ≥ 0.16 emits, and removes any inherited types
  6. Cross-platform support: Works on macOS (Intel/ARM) and Linux (x86_64/ARM64)

Manual Installation of protoc-gen-elixir

If you prefer to install the Elixir protoc plugin manually:

# Install the protobuf escript globally

mix escript.install hex protobuf



# The plugin will be installed to ~/.mix/escripts/protoc-gen-elixir

# Make sure ~/.mix/escripts is in your PATH

export PATH="$HOME/.mix/escripts:$PATH"

Project Structure

googleapis_proto_ex/

├── lib/                          # Generated Elixir modules

│   └── google/

│       ├── api/                  # Only monitoring types missing from :googleapis

│       ├── iam/                  # IAM v1 policy types

│       ├── longrunning/          # (inherited from :googleapis at runtime)

│       ├── logging/              # Cloud Logging v2

│       ├── datastore/            # Datastore APIs

│       ├── bigtable/             # Bigtable APIs

│       ├── pubsub/               # Pub/Sub APIs

│       ├── storage/              # Storage APIs

│       ├── cloud/                # Other Cloud APIs (KMS, Tasks, Secret Manager, BigQuery, …)

│       └── ai/                   # AI/ML APIs

│                                 # NOTE: google/{protobuf,rpc,type} are provided by deps,

│                                 #       not generated here (see "Shared modules" above)

├── staging_folder/               # Temporary build directory

│   ├── googleapis/               # Cloned googleapis repo

│   └── protoc/                   # Downloaded protoc binary

├── update-protos.sh              # Update script

├── mix.exs

└── README.md

Dependencies

This package depends on:

Troubleshooting

Permission Issues

chmod +x update-protos.sh

Missing protoc-gen-elixir

The script will automatically install it, but you can also install manually:

mix escript.install hex protobuf --force

Platform Support

The update script supports:

Build Issues

If you encounter compilation issues, try:

  1. Clean the project: mix clean
  2. Remove staging folder: rm -rf staging_folder
  3. Re-run the update script: ./update-protos.sh

Contributing

  1. Fork the repository
  2. Make your changes
  3. Update protobuf files if needed: ./update-protos.sh
  4. Run tests: mix test
  5. Submit a pull request

License

Apache 2.0 License - see LICENSE file for details.

Links