Edifice

Hex.pm Hex Docs CI Coverage License

A comprehensive ML architecture library for Elixir, built on Nx and Axon.

268 neural network architectures across 26 families — from MLPs to Mamba, transformers to graph networks, VAEs to spiking neurons, audio codecs to robotics, scientific ML to 3D generation.

What is this repo?

I wanted to build Neural Networks of various architectures. I saw that Nx exists (check out elixir-nx's github page for all the various projects). I vibe coded a scaffold, an 'edifice' to put together the architectures I wanted to support a bot that plays melee. That project is called 'exphil' which is a play on elixir and philip, the colloqial name for slippi-ai's latest bot that combines imitation learning with reinforcement learning to create the currently most capable melee bots that I am aware of. This work to support implementations of various different kinds of architectures mostly coincided with me learning about the existence of the ideas that led to these architectures. The process for implementing them was to ask the coding agent of the time (encoded in the commit themselves, if you want to know more specifically) about the different options for ML neural net architectures. There are papers that lead to the popularization of various architectures, and ML is a very very new field all in all.

Big Asterisk

This is vibe coded. To suit the needs of a personal project. But this repo could be useful to many people. Take whatever you need. Fork it, copy wholesale, whatever. Contributions are welcome. I have a license listed, but the intent is to make it so if people want to make stuff that uses ideas from this, feel free. If you make money off of this, hey kick some back if you feel its worth it, but you don't have to. Open source code. If there's a situation where it's complicated for some reason, act reasonably, don't be an idiot, it's probably fine. If you use this code to do bad things, that's on you. I don't see how it is super likely to be the case though.

If you see some part of this repo that doesn't work correctly for you, open an issue. I'll do my best to fix it. The first version of many of these architectures was broken fundamentally in some way and its a process of using what is created and running into the problems that are there, that it is fixed. Some of these architectures I used a cheap (at the time) model on and some I used more expensive ones. We're all just trying out stuff when it comes to coding agents.

I do find that the process of how you approach a problem is more important than your model choice in producing usable code. If you don't think that's the case, you're doing it wrong. Or whatever model they have is much more capable than the models today and being thoughtful doesn't even matter anymore. I doubt that will ever happen.

Why Edifice?

The Elixir ML ecosystem has excellent numerical computing (Nx) and model building (Axon) foundations, but no comprehensive collection of ready-to-use architectures. Edifice fills that gap:

Installation

AI-generated release compatibility note. The default dependencies use published Nx/EXLA 1.0 packages. EXLA 1.0's XLA 0.10 has a known CPU f32 erf/GELU error. The fixed upstream setup remains available; see dependency limits and setup. Whisper's corrected numerical results apply to that fixed setup. The accompanying authorship poem is in the validation guide.

Add edifice to your dependencies in mix.exs:

def deps do
  [
    {:edifice, "~> 0.3.0"}
  ]
end

Edifice requires Nx ~> 1.0 and Axon ~> 0.9. For GPU acceleration, add EXLA:

{:exla, "~> 1.0"}

EXLA is optional for consumers: core architectures run on Nx.BinaryBackend without a native compiler. Install EXLA for XLA compilation, GPU execution, sharding, and allocator memory profiling. The fused custom-call integration is compiled only when EXLA's protocol is available. After adding EXLA to an existing consumer, recompile Edifice (mix deps.compile edifice --force) to enable it.

The development project installs optional dependencies for testing. To check the consumer configuration without EXLA, run bash scripts/check_optional_exla.sh after mix deps.get.

For the native Mamba selective scan, build Edifice's separate FFI library with nvcc and an EXLA build that exposes EXLA.CustomCall and the EXLA dynamic-library loader (EXLA.load_dylib/1):

devenv shell -- mix edifice.cuda.build --arch sm_120  # RTX 5090; choose your GPU target
devenv shell -- env EDIFICE_FUSED_CUSTOM_CALL=1 EDIFICE_AUTOTUNE=0 \
  mix test test/edifice/cuda/fused_custom_call_gpu_test.exs --include exla_only

The build uses the installed EXLA dependency's XLA headers and library without modifying EXLA. The loader registers the handlers once per VM, before compiling a supported CUDA call. Only the f32 selective scan (state size ≤ 32) is enabled; every other fused op uses its Nx fallback. The kernel pays off at batch sizes above a handful (3-5x inside Mamba at batch 32, a wash at batch 1); see docs/kernel_tier_benchmarks.md for the measurements and for why the GRU handler was retired. The flag is opt-in, and a missing library gives a build instruction. Restart running VMs after rebuilding kernels. Rebuild after changing XLA versions. For a separately installed library set EDIFICE_CUDA_LIBRARY to its absolute path; legacy EXLA builds with these handlers already linked use linked. Real activation checkpointing remains pending the upstream Nx checkpoint work.

Native kernels also allocate scratch memory outside XLA's pool. Leave headroom before starting EXLA (the GPU tests use this setting):

config :exla, :clients, cuda: [platform: :cuda, memory_fraction: 0.7]

XLA's default 90% reservation can cause native-kernel allocation failures even when XLA still has unused pool memory. Size the fraction for your workload.

Tip: On Elixir 1.19+, set MIX_OS_DEPS_COMPILE_PARTITION_COUNT=4 to compile dependencies in parallel (up to 4x faster first build).

Quick Start

How much has each architecture been tested?

AI-generated evidence summary. Coverage varies. Canonical Mamba-2, GRU, Mamba and MinGRU have independent numerical checks within documented limits. Selected vision models have real-checkpoint comparisons. Many other models have local tests, but have not been independently certified.

The architecture index now labels each listed architecture's reviewed evidence. The validation guide explains what was compared, which backend was used, and what remains unknown. Unreviewed means the evidence has not yet been assessed, not that no tests exist. Shape tests and speed measurements alone do not establish correctness. The validation guide includes the authorship poem for these additions.

Build a model

# Build any architecture by name
model = Edifice.build(:mamba, embed_size: 256, hidden_size: 512, num_layers: 4)

# Or use the module directly for more control
model = Edifice.SSM.Mamba.build(
  embed_size: 256,
  hidden_size: 512,
  state_size: 16,
  num_layers: 4,
  window_size: 60
)

# Build and run
{init_fn, predict_fn} = Axon.build(model)
params = init_fn.(Nx.template({1, 60, 256}, :f32), Axon.ModelState.empty())
output = predict_fn.(params, input)

# Explore what's available
Edifice.list_architectures()
# => [:attention, :bayesian, :capsule, :deep_sets, :densenet, :diffusion, ...]

Edifice.list_families()
# => %{ssm: [:mamba, :mamba_ssd, :s5, ...], attention: [:attention, :retnet, ...], ...}

Architecture Families

268 architectures across 26 families, plus 20 shared building blocks. See the full architecture index for every module.

Not every architecture has seen the same scrutiny. The ones that have been trained and deployed in ExPhil, or checked against a PyTorch reference with real or random weights, are listed under Verified in use in the index, and Edifice.verification/1 reports the same from code. The rest are faithful readings of their papers backed by shape, gradient and sweep tests.

Family Count Highlights
Attention 48 Multi-Head, GQA, MLA, Perceiver, RetNet, RWKV, GLA, Griffin, NSA, Ring Attention, Conformer
Meta 36 MoE, LoRA, DoRA, DPO, GRPO, Capsules, Speculative Decoding, Mixture of Depths, QAT
Generative 33 VAE, GAN, Diffusion, DiT, MMDiT, Flow Matching, Transfusion, CogVideoX, MDLM, VAR
SSM 23 Mamba, Mamba-2 (full mixer and SSD), Mamba-3, S4, S5, Hyena, Hymba, Jamba, StripedHyena, Zamba
Recurrent 21 LSTM, GRU, xLSTM, MinGRU, DeltaNet, Gated DeltaNet, TTT, Titans
Vision 18 ViT, DeiT, Swin, U-Net, ConvNeXt, MLP-Mixer, DINOv2, DINOv3, NeRF, MambaVision
Graph 13 GCN, GAT, GIN, GraphSAGE, SchNet, EGNN, SE(3)-Transformer, GPS
Interpretability 12 Sparse Autoencoder, Gated / JumpReLU / Matryoshka SAE, Transcoder, Crosscoder, LEACE, Probes
Contrastive 9 SimCLR, BYOL, MAE, VICReg, JEPA, V-JEPA 2, SigLIP
Audio 6 EnCodec, VALL-E, SoundStorm, Whisper, wav2vec 2.0, F5-TTS
Convolutional 6 ResNet, DenseNet, TCN, MobileNet, EfficientNet
Feedforward 5 MLP, KAN, KAT, TabNet, BitNet
Transformer 5 Decoder-Only, Multi-Token Prediction, BLT, Nemotron-H, Free Transformer
Memory 4 NTM, Memory Networks, Memory Layer, Engram
Sets 4 DeepSets, PointNet, PointNet++, Entity Encoder
RL 4 PolicyValue, Decision Transformer, Pointer Network, Autoregressive Head
Scientific 3 FNO, DeepONet, TNO
Detection 3 DETR, RT-DETR, SAM 2
Energy 3 EBM, Hopfield, Neural ODE
Probabilistic 3 Bayesian, MC Dropout, Evidential
Robotics 3 ACT, Diffusion Policy, OpenVLA
Neuromorphic 2 SNN, ANN2SNN
+ 4 more 4 Liquid NN, World Model, Medusa, Multimodal Fusion

Guides

New to ML?

Start here if you're new to machine learning. These guides build from zero to fluency with Edifice's API and architecture families.

  1. ML Foundations — What neural networks are, how they learn, tensors and shapes
  2. Core Vocabulary — Essential terminology used across all guides
  3. The Problem Landscape — Classification, generation, sequence modeling — which architectures solve which problems
  4. Reading Edifice — The build/init/predict pattern, Axon graphs, shapes, and runnable examples
  5. Learning Path — A guided tour through the architecture families

Reference

Architecture Guides

Conceptual guides covering theory, architecture evolution, and decision tables for each family.

Sequence Processing

Representation Learning

Generative & Dynamic

Composition & Enhancement

Examples

See examples/ for runnable scripts including mlp_basics.exs, sequence_comparison.exs, graph_classification.exs, vae_generation.exs, and architecture_tour.exs.

Mamba for Sequence Modeling

model = Edifice.SSM.Mamba.build(
  embed_size: 128,
  hidden_size: 256,
  state_size: 16,
  num_layers: 4,
  window_size: 100
)

{init_fn, predict_fn} = Axon.build(model)
params = init_fn.(Nx.template({1, 100, 128}, :f32), Axon.ModelState.empty())
output = predict_fn.(params, Nx.broadcast(0.5, {1, 100, 128}))
# => {1, 256}

Graph Classification with GCN

model = Edifice.Graph.GCN.build_classifier(
  input_dim: 16,
  hidden_dims: [64, 64],
  num_classes: 2,
  pool: :mean
)

{init_fn, predict_fn} = Axon.build(model)

params = init_fn.(
  %{
    "nodes" => Nx.template({4, 10, 16}, :f32),
    "adjacency" => Nx.template({4, 10, 10}, :f32)
  },
  Axon.ModelState.empty()
)

output = predict_fn.(params, %{
  "nodes" => Nx.broadcast(0.5, {4, 10, 16}),
  "adjacency" => Nx.eye(10) |> Nx.broadcast({4, 10, 10})
})
# => {4, 2}

VAE with Reparameterization

{encoder, decoder} = Edifice.Generative.VAE.build(
  input_size: 784,
  latent_size: 32,
  encoder_sizes: [512, 256],
  decoder_sizes: [256, 512]
)

# Encoder outputs mu and log_var
{init_fn, predict_fn} = Axon.build(encoder)
params = init_fn.(Nx.template({1, 784}, :f32), Axon.ModelState.empty())
%{mu: mu, log_var: log_var} = predict_fn.(params, Nx.broadcast(0.5, {1, 784}))

# Sample latent vector (requires PRNG key for stochastic sampling)
key = Nx.Random.key(42)
{z, _new_key} = Edifice.Generative.VAE.reparameterize(mu, log_var, key)

# KL divergence for training
kl_loss = Edifice.Generative.VAE.kl_divergence(mu, log_var)

Permutation-Invariant Set Processing

model = Edifice.Sets.DeepSets.build(
  input_dim: 3,
  hidden_dim: 64,
  output_dim: 10,
  pool: :mean
)

{init_fn, predict_fn} = Axon.build(model)
params = init_fn.(Nx.template({4, 20, 3}, :f32), Axon.ModelState.empty())
# Process sets of 20 3D points
output = predict_fn.(params, Nx.broadcast(0.5, {4, 20, 3}))
# => {4, 10}

API Design

Every architecture module follows the same pattern:

# Module.build(opts) returns an Axon model
model = Edifice.SSM.Mamba.build(embed_size: 256, hidden_size: 512)

# Some modules expose layer-level builders for composition
layer = Edifice.Graph.GCN.gcn_layer(nodes, adjacency, output_dim)

# Generative models may return tuples
{encoder, decoder} = Edifice.Generative.VAE.build(input_size: 784)

# Utility functions for training
loss = Edifice.Generative.VAE.loss(reconstruction, target, mu, log_var)
energy = Edifice.Energy.Hopfield.energy(query, patterns, beta)

The unified registry lets you build any architecture by name:

# Useful for hyperparameter search, config-driven experiments
for arch <- [:mamba, :retnet, :griffin, :gla] do
  model = Edifice.build(arch, embed_size: 256, hidden_size: 512, num_layers: 4)
  # ... train and evaluate
end

Requirements

License

MIT License. See LICENSE for details.