MicrogradEx

MicrogradEx is a small educational scalar autodiff and neural-network library in Elixir, inspired by Andrej Karpathy's micrograd.

It implements reverse-mode autodiff over scalar Value nodes, a tiny neural-network layer, and a Livebook-first recreation of the classic micrograd two-moons classification demo.

The goal is not performance. The goal is to make the mechanics of backpropagation visible in idiomatic Elixir.

What is this?

MicrogradEx is a pure-Elixir learning project for reverse-mode automatic differentiation. It keeps the same educational scope as the original Python micrograd: scalar values, explicit computation graphs, neurons, layers, and small MLPs.

The main user experience is notebooks/micrograd_demo.livemd, which mirrors the official micrograd workflow in pure Elixir: make a two-moons dataset, train MLP.new(2, [16, 16, 1]), plot loss and accuracy, and visualize a decision boundary.

What this repo includes

• scalar Value objects with reverse-mode autodiff;
• immutable Gradients tables returned from Value.backward/1;
• tiny neural-network modules: Neuron, Layer, and MLP;
• deterministic pure-Elixir dataset generators: moons, spiral, and blobs;
• max-margin classification loss with L2 regularization;
• immutable SGD-style training;
• plot-data helpers for Livebook/Vega-Lite;
• scalar graph inspection and DOT export;
• a flagship Livebook demo;
• ExDoc guides.

Quick start: Livebook demo

Install livebook if needed:

mix do local.rebar --force, local.hex --force
mix escript.install hex livebook

Path may not be found. If you're using asdf and if needed:

asdf reshim elixir

Clone the repo and open the main notebook:

git clone <repo-url>
cd micrograd_ex
livebook server

Then open:

notebooks/micrograd_demo.livemd

Run the notebook from top to bottom. It will:

1. build a two-moons dataset;
2. inspect a scalar autodiff graph;
3. initialize MLP.new(2, [16, 16, 1]);
4. confirm the model has 337 parameters;
5. train for 100 steps;
6. plot loss and accuracy;
7. visualize the learned decision boundary.

The notebook uses Mix.install/2 for Kino and Vega-Lite. Those visualization packages are not runtime dependencies of the core library.

Extra experiments

After running the main demo, open:

notebooks/micrograd_extras.livemd

The extras notebook explores dataset noise, model size, regularization, learning-rate schedules, decision-boundary resolution, and the spiral dataset. It is optional; the main demo remains the official parity path.

Quick start: IEx

iex -S mix

alias MicrogradEx.NN.MLP
alias MicrogradEx.{Datasets, Losses, Trainer}

dataset = Datasets.moons(100, noise: 0.1, seed: {1337, 1337, 1337})
model = MLP.new(2, [16, 16, 1], seed: {1337, 1337, 1337})

initial = Losses.max_margin(model, dataset.xs, dataset.ys)

run =
  Trainer.train(model, dataset,
    steps: 100,
    alpha: 1.0e-4
  )

%{
  initial_loss: initial.total_loss.data,
  final_loss: run.final_loss,
  final_accuracy: run.final_accuracy
}

Official micrograd demo parity

The main Livebook mirrors the original micrograd demo workflow. It does not byte-match sklearn or Matplotlib output; it reproduces the educational workflow in pure Elixir.

Expected visuals in the notebook:

• a scatter plot of the two-moons dataset;
• a loss chart showing total, data, and regularization loss;
• an accuracy chart in percent;
• a decision-boundary plot behind the training points.

Elixir design differences

Python micrograd stores mutable state directly on each Value:

• value.data;
• value.grad.

MicrogradEx keeps Value immutable. A backward pass returns a separate Gradients table:

gradients = MicrogradEx.Value.backward(loss)
dx = MicrogradEx.Value.grad(x, gradients)

Model updates are immutable too:

next_model = MicrogradEx.NN.apply_gradients(model, gradients, learning_rate)

Because gradients are returned fresh from each backward pass, there is no zero_grad step.

Project layout

lib/
  micrograd_ex/
    value.ex          # scalar autodiff values
    gradients.ex      # immutable gradient table
    nn.ex             # Neuron, Layer, MLP
    datasets.ex       # pure-Elixir toy datasets
    losses.ex         # max-margin loss
    trainer.ex        # immutable training loop
    plot_data.ex      # plain rows for plotting
    graph.ex          # graph inspection and DOT export

notebooks/
  micrograd_demo.livemd
  micrograd_extras.livemd

guides/
  getting_started_with_livebook.md
  micrograd_demo_walkthrough.md
  elixir_design_notes.md
  api_reference.md
  troubleshooting.md

test/

Development

mix deps.get
mix format --check-formatted
mix test
mix credo
mix dialyzer

mix quality runs the formatter check, Credo, and Dialyzer together.

Validate the Livebook structure:

bash scripts/validate_livebook.sh

Validate documentation:

bash scripts/validate_docs.sh

Documentation

Generate local docs with:

mix docs

The docs include generated API pages and the guides in guides/.

Guides

• Getting started with Livebook
• Micrograd demo walkthrough
• Elixir design notes
• API reference
• Troubleshooting

Troubleshooting

If Livebook cannot find the local package, confirm the notebook is still at notebooks/micrograd_demo.livemd. Its setup cell uses Mix.install([{:micrograd_ex, path: ".."}]).

If training or decision-boundary plotting is slow, remember that this is scalar educational autodiff. Reduce sample count, hidden width, training steps, or use a coarser boundary grid such as h: 0.35.

If the parameter count is not 337, confirm the architecture is exactly:

MicrogradEx.NN.MLP.new(2, [16, 16, 1])

More setup notes are in Troubleshooting.

Attribution

MicrogradEx is inspired by Andrej Karpathy's micrograd. The Elixir implementation preserves the scalar educational spirit while adapting state flow to immutable data.

License

See the LICENSE file.