Margarine 🧈

Margarine Logo

AI-powered image generation for Elixir using FLUX and SDXL.

Margarine brings state-of-the-art text-to-image and image-to-image generation to the Elixir ecosystem with a clean, native API. Generate beautiful images from text prompts or transform existing images with just a few lines of code.

Features

Quick Start

# Add to mix.exs
def deps do
  [
    {:margarine, "~> 0.2.0"},

    # REQUIRED: Choose ONE Nx backend based on your hardware
    {:emlx, "~> 0.1"}  # For Apple Silicon (M1/M2/M3/M4)
    # {:exla, "~> 0.10"}  # For NVIDIA/AMD GPU or CPU
  ]
end

Important: You must install either EMLX or EXLA alongside Margarine. The backend handles GPU/CPU acceleration for tensor operations.

# Simple text-to-image with FLUX
{:ok, image} = Margarine.generate("a red panda eating bamboo")
Margarine.Image.save(image, "panda.png")

# Photorealistic with SDXL
{:ok, image} = Margarine.generate("a serene mountain landscape at sunset",
  model: :sdxl_turbo,   # Fast photorealistic generation
  steps: 1
)

# Image-to-image transformation
{:ok, image} = Margarine.img2img(
  "turn into a watercolor painting",
  "photo.png",
  denoising_strength: 0.5,  # How much to change (0.0-1.0)
  model: :sdxl_base
)

🚀 Interactive Tutorials (Recommended!)

The best way to learn Margarine is through our interactive Livebook tutorials. These step-by-step guides run live code in your browser and let you experiment as you learn.

Start Here: Choose Your Model

We have two comprehensive tutorials, one for each model type:

📘 FLUX Tutorial (Artistic Generation)

livebook open notebooks/flux_getting_started.livemd

You'll learn:

Best for: Illustrations, artistic images, creative concepts

📗 SDXL Tutorial (Photorealistic Generation)

livebook open notebooks/sdxl_getting_started.livemd

You'll learn:

Best for: Realistic photos, detailed scenes, portraits

First Time Using Livebook?

Install it first:

mix escript.install hex livebook

Then open either tutorial above. The notebooks include everything you need, including dependency installation!

Installation

1. Add Dependencies

Add margarine and an Nx backend to your mix.exs:

def deps do
  [
    {:margarine, "~> 0.1.0"},

    # Choose ONE backend:
    {:emlx, "~> 0.1"}   # Apple Silicon (M1/M2/M3/M4) - Recommended for Macs
    # {:exla, "~> 0.10"}  # NVIDIA/AMD GPU or CPU
    # {:torchx, "~> 0.7"}  # PyTorch backend (experimental)
  ]
end

2. Configure Nx Backend

In config/config.exs:

# For Apple Silicon
config :nx,
  default_backend: EMLX.Backend,
  default_defn_options: [compiler: EMLX]

# OR for NVIDIA/AMD GPU
# config :nx,
#   default_backend: EXLA.Backend,
#   default_defn_options: [compiler: EXLA]

For NVIDIA GPU users on Linux: See the EXLA Setup Guide for Linux (NVIDIA GPU) for detailed CUDA installation and configuration instructions.

3. First Run Setup

No manual installation required! Margarine automatically handles everything:

  1. UV Package Manager - Automatically downloaded and installed by Pythonx
  2. Python 3.11+ - Downloaded via UV (~100MB)
  3. PyTorch & Dependencies - Installed via UV (~500MB)
  4. Everything cached - Instant subsequent runs

This takes 2-5 minutes on first run. All you need is an internet connection and ~15GB disk space.

For production deployments, we recommend running a "warm-up" generation when your server starts:

# In your application startup
defmodule MyApp.Application do
  def start(_type, _args) do
    # Warm up Margarine to avoid first-request timeout
    Task.start(fn ->
      Margarine.check_environment()
    end)

    # ... rest of your application setup
  end
end

System Requirements

Minimum Requirements

Recommended for Best Performance

Model Memory Requirements

Usage Examples

Basic Generation

# Simple prompt
{:ok, image} = Margarine.generate("a cute red panda")

# Check the result
IO.inspect(Nx.shape(image))  # {1024, 1024, 3}
IO.inspect(Nx.type(image))   # {:u, 8} (RGB values 0-255)

# Save to file
Margarine.Image.save(image, "panda.png")

Reproducible Generation

# Use a seed for reproducible results
opts = [seed: 42, model: :flux_schnell]

{:ok, image1} = Margarine.generate("a mountain landscape", opts)
{:ok, image2} = Margarine.generate("a mountain landscape", opts)

# image1 and image2 will be identical

High-Quality Generation

# Use FLUX Dev for artistic quality
{:ok, image} = Margarine.generate("a cyberpunk cityscape",
  model: :flux_dev,
  steps: 28,
  guidance_scale: 3.5,
  size: {1024, 1024}
)

# Use SDXL Base for photorealistic quality
{:ok, image} = Margarine.generate("a photorealistic portrait",
  model: :sdxl_base,
  steps: 20,
  guidance_scale: 7.5,
  size: {1024, 1024}
)

Image-to-Image Transformation

# Subtle style change (keep most of original)
{:ok, image} = Margarine.img2img(
  "watercolor painting style",
  "photo.png",
  denoising_strength: 0.3,
  model: :sdxl_base
)

# Moderate transformation
{:ok, image} = Margarine.img2img(
  "oil painting, impressionist style",
  "photo.png",
  denoising_strength: 0.6,
  model: :flux_schnell
)

# Heavy modification (only rough composition remains)
{:ok, image} = Margarine.img2img(
  "cyberpunk city at night",
  "photo.png",
  denoising_strength: 0.8,
  model: :sdxl_base
)

Error Handling

case Margarine.generate(prompt, opts) do
  {:ok, image} ->
    Margarine.Image.save(image, "output.png")
    IO.puts("Image generated successfully!")

  {:error, reason} ->
    IO.puts("Generation failed: #{reason}")
end

Health Checks

# Check if Python environment is ready
env = Margarine.check_environment()

case env do
  %{pythonx_initialized: true, python_version: version} ->
    IO.puts("Ready! Python #{version}")

  %{pythonx_initialized: false} ->
    IO.puts("Python not initialized yet")
end

Configuration Options

Text-to-Image Options (Margarine.generate/2)

Option Type Default Description
:model:flux_schnell | :flux_dev | :sdxl_turbo | :sdxl_base:flux_schnell Model to use
:stepspos_integer() Model-specific Number of denoising steps (4 for schnell, 28 for dev, 1 for turbo, 20 for base)
:guidance_scalefloat() Model-specific Guidance strength (0.0 for schnell/turbo, 3.5 for dev, 7.5 for base)
:seedinteger() | nilnil (random) Random seed for reproducibility
:size{height, width}{1024, 1024} Image dimensions (must be divisible by 8)

Image-to-Image Options (Margarine.img2img/3)

Option Type Default Description
:model:flux_schnell | :flux_dev | :sdxl_turbo | :sdxl_base:flux_schnell Model to use
:denoising_strengthfloat() (0.0-1.0) 0.75 How much to modify (0.0 = no change, 1.0 = complete regeneration)
:stepspos_integer() Model-specific Number of denoising steps
:guidance_scalefloat() Model-specific Guidance strength
:seedinteger() | nilnil (random) Random seed for reproducibility
:size{height, width} | nil Auto-detect from image Target dimensions (defaults to original image size, rounded to multiple of 8)

Architecture

Margarine uses a modular architecture:

All Python dependencies are managed automatically via UV. No manual pip install or virtualenv management required!

Development

This project follows strict Test-Driven Development (TDD) practices.

Running Tests

# Fast unit tests (default)
mix test

# With coverage report
mix test --cover

# Integration tests (requires model download, ~12GB)
mix test --only integration

# All tests including integration
mix test --include integration

# Linting
mix credo --strict

Test Coverage

We maintain 80%+ code coverage. Current coverage: 75.3%

Contributing

We welcome contributions! Please:

  1. Write tests first (TDD)
  2. Ensure all tests pass (mix test)
  3. Run Credo (mix credo --strict)
  4. Update documentation as needed

Roadmap

Phase 1: FLUX MVP ✅ (v0.1.0)

Phase 2: SDXL + Image-to-Image ✅ (v0.2.0)

Phase 3: Inpainting (Planned)

Phase 4: Advanced Features (Future)

Troubleshooting

"Python environment not initialized"

The first run takes 2-5 minutes to download Python and dependencies. Subsequent runs are instant.

Out of Memory Errors

Slow Generation

Timeout Errors

If you're getting timeout errors on slower systems or with limited memory (e.g., 24GB):

  1. Increase the timeout in your config/config.exs: 

    config :margarine,
  timeout: 600_000  # 10 minutes (default is 5 minutes)

  2. For very slow systems, increase even further: 

    config :margarine,
  timeout: 900_000  # 15 minutes

  3. Reduce memory pressure:

    • Use smaller image sizes: size: {512, 512} instead of {1024, 1024}
    • Use SDXL models (7GB) instead of FLUX (14GB) on 24GB systems
    • Close other applications to free up memory

Documentation

License

MIT License - see LICENSE for details.

Acknowledgments

Built with ❤️ by the Elixir community. Logo generated with FLUX - a perfect example of what this library can do!