Crucible Framework Logo

CrucibleFramework

Thin orchestration layer for ML experiment pipelines

ElixirOTPLicense

What's New (v0.5.2 - 2025-12-28)

What's New (v0.5.1 - 2025-12-27)

What's New (v0.5.0 - 2025-12-27)

See CHANGELOG.md for the complete migration guide.

What's New (v0.4.1 - 2025-12-26)

What's New (v0.4.0 - 2025-12-25)

Purpose

CrucibleFramework provides:

This library focuses purely on orchestration. Domain-specific functionality belongs in specialized packages:

Domain Package
Training crucible_train (future)
CNS Dialectics cns_crucible
Fairness ExFairness
XAI crucible_xai

Quick Start

def deps do
  [
    {:crucible_framework, "~> 0.5.1"}
  ]
end

Define and Run an Experiment

experiment = %CrucibleIR.Experiment{
  id: "my-experiment",
  backend: %CrucibleIR.BackendRef{id: :my_backend},
  pipeline: [
    %CrucibleIR.StageDef{name: :validate},
    %CrucibleIR.StageDef{name: :data_checks},
    %CrucibleIR.StageDef{name: :bench},
    %CrucibleIR.StageDef{name: :report}
  ]
}

{:ok, ctx} = CrucibleFramework.run(experiment)

Examples

Runnable scripts live under examples/. Start with:

mix run examples/01_core_pipeline.exs

Run the full set with:

./examples/run_all.sh

See examples/README.md for descriptions and optional dependency notes.

Core Modules

Crucible.Context

Runtime context threaded through pipeline stages. Uses Phoenix-style assigns for domain-specific data:

ctx = %Crucible.Context{
  experiment_id: "exp-1",
  run_id: "run-1",
  experiment: experiment
}

# Add metrics
ctx = Crucible.Context.put_metric(ctx, :accuracy, 0.95)

# Store domain data in assigns (training stages, CNS stages, etc.)
ctx = Crucible.Context.assign(ctx, :dataset, my_data)
ctx = Crucible.Context.assign(ctx, :backend_session, session)
ctx = Crucible.Context.assign(ctx, :snos, extracted_snos)

# Track stage completion
ctx = Crucible.Context.mark_stage_complete(ctx, :data_load)

Context Helper Functions

Category Functions
Metrics put_metric/3, get_metric/3, update_metric/3, merge_metrics/2, has_metric?/2
Outputs add_output/2, add_outputs/2
Artifacts put_artifact/3, get_artifact/3, has_artifact?/2
Assigns assign/2, assign/3
Stages mark_stage_complete/2, stage_completed?/2, completed_stages/1

Crucible.Stage

Behaviour for pipeline stages. All stages must implement both run/2 and describe/1:

defmodule MyApp.Stage.CustomStage do
  @behaviour Crucible.Stage

  @impl true
  def run(%Crucible.Context{} = ctx, opts) do
    # Do work, update ctx
    {:ok, updated_ctx}
  end

  @impl true
  def describe(_opts) do
    %{
      name: :custom,
      description: "My custom stage",
      required: [:input_path],
      optional: [:format, :verbose],
      types: %{
        input_path: :string,
        format: {:enum, [:json, :csv]},
        verbose: :boolean
      },
      defaults: %{
        format: :json,
        verbose: false
      }
    }
  end
end

Stage Contract

All stages must implement describe/1 returning a canonical schema:

Field Required Type Description
name Yes atom Stage identifier
description Yes string Human-readable description
required Yes list of atoms Required option keys
optional Yes list of atoms Optional option keys
types Yes map Type specifications for options
defaults No map Default values for optional fields
version No string Stage version
__extensions__ No map Domain-specific metadata

Use mix crucible.stages to list available stages and their schemas:

$ mix crucible.stages
$ mix crucible.stages --name bench

Built-in Stages

Stage Purpose
Crucible.Stage.Validate Pre-flight pipeline validation
Crucible.Stage.DataChecks Lightweight data validation (reads from assigns[:examples])
Crucible.Stage.Guardrails Safety checks via adapters
Crucible.Stage.Bench Statistical analysis (requires crucible_bench)
Crucible.Stage.Report Output generation

Crucible.Registry

Stage module resolution from config:

# In config.exs
config :crucible_framework,
  stage_registry: %{
    validate: Crucible.Stage.Validate,
    bench: Crucible.Stage.Bench,
    my_stage: MyApp.Stage.Custom
  }

Architecture

CrucibleFramework.run(experiment)
    |
    v
Crucible.Pipeline.Runner
    |
    +-> Stage 1: Validate
    +-> Stage 2: CustomDataLoader (domain-specific)
    +-> Stage 3: CustomBackendCall (domain-specific)
    +-> Stage 4: Bench
    +-> Stage 5: Report
    |
    v
{:ok, final_context}

Domain-Specific Stages

Training, CNS, and other domain-specific stages should be implemented in their respective packages and registered via config:

# crucible_train would provide:
config :crucible_framework,
  stage_registry: %{
    data_load: CrucibleTrain.Stage.DataLoad,
    backend_call: CrucibleTrain.Stage.BackendCall,
    # ...
  }

# cns_crucible would provide:
config :crucible_framework,
  stage_registry: %{
    cns_extract: CnsCrucible.Stage.SNOExtraction,
    cns_topology: CnsCrucible.Stage.TopologyAnalysis,
    # ...
  }

Configuration

Database Configuration (Oban Pattern)

CrucibleFramework uses dynamic repo injection - your host application provides the Repo:

# config/config.exs
config :crucible_framework,
  repo: MyApp.Repo,  # Required: host app's Repo module
  stage_registry: %{
    validate: Crucible.Stage.Validate,
    data_checks: Crucible.Stage.DataChecks,
    guardrails: Crucible.Stage.Guardrails,
    bench: Crucible.Stage.Bench,
    report: Crucible.Stage.Report
  },
  guardrail_adapter: Crucible.Stage.Guardrails.Noop

# Your host app's Repo configuration
config :my_app, MyApp.Repo,
  database: "my_app_dev",
  username: "postgres",
  password: "postgres",
  hostname: "localhost"

Then start your Repo in your application's supervision tree:

# lib/my_app/application.ex
children = [
  MyApp.Repo,
  # ... other children
]

Migrations

Copy migrations from deps/crucible_framework/priv/repo/migrations/ or run:

mix crucible_framework.install

Legacy Mode

For backwards compatibility, set start_repo: true to auto-start CrucibleFramework.Repo:

config :crucible_framework,
  start_repo: true,
  ecto_repos: [CrucibleFramework.Repo]

config :crucible_framework, CrucibleFramework.Repo,
  database: "crucible_dev",
  username: "crucible_dev",
  password: "crucible_dev_pw",
  hostname: "localhost"

Dependencies

Optional Dependencies

CrucibleFramework runs without the optional packages below; they enable specific features.

crucible_bench

crucible_trace

Enabling optional packages

def deps do
  [
    {:crucible_framework, "~> 0.5.1"},
    {:crucible_bench, "~> 0.4.0"},
    {:crucible_trace, "~> 0.3.0"}
  ]
end

If you do not need bench or tracing, omit those deps and remove :bench from your pipeline (or from stage_registry) to keep the core slim. See examples/02_bench_optional.exs and examples/03_trace_optional.exs for optional-dep usage.

Development

# Setup
mix deps.get && mix compile

# Tests
mix test

# Integration tests (persistence; requires CRUCIBLE_DB_ENABLED=true)
CRUCIBLE_DB_ENABLED=true MIX_ENV=test mix test --include integration

# Quality checks
mix format
mix credo --strict
mix dialyzer

Related Repositories

Repository Purpose
crucible_ir Shared IR structs
crucible_bench Statistical testing
crucible_trace Causal transparency
cns CNS dialectical reasoning
cns_crucible CNS + Crucible integration

License

MIT. See LICENSE.