PropertyDamage

Controlled chaos from the outside in: break your systems before your users do it in prod.

A stateful property-based testing (SPBT) framework for Elixir.

PropertyDamage generates random sequences of operations against your system and verifies that invariants hold throughout. When a failure is found, it automatically shrinks the sequence to the minimal reproduction case.

Features

• Stateful Testing: Generate sequences of commands, not just individual inputs
• Automatic Shrinking: Failed sequences are minimized to the smallest reproduction
• Symbolic References: Commands can reference results from earlier commands
• Parallel Execution: Branching sequences for race condition detection
• Linearization Checking: Verify parallel results are sequentially explainable
• Idempotency Testing: Built-in stutter testing for retry safety
• Rich Failure Reports: Comprehensive diagnostics when tests fail
• Failure Persistence: Save failures for later analysis and regression testing
• Step-by-Step Replay: Debug failures by executing commands one at a time
• Seed Library: Track and share interesting seeds across your team
• Coverage Metrics: Know how thoroughly your model is being exercised
• Flakiness Detection: Identify non-deterministic behavior in your SUT
• Load Testing: Generate realistic load using SPBT traffic patterns
• Visual Diagrams: Sequence diagrams in Mermaid, PlantUML, WebSequence formats
• Diff Debugging: Compare passing vs failing runs to find divergence
• Failure Export Hub: Convert failures to portable artifacts (scripts, tests, notebooks)
• Mutation Testing: Verify your tests catch bugs by injecting faults
• Invariant Suggestions: Get AI-powered suggestions for missing checks
• Failure Intelligence: Pattern detection, similarity analysis, and fix verification
• OpenAPI Scaffolding: Generate command modules from API specifications
• Telemetry Dashboard: Real-time monitoring of test runs with LiveView integration
• Livebook Integration: Interactive exploration with rich visualizations and charts
• Chaos Engineering: Built-in nemesis operations for network, resource, time, and process faults
• Differential Testing: Compare implementations against oracles, baselines, or each other

Installation

Add property_damage to your list of dependencies in mix.exs:

def deps do
  [
    {:property_damage, "~> 0.1.0"}
  ]
end

Quick Start

1. Define Commands

Commands represent operations that can be executed against your system:

defmodule MyApp.Commands.CreateUser do
  use PropertyDamage.Command

  defstruct [:name, :email]

  @impl true
  def new!(state, generators) do
    %__MODULE__{
      name: Faker.Person.name(),
      email: Faker.Internet.email()
    }
  end

  @impl true
  def precondition(_state), do: true

  @impl true
  def events(command, response) do
    [%MyApp.Events.UserCreated{
      id: response["id"],
      name: command.name,
      email: command.email
    }]
  end

  @impl true
  def ref(_command, response), do: response["id"]
end

2. Define Projections

Projections maintain state by processing events:

defmodule MyApp.Projections.Users do
  use PropertyDamage.Model.Projection

  def init, do: %{}

  def handles?(%MyApp.Events.UserCreated{}), do: true
  def handles?(_), do: false

  def apply(state, %MyApp.Events.UserCreated{} = event) do
    Map.put(state, event.id, %{name: event.name, email: event.email})
  end
end

3. Define Assertions (Invariants)

Assertions verify that invariants hold after each command. Use Model.Projection to define assertions with optional state tracking:

defmodule MyApp.Assertions.UniqueEmails do
  use PropertyDamage.Model.Projection

  # Track users state (optional - defaults to %{})
  def init, do: %{users: %{}}

  # Update state on events (optional - defaults to returning state unchanged)
  def apply(state, %UserCreated{id: id, email: email}) do
    put_in(state, [:users, id], %{email: email})
  end
  def apply(state, _), do: state

  # Assert unique emails after every step
  @trigger every: 1
  def assert_unique_emails(state, _cmd_or_event) do
    emails = Map.values(state.users) |> Enum.map(& &1.email)
    unless length(emails) == length(Enum.uniq(emails)) do
      PropertyDamage.fail!("Duplicate emails found", emails: emails)
    end
  end
end

For simpler assertions that don't need state tracking, you can skip init/0 and apply/2:

defmodule MyApp.Assertions.ValidEmails do
  use PropertyDamage.Model.Projection

  # Just define assertions - defaults are injected
  @trigger every: CreateUser
  def assert_valid_email(_state, %CreateUser{email: email}) do
    unless String.contains?(email, "@") do
      PropertyDamage.fail!("Invalid email", email: email)
    end
  end
end

4. Define a Model

The model ties everything together:

defmodule MyApp.TestModel do
  @behaviour PropertyDamage.Model

  @impl true
  def commands do
    [
      {MyApp.Commands.CreateUser, weight: 10},
      {MyApp.Commands.UpdateUser, weight: 5},
      {MyApp.Commands.DeleteUser, weight: 3}
    ]
  end

  @impl true
  def state_projection, do: MyApp.Projections.Users

  @impl true
  def extra_projections do
    [MyApp.Assertions.UniqueEmails, MyApp.Assertions.ValidEmails]
  end
end

5. Define an Adapter

The adapter executes commands against your actual system:

defmodule MyApp.TestAdapter do
  @behaviour PropertyDamage.Adapter

  @impl true
  def execute(%MyApp.Commands.CreateUser{} = cmd, config) do
    Req.post!("#{config.base_url}/users", json: %{
      name: cmd.name,
      email: cmd.email
    }).body
  end

  # ... other commands
end

6. Run Tests

PropertyDamage.run(
  model: MyApp.TestModel,
  adapter: MyApp.TestAdapter,
  adapter_config: %{base_url: "http://localhost:4000"},
  max_commands: 50,
  max_runs: 100
)

Debugging Failures

When PropertyDamage finds a failure, it provides rich tools for understanding what went wrong.

Understanding Failure Reports

{:error, failure} = PropertyDamage.run(model: M, adapter: A)

# Get a quick explanation
explanation = PropertyDamage.explain(failure)
IO.puts(PropertyDamage.Analysis.format_explanation(explanation))

# Find what triggered the failure
{:ok, trigger} = PropertyDamage.isolate_trigger(failure)
IO.puts("Cause: #{trigger.likely_cause}")

# Generate a reproducible test
test_code = PropertyDamage.generate_test(failure, format: :exunit)
File.write!("test/regression_test.exs", test_code)

Interactive Shrinking

If the initial shrinking didn't produce a minimal sequence:

# Try harder to shrink
{:ok, smaller} = PropertyDamage.shrink_further(failure,
  strategy: :exhaustive,
  max_time_ms: 120_000
)

Strategies:

• :quick - Fast, may miss some reductions
• :thorough - Balanced approach (default)
• :exhaustive - Try all possible reductions

Step-by-Step Replay

Execute commands one at a time to see exactly what happens:

{:ok, steps} = PropertyDamage.replay(failure)

for step <- steps do
  IO.puts("[#{step.index}] #{step.command_name}")
  IO.inspect(step.projections, label: "State after")

  case step.result do
    :ok -> IO.puts("  OK")
    {:check_failed, check, msg} -> IO.puts("  FAILED: #{msg}")
  end
end

For interactive debugging:

alias PropertyDamage.Replay

{:ok, session} = Replay.start(failure)
{:ok, session, step1} = Replay.step(session)
IO.inspect(Replay.current_state(session))
{:ok, session, step2} = Replay.step(session)
# ... continue stepping
Replay.stop(session)

Visual Debugging Tools

For complex failures, PropertyDamage provides visual tools to understand execution flow:

# Generate a sequence diagram from a failure
diagram = PropertyDamage.Diagram.from_failure_report(failure, :mermaid)
IO.puts(diagram)  # Paste into GitHub markdown, Notion, etc.

# Compare a passing run against a failing run to find the divergence
passing_trace = PropertyDamage.Diff.create_trace(passing_commands, passing_events, [], :pass)
failing_trace = PropertyDamage.Diff.create_trace(failing_commands, failing_events, [], {:fail, :test})
diff = PropertyDamage.Diff.compare_traces(passing_trace, failing_trace)
IO.puts(PropertyDamage.Diff.format(diff, format: :terminal))

See Visual Sequence Diagrams and Diff-Based Debugging for detailed documentation.

Failure Persistence

Save failures for later analysis or to build a regression suite:

# Save a failure
{:error, failure} = PropertyDamage.run(model: M, adapter: A)
{:ok, path} = PropertyDamage.save_failure(failure, "failures/")
# => {:ok, "failures/20251226T143000-check_failed-UniqueEmails-seed512902757.pd"}

# Load and analyze later
{:ok, loaded} = PropertyDamage.load_failure(path)
{:ok, steps} = PropertyDamage.replay(loaded)

# List all saved failures
failures = PropertyDamage.list_failures("failures/", sort: :newest)

# Delete old failures
PropertyDamage.delete_failure(path)

Seed Library

Track seeds that have found bugs for regression testing:

# Create or load a seed library
{:ok, library} = PropertyDamage.load_seed_library("seeds.json")

# Add a failure to the library
{:error, failure} = PropertyDamage.run(model: M, adapter: A)
{:ok, library} = PropertyDamage.add_to_seed_library(library, failure,
  tags: [:currency, :capture],
  description: "Currency mismatch in capture"
)

# Save the library
PropertyDamage.save_seed_library(library, "seeds.json")

# Get seeds to run in CI
alias PropertyDamage.SeedLibrary
failing_seeds = SeedLibrary.seed_values(library, status: :failing)

# Update status after running
library = SeedLibrary.record_run(library, seed, failed: false)

# View statistics
IO.puts(SeedLibrary.format(library))

Coverage Metrics

Track how thoroughly your model is being exercised:

alias PropertyDamage.Coverage

# Single run coverage
result = PropertyDamage.run(model: M, adapter: A)
coverage = PropertyDamage.coverage(result, M)
IO.puts(Coverage.format(coverage))

# Track across multiple runs
tracker = Coverage.new(M)
tracker = Coverage.record(tracker, result1)
tracker = Coverage.record(tracker, result2)

# Check thresholds in CI
unless Coverage.meets_threshold?(tracker, command: 80, transition: 50) do
  raise "Coverage threshold not met!"
end

# Find untested commands
untested = Coverage.untested_commands(tracker)

Format Options

Coverage supports multiple output formats:

# Summary - basic stats
IO.puts(Coverage.format(tracker, :summary))

# Matrix - shows command transition coverage
IO.puts(Coverage.format(tracker, :matrix))

# Full - includes everything
IO.puts(Coverage.format(tracker, :full))

# State classes (when classifier is set)
IO.puts(Coverage.format(tracker, :state_classes))

Transition Coverage

Track which command pairs (transitions) have been tested:

# Get a transition matrix showing which A→B pairs were tested
matrix = Coverage.transition_matrix(tracker)
# => %{CreateAccount => %{CreateAccount => 5, CreditAccount => 12, DebitAccount => 8}, ...}

# Find untested transitions
untested = Coverage.untested_transitions(tracker)
# => [{CreateAccount, DeleteAccount}, {DebitAccount, CloseAccount}, ...]

# Get most frequent transitions
top = Coverage.top_transitions(tracker, 5)
# => [{{CreateAccount, CreditAccount}, 42}, {{CreditAccount, DebitAccount}, 38}, ...]

State Class Coverage

For more meaningful coverage, define a state classifier to group concrete states into abstract classes:

# Define a classifier function
classifier = fn state ->
  cond do
    state.accounts == %{} -> :no_accounts
    Enum.all?(state.accounts, fn {_, a} -> a.balance == 0 end) -> :all_zero_balance
    Enum.any?(state.accounts, fn {_, a} -> a.balance < 0 end) -> :has_negative
    true -> :has_positive
  end
end

# Create tracker with classifier
tracker = Coverage.new(MyModel, state_classifier: classifier)
tracker = Coverage.record(tracker, result1)
tracker = Coverage.record(tracker, result2)

# View state class distribution
counts = Coverage.state_class_counts(tracker)
# => %{no_accounts: 5, all_zero_balance: 12, has_positive: 83}

# View state class transitions (what state classes lead to what)
transitions = Coverage.state_class_transitions(tracker)
# => %{{:no_accounts, :all_zero_balance} => 5, {:all_zero_balance, :has_positive} => 10, ...}

# Get state class matrix for visualization
state_matrix = Coverage.state_class_matrix(tracker)

# Format with state class matrix
IO.puts(Coverage.format(tracker, :state_classes))

State class coverage helps answer: "Have we tested all interesting state configurations?"

Flakiness Detection

Detect non-deterministic behavior in your system:

# Check if a specific seed is flaky
case PropertyDamage.check_determinism(M, A, 512902757, runs: 10) do
  {:ok, :deterministic} ->
    IO.puts("Seed produces consistent results")

  {:ok, :flaky, stats} ->
    IO.puts("FLAKY: passed #{stats.passes}/#{stats.runs} times")
    IO.puts("Variance type: #{stats.variance_type}")
end

# Discover flaky seeds
flaky_seeds = PropertyDamage.discover_flaky_seeds(M, A,
  num_seeds: 20,
  runs_per_seed: 5,
  verbose: true
)

OpenAPI Scaffolding

Generate command modules from an OpenAPI specification:

# Generate from a local file
mix pd.scaffold --from openapi.json --output lib/my_app_test/commands/

# Generate from a URL
mix pd.scaffold --from https://api.example.com/openapi.json --output lib/

# Only specific operations
mix pd.scaffold --from openapi.json --operations createUser,updateUser

# Preview without writing
mix pd.scaffold --from openapi.json --dry-run

Generated commands include:

• Struct fields from request body schemas
• Type hints from OpenAPI types
• Placeholder generators based on field types
• Adapter execution hints

Model Validation

Validate your model before running tests:

mix pd.validate --model MyApp.TestModel

This checks:

• All commands implement required callbacks
• Projections handle their declared events
• Checks reference valid projections
• No circular dependencies

Configuration

Run Options

PropertyDamage.run(
  model: MyApp.TestModel,
  adapter: MyApp.TestAdapter,

  # Generation
  max_commands: 50,        # Max commands per sequence
  max_runs: 100,           # Number of test runs
  seed: 12345,             # Deterministic seed (optional)

  # Shrinking
  shrink_timeout_ms: 30_000,
  max_shrink_iterations: 1000,

  # Idempotency
  stutter_probability: 0.1,  # Retry probability

  # Adapter
  adapter_config: %{base_url: "http://localhost:4000"}
)

Model Callbacks

defmodule MyModel do
  @behaviour PropertyDamage.Model

  # Required
  def commands, do: [{CommandModule, weight: N}, ...]
  def state_projection, do: MyStateProjection
  def extra_projections, do: [MyExtraProjection, ...]  # Optional

  # Optional
  def injectable_events, do: []  # For Adapter.Injector
  def simulator, do: MySimulatorModule  # Returns module implementing Simulator behaviour
  def setup_once(config), do: :ok
  def setup_each(config), do: :ok  # Called before each run/shrink attempt
  def teardown_each(config), do: :ok
  def teardown_once(config), do: :ok
  def terminate?(state, command, events), do: false  # Custom termination
end

Parallel Execution

PropertyDamage supports branching sequences for detecting race conditions and concurrent bugs. Commands can execute in parallel branches, and the framework verifies that results are linearizable.

Enabling Branching Sequences

PropertyDamage.run(
  model: MyApp.TestModel,
  adapter: MyApp.TestAdapter,
  max_commands: 50,
  max_runs: 100,
  branching: [
    branch_probability: 0.3,   # Probability of creating branch points
    max_branches: 3,           # Max parallel branches
    max_branch_length: 5,      # Max commands per branch
    min_prefix_length: 3       # Min commands before branching
  ]
)

How It Works

A branching sequence has three parts:

1. Prefix: Commands executed sequentially before branching
2. Branches: Parallel command lists executed concurrently
3. Suffix: Commands executed after branches merge

Prefix:  [cmd1, cmd2]
                |
       +--------+--------+
       |                 |
Branch A: [cmd3a, cmd4a] | Branch B: [cmd3b]
       |                 |
       +--------+--------+
                |
Suffix: [cmd5]

Linearization Checking

After parallel execution, PropertyDamage verifies that the observed results can be explained by some sequential ordering of the commands. If no valid ordering exists, a :linearization_failed error is raised.

alias PropertyDamage.Linearization

# Check complexity before verification
case Linearization.feasibility(branches) do
  :ok -> IO.puts("Manageable linearization space")
  {:warning, count} -> IO.puts("#{count} possible orderings")
end

# Count possible linearizations
count = Linearization.linearization_count([[cmd1, cmd2], [cmd3]])
# => 3 (possible orderings: [1,2,3], [1,3,2], [3,1,2])

Shrinking Branching Sequences

The shrinker handles branching sequences with special strategies:

1. Convert to linear: If race not required for failure
2. Remove branches: Eliminate unnecessary parallel branches
3. Shrink branches: Remove commands within individual branches
4. Shrink prefix/suffix: Remove non-essential sequential commands

Ref Constraints in Parallel Execution

Symbolic references follow strict rules in branching sequences:

• Refs from prefix can be used in any branch
• Refs from one branch cannot be used in another branch
• Refs from branches can be used in suffix

# Valid: prefix ref used in branch
prefix = [CreateUser.new()]  # Creates :user_ref
branches = [[GetUser.new(user_ref: :user_ref)], [UpdateUser.new(user_ref: :user_ref)]]

# Invalid: cross-branch ref usage
branches = [[CreateItem.new()],  # Creates :item_ref
            [ViewItem.new(item_ref: :item_ref)]]  # ERROR: :item_ref not visible

Eventual Consistency (Async Support)

For systems with eventual consistency, PropertyDamage provides probe and async command semantics with automatic settle/retry logic.

Command Semantics

Commands can declare their semantics via the semantics/0 callback:

defmodule MyTest.Commands.GetOrderStatus do
  @behaviour PropertyDamage.Command

  defstruct [:order_id]

  # This is a probe - it queries state and may need to retry
  def semantics, do: :probe

  # Configure settle behavior
  def settle_config do
    %{
      timeout_ms: 5_000,    # Max time to wait
      interval_ms: 200,     # Time between retries
      backoff: :exponential # :linear or :exponential
    }
  end

  def read_only?, do: true
end

Semantics Types

Adapter Integration

Adapters return settle-compatible results for probes:

def execute(%GetOrderStatus{order_id: id}, ctx) do
  case MyAPI.get_order(id) do
    {:ok, %{status: "pending"}} ->
      {:retry, :still_pending}  # Keep trying

    {:ok, order} ->
      {:ok, order}  # Success - stop retrying

    {:error, :not_found} ->
      {:retry, :not_found}  # Keep trying

    {:error, reason} ->
      {:error, reason}  # Hard failure - stop immediately
  end
end

See Async and Eventual Consistency Guide for complete documentation including bridge commands, Adapter.Injector, and handling async operations that require polling.

Fault Injection (Nemesis)

Test system resilience by injecting faults like network partitions, latency, and node crashes.

Defining a Nemesis Command

defmodule MyTest.Nemesis.PartitionNetwork do
  @behaviour PropertyDamage.Nemesis

  defstruct [:partition_type, :duration_ms]

  @impl true
  def precondition(_state), do: true

  @impl true
  def inject(%__MODULE__{partition_type: type}, ctx) do
    :ok = Toxiproxy.partition(ctx.proxy, type)
    {:ok, [%NetworkPartitioned{type: type}]}
  end

  @impl true
  def restore(%__MODULE__{partition_type: type}, ctx) do
    Toxiproxy.restore(ctx.proxy, type)
    {:ok, [%NetworkRestored{type: type}]}
  end

  # Auto-restore after duration
  def auto_restore?, do: true
  def duration_ms(%__MODULE__{duration_ms: d}), do: d
end

Using Nemesis in Models

Add nemesis commands with lower weights:

def commands do
  [
    {CreateOrder, weight: 5},
    {ProcessPayment, weight: 3},
    {PartitionNetwork, weight: 1},   # Fault injection
    {InjectLatency, weight: 1}
  ]
end

Built-in Nemesis Operations

PropertyDamage includes ready-to-use nemesis operations for common fault injection scenarios:

Network Operations

# Add network latency
alias PropertyDamage.Nemesis.NetworkLatency

def commands do
  [
    {CreateOrder, weight: 5},
    {NetworkLatency, weight: 1}  # Uses defaults: 100ms latency, 5s duration
  ]
end

# Or customize
%NetworkLatency{latency_ms: 200, jitter_ms: 50, duration_ms: 10_000}

Resource Operations

alias PropertyDamage.Nemesis.{MemoryPressure, CPUStress}

# Create memory pressure (100MB)
%MemoryPressure{megabytes: 100, allocation_pattern: :bulk}

# Create CPU stress (intensity 1-10)
%CPUStress{intensity: 5, schedulers: :all, duration_ms: 5000}

Time Operations

alias PropertyDamage.Nemesis.ClockSkew

# Jump 1 minute into the future
%ClockSkew{skew_ms: 60_000, mode: :instant}

# Gradual drift (10% fast)
%ClockSkew{skew_ms: 0, drift_rate: 1.1, mode: :gradual}

# In your adapter, use the virtual clock:
def get_current_time do
  ClockSkew.now()  # Returns skewed time when active
end

Process Operations

Security Operations

| CertificateExpiry | Simulate TLS certificate failures (expired, wrong host, self-signed, revoked)

alias PropertyDamage.Nemesis.CertificateExpiry

# Simulate expired certificate
%CertificateExpiry{failure_type: :expired}

# Simulate hostname mismatch
%CertificateExpiry{failure_type: :wrong_host, target: :api}

# In your adapter:
def connect(host, port, opts) do
  if CertificateExpiry.should_fail?() do
    CertificateExpiry.get_ssl_error()  # Returns {:error, {:tls_alert, ...}}
  else
    :ssl.connect(host, port, opts)
  end
end

Process Operations (continued)

alias PropertyDamage.Nemesis.{ProcessKill, SlowIO}

# Kill a specific named process
%ProcessKill{target: {:name, :my_worker}, signal: :kill}

# Kill random processes from supervised children
%ProcessKill{target: {:supervised_by, MyApp.WorkerSupervisor}}

# Slow down I/O operations
%SlowIO{delay_ms: 100, target: :all}  # :reads, :writes, or :all

# In your adapter:
def read_data(path) do
  if SlowIO.should_delay?(:reads), do: SlowIO.apply_delay()
  File.read(path)
end

Integration with Toxiproxy

Network operations integrate with Toxiproxy when available:

# Configure in adapter context
context = %{
  toxiproxy: %{
    proxy_name: "my_service",
    api_url: "http://localhost:8474"
  }
}

# Nemesis operations will automatically use Toxiproxy
# Falls back to simulated mode if not configured

Adjusting Invariants During Faults

@trigger every: 1
def assert_latency_sla(state, _cmd_or_event) do
  # Skip SLA check during partition
  unless Map.get(state.active_faults, :network_partition) do
    unless state.last_latency_ms < 100 do
      PropertyDamage.fail!("SLA violated", latency_ms: state.last_latency_ms)
    end
  end
end

Production Forensics

Replay production event logs through your model to analyze incidents.

Basic Usage

# Fetch events from your observability system
{:ok, events} = ProductionLogs.fetch(trace_id: "abc123")

# Replay through model projections
result = PropertyDamage.Forensics.analyze(
  events: events,
  model: OrderModel
)

case result do
  {:ok, %{final_state: state, events_processed: n}} ->
    IO.puts("Processed #{n} events - no violations")

  {:error, failure} ->
    IO.puts("Violation at event ##{failure.failure_step}")
    IO.puts(PropertyDamage.Forensics.format_report(failure))
end

Event Mapping

Translate production event formats to your model's event structs:

defmodule MyEventMapping do
  @behaviour PropertyDamage.Forensics.EventMapping

  @impl true
  def map(%{"type" => "order.created", "payload" => p}) do
    {:ok, %OrderCreated{
      order_id: p["order_id"],
      amount: p["total"]
    }}
  end

  def map(%{"type" => "internal.metric"}), do: :skip
  def map(_), do: {:skip, :unknown_event}
end

# Use with analyze
Forensics.analyze(
  events: production_events,
  model: OrderModel,
  event_mapping: MyEventMapping
)

Generate Regression Tests

Create test cases from production failures:

{:error, failure} = Forensics.analyze(events: events, model: MyModel)
test_code = Forensics.generate_regression_test(failure, MyModel)
File.write!("test/regressions/incident_2025_01_15_test.exs", test_code)

Liveness Checking

Detect deadlocks, livelocks, and starvation with the Liveness projection.

Configuration

defmodule MyModel do
  def extra_projections do
    [
      {PropertyDamage.Model.Projection.Liveness, [
        max_pending_duration_ms: 10_000,
        check_interval: 10,
        required_completions: %{
          CreateTransfer => [TransferCompleted, TransferFailed],
          CreateOrder => [OrderConfirmed, OrderRejected]
        }
      ]}
    ]
  end
end

How It Works

1. Track starts: When CreateTransfer executes, mark operation as pending
2. Track completions: When TransferCompleted or TransferFailed arrives, mark complete
3. Check timeouts: Periodically check for operations pending too long
4. Report stuck: If any operation exceeds max_pending_duration_ms, fail

What It Detects

Load Testing

Generate realistic load against your system using SPBT-generated traffic. Unlike synthetic benchmarks, each simulated user session follows valid state transitions with command weights that model real usage patterns.

Basic Usage

{:ok, report} = PropertyDamage.LoadTest.run(
  model: MyModel,
  adapter: HTTPAdapter,
  adapter_config: %{base_url: "http://localhost:4000"},
  concurrent_users: 50,
  duration: {2, :minutes}
)

# Print formatted report
IO.puts(PropertyDamage.LoadTest.format(report, :terminal))

Advanced Configuration

{:ok, report} = PropertyDamage.LoadTest.run(
  model: MyModel,
  adapter: HTTPAdapter,
  adapter_config: %{base_url: "http://localhost:4000"},

  # Load configuration
  concurrent_users: 100,
  duration: {5, :minutes},

  # Ramp strategies: :immediate, {:linear, duration}, {:step, N, interval}, {:exponential, duration}
  ramp_up: {:linear, {30, :seconds}},
  ramp_down: {:linear, {10, :seconds}},

  # Session behavior
  commands_per_session: {10, 50},  # {min, max} commands per sequence
  think_time: {100, 500},          # {min, max} ms between commands

  # Live metrics callback (called every interval)
  metrics_interval: {1, :seconds},
  on_metrics: fn m ->
    IO.puts("RPS: #{m.requests_per_second}, p95: #{m.latency_p95}ms, errors: #{m.error_rate}%")
  end,

  # Called when test completes
  on_complete: fn report ->
    PropertyDamage.LoadTest.save(report, "load_test.md", :markdown)
  end,

  # Assertion mode: :disabled (default), :record, or :log
  assertion_mode: :record  # Track assertion failures in metrics
)

Ramp Strategies

Metrics Collected

• Throughput: Total requests, requests/second
• Latency: p50, p95, p99, min, max, mean (in milliseconds)
• Errors: Total count, error rate, breakdown by type
• Assertions: Failures count, rate, by assertion name (when enabled via assertion_mode)
• Per-Command: Individual metrics for each command type
• History: Time series for trend analysis

Report Formats

# Terminal output with ASCII charts
IO.puts(PropertyDamage.LoadTest.format(report, :terminal))

# Markdown for documentation
PropertyDamage.LoadTest.save(report, "report.md", :markdown)

# JSON for programmatic analysis
json = PropertyDamage.LoadTest.format(report, :json)

Async Control

# Start without blocking
{:ok, runner} = PropertyDamage.LoadTest.start(opts)

# Monitor progress
status = PropertyDamage.LoadTest.status(runner)
# => %{phase: :steady, active_sessions: 50, progress_percent: 45.0, ...}

# Get live metrics
metrics = PropertyDamage.LoadTest.get_metrics(runner)

# Stop early if needed
{:ok, report} = PropertyDamage.LoadTest.stop(runner)

# Or wait for completion
{:ok, report} = PropertyDamage.LoadTest.await(runner)

Visual Sequence Diagrams

Generate sequence diagrams from failure reports to visualize command flows and pinpoint failures.

Supported Formats

Basic Usage

# From a failure report
{:error, report} = PropertyDamage.run(model: MyModel, adapter: MyAdapter)
diagram = PropertyDamage.Diagram.from_failure_report(report, :mermaid)
IO.puts(diagram)

# From sequence and event log
diagram = PropertyDamage.Diagram.generate(sequence, event_log, :plantuml,
  title: "Account Creation Flow",
  highlight_failure: true
)

# Save to file
PropertyDamage.Diagram.save(diagram, "failure_diagram", :mermaid)
# Creates: failure_diagram.md

Example Output (Mermaid)

sequenceDiagram
    title Failure: NonNegativeBalance (seed: 12345)
    participant Test
    participant SUT

    Test->>SUT: CreateAccount(name: "Alice")
    SUT-->>Test: AccountCreated(id: "acc_123", balance: 0)

    Test->>SUT: Deposit(amount: 100)
    SUT-->>Test: DepositSucceeded(new_balance: 100)

    Note over Test,SUT: ❌ FAILURE at command 2
    Test-xSUT: Withdraw(amount: 200)
    Note right of SUT: Balance went negative

Options

• :title - Custom diagram title
• :show_state - Include state participant
• :max_value_length - Truncate long values (default: 50)
• :highlight_failure - Visual failure markers (default: true)

Diff-Based Debugging

Compare passing and failing test runs to identify exactly what changed.

Comparing Traces

# Compare two failure reports
passing = PropertyDamage.run(model: M, adapter: A, seed: 123) |> elem(1)
failing = PropertyDamage.run(model: M, adapter: A, seed: 456) |> elem(1)

diff = PropertyDamage.Diff.compare_reports(passing, failing)
IO.puts(PropertyDamage.Diff.format(diff))

Output Formats

# Terminal (default) - ASCII boxes
PropertyDamage.Diff.format(diff, format: :terminal)

# Markdown - tables for documentation
PropertyDamage.Diff.format(diff, format: :markdown)

# JSON - for programmatic analysis
PropertyDamage.Diff.format(diff, format: :json)

Example Terminal Output

╔══════════════════════════════════════════════════════════════════════╗
║                         EXECUTION DIFF                               ║
╚══════════════════════════════════════════════════════════════════════╝

Summary: Divergence at command 2: Withdraw. Events differ.

┌─ Event Differences ─────────────────────────────────────────────────┐
│ Cmd 2 ≠: LEFT: [WithdrawSucceeded]                                  │
│         RIGHT: [WithdrawFailed]                                     │
└──────────────────────────────────────────────────────────────────────┘

┌─ State Differences ─────────────────────────────────────────────────┐
│ After command 2:                                                    │
│   balance: -50 → 100                                                │
└──────────────────────────────────────────────────────────────────────┘

What It Detects

Failure Export Hub

Convert failure reports into portable artifacts for sharing, regression testing, and interactive exploration.

Export Formats

Basic Usage

{:error, failure} = PropertyDamage.run(model: MyModel, adapter: MyAdapter)

# Generate ExUnit regression test
test_code = PropertyDamage.Export.to_exunit(failure)
File.write!("test/regressions/seed_#{failure.seed}_test.exs", test_code)

# Generate standalone scripts
elixir_script = PropertyDamage.Export.to_script(failure, :elixir,
  base_url: "http://localhost:4000",
  adapter: MyHTTPAdapter
)

curl_script = PropertyDamage.Export.to_script(failure, :curl,
  base_url: "http://localhost:4000",
  adapter: MyHTTPAdapter
)

python_script = PropertyDamage.Export.to_script(failure, :python,
  base_url: "http://localhost:4000",
  adapter: MyHTTPAdapter
)

# Generate LiveBook notebook
notebook = PropertyDamage.Export.to_livebook(failure,
  base_url: "http://localhost:4000",
  adapter: MyHTTPAdapter
)

File Operations

# Save single format
{:ok, path} = PropertyDamage.Export.save(failure, "exports/", :exunit)
# => {:ok, "exports/reproduce_512902757.exs"}

{:ok, path} = PropertyDamage.Export.save(failure, "exports/", {:script, :curl},
  base_url: "http://localhost:4000",
  adapter: MyHTTPAdapter
)
# => {:ok, "exports/reproduce_512902757.sh"}

# Save all formats at once
{:ok, paths} = PropertyDamage.Export.save_all(failure, "exports/",
  base_url: "http://localhost:4000",
  adapter: MyHTTPAdapter,
  script_languages: [:elixir, :curl, :python]
)
# => {:ok, %{
#   exunit: "exports/reproduce_512902757.exs",
#   livebook: "exports/reproduce_512902757.livemd",
#   script_elixir: "exports/reproduce_512902757.exs",
#   script_curl: "exports/reproduce_512902757.sh",
#   script_python: "exports/reproduce_512902757.py"
# }}

HTTPSpec for Script Generation

For scripts to make HTTP calls, your adapter needs to implement http_spec/2:

defmodule MyHTTPAdapter do
  @behaviour PropertyDamage.Adapter

  alias PropertyDamage.Export.HTTPSpec

  # Standard adapter callbacks...
  def execute(cmd, ctx), do: # ...

  # Optional: HTTP mapping for export
  def http_spec(%CreateAccount{currency: curr}, _ctx) do
    %HTTPSpec{
      method: :post,
      path: "/api/accounts",
      body: %{currency: curr}
    }
  end

  def http_spec(%CreditAccount{account_ref: ref, amount: amt}, _ctx) do
    %HTTPSpec{
      method: :post,
      path: "/api/accounts/:account_id/credit",
      path_params: %{account_id: ref},
      body: %{amount: amt}
    }
  end

  def http_spec(%DebitAccount{account_ref: ref, amount: amt}, _ctx) do
    %HTTPSpec{
      method: :post,
      path: "/api/accounts/:account_id/debit",
      path_params: %{account_id: ref},
      body: %{amount: amt}
    }
  end
end

LiveBook Features

Generated LiveBook notebooks include:

• Setup section: Installs dependencies (Req, Jason)
• State tracking: Tracks refs and model state alongside execution
• Step-by-step commands: Each command in its own cell with HTTP call
• Failure marker: Highlights the command that caused the failure
• Exploration section: Space to experiment with variations

# Exclude exploration section if not needed
notebook = PropertyDamage.Export.to_livebook(failure,
  base_url: "http://localhost:4000",
  adapter: MyHTTPAdapter,
  include_exploration: false
)

Example Generated Script (curl)

#!/bin/bash
# Failure Reproduction Script
# Generated: 2025-12-26T14:30:00Z
# Failure: NonNegativeBalance check failed
# Seed: 512902757

set -e
BASE_URL="${BASE_URL:-http://localhost:4000}"

echo "=== Step 1: CreateAccount ==="
RESP1=$(curl -s -X POST "$BASE_URL/api/accounts" \
  -H "Content-Type: application/json" \
  -d '{"currency": "USD"}')
echo "$RESP1"
REF_account_0=$(echo "$RESP1" | jq -r '.data.id // .id // empty')

echo "=== Step 2: CreditAccount ==="
RESP2=$(curl -s -X POST "$BASE_URL/api/accounts/$REF_account_0/credit" \
  -H "Content-Type: application/json" \
  -d '{"amount": 100}')
echo "$RESP2"

echo "=== Step 3: DebitAccount (FAILURE POINT) ==="
RESP3=$(curl -s -X POST "$BASE_URL/api/accounts/$REF_account_0/debit" \
  -H "Content-Type: application/json" \
  -d '{"amount": 200}')
echo "$RESP3"

Mutation Testing

Verify that your property tests are actually effective at catching bugs. Mutation testing injects faults into adapter responses and checks if your tests detect them.

Basic Usage

{:ok, report} = PropertyDamage.Mutation.run(
  model: MyModel,
  adapter: MyAdapter,
  adapter_config: %{base_url: "http://localhost:4000"},
  target_score: 0.80
)

# Check results
IO.puts(PropertyDamage.Mutation.format(report))

# Get detailed analysis
if not PropertyDamage.Mutation.passes?(report) do
  analysis = PropertyDamage.Mutation.analyze(report)
  IO.puts(PropertyDamage.Mutation.Analysis.format(analysis))
end

Understanding Results

• Killed mutant: Your tests detected the simulated bug (good)
• Survived mutant: Your tests missed the bug (bad - weak tests)
• Mutation score: killed / total - aim for 80%+

Mutation Operators

Options

PropertyDamage.Mutation.run(
  model: MyModel,
  adapter: MyAdapter,
  adapter_config: %{base_url: "http://localhost:4000"},

  # Which operators to use (default: all)
  operators: [:value, :omission, :status],

  # Mutations per command type (default: 5)
  mutations_per_command: 10,

  # PropertyDamage runs per mutation (default: 10)
  max_runs: 20,

  # Target score to pass (default: 0.80)
  target_score: 0.80,

  # Timeout per mutation test (default: 30000)
  timeout_ms: 60_000,

  # Print progress
  verbose: true
)

Example Report

╔══════════════════════════════════════════════════════════════════════╗
║                       MUTATION TESTING REPORT                        ║
╚══════════════════════════════════════════════════════════════════════╝

Mutation Score: 85% (17/20 killed)  ✓ PASS (target: 80%)

┌─ By Command ────────────────────────────────────────────────────────┐
│ CreateAccount    ████████████████████ 100% (5/5)                    │
│ CreditAccount    ██████████████░░░░░░  86% (6/7)                    │
│ DebitAccount     ████████████░░░░░░░░  75% (6/8)                    │
└─────────────────────────────────────────────────────────────────────┘

┌─ Survived Mutations (Weaknesses) ───────────────────────────────────┐
│ 1. CreditAccount: amount 100→99 (off-by-one not detected)           │
│ 2. DebitAccount: omitted 'timestamp' field not detected             │
└─────────────────────────────────────────────────────────────────────┘

Analysis & Suggestions

analysis = PropertyDamage.Mutation.analyze(report)

# Weak commands (low kill rates)
for {cmd, score} <- analysis.weak_commands do
  IO.puts("#{cmd}: #{Float.round(score * 100, 1)}%")
end

# Fields that aren&#39;t being validated
IO.inspect(analysis.unchecked_fields)

# Actionable suggestions
for suggestion <- analysis.suggestions do
  IO.puts("• #{suggestion}")
end

Property & Invariant Suggestions

Automatically analyze your model and get suggestions for missing checks and invariants.

Basic Usage

# Analyze a model
suggestions = PropertyDamage.Suggestions.analyze(MyModel)

# Print formatted suggestions
IO.puts(PropertyDamage.Suggestions.format(suggestions))

# Get high-priority suggestions only
high_priority = PropertyDamage.Suggestions.high_priority(suggestions)

# Filter by field or event
balance_suggestions = PropertyDamage.Suggestions.for_field(suggestions, :balance)

What It Detects

The suggestion system examines your events and existing checks to identify gaps:

Example Output

╔════════════════════════════════════════════════════════════════════════╗
║             PROPERTY & INVARIANT SUGGESTIONS                           ║
╚════════════════════════════════════════════════════════════════════════╝

Model: MyApp.TestModel
Events analyzed: 12
Existing checks: 3
Field coverage: 40%

Suggestions: 8 total
  ▸ 2 high priority (should address)
  ▸ 4 medium priority (consider adding)
  ▸ 2 low priority (nice to have)

┌─ Suggestions ──────────────────────────────────────────────────────────┐
│ ▶ HIGH PRIORITY ───────────────────────────────────────────────────────│
│   1. Add non-negative check for balance (balance)                      │
│   2. Add currency consistency check (currency)                         │
│                                                                        │
│ ▶ MEDIUM PRIORITY ─────────────────────────────────────────────────────│
│   3. Add reference existence check for account_ref (account_ref)       │
│   4. Add status transition validation for status (status)              │
└────────────────────────────────────────────────────────────────────────┘

Output Formats

# Terminal - ASCII boxes (default)
PropertyDamage.Suggestions.format(suggestions, :terminal)

# Markdown - tables with example code
PropertyDamage.Suggestions.format(suggestions, :markdown)

# JSON - for programmatic analysis
PropertyDamage.Suggestions.format(suggestions, :json)

Options

PropertyDamage.Suggestions.analyze(MyModel,
  # Include low-priority suggestions (default: true)
  include_low_priority: true,

  # Maximum suggestions to return (default: 20)
  max_suggestions: 10,

  # Focus on specific areas (default: :all)
  # Options: :all, :numeric, :references, :consistency
  focus: :numeric
)

Integration with Mutation Testing

Use suggestions to improve your mutation testing score:

# Run mutation testing
{:ok, mutation_report} = PropertyDamage.Mutation.run(model: MyModel, adapter: MyAdapter)

# If score is low, get suggestions for improvement
if mutation_report.mutation_score < 0.8 do
  suggestions = PropertyDamage.Suggestions.analyze(MyModel)
  IO.puts(PropertyDamage.Suggestions.format(suggestions, :markdown))
end

Failure Intelligence

Analyze, cluster, and verify fixes for failures using fingerprinting and similarity detection.

Pattern Detection

When you have multiple failures, identify patterns to find root causes:

# Analyze a set of failures
failures = [failure1, failure2, failure3, ...]
analysis = PropertyDamage.FailureIntelligence.analyze(failures)

IO.puts(analysis.pattern_summary)
# => "Analyzed 15 failures:
#     - 3 distinct patterns (12 failures)
#     - 3 unique failures (no pattern match)
#
#     Top patterns:
#       - Check failure in :balance_valid during DebitAccount (5 occurrences)
#       - Invariant violation during CreditAccount (4 occurrences)"

# Get individual clusters
for cluster <- analysis.clusters do
  IO.puts("Pattern: #{cluster.pattern.description}")
  IO.puts("Occurrences: #{cluster.size}")
end

Similarity Detection

Compare failures to identify duplicates and related issues:

# Check if two failures are similar
if PropertyDamage.FailureIntelligence.similar?(failure1, failure2) do
  IO.puts("These failures likely have the same root cause")
end

# Get similarity score (0.0 to 1.0)
score = PropertyDamage.FailureIntelligence.similarity_score(failure1, failure2)
# => 0.85

# Detailed comparison
comparison = PropertyDamage.FailureIntelligence.compare(failure1, failure2)
# => %{
#   score: 0.85,
#   breakdown: %{failure_type: 1.0, check_name: 1.0, command_type: 0.8, ...},
#   is_similar: true
# }

# Find similar failures from a list
similar = PropertyDamage.FailureIntelligence.find_similar(new_failure, known_failures,
  threshold: 0.80,
  limit: 5
)

Fingerprinting

Fingerprints capture the essential characteristics of a failure:

# Get a fingerprint for quick comparison
fingerprint = PropertyDamage.FailureIntelligence.fingerprint(failure)
# => %Fingerprint{
#   failure_type: :check_failed,
#   check_name: :balance_non_negative,
#   command_type: DebitAccount,
#   event_types: [AccountDebited],
#   sequence_shape: [CreateAccount, CreditAccount, DebitAccount],
#   error_category: :check_violation,
#   ...
# }

# Get a short hash for display
hash = PropertyDamage.FailureIntelligence.fingerprint_hash(failure)
# => "a1b2c3d4"

# Group failures by fingerprint
groups = PropertyDamage.FailureIntelligence.group_by_fingerprint(failures)
for {hash, group} <- groups do
  IO.puts("Hash #{hash}: #{length(group)} failures")
end

# Find potential duplicates (> 90% similar)
duplicates = PropertyDamage.FailureIntelligence.find_duplicates(failures)
for {f1, f2, score} <- duplicates do
  IO.puts("Seeds #{f1.seed} and #{f2.seed} are #{score * 100}% similar")
end

Fix Verification

When you believe a bug is fixed, verify the fix is robust:

result = PropertyDamage.FailureIntelligence.verify_fix(failure, MyModel,
  adapter: MyAdapter,
  adapter_config: %{base_url: "http://localhost:4000"},
  max_variations: 20  # Test 20 seed variations
)

case result.status do
  :verified ->
    IO.puts("Fix verified with #{result.confidence * 100}% confidence")

  :still_failing ->
    IO.puts("Original failure still reproduces!")

  :partially_fixed ->
    IO.puts("Fix incomplete. #{result.variations_failed} variations still fail")

  :flaky ->
    IO.puts("Intermittent failures detected. May be timing-related.")
end

# Format for display
IO.puts(PropertyDamage.FailureIntelligence.format_verification(result))

Verification Result

%{
  status: :verified | :still_failing | :partially_fixed | :flaky,
  original_seed: 12345,
  original_passes: true,
  variations_run: 20,
  variations_passed: 18,
  variations_failed: 2,
  failed_variations: [12346, 12400],
  confidence: 0.95,
  summary: "Fix verified! Original seed and all 18 variations pass."
}

Quick Checks

# Quick check if a seed still fails
if PropertyDamage.FailureIntelligence.still_fails?(12345, MyModel, MyAdapter) do
  IO.puts("Bug not fixed yet!")
end

# Verify multiple fixes at once
results = PropertyDamage.FailureIntelligence.verify_fixes(failures, MyModel,
  adapter: MyAdapter
)
for {failure, result} <- results do
  IO.puts("Seed #{failure.seed}: #{result.status}")
end

Example Workflow

# 1. Collect failures from test runs
failures = collect_failures_from_ci()

# 2. Analyze to find patterns
analysis = PropertyDamage.FailureIntelligence.analyze(failures)
IO.puts("Found #{length(analysis.clusters)} distinct failure patterns")

# 3. Work on the most common pattern first
if pattern = analysis.most_common_pattern do
  IO.puts("Most common: #{pattern.description}")
end

# 4. After fixing, verify the fix
{:ok, fixed_failure} = PropertyDamage.load_failure("failures/issue_123.pd")
result = PropertyDamage.FailureIntelligence.verify_fix(fixed_failure, MyModel,
  adapter: MyAdapter,
  max_variations: 50
)

if result.status == :verified do
  IO.puts("Fix confirmed! Safe to merge.")
  PropertyDamage.delete_failure("failures/issue_123.pd")
end

Automatic Regression Management

Automatically save failures to seed libraries and generate regression tests when bugs are found.

Basic Usage

Use the :regression option in PropertyDamage.run/1:

PropertyDamage.run(
  model: MyModel,
  adapter: MyAdapter,
  regression: [
    save_failures: "failures/",           # Save failure files
    seed_library: "seeds.json",           # Add to seed library
    generate_tests: "test/regressions/",  # Generate ExUnit tests
    tags: [:auto_detected],               # Tags for seed library
    dedup: true                           # Skip similar failures
  ]
)

When a failure is found, PropertyDamage will automatically:

1. Save the failure file to the specified directory
2. Add the seed to your seed library
3. Generate an ExUnit regression test

Deduplication

Avoid noise from multiple runs finding the same bug:

PropertyDamage.run(
  model: MyModel,
  adapter: MyAdapter,
  regression: [
    save_failures: "failures/",
    dedup: true,                 # Enable deduplication
    dedup_threshold: 0.90        # 90% similarity threshold
  ]
)

Using Handlers Directly

For more control, use handlers with :on_failure:

alias PropertyDamage.Regression

# Single handler
PropertyDamage.run(
  model: MyModel,
  adapter: MyAdapter,
  on_failure: Regression.save_failure("failures/")
)

# Compose multiple handlers
PropertyDamage.run(
  model: MyModel,
  adapter: MyAdapter,
  on_failure: Regression.compose([
    Regression.save_failure("failures/"),
    Regression.add_to_library("seeds.json", tags: [:critical]),
    fn report -> Logger.warning("Failure found: #{report.seed}") end
  ])
)

Batch Processing

Process multiple failures at once with deduplication:

failures = [failure1, failure2, failure3]

results = PropertyDamage.Regression.process_batch(failures,
  seed_library: "seeds.json",
  dedup: true,
  dedup_threshold: 0.90
)

summary = PropertyDamage.Regression.batch_summary(results)
IO.puts(PropertyDamage.Regression.format_batch_summary(summary))

Options

Differential Testing

Compare multiple implementations by running the same command sequences against them. Use cases include oracle testing, performance comparison, migration validation, and regression testing.

Basic Usage

# Oracle testing - compare against reference implementation
PropertyDamage.Differential.run(
  model: MyModel,
  targets: [
    {ReferenceAdapter, role: :reference},
    {SUTAdapter, name: "new-impl"}
  ],
  compare: :correctness,
  max_runs: 100
)

# Performance comparison
PropertyDamage.Differential.run(
  model: MyModel,
  targets: [
    {RedisAdapter, name: "redis-backend"},
    {PostgresAdapter, name: "postgres-backend"}
  ],
  compare: :performance
)

# Same adapter, different configurations (e.g., staging vs prod)
PropertyDamage.Differential.run(
  model: MyModel,
  targets: [
    {HTTPAdapter, role: :reference, opts: [base_url: "https://prod.example.com"]},
    {HTTPAdapter, name: "staging", opts: [base_url: "https://staging.example.com"]}
  ],
  compare: :correctness
)

Time-Separated Comparison

Save results now, compare later:

# Export baseline before deployment
PropertyDamage.Differential.run(
  model: MyModel,
  targets: [{ProdAdapter, name: "v2.3"}],
  compare: :performance,
  export_to: "baselines/v2.3.json",
  seed: 12345
)

# Compare against baseline after deployment
PropertyDamage.Differential.run(
  model: MyModel,
  targets: [{ProdAdapter, name: "v2.4"}],
  compare: :performance,
  baseline: "baselines/v2.3.json"
)

Equivalence Strategies

# Exact matching (default)
compare: :correctness, equivalence: :exact

# Structural - ignores IDs, timestamps, UUIDs
compare: :correctness, equivalence: :structural

# Custom comparison function
compare: :correctness, equivalence: fn ref, target ->
  ref.status == target.status && ref.amount == target.amount
end

See Differential Testing Guide for complete documentation.

Telemetry Dashboard

PropertyDamage emits telemetry events during test execution that can be used for real-time monitoring via a LiveView dashboard.

Setup

1. Add the Collector to your application supervisor:

# In your application.ex
def start(_type, _args) do
  children = [
    # ... your other children
    PropertyDamage.Telemetry.Collector
  ]

  opts = [strategy: :one_for_one, name: MyApp.Supervisor]
  Supervisor.start_link(children, opts)
end

1. Create a LiveView for the dashboard:

defmodule MyAppWeb.PropertyDamageDashboardLive do
  use MyAppWeb, :live_view

  alias PropertyDamage.Telemetry.{Collector, Dashboard}

  def mount(_params, _session, socket) do
    if connected?(socket) do
      Collector.subscribe()
    end

    state = Collector.get_state()

    {:ok,
     assign(socket,
       page_title: "PropertyDamage Dashboard",
       state: state,
       view_mode: :overview
     )}
  end

  def handle_info({:telemetry_update, _event_type, _data, state}, socket) do
    {:noreply, assign(socket, :state, state)}
  end

  def handle_event("reset", _params, socket) do
    Collector.reset()
    {:noreply, socket}
  end

  def handle_event("set_view_mode", %{"mode" => mode}, socket) do
    {:noreply, assign(socket, :view_mode, String.to_existing_atom(mode))}
  end

  def render(assigns) do
    Dashboard.render(assigns)
  end
end

1. Add a route:

# In your router.ex
live "/property-damage", PropertyDamageDashboardLive

Dashboard Views

Telemetry Events

PropertyDamage emits these telemetry events:

Custom Telemetry Handlers

You can attach custom handlers to these events:

:telemetry.attach(
  "my-metrics-handler",
  [:property_damage, :command, :stop],
  fn _event, measurements, metadata, _config ->
    # Record command execution time to your metrics system
    MyMetrics.histogram(
      "property_damage.command.duration",
      measurements.duration,
      tags: [command: metadata.command]
    )
  end,
  nil
)