PropWise

An AST-based analyzer for identifying property-based testing candidates in Elixir codebases.

Overview

PropWise analyzes your Elixir code to find functions that would benefit from property-based testing. It examines the Abstract Syntax Tree (AST) of your code to:

Features

Purity Analysis

Detects side effects by analyzing function calls:

Pattern Detection

Identifies functions with characteristics ideal for property testing:

Inverse Pair Detection

Finds function pairs that are inverses of each other:

Concrete Test Generation

Generates ready-to-use property-based test code:

Installation

As a Library

Add propwise to your list of dependencies in mix.exs:

def deps do
  [
    {:propwise, "~> 0.2"}
  ]
end

As a Command-Line Tool

Option 1: escript (Recommended for standalone use)

Build and install the standalone executable:

cd propwise
mix deps.get
mix escript.build

# Copy to a directory in your PATH
sudo cp propwise /usr/local/bin/
# Or just use it directly
./propwise

The escript bundles all dependencies and works without Mix or any additional setup.

Option 2: Mix archive

Install globally from Hex as a Mix archive:

mix archive.install hex propwise

This makes the mix propwise task available in any project. Note: This requires jason to be available in your Mix environment.

To uninstall:

mix archive.uninstall propwise

Option 3: As a dependency

When added as a project dependency, PropWise provides a Mix task:

mix propwise

Usage

Command Line

Using escript

# Analyze current project
./propwise .

# Analyze with custom minimum score
./propwise --min-score 5 ./my_project

# Output as JSON
./propwise --format json ./my_project

# Use PropEr instead of stream_data
./propwise --library proper ./my_project

# Show help
./propwise --help

Using Mix task

# Analyze current project
mix propwise

# Analyze with custom minimum score
mix propwise --min-score 5

# Output as JSON
mix propwise --format json

# Use PropEr instead of stream_data
mix propwise --library proper

# Analyze another project
mix propwise ../other_project

# Show help
mix propwise --help

As a Library

# Analyze a project
result = PropWise.analyze("./my_project")

# Analyze with custom options
result = PropWise.analyze("./my_project", min_score: 5, library: :proper)

# Print the report
PropWise.print_report(result)

# Print as JSON
PropWise.print_report(result, format: :json)

Example Output

================================================================================
PropWise Analysis Report
================================================================================

Summary:
  Total functions analyzed: 143
  Property test candidates: 24
  Candidates dropped (below threshold): 12
  Coverage: 16.8%

--------------------------------------------------------------------------------
Inverse Function Pairs Detected:
--------------------------------------------------------------------------------

  MyApp.Encoder.encode/1 <-> decode/1
  Suggestion: Test round-trip property: decode(encode(x)) == x

--------------------------------------------------------------------------------
Top Candidates (sorted by score):
--------------------------------------------------------------------------------

MyApp.Parser.parse_json/1
  Score: 6
  Location: lib/my_app/parser.ex:42
  Type: public
  Patterns:
    - Parser: Parser function
  Testing suggestions:
    - property "parse returns expected structure" do
        check all input <- string(:alphanumeric) do
          case Parser.parse_json(input) do
            {:ok, result} ->
              # TODO: Add structural assertions for parsed output.
              assert result != nil
            {:error, _} -> true
          end
        end
      end

MyApp.List.merge_sorted/2
  Score: 8
  Location: lib/my_app/list.ex:15
  Type: public
  Patterns:
    - Collection Operation: Uses Enum collection operations
    - Algebraic Structure: Potentially algebraic operation
  Testing suggestions:
    - property "associativity" do
        check all a <- term(), b <- term(), c <- term() do
          assert List.merge_sorted(List.merge_sorted(a, b), c) ==
                 List.merge_sorted(a, List.merge_sorted(b, c))
        end
      end

Scoring System

Functions are scored based on multiple factors:

Default minimum score is 4, but this can be adjusted based on your needs.

For detailed information about all detection criteria and scoring rules, see Scoring.

Configuration

You can customize PropWise's behavior by creating a .propwise.exs file in your project root.

Example Configuration

# .propwise.exs
%{
  # Directories to analyze (relative to project root)
  # Default: ["lib"]
  analyze_paths: ["lib"],

  # Property-based testing library to use for suggestions
  # Options: :stream_data (default) or :proper
  library: :stream_data

  # You can analyze multiple directories:
  # analyze_paths: ["lib", "src", "apps/my_app/lib"]
}

Configuration Options

If no .propwise.exs file is present, PropWise will use the defaults.

Options

CLI Options

Note: CLI options override configuration file settings.

Library Options

How It Works

  1. Parse: Recursively finds all .ex files in configured directories (default: lib)
  2. Extract: Parses each file's AST and extracts function definitions
  3. Analyze Purity: Walks the AST to detect side effects
  4. Detect Patterns: Looks for common patterns in function structure and naming
  5. Score: Calculates a testability score for each function
  6. Find Pairs: Identifies inverse function pairs across the codebase
  7. Generate Suggestions: Creates concrete property-based test examples using your chosen library
  8. Report: Presents findings with ready-to-use test code

Limitations

Security Note

PropWise loads configuration from .propwise.exs files using Code.eval_file/1, which executes arbitrary Elixir code. Only analyze projects you trust.

Contributing

Contributions are welcome! Areas for improvement:

License

MIT