mutare_ecto

A mutation-testing plugin for Ecto, built as a custom Mutare mutator.

Mutation testing checks how good your tests actually are: it makes small, deliberate changes to your code — a > becomes a >=, a where clause is dropped, a validate_required is removed — and reruns your suite. If the tests still pass, that mutation survived, and you've found a gap your assertions don't cover.

mutare_ecto aims that lens at the Ecto code you write — Repo calls, changeset pipelines, and the from/query DSL — so a survivor tells you something concrete: "no test would notice if this filter, this sort order, or this validation quietly changed."

Why a dedicated Ecto mutator

An Ecto where clause looks like Elixir, but it isn't — it's a fragment of SQL, and SQL runs under three-valued logic (NULL is neither true nor false). A general-purpose mutator that treats a < b or a > b as ordinary Elixir will "helpfully" conclude it's equivalent to a != b and skip the mutation. In SQL that's wrong: when a or b is NULL, the two differ — and that's exactly the untested edge you'd want flagged.

So mutare_ecto ships its own SQL-semantics mutation catalog and never borrows Mutare's Elixir-semantics mutators inside a query. Every mutation it emits is one a real SQL engine will run, and its equivalence reasoning is SQL's, not Elixir's.

Installation

Add both Mutare and this plugin to the app you want to test, in :dev/:test:

# mix.exs
defp deps do
  [
    {:mutare, "~> ..."},
    {:mutare_ecto, "~> ..."}
  ]
end

It must run as a dependency of the app under test (not against an external source path), so your Repo and schemas are loadable in the Mutare process — that's what lets use Ecto.Schema expand and the query macros resolve. External-source operation is unsupported: the plugin declares the Ecto surface it needs (Ecto.Schema/Ecto.Query) via Mutare.Ecto.required_modules/0, and Mutare checks it once at startup, aborting with a Mutare.EnvironmentError when a module is not loadable. Beyond that guard, unresolved target-app modules can still make routing incomplete or invalid.

Usage

Enable it in .mutare.exs, naming your Repo when Repo-call mutations are needed:

# .mutare.exs
[
  mutators: [
    :all,                            # Mutare's built-ins for ordinary Elixir
    {Mutare.Ecto, repo: MyApp.Repo}  # the Ecto surface
  ]
]

Then run Mutare as usual. Listing the entry both registers the plugin's query-DSL routing and enables its mutations; repo: (one module, or a list when the app has several) is optional for query and changeset mutations, and is what lets it recognise Repo.* calls regardless of how they're aliased or imported.

What it mutates

Every mutation is tagged with a family, so you can enable or report on them individually (see Configuration). They cover three surfaces:

Inside where / having conditions — delivered through Ecto's ^/dynamic injection so the query still compiles once and the active mutant is chosen at build time:

† Off by default (opt-in). A string, atom, or boolean literal mutant is the most likely to be a noisy survivor — a string/atom because its value space is large (an in-fragment string the broadest), a boolean because a direct boolean literal in a condition is rarely idiomatic. Enable them with families: :all or by naming them in an explicit list (see Configuration). The numeric arms (integer_literal/float_literal) are on by default. Whatever the selection, a literal at a structural position of a known Ecto DSL form — the fragment template, the interval unit of datetime_add/date_add/from_now/ago, the cast type of type/2, the name in field/2, as/1/parent_as/1, or selected_as — is never mutated (it shapes the SQL, so a mutant would just be a broken query, not a test signal).

Query shape — ordering, pagination, joins, aggregates, and the query terminals:

Repo writes and changesets — plain calls, no query DSL involved:

Both query syntaxes are covered — the from(u in User, where: …) keyword form (piped too: User |> from(as: :u, where: …)) and the composable pipe form (q |> where([u], …)) — as are direct, aliased, and import/use-bundled call styles. Schema definitions (schema/embedded_schema) are left untouched: a mutated field name is a broken schema, not an interesting mutant.

Configuration

Each {Mutare.Ecto, …} entry takes:

{Mutare.Ecto,
 repo: MyApp.Repo,                 # optional — identifies Repo.* calls; a module or a list
 families: :default,               # the default; or :all, a list, or {:default | :all, except: […]}
 dialects: [:postgres]}            # gate dialect-specific mutations (default: portable core)

• repo: — identify the Repo module for aggregate and write-call mutations — one module, or a list (repo: [MyApp.Repo, MyApp.ReplicaRepo]) when the app has several. Omit it when only query/changeset families are needed; Repo-call families then produce no mutations.

• families: — select the catalog. Every family above is independently toggleable; an unknown name fails loudly. Accepts:

  • :default (the unset default) — every family except the opt-in string_literal / atom_literal / boolean_literal arms (see the † note above);
  • :all — every family, including those opt-in arms;
  • an explicit list, e.g. [:comparison, :null_predicate] (name the opt-in arms here to add them);
  • {:default | :all, except: [families]} — a base set minus exclusions; the easy way to disable a default-on arm, e.g. {:default, except: [:integer_literal]}.

  Mutare.Ecto.families/0 returns the full set and Mutare.Ecto.default_families/0 the default subset.

• dialects: — enable mutations that aren't portable across all adapters. The default [] is the portable core (safe on SQLite, Postgres, MySQL alike). :postgres adds like↔ilike; :postgres/:mysql add the LEFT↔RIGHT join swap (SQLite has no RIGHT JOIN).

• as: — rename the family in the report. List the plugin more than once with different families:/as: to report a sub-family under its own name, or different repo:/as: to report each repo's mutants separately (a single repo: [A, B] entry reports both as ecto).

Unknown plugin option names raise an ArgumentError; as: is handled and removed by Mutare before the remaining options reach this plugin.

Equivalence reporting

Some survivors are honest signal rather than a flat "your test is missing." A surviving >=↔>, and↔or, or is_nil mutant may be legitimately unkillable without the right fixture — a boundary row, a NULL, an orphan — not an oversight. The plugin marks these families and gives each a note naming the specific data a kill needs, so the report reads:

… SURVIVED  — kill may require a row whose value sits exactly on the bound — …
… SURVIVED  — kill may require NULL rows in the ordered column — …
… SURVIVED  — kill may require an orphan row — …

The reasons are distinct — a boundary value, NULL exclusion (==/!=), three-valued and/or, an arithmetic identity operand (0 for +/-, ±1 for *//), a NULL row for the coalesce default, a near-now row for the ago/from_now flip, NULL ordering, join cardinality — so the notes are too, rather than one catch-all string.

Mutare.Ecto.equivalence_sensitive_families/0 returns that set, and with as: you can group them under their own report name to separate "needs a boundary fixture" from "needs any test at all":

# .mutare.exs
[
  mutators: [
    :all,
    {Mutare.Ecto, repo: MyApp.Repo, dialects: [:postgres],
     families: Mutare.Ecto.equivalence_sensitive_families(), as: :ecto_boundary_null},
    {Mutare.Ecto, repo: MyApp.Repo, dialects: [:postgres]}
  ]
]

Because the catalog is SQL-native, it also never emits the always-equivalent mutations (like x * 1) that would otherwise inflate your denominator and dilute the score.