Provenance

Universal artifact lineage tracking for Elixir. Tracks what code produced what data, across modules, queries, requests, and jobs.

Every artifact gets a deterministic kind:identifier ID. An ETS-backed graph store records producer/consumer relationships. A compile tracer automatically discovers modules and classifies them into architectural layers.

Designed for use with Sentinel but works standalone in any Elixir project.

Installation

Add to your mix.exs:

def deps do
  [
    {:provenance, "~> 0.1"},
    {:ecto, "~> 3.10", optional: true},
    {:plug, "~> 1.15", optional: true}
  ]
end

Quick Start

Registering Modules

Modules can be registered manually or discovered automatically via the compile tracer.

Provenance.register_module(MyApp.Orders,
  source_file: "lib/my_app/orders.ex",
  layer: :context,
  dependencies: ["mod:MyApp.Repo"]
)

Provenance.register_module(MyApp.Orders.Order,
  source_file: "lib/my_app/orders/order.ex",
  layer: :schema
)

Provenance.lookup("mod:MyApp.Orders")
# => {:ok, %{id: "mod:MyApp.Orders", kind: :mod, layer: :context, ...}}

Recording Provenance

Record that an artifact was produced by another:

Provenance.record("rec:orders:42", origin: "fn:MyApp.Orders.create/1")
Provenance.record("rec:order_items:99", origin: "fn:MyApp.Orders.add_item/2")

Querying Related Artifacts

Provenance.related("fn:MyApp.Orders.create/1")
# => ["rec:orders:42"]

Provenance.related("mod:MyApp.Orders")
# => ["mod:MyApp.Repo"]

Provenance.module_graph()
# => %{"mod:MyApp.Orders" => ["mod:MyApp.Repo"], ...}

Process Context

Attach provenance context to the current process. Useful for tracing requests, jobs, or any unit of work:

Provenance.put_context(request_id: "req:F1a2b3c4d5", user_id: "usr:42")
Provenance.get_context()
# => %{request_id: "req:F1a2b3c4d5", user_id: "usr:42"}

# Context merges -- existing keys are preserved, new ones added
Provenance.put_context(trace_id: "trc:abc")
Provenance.get_context()
# => %{request_id: "req:F1a2b3c4d5", user_id: "usr:42", trace_id: "trc:abc"}

Artifact ID Format

All IDs follow the kind:identifier scheme:

ID helpers:

Provenance.module_id(MyApp.Orders)       # => "mod:MyApp.Orders"
Provenance.function_id(MyApp.Orders, :get_order, 1)  # => "fn:MyApp.Orders.get_order/1"
Provenance.table_id("orders")            # => "tbl:orders"
Provenance.record_id("orders", 42)       # => "rec:orders:42"

Compile Tracer

Automatically registers modules and records inter-module dependencies at compile time.

Enable in mix.exs:

def project do
  [
    # ...
    compilers: Mix.compilers(),
    tracers: [Provenance.CompileTracer]
  ]
end

Or in config:

config :elixir, :tracers, [Provenance.CompileTracer]

The tracer classifies each module into an architectural layer based on file path and module characteristics:

Architecture

+---------------------+
|   Your Application  |
+---------------------+
         |
    register / record / query
         |
+---------------------+     +------------------------+
|    Provenance API    |---->|  Provenance.Store      |
|  (lib/provenance.ex) |     |  (ETS GenServer)       |
+---------------------+     |                        |
                             |  :provenance_artifacts |
+---------------------+     |  :provenance_edges     |
| CompileTracer        |---->|  :provenance_modules   |
| (compile-time hook)  |     +------------------------+
+---------------------+

Process Dictionary
+---------------------+
| put_context/1        |  Per-process provenance context
| get_context/0        |  (:provenance_context key)
+---------------------+

• Provenance -- Public API. Delegates to Provenance.Store for persistence and provides ID builder functions.
• Provenance.Store -- GenServer owning three ETS tables: artifacts (set), edges (duplicate_bag, bidirectional), and modules (set).
• Provenance.CompileTracer -- Elixir compiler tracer. On each compiled module, registers it with layer classification and records remote function call edges.
• Process context -- Stored in the process dictionary. Designed for inclusion in provenance recordings during request/job execution.

Ecto Integration (Planned)

Ecto is an optional dependency. Current support is limited to detecting Ecto schemas in the compile tracer for layer classification. Future versions will add:

• Automatic query-level provenance via Ecto telemetry
• Schema field to artifact ID mapping
• Migration tracking

API Reference

Core

ID Builders

Process Context

License

MIT -- see LICENSE.