Fathom

A program database for Elixir codebases.

Fathom compiles your project under a compiler tracer and writes what the compiler resolved into a SQLite file: every definition, every call, module dependencies and their kind, Ecto schemas, Phoenix routes, behaviours, protocols, and the places where dispatch is dynamic enough that the call graph stops being complete.

Then an agent can ask the questions it actually has, in SQL:

WITH RECURSIVE up(mfa) AS (
  SELECT caller FROM calls WHERE callee = 'MyApp.Repo.delete_all/1'
  UNION
  SELECT c.caller FROM calls c JOIN up ON c.callee = up.mfa
)
SELECT f.mfa, f.file, f.line
FROM functions f JOIN up USING (mfa)
WHERE f.kind = 'def';

Which public functions can delete everything? That question has no grep, and no LSP request either. "Find references" gives you one hop, and you walk the rest by hand.

Why not an LSP

This follows a suggestion from José Valim. The Language Server Protocol was built for editors, so its operations are addressed by file, line and column, coordinates an agent does not track and has to go and find before it can ask anything. It also answers one hop at a time, because that is how a human reads code: jump to definition, look around, jump again.

An agent has neither constraint. It will happily write a query, and it can compose questions no IDE would ever put in a menu. The information a language server already has is better exposed as a database than as a protocol.

Install

Add Fathom as a development dependency:

def deps do
  [{:fathom, "~> 0.1", only: [:dev], runtime: false}]
end

mix deps.get
mix fathom.install   # writes the schema and example queries into AGENTS.md
mix fathom.build     # compiles under the tracer, writes .fathom/program.db

It has to be a dependency rather than an archive or an escript. The compiler loads the tracer from the code path while compiling your files, so it must already be built, and built by the same toolchain: BEAM files from OTP 28 will not load on OTP 27.

If you would rather not touch mix.exs, you can put a build of Fathom on the code path from outside, as long as it was compiled with the same Elixir and OTP as the target project:

ERL_LIBS=/path/to/fathom/_build/dev/lib mix fathom.build

Use

mix fathom.build                     # rebuild after changing code
mix fathom.query "SELECT ..."        # query without needing sqlite3
sqlite3 .fathom/program.db "SELECT ..."

mix fathom.install writes the schema and a set of worked queries into AGENTS.md, between markers so re-running it updates in place. That file is what makes an agent use the database instead of falling back to grep.

What is in it

modules          functions        calls            module_deps
struct_uses      alias_refs       use_sites        compile_envs
behaviours       callbacks        types            impls
schemas          schema_fields    schema_assocs    routes
dynamic_sites    meta

mfa columns are strings like "MyApp.Accounts.create_user/2" and join everything together. The schema is denormalised on purpose: every row that refers to a function also carries its module, so most questions need no join.

Facts come from three places. A compiler tracer records references after macro expansion, which is the only way to see what use, import and framework macros actually generated. While each module is still open, Module.definitions_in/1 yields every definition including private ones, which cannot be recovered from the BEAM file afterwards. After compilation, reflection fills in docs, specs, behaviours, and whatever Ecto and Phoenix will tell you about themselves.

Two things worth knowing

Macro-generated definitions are marked. Running this over Credo, 3,016 of its 4,701 function definitions came from a macro rather than from someone typing them. Credo.Service.SourceFileAST is a four-line module whose entire body is use Credo.Service.ETSTableHelper, and use Ecto.Repo contributes about seventy public functions to any project that has one. They are flagged with generated = 1, because otherwise they swamp any question about the code a person actually wrote.

Gaps in the call graph are recorded, not hidden. apply/3, a module held in a variable or read from config, a protocol dispatch. None of these produce an edge, and some produce no trace event at all. Fathom finds them by walking the expanded AST and records them in dynamic_sites. An agent asking "who calls this?" can also ask "does this module dispatch dynamically?" and know how far to trust the answer. That is more useful than a graph that claims to be total.

Beyond that: the database covers your project and not its dependencies, and it is built from one MIX_ENV, so a dev build does not see test code.

Does it beat grep?

bench/compare.exs runs real navigation questions against both, on the same checkout, measuring round trips, output size and wall time. The questions live in a separate file because they have to name real modules; the committed set is written against Credo so the numbers below can be reproduced.

git clone https://github.com/rrrene/credo && cd credo
mix deps.get && mix fathom.build
elixir path/to/fathom/bench/compare.exs . .fathom/program.db

Of nine questions, grep answered one correctly, three approximately, four wrongly, and could not attempt one at all. Across the eight it could attempt, Fathom needed 8 tool calls against 13 and 86ms against 357ms.

Fathom returns more output, not less, and that is the honest shape of the result: it answers completely where grep returns a partial match list. Ask which public functions reach Credo.Code.Block.do_block_for!/1 and grep finds the nine direct call sites, where the real answer is sixty-eight.

Grep wins on a repository you have just cloned and will ask two questions about, because it needs no index. The build is a full forced compile: 2.5 seconds for Credo's 266 files, producing 79,057 facts.

Pass your own scenario file as a third argument to run it elsewhere. Scenarios marked requires: are skipped when the project has no such facts, so the Phoenix and Ecto questions drop out rather than counting as a win over an empty table.

Is it correct?

bench/verify_against_xref.sh checks Fathom's references against mix xref callers. That is a genuine oracle rather than a tautology: xref reads the same compiler output through Mix's own manifests, not through a tracer. On every module tried, Fathom found a strict superset of what xref found, the extra being self-references, which xref excludes by design.

bench/verify_against_xref.sh /path/to/repo /path/to/program.db MyApp.Repo

Incremental builds

Not yet. mix fathom.build is a full rebuild every time, which on a mid-sized application costs under ten seconds.

The schema is already shaped for it: every table can be deleted by module, which is the unit the compiler recompiles. An incremental build drops --force, collects the modules that produced definitions this time round, deletes their rows across every table, inserts the new facts, and purges any module with no BEAM file left on disk. The compiler already propagates compile-time dependencies, so it decides what to re-trace. The reason it is not here yet is that the correctness bugs all live in that path, and a full rebuild that is certainly right beats an incremental one that is subtly stale.

License

MIT.