Tuist for Elixir

This package connects an Elixir project to Tuist: it reports your test runs and builds to the dashboard, flags flaky tests, and splits a test suite across machines. It adds mix tuist.login, mix tuist.test, mix tuist.compile and mix tuist.test.build, and writes to the same Tuist server the command line tool and the Gradle plugin use.

The guides live in the Tuist documentation: start with Elixir project, and see Install the Hex package for every option.

Configure the server and project

You can share the Tuist server address and the project handle with your team in mix.exs:

def project do
  [
    app: :my_app,
    tuist: [
      url: "https://tuist.example.com",
      project: "your-account/your-project"
    ]
  ]
end

TUIST_URL and TUIST_PROJECT override those values, as they do in the Tuist command line tool. --url and --project are also accepted when the env overrides are unset. The default URL is https://tuist.dev.

Log in

mix tuist.login
mix tuist.login --email person@example.com --password secret
mix tuist.login --url https://tuist.example.com

Without credentials, the task opens a browser and waits for authorization. If you provide only --email or --password, it prompts for the missing value. On a continuous integration provider, it exchanges an OpenID Connect identity token from GitHub Actions, CircleCI, or Bitrise, matching the Tuist command line flow. On those providers a login step is optional: when no other credentials are available, the reporting tasks make the same exchange the first time they need a token. On GitHub Actions the job needs the id-token: write permission.

Credentials are saved under the Tuist configuration directory. The task writes them atomically with access restricted to the current user. Refreshes take a lock at the Tuist command line tool's lock path and reread the file before exchanging the refresh token, so parallel processes do not refresh the same token twice.

Run tests with analytics

mix tuist.test
mix tuist.test --project your-account/your-project
mix tuist.test --trace        # forwards --trace to mix test

mix tuist.test installs an ExUnit formatter alongside the default one and runs mix test. When the suite finishes, it POSTs a summary of the run, each module, each suite (from describe blocks), and each test case, including failure messages and source locations, to the Tuist server. Test timings are converted from microseconds to milliseconds. Every argument other than the task's own (--url, --project, --retries) is forwarded to mix test verbatim.

Elixir, OTP, Mix environment, the current git ref, and any recognised continuous integration provider metadata travel with the payload so a dashboard can group runs by branch, commit, and CI run.

Analytics submission never changes the exit code of mix test. If a request fails, set TUIST_DEBUG=1 to print the reason to standard error. Authentication uses the TUIST_TOKEN environment variable if set, then the credentials from mix tuist.login, then an OpenID Connect exchange on a supported continuous integration provider.

Keep using mix test and mix compile

The Tuist tasks take the same command line as the tasks they wrap, so you can alias them once and nobody, person or coding agent, has to learn a different command:

def project do
  [
    app: :my_app,
    aliases: [test: "tuist.test", compile: "tuist.compile"],
    tuist: [project: "your-account/your-project"]
  ]
end

mix test test/my_test.exs:12 --trace then runs and reports exactly as mix tuist.test would. The alias also gets the environment right: Mix runs mix test in the test environment, whereas a task typed as mix tuist.test starts in dev and has to start over in a new process. If you call the tasks by name, set MIX_ENV=test or add preferred_envs: ["tuist.test": :test, "tuist.test.build": :test] to def cli in your mix.exs. Aliasing compile covers every compile, including the ones mix test or mix phx.server trigger; a compile that finds nothing to do is not reported.

Detect flaky tests with retries

ExUnit runs each test once, so a test that fails intermittently looks the same as a broken one. Turn retries on and the tests that failed are run again with mix test --failed, in a fresh process, up to the number of times you set:

tuist: [project: "your-account/your-project", test_retries: 2]

--retries 2 and TUIST_TEST_RETRIES=2 do the same for one invocation. A test that passes on a retry is reported with every attempt and shows up as flaky in the dashboard, and the command succeeds if every failed test eventually passes. Retries are off by default, in which case the exit code is always that of mix test.

A run that --max-failures cut short is never retried: the tests it did not reach are unknown, so the run is reported as it is and stays failed. Retries are also skipped with --warnings-as-errors, which mix test would no longer enforce, and from an umbrella's root: run the tests inside one of its applications to retry them.

Without retries, Tuist still flags a failure as flaky when the same test has also passed on the same commit in continuous integration, for example after someone reruns a failed job.

Build once, test on several machines

Sharding splits your tests across machines, balanced by how long each test module took in earlier runs, and reports them as one test run. Only one machine compiles; the others download its build.

On one machine, once per pipeline:

mix tuist.test.build --shard-max 4

This compiles the project for testing, asks Tuist to split the test files into at most four shards, and uploads the _build/test directory. A test file runs as a whole and is planned under the first module it defines; a file that cannot be parsed stops the plan, since the suite could not run either. On GitHub Actions it writes matrix={"shard":[0,1,2,3]} to the job outputs; elsewhere it writes the plan to .tuist-shard-matrix.json.

Then on each shard, after checking out the code and running mix deps.get:

TUIST_SHARD_INDEX=0 mix tuist.test

or plain mix test if you aliased it. The task downloads the build, runs only that shard's test files without compiling, and reports the results. Shards need the same Elixir and Erlang versions as the machine that built.

The shards find their plan by a reference derived from the pipeline run on GitHub Actions, GitLab, CircleCI and Buildkite. Anywhere else, set TUIST_SHARD_REFERENCE to the same value on every machine. Pass --no-upload to mix tuist.test.build if you would rather have each shard compile.

Test paths given on a shard's command line narrow what that shard runs. Run both commands inside an application: sharding from an umbrella's root is not supported yet.

Report code coverage

Pass --cover and the run's line coverage is reported with it:

mix tuist.test --cover

Tuist reads the counters Erlang's cover collected once the suite finishes, so your coverage tool keeps working as before: mix test's HTML report and threshold, or excoveralls. Each covered file is reported with the OTP application it belongs to and the Git blob it had, and files under your test paths count as test code, which no figure includes. A run that picks its tests (--only, --exclude, --failed, --stale, or test files on the command line) is reported as partial and stays out of the branch's coverage trend. Shards each report their share and Tuist merges them.

With coverage, the run also reports its Git history: the merge base with the branch a pull request merges into, the files it changed, and the commits Tuist does not have yet. A shallow clone is deepened, within a time budget, until the merge base is found. Once every job of the pipeline that measures coverage has finished, tell Tuist the commit's coverage is complete:

mix tuist.coverage.complete

Until then the commit is shown as in progress and stays out of the trend.

Tag runs with custom metadata

Every submitted run can carry customer-supplied tags (labels for filtering) and values (key/value pairs). Sources merge with the environment last:

def project do
  [
    app: :my_app,
    tuist: [
      url: "https://tuist.example.com",
      project: "your-account/your-project",
      tags: ["nightly"],
      values: %{"team" => "platform"}
    ]
  ]
end

TUIST_TAGS (comma-separated) and TUIST_VALUES (key=value pairs, comma-separated) override the mix.exs config. The Tuist dashboards use those labels to group and filter runs across ecosystems.

Compile with analytics

mix tuist.compile
mix tuist.compile --force

mix tuist.compile wraps mix compile, attaches an after-compiler hook to the Elixir and .app compilers, and submits a build record to your Tuist dashboard when the compile finishes. The record carries duration and status, each warning and error as a distinct diagnostic (file, module, message, line, and column), a per-file profile (when each file started compiling, how long it compiled, the project files it depends on and how strongly, and how long it sat paused waiting on other files) and the other work the compiler reports (type checking each module, writing modules to disk, the other Mix compilers), which together feed the build's timeline, machine samples taken once a second (processor, memory, network and disk throughput), plus the same environment metadata (Elixir and Open Telecom Platform versions, Mix environment, git ref, and continuous integration provider) that mix tuist.test submits. Every argument other than --url and --project is forwarded to mix compile. Submission failures never change the exit code of mix compile.

Development

Requires Elixir 1.18 or later. From tuist_ex/:

mix deps.get
mix format --check-formatted
mix compile --warnings-as-errors
mix test --warnings-as-errors
mix hex.build
mix docs --warnings-as-errors

Quokka is configured as a formatter plugin. mix format applies its fixes; mix format --check-formatted enforces them in automated checks. Mimic is available for test mocks. Register modules with Mimic.copy/1 in test/test_helper.exs as tests are added. The package currently has focused authentication and locking tests.

Automated checks and releases

The check workflow runs compilation, documentation, tests, formatting, and package assembly as separate jobs. Compilation checks Elixir 1.18 and the repository's current Elixir version; the other jobs use the current version.

The tuist-ex release component uses tuist-ex@ tags and scoped commits such as feat(tuist-ex): add build instrumentation. On main, the release workflow checks for a version bump, runs the same validation, publishes the package and documentation to Hex, and creates a GitHub release. Publication is serialized and never cancelled mid-release. If package publication succeeds but the GitHub release fails, a retry verifies the published package contents and resumes documentation and release creation without replacing the package. Manual releases are restricted to main.

The release workflow sets the package version from the shared release checker; the version in mix.exs is the development baseline. Release notes are generated from scoped commits using cliff.toml.

Publication reuses the repository's HEX_API_KEY secret, which also publishes Noora. Its Hex account must be the intended owner of tuist_ex and the key must have package publishing permission. The first publication requires the package name to be available. TUIST_RELEASE_GITHUB_TOKEN is used for GitHub releases when available, with the workflow token as the fallback.