ResponseSnapshot

ResponseSnapshot is a testing tool for Elixir that captures the output of responses and ensures that they do not change in between test runs. The output is saved to disk, meant to be checked into source control. This output can be used by frontend and other tests to ensure proper integration between frontend and backend code, or to ensure that endpoint responses do not change over time.

Read more in this introduction blog post.

Installation

Add the following to your dependency list:

def deps() do
  [
    ...
    {:response_snapshot, "~> 1.0", only: :test}
  ]
end

Usage

The most basic is a simple call to store_and_compare! as such:

  response_json
    |> ResponseSnapshot.store_and_compare!(path: "test/location/i/want/output.json")

This will cause the output to be written to disk the first time, and then compared using exact match in all future tests.

The response will be stored as JSON, always, which brings along some niceities like serialization and deserialization. This means that your JSON responses will work out of the box and give you benefits like ignored keys. However, string responses are also valid JSON and so you can use this library to capture the output of strings. The string comparison is exact match only.

Options

Application Config

In addition to being able to set configuration for each call, certain configuration can be achieved using the Application module. The following options are available:

Option values passed into the store_and_compare! function are used over the Application config values.

Comparison Modes

The store_and_compare! interface has 2 different modes, exact and keys. The :exact mode is default and requires both key and value of the comparison to match the stored snapshot. The :keys mode requires only the keys of the comparison to match the stored snapshot. This can be useful in testing that the shape of an endpoint doesn’t change over time, without worrying about the test input.

Ignored Keys

It is possible to ignore keys that will change between test runs. This is most common for dynamic fields such as ids, timestamps, etc. Ignored keys can be done via an exact string comparison, or a wildcard-like implementation.

  response_json
    |> ResponseSnapshot.store_and_compare!(path: path, ignored_keys: ["exact.example", {"partial", :any_nesting}])

The exact.example key requires that the shape of the JSON is exact -> key. The partial key allows for matches such as “partial”, “partial.nested”, or “nested.partial”.

Ignored keys will only ignore value changes, not key additions or removals. This is due to an addition or removal affecting the output shape, which would go against the goals of this library.

TODO