ESpec

ESpec is a BDD test framework for Elixir.

It is NOT a wrapper around ExUnit but a completely new test framework written from scratch.

ESpec is inspired by RSpec and the main idea is to be close to its perfect DSL.

Warning! Since 0.8.0 '__' is replaced by 'shared'.

Features

• Test organization with describe, context, it, and etc blocks.
• Familiar matchers: eq, be_close_to, raise_exception, etc.
• Possibility to add custom matchers.
• RSpec expectation syntax:
  • With expect helper: expect(smth1).to eq(smth2) or is_expected.to eq(smth) when subject is defined;
  • With old-style should: smth1 |> should eq smth2 or should eq smth when subject is defined.
• before and finally blocks (like RSpec before and after).
• let, let! and subject.
• Shared examples.
• Async examples.
• Mocks with Meck.
• Doc specs.
• HTML and JSON outputs.
• Etc and etc.

Contents

• Installation
• Run specs
• Context blocks
• Examples
• Filters
• 'before' and 'finally'
• 'shared'
• 'let' and 'subject'
• Shared examples
• Async examples
• Matchers
• Custom matchers
• Mocks
• Doc specs
• Configuration and options
• Contributing

Installation

Add espec to dependencies in the mix.exs file:

def deps do
  ...
  {:espec, "~> 0.8.0", only: :test},
  #{:espec, github: "antonmi/espec", only: :test}, to get the latest version
  ...
end

mix deps.get

Then run:

MIX_ENV=test mix espec.init

The task creates spec/spec_helper.exs and spec/example_spec.exs.

Set preferred_cli_env for espec in the mix.exs file:

def project do
  ...
  preferred_cli_env: [espec: :test],
  ...
end

Or run with MIX_ENV=test:

MIX_ENV=test mix espec

Place your _spec.exs files into spec folder. use ESpec in the 'spec module'.

defmodule SomeSpec do
  use ESpec
  it do: expect(1+1).to eq(2)
  it do: (1..3) |> should have 2
end

Run specs

mix espec

Run specific spec:

mix espec spec/some_spec.exs:25

Read the help:

MIX_ENV=test mix help espec

Context blocks

There are three macros with the same functionality: context, describe, and example_group.

Context can have description and options.

defmodule SomeSpec do
  use ESpec

  example_group do
    context "Some context" do
      it do: expect("abc").to match(~r/b/)
    end

    describe "Some another context with opts", focus: true do
      it do: 5 |> should be_between(4,6)
    end
  end
end

Available options are:

• skip: true or skip: "Reason" - skips examples in the context;
• focus: true - sets focus to run with --focus option.

There are also xcontext, xdescribe, xexample_group macros to skip example groups. And fcontext, fdescribe, fexample_group for focused groups.

'spec' module is also a context with module name as description. One can add options for this context after use ESpec:

defmodule SomeSpec do
  use ESpec, skip: "Skip all examples in the module"
  ...
end

Examples

example, it, and specify macros define the 'spec example'.

defmodule SomeSpec do

  example do: expect([1,2,3]).to have_max(3)

  it "Test with description" do
    4.2 |> should be_close_to(4, 0.5)
  end

  specify "Test with options", [pending: true], do: "pending"
end

You can use skip, pending or focus options to control evaluation. There are also macros:

• xit, xexample, xspecify - to skip;
• fit, fexample, fspecify, focus - to focus;
• pending/1, example/1, it/1, specify/1 - for pending examples.

defmodule SomeSpec do
  use ESpec

  xit "skip", do: "skipped"
  focus "Focused", do: "Focused example"

  it "pending example"
  pending "it is also pending example"
end

##Filters The are --only, --exclude and --string command line options.

One can tag example or context and then use --only or --exclude option to run (or exclude) tests with specific tag.

defmodule SomeSpec do
  use ESpec

  context "context with tag", context_tag: :some_tag do
    it do: "some example"
    it "example with tag", example_tag: true do
     "another example"
    end
  end
end

mix espec spec/some_spec.exs --only context_tag:some_tag --exclude example_tag

This runs only one test "some example"

You can also filter examples by --string option which filter examples which contain given string it their nested description.

mix espec spec/some_spec.exs --string 'context with tag'

before and finally

before blocks are evaluated before the example and finally runs after the example.

The blocks can return {:ok, key: value, ...}, so the keyword list will be saved in the dictionary and can be accessed in other before blocks, in the example, and in finally blocks through 'shared`:

defmodule SomeSpec do
  use ESpec

  before do: {:ok, a: 1}

  context "Context" do
    before do: {:ok, b: shared[:a] + 1}
    finally do: "#{shared[:b]} == 2"

    it do: shared.a |> should eq 1
    it do: shared.b |> should eq 2

    finally do: "This finally will not be run. Define 'finally' before the example"
  end
end  

Note, that finally blocks must be defined before the example. You can configure 'global' before and finally in spec_helper.exs:

ESpec.start

ESpec.configure fn(config) ->
  config.before fn -> {:ok, answer: 42} end  #can assign values in dictionary
  config.finally fn(shared) -> shared.answer  end     #can access assigns
end

These functions will be called before and after each example which ESpec runs.

'shared'

shared is used to share data between spec blocks. You can access data by shared.some_key or shared[:some_key]. shared.some_key will raise exception if the key 'some_key' does not exist, while shared[:some_key] will return nil.

The shared variable appears in your before, finally, in config.before and config.finally, in let and example blocks.

before and finally blocks (including 'global') can modify the dictionay when return {:ok, key: value}. The example bellow illustrate the life-cycle of shared:

spec_helper.exs

ESpec.start

ESpec.configure fn(config) ->
  config.before fn -> {:ok, answer: 42} end         # shared == %{anwser: 42}
  config.finally fn(shared) -> IO.puts shared.answer  end    # it will print 46   
end

some_spec.exs:

defmodule SomeSpec do
  use ESpec

  before do: {:ok, answer: shared.answer + 1}          # shared == %{anwser: 43}       
  finally do: {:ok, answer: shared.answer + 1}             # shared == %{anwser: 46}

  context do
    before do: {:ok, answer: shared.answer + 1}        # shared == %{anwser: 43}
    finally do: {:ok, answer: shared.answer + 1}           # shared == %{anwser: 45}
    it do: shared.answer |> should eq 44
  end
end

So, 'config.finally' will print 46. Pay attention to how finally blocks are defined and evaluated.

let and subject

let and let! have the same behaviour as in RSpec. Both defines memoizable functions in 'spec module'. The value will be cached across multiple calls in the same example but not across examples. let is not evaluated until the first time the function it defines is invoked. Use let! to force the invocation before each example.

The shared is available in 'lets' but neither let nor let! can modify the dictionary.

defmodule SomeSpec do
  use ESpec

  before do: {:ok, a: 1}
  let! :a, do: shared.a
  let :b, do: shared.a + 1

  it do: expect(a).to eq(1)
  it do: expect(b).to eq(2)
end  

subject and subject! are just aliases for let :subject, do: smth and let! :subject, do: smth. You can use is_expected macro (or a simple should expression) when subject is defined.

defmodule SomeSpec do
  use ESpec

  subject(1+1)
  it do: is_expected.to eq(2)
  it do: should eq 2

  context "with block" do
    subject do: 2+2
    it do: is_expected.to_not eq(2)
    it do: should_not eq 2
  end
end

Shared Examples

One can reuse the examples defined in spec module.

defmodule SharedSpec do
  use ESpec, shared: true

  subject shared.hello
  it do: should eq("world!")
end

shared: true marks examples in the module as shared, so the examples will be skipped until you reuse them. You can use the examples with it_behaves_like macro:

defmodule UseSharedSpec do
  use ESpec

  before do: {:ok, hello: "world!"}
  it_behaves_like(SharedSpec)
end

Async examples

There is an async: true option you can set for the context or for the individual example:

defmodule AsyncSpec do
  use ESpec, async: true
  it do: "async example"

  context "Sync", async: false do
    it do: "sync example"

    it "async again", async: true do
      "async"
    end
  end
end

The examples will be partioned into two queries. Examples in asynchronous query will be executed in parallel in different processes.

Don't use async: true if you change the global state in your specs!

Matchers

Equality

expect(actual).to eq(expected)  # passes if actual == expected
expect(actual).to eql(expected) # passes if actual === expected
expect(actual).to be_close_to(expected, delta)
expect(actual).to be_between(hard_place, rock)

Comparisons

Can be used with :>, :<, :>=, :<=, and etc.

expect(actual).to be operator, value

Passes if apply(Kernel, operator, [actual, value]) == true

Booleans

expect(actual).to be_true
expect(actual).to be_truthy
expect(actual).to be_false
expect(actual).to be_falsy

Regular expressions

expect(actual).to match(~r/expression/)
expect(actual).to match("string")

Enumerable

There are many helpers to test enumerable collections:

expect(collection).to be_empty #Enum.count(collection) == 0
... have(value)                #Enum.member?(collection, value)
... have_all(func)             #Enum.all?(collection, func)
... have_any(func)             #Enum.any?(collection, func)
... have_at(position, value)   #Enum.at?(collection, position) == value
... have_count(value)          #Enum.count(collection) == value
... have_size(value)           #alias
... have_length(value)         #alias
... have_count_by(func, value) #Enum.count(collection, func) == value
... have_max(value)            #Enum.max(collection) == value
... have_max_by(func, value)   #Enum.max_by(collection, fun) == value
... have_min(value)            #Enum.min(collection) == value
... have_min_by(func, value)   #Enum.min_by(collection, fun) == value

List

expect(list).to have_first(value) #List.first(list) == value
... have_last(value)              #List.last(list) == value
... have_hd                       #hd(list) == value
... have_tl                       #tl(list) == value

String

expect(string).to have_first(value)  #String.first(string) == value
... have_last(value)                 #String.last(string) == value
... start_with(value)                #String.starts_with?(string, value)
... end_with(value)                  #String.end_with?(string, value)
... have(value)                      #String.contains?(string, value)    
... have_at(pos, value)              #String.at(string, pos) == value
... have_length(value)               #Stirng.length(string) == value
... have_size(value)                 #alias
... have_count(value)                #alias
... be_valid_string                  #String.valid?(string)
... be_printable                     #String.printable?(string)

Dict

expect(dict).to have_key(value)     #Dict.has_key?(value)
expect(dict).to have_value(value)   #Enum.member?(Dict.values(dict), value)

Type checking

expect(:espec).to be_atom  #is_atom(:espec) == true
... be_binary
... be_bitstring
... be_boolean
... ...
... ...
... be_tuple
... be_function
... be_function(arity)
... be_struct
... be_struct(StructExample)

Exceptions

expect(function).to raise_exception
expect(function).to raise_exception(ErrorModule)
expect(function).to raise_exception(ErrorModule, "message")

Throws

expect(function).to throw_term
expect(function).to throw_term(term)

Change state

Test if call of function1 change the function2 returned value to smth or from to smth

expect(function1).to change(function2, to)
expect(function1).to change(function2, from, to)

Custom matchers

You can define your own matchers! The only functions you should implement is match/2, success_message/4, and error_message. Read the wiki page for detailed instructions. There is an example custom_assertion_spec.exs.

Mocks

ESpec uses Meck to mock functions. You can mock the module with 'allow accept':

defmodule SomeSpec do
  use ESpec
  before do: allow(SomeModule).to accept(:func, fn(a,b) -> a+b end)
  it do: expect(SomeModule.func(1, 2)).to eq(3)
end

If you don't specify the function to return ESpec creates stubs with arity 0 and 1: fn -> end and fn(_) -> end, which return nil.

defmodule SomeSpec do
  use ESpec
  before do: allow(SomeModule).to accept(:func)
  it do: expect(SomeModule.func).to be_nil
  it do: expect(SomeModule.func(42)).to be_nil
end

Behind the scenes 'allow accept' makes the following:

:meck.new(module, [:non_strict, :passthrough])
:meck.expect(module, name, function)

Find the explanation aboute the :non_strict and :passthrough options here. The default options ([:non_strict, :passthrough]) can be overridden:

allow(SomeModule).to accept(:func, fn(a,b) -> a+b end, [:non_strict, :unstick])

All the mocked modules are unloaded whith :meck.unload(modules) after each example.

You can also pass a list of atom-function pairs to the accept function:

allow(SomeModule).to accept(f1: fn -> :f1 end, f2: fn -> :f2 end)

One can use passthrough/1 function to call the original function:

  before do
    allow(SomeModule).to accept(:fun, fn
      :mocked -> "mock!"
      _ -> passthrough([args])
    end)
  end

  it do: expect(SomeModule.fun(:mocked)).to eq("mock!")
  it do: expect(SomeModule.fun(2)).to eq(3)

The passthrough/1 just calls the :meck.passthrough/1 from the :meck module.

There is also an expectation to check if the module accepted a function call:

accepted(func, args \\ :any, opts \\ [pid: :any, count: :any])

So, the options are:

• test if the function is called with some particular arguments of with any;
• specify the pid of the process which called the function;
• test the count of function calls.

defmodule SomeSpec do
  use ESpec
  before do
    allow(SomeModule).to accept(:func, fn(a,b) -> a+b end)
    SomeModule.func(1, 2)
  end  

  it do: expect(SomeModule).to accepted(:func)
  it do: expect(SomeModule).to accepted(:func, [1,2])

  describe "with options" do
    defmodule Server do
      def call(a, b) do
        ESpec.SomeModule.func(a, b)
        ESpec.SomeModule.func(a, b)
      end
    end

    before do
      pid = spawn(Server, :call, [1, 2])
      :timer.sleep(100)
      {:ok, pid: pid}
    end

    it do: expect(ESpec.SomeModule).to accepted(:func, [1,2], pid: shared.pid, count: 2)
  end
end

accepted assertion checks :meck.history(SomeModule). See meck documentation.

Don't use async: true when using mocks!

Doc specs

ESpec has functionality similar to ExUnit.DocTest. Read more about docs syntax here The functionality is implemented by two modules: ESpec.DocExample parses module documentation and ESpec.DocTest creates 'spec' examples for it. ESpec.DocExample functions is just copy-paste of ExUnit.Doctest parsing functionality. ESpec.DocTest implement doctest macro which identical to ExUnit analogue.

defmodule SomeSpec do
  use ESpec
  doctest MySuperModule
end  

There are three options (similar to ExUnit.DocTest):

:except - generate specs for all functions except those listed (list of {function, arity} tuples).

defmodule SomeSpec do
  use ESpec
  doctest MySuperModule, except: [fun: 1, func: 2]
end  

:only — generate specs only for functions listed (list of {function, arity} tuples).

And :import to test a function defined in the module without referring to the module name.Default is false. Use this option with care because you can clash with another modules.

There are three types of specs can be generated based on docs.

• Examples where input and output can be evaluated. For example:

@doc """
iex> Enum.map [1, 2, 3], fn(x) ->
...>   x * 2
...> end
[2,4,6]
"""

Such examples will be converted to:

it "Example description" do
  expect(input).to eq(output)
end  

• Examples which return complex structure so Elixir prints it as #Name<...>.:

@doc """
iex> Enum.into([a: 10, b: 20], HashDict.new)
#HashDict<[b: 20, a: 10]>
"""

The examples will be converted to:

it "Example description" do
  expect(inspect input).to eq(output)
end

• Examples with exceptions:

@doc """
iex(1)> String.to_atom((fn() -> 1 end).())
** (ArgumentError) argument error
"""

The examples will be tested as:

it "Example description" do
  expect(fn -> input end).to raise_exception(error_module, error_message)
end

Configuration and options

`MIX_ENV=test mix help espec`

Spec paths and pattern

You can change (in mix.exs file) the folder where your specs are and the pattern to match the files.

 def project do
  ...
  spec_paths: ["my_specs", "espec"],
  spec_pattern: "*_espec.exs",
  ...
 end

Coverage

One can run specs with coverage:

mix espec --cover

Find the results in /cover folder. ESpec, like ExUnit, uses very simple wrapper around OTP's cover. But you can override this.

Take a look to coverex as a perfect example.

Output formats

There are three formatters in ESpec: 'doc', 'json' and 'html'.

Example:

mix espec --format=doc

The 'doc' format will print detailed description of example and its context.

--trace option is an alias for --format=doc.

mix espec --trace

'html' and 'json' formatters prepare pretty HTML and JSON outputs.

You may use --format with --out option to write output to the file.

mix espec --format=html --out=spec.html

Contributing

Contributions are welcome and appreciated!

Request a new feature by creating an issue.

Create a pull request with new features and fixes.

ESpec is tested using ExUnit and ESpec. So run:

mix test
mix espec