doctest

An Erlang library to test @doc tags and -moduledoc and -doc attributes.

It is compatible with the doctests syntax introduced in OTP-28.

Installation

% rebar.config
% {minimum_otp_vsn, "24"}.
{profiles, [
    {test, [
        % 'debug_info' is required to extract doc chunks.
        {erl_opts, [debug_info]},
        {deps, [{doctest, "~> 0.11"}]}
    ]}
]}.
% 'doctest_eunit_report' is required to pretty print and correctly displays the failed Eunit tests.
{eunit_opts, [no_tty, {report, {doctest_eunit_report, []}}]}.

{profiles, [
    {test, [
        {cover_enabled, true},
        % ...
    ]}
]}.

Overview

Erlang documentation can be written:

• Via EDoc by using the @doc tag, e.g.:

  %% @doc Prints "Hello, Joe!".
  %%
  %% Example:
  %%
  %% ```
  %% > print().
  %% "Hello, Joe!"
  %% '''
  print() -> "Hello, Joe!".
  

• Or via ExDoc, by using the -moduledoc and -doc attributes introduced in OTP 27, e.g.:

  -doc """
  Prints "Hello, Joe!".
  
  Example:
  
  ```
  > print().
  "Hello, Joe!"
  ```
  """.
  print() -> "Hello, Joe!".
  

There are some rules to test documentation. One rule is that only code blocks are testable. Via EDoc/tags, code blocks are code between ``` and ''' (triple backticks and triple single quotes), and via ExDoc/attributes, they are code between ``` and ``` (triple quotes and triple quotes). The code of the code blocks follows the same rules as the current Erlang shell, for example:

> % - Comments and multiline expressions are allowed;
  % - Multiline expressions must be aligned;
  % - Invalid syntaxes are skipped.
  print().
"Hello, Joe!"
> % All tests compare the equality between the expression and the result.
  % The example below is translated to an `?assertEqual` macro result:
  % > ?assertEqual(print() =/= "Hello, World", true).
  print() =/= "Hello, World!".
true

Optionally

1. Variable outputs can be skipped by replacing the right side (result) by an underscore (_) or without giving a result to it by starting a new expression
2. The erlang language in code blocks is not required
3. The syntax can contains the line numbers and dots between multiple lines, like in the erlang shell

For example:

-doc """
```erlang
1> Foo =
.. foo.
_
2> Bar = Foo.
3> foo(Bar).
foo
```
""".
foo(Foo) -> Foo.

Usage

There are two ways to test your documentation:

• Manually calling doctest:module/1,2 functions in the Erlang shell, e.g.:

  > doctest:module(greeting, #{
     % Options (please see the options below)
  }).
  

• Or via parse transformation by using the doctest_transform module included in the doctest/include/doctest.hrl and then running rebar3 eunit, e.g.:

  -ifdef(TEST).
  % The doctest header sets `doctest_transform` as a parse_transform:
  -include_lib("doctest/include/doctest.hrl").
  -doctest #{
      % Options (please see the options below)
  }.
  -endif.
  

Common Test (CT) usage

Example how to use doctest via Common Test:

-module(mymodule_SUITE).
-behaviour(ct_suite).
-export([all/0, doctest/1]).

all() -> [doctest].

doctest(Config) when is_list(Config) ->
    ok = doctest:module(mymodule). % Use `doctest:module/2` to pass options

Eunit usage

Example how to use doctest via Eunit:

-module(mymodule).
-export([sum/2]).

-ifdef(TEST).
-include_lib("doctest/include/doctest.hrl").
% Use the -doctest attribute to pass options
-endif.

% The `sum/2` will be tested by default because it contains a valid Markdown
% code block. Multiple Markdown code blocks can be defined.
-doc """
```
> mymodule:sum(1, 1).
2
```
""".
sum(A, B) ->
    A + B.

Options

The options are passed via a map:

#{
    % Enable or turn off any test.
    % Default: true.
    enabled => boolean(),

    % Enable or turn off module doc tests.
    % Default: true.
    moduledoc => boolean(),

    % Enable or turn off functions doc tests or define a list of functions
    % to be tested.
    % Default: true.
    doc => boolean() | [{atom(), arity()}],

    % Set the EUnit options. 'rebar3_config' tries to resolve the options
    % defined in the rebar3.
    % Default: rebar3_config.
    eunit_opts => rebar3_config | [term()],

    % Overrides the code blocks extractors. See the 'doctest_extract'
    % behavior. Custom extractors are allowed.
    % Default:
    % - OTP < 27: [doctest_extract_tag];
    % - OTP >= 27: [doctest_extract_attr, doctest_extract_tag].
    extractors => [module()]
}

In a module, the -doctest attribute is used to override the default settings via a map, e.g., -doctest #{enabled => true}., or via some shortcuts, for example:

• {enabled, boolean()} or boolean(): equivalent to enabled option.

  -doctest true.
  

• {moduledoc, boolean()}: equivalent to moduledoc option.

  -doctest {moduledoc, true}.
  

• {doc, boolean() | [{atom(), arity()}]} or [{atom(), arity()}]: equivalent to doc option.

  -doctest [print/0].
  

• {eunit_opts, rebar3_config | term()}: equivalent to eunit_opts option.

  -doctest {eunit_opts, rebar3_config}.
  

• {extractors, [module()]}: equivalent to extractors option.

  -doctest {extractors, [doctest_extract_attr, doctest_extract_tag]}.
  

Global Options

Options can be globally defined via a config file, e.g.:

% config/sys.config
[{doctest, [
    {enabled, true},
    {moduledoc, true},
    {doc, true},
    {eunit_opts, rebar3_config},
    {extractors, [doctest_extract_attr, doctest_extract_tag]}
]}].

Please make sure to add the config file to the rebar3 config, e.g.:

{shell, [{config, "config/sys.config"}]}.
{eunit_opts, [{sys_config, ["config/sys.config"]}]}.

Example

Take this module:

 1 │ -module(greeting).
 2 │ -moduledoc """
 3 │ Module documentation are testable.
 4 │
 5 │ ```
 6 │ > greeting:print() =:= "Hello, Joe!".
 7 │ true
 8 │ ```
 9 │ """.
10 │
11 │ -export([print/0]).
12 │
13 │ -ifdef(TEST).
14 │ -include_lib("doctest/include/doctest.hrl").
15 │ -endif.
16 │
17 │ -doc """
18 │ ```
19 │ > greeting:print().
20 │ "Hello, World!"
21 │ ```
22 │ """.
23 │ print() ->
24 │     hello().
25 │
26 │ %% @doc Non-exported functions are testable.
27 │ %%
28 │ %% ```
29 │ %% > % Bound variables to a value is valid, e.g.:
30 │ %%   Greeting = hello().
31 │ %% "Hello, Joe!"
32 │ %% > Greeting =:= "Hello, World!".
33 │ %% true
34 │ %% '''
35 │ hello() ->
36 │     "Hello, Joe!".

As mentioned before, there are two ways to run the tests.

• Via doctest:module/1,2 in the Erlang shell, e.g.:

  $ rebar3 as test shell
  

  > doctest:module(greeting).
  

• Or via rebar3 eunit

Both produce the same output:

 PASS  ./src/greeting.erl:6 -moduledoc
 FAIL  ./src/greeting.erl:19 -doc

    ❌ assertEqual

    Expected: "Hello, World!"
    Received: "Hello, Joe!"

    │
 19 │ > greeting:print().
 20 │ "Hello, World!"
    │
    └── at ./src/greeting.erl:19

 PASS  ./src/greeting.erl:29 @doc
 FAIL  ./src/greeting.erl:32 @doc

    ❌ assertEqual

    Expected: true
    Received: false

    │
 32 │ %% > Greeting =:= "Hello, World!".
 33 │ %% true
    │
    └── at ./src/greeting.erl:32



Tests: 2 failed, 2 passed, 4 total
 Time: 0.014 seconds

Doctest EUnit Reporter

There is a built-in EUnit reporter called doctest_eunit_report to display the tests results correctly. Set it in the EUnit options of the project options, e.g.:

% rebar3.config
{eunit_opts, [
    no_tty,
    {report, {doctest_eunit_report, [
        % Default options
        {print_depth, 15}
    ]}}
]}.

An example of the doctest_eunit_report output:

Sponsors

If you like this tool, please consider sponsoring me. I'm thankful for your never-ending support ❤️

I also accept coffees ☕

Contributing

Issues

Feel free to submit an issue on Github.

License

Copyright (c) 2024 William Fank Thomé

doctest is 100% open source and community-driven. All components are available under the Apache 2 License on GitHub.

See LICENSE.md for more information.