markdownz for Erlang

This is an Erlang/OTP Markdown parser inspired by markdown-it. The upstream TypeScript project is used as a behavioral reference but is not included in this repository. The Erlang implementation is intentionally functional: it consumes binaries through recursive pattern matching, builds immutable HTML trees, traverses those trees with a zipper, and returns HTML as iodata().

The project currently implements the practical CommonMark core and the syntax used by Zotonic, including:

The parser returns the same basic terms as z_html_parse in z_stdlib:

Text = binary(),

Element = {TagBinary, [{AttributeBinary, Value}], [Text | Element]}.

Raw HTML, when enabled, uses z_stdlib's {'=', HtmlBinary} node.

Build and use

make

make test

make xref

make dialyzer

make edoc

The EUnit suite loads all 652 examples from the CommonMark 0.31.2 fixture in test/fixtures/commonmark/spec.txt. Each example is rendered and compared with its specified HTML. All 652 examples currently conform. The explicit markdownz_commonmark_tests:known_failures/0 baseline is empty, so every example is a regression check.

The suite also includes markdown-it's 13 link-normalization fixtures, 38 table fixtures, 12 typographic-replacement fixtures, and 19 smart-quote fixtures. Table output is compared semantically by element structure and text nodes, so irrelevant serializer whitespace and attribute spelling do not affect those tests. Link normalization covers percent encoding, human-readable autolink text, IDN/Punycode hostnames, protocol-relative URLs, and email links. An adapted set of 30 markdown-it pathological cases runs in bounded Erlang processes to catch algorithmic-denial-of-service regressions without risking the complete test VM.

1> markdownz:to_html(<<"# Hello *Erlang*">>).

%% iodata(), without a final flattening pass



2> markdownz:to_binary(<<"H~2~O and x^2^">>).

<<"<p>H<sub>2</sub>O and x<sup>2</sup></p>">>



3> markdownz:parse(<<"**tree**">>).

{ok,[{<<"p">>,[],[{<<"strong">>,[],[<<"tree">>]}]}]}

Configuration

markdownz:new/1 accepts a preset atom or an option map. markdownz:new/0 and markdownz:new(default) are equivalent.

Presets

Option map

An option map is merged over the default options, so only changed values need to be supplied:

Config = markdownz:new(#{

    html => true,

    linkify => false,

    table_class => <<"table table-striped">>

}).

The supported options are:

Resource-bounded parsing

parse/1,2 always enforce max_input_bytes and max_nesting. For content received from an untrusted boundary, the bounded variants additionally run the complete operation in a monitored Erlang process with timeout and heap limits:

case markdownz:to_binary_bounded(UserMarkdown, markdownz:new(zotonic)) of

    {ok, Html} ->

        Html;

    {error, input_too_large, Details} ->

        {reject, Details};

    {error, max_nesting, Details} ->

        {reject, Details};

    {error, timeout, Details} ->

        {reject, Details};

    {error, resource_limit, Details} ->

        {reject, Details}

end.

The available functions are parse_bounded/1,2, to_html_bounded/1,2, and to_binary_bounded/1,2; all return {ok, Result} or a structured error. These limits constrain CPU and memory use. They do not sanitize HTML: when html is enabled, render the result only in a trusted context or pass it through an HTML sanitizer such as Zotonic's z_sanitize:html/2.

Extensions

A configuration contains independent ordered rulers for the block, inline, and core phases. Rules are named, can be enabled or disabled, and can be inserted before or after another rule:

Parsing phases

PhaseInputPurpose
blockRemaining Markdown linesRecognizes document structure and invokes inline rules for textual content.
inlineRemaining bytes of an inline binaryRecognizes markup within textual block content.
coreThe complete HTML forestTransforms the complete tree after parsing, for example, decorating task lists.

The phase argument to add_rule/5, replace_rule/4, enable/3, or disable/3 selects one of these independent rulers. A before or 'after' anchor is resolved only within the selected phase; for example, emphasis is an inline rule name.

Block and inline rules are tried in ruler order at the current input position. Returning nomatch tries the next rule, while a successful rule must consume input. Core rules are applied in order to the complete tree. Every phase threads the parser state forward, allowing rules to share references and add extension-specific state without mutation.

Config0 = markdownz:new(),

Config1 = markdownz:add_rule(

    Config0,

    inline,

    before,

    emphasis,

    {mark, fun mark_rule/2}),

Config2 = markdownz:disable(Config1, inline, [subscript, superscript]).

The complete call has the following shape:

markdownz:add_rule(Config, Phase, Position, Anchor, {Name, Handler})

The result is a new configuration containing the inserted rule; Config itself remains unchanged. An unknown anchor or duplicate rule name raises an error.

An inline rule has this contract:

Rule(SourceBinary, State) ->

    nomatch |

    {ok, HtmlNodes, UnconsumedBinary, NewState}.

A block rule receives the remaining lines instead:

Rule(Lines, State) ->

    nomatch |

    {ok, HtmlNodes, UnconsumedLines, NewState}.

Core rules receive Tree and State as two arguments and return {NewTree, NewState}. markdownz_zipper is provided for local, immutable edits to the HTML forest; its map/2 is bottom-up, so parent rewrites can depend on transformed children.

A plugin module implements markdownz_plugin and its init/2 callback. Tag rendering can be overridden with markdownz:set_renderer/3 without changing the parsed tree.

Scope

This implementation passes the complete bundled CommonMark 0.31.2 corpus. Its public tree and ruler APIs are designed so GFM and Zotonic-specific rules can continue to evolve without changing callers.

License

This Erlang implementation is released under the MIT License.