SWAR

Byte-set guards for Elixir that classify seven bytes at a time, in pure BEAM code. No NIFs, no CPU feature checks.

SWAR means SIMD Within A Register: several bytes are packed into one integer and tested with arithmetic. SWAR.defbyteset/2 lets you describe the accepted bytes once, then generates matching byte, word and tail-padding helpers.

defmodule Scan do
  require SWAR

  SWAR.defbyteset(:unescaped, [0x20..0x7E, not ?", not ?\\])

  @word SWAR.word_bytes()

  def count(bin), do: count(bin, 0)

  defp count(<<w::size(@word)-unit(8), rest::binary>>, n) when is_unescaped_word(w),
    do: count(rest, n + @word)

  for size <- (@word - 1)..1//-1 do
    defp count(<<w::size(unquote(size))-unit(8)>>, n)
         when is_unescaped_word(unescaped_pad(w, unquote(size))),
         do: n + unquote(size)
  end

  defp count(<<byte, rest::binary>>, n) when is_unescaped(byte),
    do: count(rest, n + 1)

  defp count(_rest, n), do: n
end

What gets generated

For SWAR.defbyteset(:unescaped, spec), the default helpers are:

All generated helpers are private to the module that calls defbyteset/2.

More helpers are available with define::

SWAR.defbyteset(:digit, ?0..?9, define: :all)
SWAR.defbyteset(:plain, [not ?", not ?\\], define: [:byte, :word, :leading])

Use _leading/1 after a word guard fails when you need the accepted prefix length:

defp scan(<<w::size(@word)-unit(8), rest::binary>>, n) when is_plain_word(w),
  do: scan(rest, n + @word)

defp scan(<<w::size(@word)-unit(8), _rest::binary>>, n),
  do: n + plain_leading(w)

defp scan(tail, n), do: byte_scan(tail, n)

The byte guard, word guard, pad macro and mask macro can be used in guards. _leading/1, _trailing/1 and _count/1 expand to blocks, so use them in function bodies.

Specs

A spec is one term or a list of terms. Supported terms are:

• literal bytes: ?a, 0x2E
• literal ranges: ?a..?z, 0x20..0x7E
• charlists: ~c"-:", [?+, ?/]
• negated terms: not ?", not(?p..?r); ! also works

Positive terms are unioned. Negated terms are subtracted from that union, or from all ASCII bytes when the spec contains only negations.

SWAR.defbyteset(:base64, [?A..?Z, ?a..?z, ?0..?9, ~c"+/"])
SWAR.defbyteset(:printable, [0x20..0x7E, not ?", not ?\\])
SWAR.defbyteset(:no_cr, [not ?\r])

The generated code uses a normalized set: ranges are sorted, merged and made disjoint before expansion. These two specs generate the same checks:

[0x20..0x7E, not ?", not ?#, not ?\\]
[0x20..0x21, 0x24..0x5B, 0x5D..0x7E]

Bytes must be literal values in 0..0x7F. Other values raise ArgumentError while the macro expands.

Bytes above 0x7F

By default, specs describe ASCII only. That is what you want for alphabets such as base64 or decimal digits.

Use high: true when every byte above 0x7F should also be accepted. This is useful for delimiter scans over arbitrary text, such as reading until a quote or backslash, where non-ASCII bytes are just more non-delimiters.

SWAR.defbyteset(:plain, [not ?", not ?\\],
  high: true,
  define: [:byte, :word, :leading]
)

high: true does not decode UTF-8. It only says high bytes belong to the set.

Reading digits

SWAR.uint(word, size) reads size ASCII digits, most significant first, from a word. Validate the word first with a digit byteset.

SWAR.defbyteset(:digit, ?0..?9, define: [:byte, :word, :pad])

defp to_integer(<<w::size(7)-unit(8), rest::binary>>, acc) when is_digit_word(w),
  do: to_integer(rest, acc * 10_000_000 + SWAR.uint(w, 7))

defp to_integer(<<byte, rest::binary>>, acc) when is_digit(byte),
  do: to_integer(rest, acc * 10 + (byte - ?0))

This pays off for longer runs of digits. For short fixed fields such as dates, ordinary byte guards are usually simpler and at least as fast.

When it helps

SWAR is best when:

• the accepted set is a few ranges, not many scattered bytes
• the input usually has runs of at least seven accepted bytes
• the scan works mostly on ASCII, or high: true is correct for the problem
• the input is long enough that call overhead is not the main cost

It is usually not worth it when:

• the set has more than about eight or nine range terms
• the scan usually stops in the first few bytes
• accepted runs are shorter than a word
• high bytes need to be decoded rather than skipped

The benchmarks in bench/ cover delimiter scans, String.printable?/2-style checks, digit parsing, manual single-byte/range comparisons and sets with different numbers of range terms.

To see the direct comparison against ordinary byte-at-a-time code, run:

mix run bench/manual.exs

This benchmark uses Benchee and reports iterations per second plus relative speed for each set shape and input.

Representative results from Linux, AMD Ryzen 7 7840HS, Elixir 1.20.0, Erlang 29.0, with JIT enabled (manual / SWAR; values are average ns per call):

The exact numbers vary by machine, but the pattern is the useful result: a manual check is preferable for tiny inputs and usually for an early rejection. Once the input contains accepted runs of seven or more bytes, SWAR generally wins; on accepted 1 KB inputs it was 3.0–4.4× faster here. Multiple ranges add work to the SWAR word guard, but still beat the three manual range comparisons on long accepted input.

The short-input crossover is especially clear for mixed-case hex:

For a focused run, use BENCH_CASE=hex BENCH_INPUT=accepted mix run bench/manual.exs. The full script also includes rejection-at-first/last cases.

It compares a single-character set (byte == ?a), a negated character (byte != ?!), one contiguous range (?a <= byte <= ?z), lowercase hex (0..9, a..f), mixed-case hex (0..9, a..f, A..F) and three ranges (0..9, A..Z, a..z) over short and long binaries, with accepted input and a rejection at either end. Expect the manual version to win for very short input or an early rejection: SWAR has to load and classify a seven-byte word before it can help. SWAR starts to make sense when accepted runs are several words long, especially when the set has multiple ranges that would make the manual byte guard more expensive. Use these numbers as guidance on your machine; benchmark the actual input distribution before choosing it.

As a quick rule of thumb:

Debug output

Use debug: true to print the generated code at compile time:

SWAR.defbyteset(:printable, [?\a..?\r, ?\e, 0x20..0x7F], debug: true)

The printed code is self-contained, so it can be pasted into a project if you want the generated guard without depending on this package. It also includes a rough performance estimate based on the number of range terms.

Why seven bytes

Seven bytes fit in 56 bits, which stays inside a BEAM small integer on 64-bit systems. An eight-byte word would overflow into a heap-allocated bignum during the arithmetic and lose the optimization.

Tests

mix test

The test suite checks generated guards against independently written byte sets, random words, high-bit cases, padding, optional helpers, error paths, Elixir core guard parity, debug output and SWAR.uint/2.

License

Licensed under the Apache License, Version 2.0. See LICENSE for the full license text.