RustyXML

Ultra-fast XML parsing for Elixir. A purpose-built Rust NIF with SIMD-accelerated zero-copy structural index and full XPath 1.0 support. Drop-in replacement for both SweetXml and Saxy — one dependency replaces two.

Hex.pm

Features

Installation

def deps do
  [{:rusty_xml, "~> 0.2.3"}]
end

Precompiled binaries are available for common platforms. For source compilation, Rust 1.91+ is required.

Quick Start

import RustyXML

xml = """
<catalog>
  <book id="1"><title>Elixir in Action</title><price>45.00</price></book>
  <book id="2"><title>Programming Phoenix</title><price>50.00</price></book>
</catalog>
"""

# Get all books
RustyXML.xpath(xml, ~x"//book"l)

# Get text content
RustyXML.xpath(xml, ~x"//title/text()"s)

# Count elements
RustyXML.xpath(xml, "count(//book)")
#=> 2.0

# Extract multiple values with xmap
RustyXML.xmap(xml, [
  titles: ~x"//title",
  prices: ~x"//price"
])
#=> %{titles: [...], prices: [...]}

The ~x Sigil

The ~x sigil creates XPath expressions with optional modifiers:

import RustyXML

~x"//item"        # Basic XPath
~x"//item"l       # Return as list
~x"//item"s       # Return as string
~x"//item"i       # Cast to integer
~x"//item"f       # Cast to float
~x"//item"o       # Optional (nil on missing)
~x"//item"e       # Decode entities
~x"//item"k       # Return as keyword list
~x"//item"slo     # Combine modifiers

Usage

All parsing flows through a single optimized structural index. XPath and streaming are API features on top of that path, maintained for SweetXml compatibility.

# Parse once, query multiple times
doc = RustyXML.parse(xml)
RustyXML.xpath(doc, ~x"//item"l)
RustyXML.xpath(doc, ~x"//price"l)

# Stream large files (returns {tag_atom, xml_string} tuples)
"large.xml"
|> RustyXML.stream_tags(:item)
|> Stream.each(fn {:item, item_xml} ->
  name = RustyXML.xpath(item_xml, ~x"./name/text()"s)
  IO.puts("Processing: #{name}")
end)
|> Stream.run()

Saxy-Compatible API

RustyXML is a drop-in replacement for Saxy. SAX handler callbacks, SimpleForm, Partial incremental parsing, and XML encoding all work with the same API.

SAX Parsing

defmodule MyHandler do
  @behaviour RustyXML.Handler

  def handle_event(:start_element, {name, _attrs}, count), do: {:ok, count + 1}
  def handle_event(_event, _data, count), do: {:ok, count}
end

# Parse a string
{:ok, 4} = RustyXML.parse_string("<root><a/><b/><c/></root>", MyHandler, 0)

# Parse a stream
File.stream!("large.xml", [], 64 * 1024)
|> RustyXML.parse_stream(MyHandler, 0)

SimpleForm

{:ok, {"root", [], [{"item", [{"id", "1"}], ["text"]}]}} =
  RustyXML.SimpleForm.parse_string("<root><item id=\"1\">text</item></root>")

Incremental Parsing

{:ok, partial} = RustyXML.Partial.new(MyHandler, initial_state)
{:cont, partial} = RustyXML.Partial.parse(partial, chunk1)
{:cont, partial} = RustyXML.Partial.parse(partial, chunk2)
{:ok, final_state} = RustyXML.Partial.terminate(partial)

XML Encoding

import RustyXML.XML

doc = element("person", [{"id", "1"}], [
  element("name", [], ["John Doe"]),
  element("email", [], ["john@example.com"])
])

RustyXML.encode!(doc)
#=> "<person id=\"1\"><name>John Doe</name><email>john@example.com</email></person>"

XPath 1.0 Support

Axes (13)

Functions (27+)

Node Functions:

String Functions:

Boolean Functions:

Number Functions:

Known Limitations

API Reference

Core Functions

# Parse XML (strict by default, like SweetXml)
doc = RustyXML.parse("<root>...</root>")

# Parse with lenient mode (accepts malformed XML)
doc = RustyXML.parse("<root/>", lenient: true)

# Parse with tuple return (for pattern matching errors)
{:ok, doc} = RustyXML.parse_document("<root/>")
{:error, reason} = RustyXML.parse_document("<1bad/>")

# Execute XPath query
RustyXML.xpath(xml_or_doc, ~x"//item"l)

# Extract multiple values
RustyXML.xmap(xml_or_doc, [key: ~x"//path"])

# Get root element
RustyXML.root(doc)

Streaming

Stream large XML files with bounded memory. Returns {tag_atom, xml_string} tuples compatible with SweetXml.

# Stream specific tags from a file
RustyXML.stream_tags("data.xml", :item)
|> Enum.each(fn {:item, item_xml} ->
  # Each item_xml is a complete XML string that can be queried
  id = RustyXML.xpath(item_xml, ~x"./@id"s)
  name = RustyXML.xpath(item_xml, ~x"./name/text()"s)
  IO.puts("Item #{id}: #{name}")
end)

# Stream from enumerable (useful for network streams)
File.stream!("data.xml", [], 64 * 1024)
|> RustyXML.stream_tags(:item)
|> Stream.map(fn {:item, item} ->
  %{
    id: RustyXML.xpath(item, ~x"./@id"s),
    name: RustyXML.xpath(item, ~x"./name/text()"s)
  }
end)
|> Enum.to_list()

# Works correctly with Stream.take (unlike SweetXml issue #97)
"large.xml"
|> RustyXML.stream_tags(:item)
|> Stream.take(5)
|> Enum.to_list()

# Stream from XML string
xml_string
|> RustyXML.stream_tags(:item, chunk_size: 32 * 1024)
|> Enum.to_list()

Key features:

Low-Level Native Functions

# Streaming parser for large files
parser = RustyXML.Native.streaming_new()
RustyXML.Native.streaming_feed(parser, chunk)
RustyXML.Native.streaming_take_events(parser, 100)

# SAX-style event parsing
RustyXML.Native.sax_parse(xml)
#=> [{:start_element, "root", [], []}, ...]

Architecture

A single UnifiedScanner tokenizes input with SIMD acceleration, dispatching to a ScanHandler trait that builds the structural index, SAX events, or streaming elements.

native/rustyxml/src/
├── lib.rs                 # NIF entry points
├── core/
│   ├── scanner.rs         # SIMD byte scanning (memchr)
│   ├── unified_scanner.rs # UnifiedScanner + ScanHandler trait
│   ├── tokenizer.rs       # State machine tokenizer
│   ├── entities.rs        # Entity decoding with Cow
│   └── attributes.rs      # Attribute parsing
├── index/
│   ├── structural.rs      # StructuralIndex (core data structure)
│   ├── span.rs            # Span struct (offset, length)
│   ├── element.rs         # IndexElement, IndexText, IndexAttribute
│   ├── builder.rs         # IndexBuilder (ScanHandler impl)
│   └── view.rs            # IndexedDocumentView (DocumentAccess impl)
├── dom/
│   ├── document.rs        # Document types, validation
│   ├── node.rs            # Node types
│   └── strings.rs         # String utilities
├── xpath/
│   ├── lexer.rs           # XPath tokenizer
│   ├── parser.rs          # Recursive descent parser
│   ├── eval.rs            # Evaluation engine
│   ├── axes.rs            # All 13 axes
│   └── functions.rs       # 27+ XPath functions
├── sax/
│   ├── events.rs          # CompactSaxEvent types
│   └── collector.rs       # SaxCollector (ScanHandler impl)
├── strategy/
│   ├── streaming.rs       # Stateful streaming parser
│   └── parallel.rs        # Parallel XPath (DirtyCpu)
├── term.rs                # BEAM term building
└── resource.rs            # ResourceArc wrappers

See Architecture for full details.

Parsing Modes: Lenient vs Strict

The Reality of XML in the Wild

In theory, XML is strictly defined by the W3C specification. In practice, many real-world XML documents contain minor well-formedness violations that strict parsers reject. This creates a tension between correctness and practicality.

Common violations found in production XML: