FeedReader

FeedReader fetches and parses RSS, Atom, and RDF feeds into normalized Elixir data. It detects formats automatically, recovers from a small set of common XML defects, and offers collection with an optional item cap as well as lazy event streams.

Installation

Add FeedReader to your Mix project's dependencies:

defp deps do
  [
    {:feed_reader, "~> 0.2.0"}
  ]
end

Quick start

Pass XML directly, or use parse_file/2 for a local file. Collection APIs gather all accepted items by default; an optional :max_items cap stops parsing early:

{:ok, result} = FeedReader.parse_file("path/to/feed.xml")

result.feed.title
Enum.map(result.items, & &1.title)

For a remote URL, fetch_and_parse/2 downloads and collects the feed. You can set :max_items to stop parsing after a given number of accepted items; when that happens, result.item_limit_reached? is true. The complete response is downloaded before parsing starts. The default HTTP transport follows redirects automatically, up to five; the final response URL is available in result.fetch.final_url.

{:ok, result} =
  FeedReader.fetch_and_parse("https://example.com/feed.xml")

result.feed.title
result.fetch.etag

Application configuration

Set stable options in your application config. For example, configure an optional collection cap alongside other defaults:

config :feed_reader, :options,
  max_items: 500,
  max_xml_depth: 128,
  http_retry_attempts: 3,
  formats: [MyApp.NoticeFormat],
  http_client: MyApp.FeedTransport

The configured cap applies to collection calls that omit :max_items; per-call values override it. If no cap is configured, collection gathers all accepted items:

{:ok, result} = FeedReader.parse_file("path/to/feed.xml")
{:ok, capped_result} = FeedReader.parse_file("path/to/feed.xml", max_items: 100)

Use stream_file/2 or stream/2 to process large feeds incrementally rather than collecting all items in memory. parse_stream/2 still collects its result; its input chunks are staged before parsing.

Request-specific options such as :feed_url, :previous_etag, :previous_checksum, and :item_filter must be supplied per call. See the resource limits guide and FeedReader API docs for defaults and details. :formats adds custom formats after the built-ins; :http_client selects a custom transport. Custom transports may define and read their own additional options.

Architecture

The public API coordinates a small pipeline:

1. Remote requests use a pluggable transport. The downloader consumes raw response chunks, decodes supported content encodings, enforces download budgets, computes a decoded-body checksum, and stages the result in an owned temporary file.
2. The parser inspects a bounded prefix to select RSS, Atom, RDF, or a supplied custom format. An explicit format can be selected when detection is unsuitable.
3. Streaming XML recovery normalizes supported encoding and markup defects before Saxy parses the input. Format handlers normalize metadata and items into FeedReader.Feed and FeedReader.Item values.
4. The parser emits {:feed, feed} and {:item, item} events. Streaming APIs expose those events lazily; collection APIs gather all accepted items by default, or stop at an optional item cap.

Remote streaming downloads before returning its session; event parsing then reads the staged file lazily. A completed stream releases the temporary file. If remote enumeration stops early, close the session with FeedReader.close/1. Local sessions read caller-owned files and never remove them.

Guides

• Parsing and change detection — files, binaries, streams, remote URLs, and unchanged-source checks.
• Streaming and verified parsing — event lifecycle, cleanup, backpressure, and validation passes.
• Resource limits — parser, download, and collection budgets.
• Results and normalized data — result structs, events, normalized fields, and common outcomes.
• Customization — format detection, custom formats, transports, and HTTP behavior.
• XML recovery — supported repairs and their limits.

See the FeedReader module documentation for the complete API and option reference.

Contributing

Install Elixir 1.20.4 with Erlang/OTP 29.1.2, as specified in .tool-versions. Then fetch dependencies and run the project checks:

mix deps.get
mix precommit
mix credo --strict

Create a branch for your change and submit it as a pull request. mix precommit compiles with warnings treated as errors, checks formatting, and runs the tests.