SuperScrape

Read an Atom/RSS feed and concatenate it with deduped historical items from past versions saved in the Wayback Machine. (View Docs)

Install

1. Add superscrape to your list of dependencies in mix.exs:

  def deps do
    [{:superscrape, "~> 0.1.0"}]
  end

1. Ensure superscrape is started before your application:

  def application do
    [applications: [:superscrape]]
  end

Usage

View the online documentation for more information.

url = "https://elixir-jobs.org/feed"

SuperScrape.process url
#=> [%{...}, %{...}, %{...}]

SuperScrape.process! url
#=> ["https://elixir-jobs.org/jobs/26", "..."]

SuperScrape.process! [url, "..."]
#=> ["https://elixir-jobs.org/jobs/26", "..."]

SuperScrape.lookup url
#=> ["http://web.archive.org/web/...", "..."]

SuperScrape.open "https://elixir-jobs.org/jobs/26"
#=> %{description: "...", title: "...", ...}

License

MIT © Luke Edwards