Rover

Drive the Servo web engine from Elixir.

Rover runs each browser as its own OS process, isolated from the BEAM and from other browsers. That gives you per-instance proxy config, independent cookie jars, and crash containment — if a page crashes the renderer, your VM keeps running.

The API is modelled after Req: small, composable, and sensible by default.

Status: 0.1 — proof-of-concept. Tagged releases ship a precompiled runtime for Apple Silicon macOS and x86-64/AArch64 Linux; development builds embed a pinned Servo checkout from source.

Why Rover?

If you just want the bytes of an HTTP response, use Req. Rover is for the cases where Req is not enough:

JS-rendered pages — content only exists after <script> tags execute.
Per-instance proxies — route different browsers through different egresses (customer machines, regional proxies, etc.).
Input automation — click, fill, submit forms, then read the result.
Visual capture — screenshots for verification or diffing.

Installation

Add :rover to your deps in mix.exs:

def deps do
  [
    {:rover, "~> 0.1", hex: :rover_browser}
  ]
end

Rover ships a companion Rust binary, rover_runtime, that embeds Servo. On a supported host, mix compile downloads the matching binary from the GitHub release and verifies its SHA-256 checksum automatically:

mix deps.get
mix compile

Precompiled targets are Apple Silicon macOS and x86-64/AArch64 glibc Linux. To work on Rover itself, fetch the pinned Servo revision and build the runtime from source (the first build can take 10–30 minutes):

scripts/fetch-servo.sh
ROVER_BUILD=1 mix rover.build

The binary is discovered automatically. You can override its location with:

ROVER_RUNTIME_BIN=/absolute/path/to/rover_runtime iex -S mix

Quick start

{:ok, result} = Rover.fetch("https://example.com")
result.body   # rendered HTML
result.title  # document.title
result.url    # final URL after redirects

Usage guide

Rover has two ways in: the one-shotRover.fetch/2 for single-page work, and the long-lived browser (Rover.start_link/1) for multi-step automation. Pick whichever matches the shape of your task — not both at once.

One-shot: `Rover.fetch/2`

fetch/2 spins up a browser, does the work, tears it down. Use it for stateless scraping where you don't need to carry cookies or navigate between pages.

{:ok, result} = Rover.fetch("https://example.com")

result.status    # :ok
result.url       # "https://example.com/"  (after redirects)
result.title     # "Example Domain"
result.body      # "<!doctype html>…"  (rendered HTML)

Waiting for JS-rendered content

Many sites don't fully populate the DOM until some script has run. Pass :wait_for with a CSS selector that only exists after the page is ready:

{:ok, result} =
  Rover.fetch("https://movies.example/now-showing",
    wait_for: ".movie-list",      # poll until this appears
    wait_timeout: 15_000           # then give up
  )

Without :wait_for, Rover returns as soon as document.readyState == complete — which is typically too early for SPAs that fetch data after first paint.

Evaluating JavaScript

evaluate runs an expression in the page and round-trips the result as an Elixir term:

{:ok, result} =
  Rover.fetch("https://shop.example/product/42",
    evaluate: "JSON.parse(document.querySelector('#product-data').textContent)"
  )

result.evaluated
# %{
#   "price" => 4999,
#   "currency" => "EUR",
#   "in_stock" => true,
#   "tags" => ["leather", "handmade"]
# }

Strings, numbers, booleans, null, arrays, and objects all convert cleanly. Functions, DOM nodes, and promises don't — wrap them in something JSON-able first.

Extracting multiple selectors

extract is a convenience for grabbing text from several selectors in one trip:

{:ok, result} =
  Rover.fetch("https://news.example/",
    wait_for: "article",
    extract: [
      headline: "h1.lead",
      subheads: "h2.section-header",
      bylines: ".author"
    ]
  )

result.extracted
# %{
#   headline: "Markets surge on surprise rate cut",
#   subheads: ["Global reaction", "Sector breakdown"],
#   bylines: ["J. Cooper", "R. Singh"]
# }

Single-match selectors give a string; multi-match selectors give a list.

Screenshot

{:ok, result} =
  Rover.fetch("https://dashboard.example/metrics",
    wait_for: ".chart-loaded",
    screenshot: :png
  )

File.write!("dashboard.png", result.screenshot)

screenshot: :png is the default; pass :jpeg for smaller files. JPEG quality is configurable on the long-lived browser only (Rover.screenshot/2).

Routing through a proxy

{:ok, result} =
  Rover.fetch("https://ipinfo.io/json",
    proxy: "http://eu.proxy.example:8080",
    evaluate: "JSON.parse(document.body.innerText)"
  )

result.evaluated["country"]  # "DE"

The proxy URI accepts http://user:pass@host:port for basic auth. It's baked into the browser at startup and applies to every request, including CONNECTs for HTTPS. For per-request routing, use multiple browsers (see below).

Long-lived browser: `Rover.start_link/1`

When you need to log in, then click around, then scrape — use a long-lived browser. Each browser keeps its cookies, auth state, and page position across commands.

{:ok, browser} = Rover.start_link(proxy: "http://proxy:8080")

try do
  :ok = Rover.navigate(browser, "https://app.example/login")
  :ok = Rover.fill(browser, "#email", "user@example.com")
  :ok = Rover.fill(browser, "#password", "hunter2")
  :ok = Rover.click(browser, "button[type=submit]")
  :ok = Rover.wait_for(browser, ".dashboard")

  {:ok, html} = Rover.content(browser)
  # …do stuff with html…
after
  Rover.stop(browser)
end

Putting the browser under supervision

Rover.Browser is a plain GenServer — add it to your supervision tree and let OTP manage restarts:

children = [
  {Rover.Browser, name: MyApp.EUBrowser, proxy: "http://eu-proxy:8080"},
  {Rover.Browser, name: MyApp.USBrowser, proxy: "http://us-proxy:8080"}
]

Supervisor.start_link(children, strategy: :one_for_one)

# Later, anywhere:
Rover.navigate(MyApp.EUBrowser, "https://site.example/eu-only")

If the Servo subprocess crashes, the Port dies, the Rover.Browser GenServer exits with {:port_died, _}, and your supervisor restarts it — fresh state, same proxy config. The BEAM never sees the failure.

Per-instance routing (the point of Rover)

Two browsers, two egresses:

{:ok, eu} = Rover.start_link(proxy: "http://eu-egress:8080")
{:ok, us} = Rover.start_link(proxy: "http://us-egress:8080")

locations =
  [eu, us]
  |> Task.async_stream(fn browser ->
    {:ok, _page} = Rover.navigate(browser, "https://ipinfo.io/json")
    {:ok, json}  = Rover.get_text(browser, "pre")
    Jason.decode!(json)
  end)
  |> Enum.map(fn {:ok, v} -> v end)

# locations = [%{"country" => "DE", …}, %{"country" => "US", …}]

Because each browser is its own OS process, the two flows never share cookies, connection pools, or DNS cache. No amount of Set-Cookie from one browser leaks into the other.

Filling forms with real input

fill and click dispatch the same events a user's keyboard and mouse would. That matters for frontends that listen on input / change / submit:

:ok = Rover.fill(browser, "#search", "elixir")
:ok = Rover.wait_for(browser, ".autocomplete-ready")
:ok = Rover.click(browser, ".suggestion:nth-child(1)")

fill is safe for <input>, <textarea>, and contenteditable regions. select_option works on <select>:

:ok = Rover.select_option(browser, "#country", "IE")

Cookies

Rover talks to Servo's SiteDataManager directly — no document.cookie round-trips. HttpOnly cookies are visible.

{:ok, _} = Rover.navigate(browser, "https://app.example/")
{:ok, cookies} = Rover.get_cookies(browser)

session_cookie = Enum.find(cookies, fn c -> c["name"] == "sid" end)

# Set a cookie manually (e.g., to skip a login screen)
:ok = Rover.set_cookie(browser, "sid=abc123; path=/; Secure; HttpOnly")

# Nuke the jar
:ok = Rover.clear_cookies(browser)

Error handling

Rover.fetch/2 and the long-lived-browser calls never raise. They return {:ok, result} | {:error, %Rover.Error{}}. Pattern match on :reason to dispatch recovery:

case Rover.fetch(url, proxy: proxy_uri) do
  {:ok, result} ->
    handle(result)

  {:error, %Rover.Error{reason: :timeout}} ->
    Logger.warning("slow page: #{url}")
    :skip

  {:error, %Rover.Error{reason: :proxy}} ->
    retry_direct(url)

  {:error, %Rover.Error{reason: :selector_timeout, message: m}} ->
    Logger.error("page never rendered: #{m}")
    :skip
end

Reasons: :timeout, :navigation, :selector_not_found, :selector_timeout, :evaluation, :proxy, :invalid_argument, :runtime, :shutdown, :port_died.

Capability summary

Operation	Elixir	Notes
Navigate	`Rover.navigate(b, url)`	Waits for `LoadStatus::Complete`.
Content	`Rover.content(b)`	Rendered `outerHTML`.
Title	`Rover.title(b)`
Wait	`Rover.wait_for(b, "sel", timeout: 5_000)`	Polls via `querySelector`.
Evaluate JS	`Rover.evaluate(b, "1 + 2")`	Round-trips strings, numbers, arrays, maps.
Extract text	`Rover.get_text(b, "h1")`	Also `get_texts`, `get_attribute`.
Click / fill	`Rover.click(b, "sel")`	Dispatches real mouse events at element centre.
Hover / select	`Rover.hover(b, ".tip")`	`select_option` fires change events.
Screenshot	`Rover.screenshot(b, format: :png)`	PNG or JPEG.
Cookies	`Rover.get_cookies(b) / set_cookie / clear_*`	Direct via Servo's `SiteDataManager`.

Architecture

The shape

┌─ BEAM ──────────────────────────────────────────────────────────┐
│                                                                 │
│  Your Supervisor                                                │
│      │                                                          │
│  Rover.Browser (GenServer)                                      │
│      │ owns a Port with `packet: 4`                             │
│      │   length-prefixed MessagePack frames                     │
└──────┼──────────────────────────────────────────────────────────┘
       │ stdin/stdout                                             
       │ stderr → Logger                                          
┌──────▼──────────────────────────────────────────────────────────┐
│  rover_runtime (Rust, one OS process per browser)               │
│    main thread: read frame → dispatch → write reply             │
│    owns: Servo instance + one WebView + SoftwareRenderingContext│
└─────────────────────────────────────────────────────────────────┘

Every browser is:

An Elixir Rover.Browser GenServer under your supervision tree.
A dedicated rover_runtime OS subprocess, spawned via Port with packet: 4 framing.
Inside that subprocess: one Servo instance, one WebView, one SoftwareRenderingContext.

Elixir and Rust speak length-prefixed MessagePack over stdin/stdout. Stderr is not merged — it goes to the BEAM's stderr for log capture.

Why a separate OS process per browser?

This was the main design decision, and it was the second architecture we tried. The short version: Servo has process-global state that makes multiple Servo instances in one address space either broken or dangerous, and the cleanest way to isolate it is the same way Firefox and Chrome do — put each browser in its own process.

What we tried first: in-process NIF, many Servos

The obvious Elixir design is a Rustler NIF: one BEAM process, multiple Servo instances, each with its own proxy config. That's how BrowseServo ships (though BrowseServo uses a singleton engine and doesn't support per-instance proxies — which is exactly the gap we were trying to fix).

Looking at Servo's source, the first problem surfaces immediately. In components/servo/servo.rs:

fn new(builder: ServoBuilder) -> Self {
    // Global configuration options, parsed from the command line.
    let opts = builder.opts.map(|opts| *opts);
    opts::initialize_options(opts.unwrap_or_default());
    …
}

And components/config/opts.rs:

pub fn initialize_options(opts: Opts) {
    OPTIONS.set(opts).expect("Already initialized");
}

OnceLock::set().expect(...) panics on the second call. Creating a second Servo in the same process panics — so multi-Servo-in-one-NIF is not possible without patching Servo.

A one-line fix (swap .set().expect() for .get_or_init()) gets past that, but the deeper audit uncovered more:

script::init() (called unconditionally from Servo::new()) initialises SpiderMonkey: DisableJitBackend, RegisterProxyHandlers, InitAllStatics, InitializeMemoryReporter. SpiderMonkey is designed to init exactly once per process. Running it twice isn't supported and would wander into undefined behaviour inside the JS engine.
servo_config::prefs::set() writes to a global RwLock<Preferences>. The ProxyConnector reads pref!(network_http_proxy_uri) once, at HttpState construction. HttpState is shared across every WebView of a Servo instance, so proxy config is per-Servo-instance at best even if we could create multiple Servos.

So Option A (patch Servo, make it multi-tenant in one process) turned into "patch SpiderMonkey init semantics and hope class-vtable re-registration is harmless." A change where the failure mode is silent heap corruption.

The pivot: one process per browser

Option B — what Rover actually does — is to treat each Servo as its own OS process. The Elixir side spawns rover_runtime via Port; the Rust side calls Servo::new() exactly once in main(), before entering the IPC loop. Every constraint Servo has about "once per process" is automatically satisfied, because every browser is a process.

That gives us:

Isolation by construction.OPTIONS, Preferences, SpiderMonkey statics, the cookie jar, the DNS cache, connection pools — all live inside one process. Nothing leaks between browsers.
Per-instance proxy. Set Preferences.network_http_proxy_uri in main(), build the Servo, you're done. No shared state to worry about.
Crash containment. A renderer segfault takes down its OS process. The Port reports the exit status, Rover.Browser raises a typed :port_died, the supervisor restarts it. The BEAM never faults.
Unmodified upstream Servo. Rover depends on Servo as a regular Rust path dep with no patches. We can cargo update when Servo releases a new version and pick up fixes without re-doing a fork.

The cost is an IPC boundary. Every command is a Port.command (Elixir) → stdin write (Rust) → rmp-serde decode → dispatch → reply write → Msgpax.unpack! (Elixir). That adds hundreds of microseconds per command. On a browser that takes hundreds of milliseconds to paint a page, this doesn't matter. If it ever does, the architecture admits an in-process design later without breaking the public API — but we'd need SpiderMonkey to gain multi-tenant support first, and that's not our fight.

Why MessagePack (not JSON, not ETF)

The wire format only needs to carry structured values, some strings, and the occasional large binary (screenshots). Three options:

JSON. Universal but ~1.5× larger, slow to parse at size, and represents binaries as Base64 — which doubles screenshot payloads again.
Erlang External Term Format (ETF). Zero parsing on the Elixir side, but pulling it into Rust means vendoring an ETF encoder. Extra surface area for little gain.
MessagePack. Compact, fast, first-class binary type, mature libraries on both sides (rmp-serde in Rust, Msgpax in Elixir). Port handles framing via packet: 4, so we only ever deal with payload bytes.

MessagePack won because the framing problem is already solved by Port, and the binary type means screenshots aren't paying a Base64 tax.

Why not a pool of browsers per proxy from day one?

The plan calls for Rover.Pool — a NimblePool keyed by proxy URI. It's deferred because:

The interesting problem is proving isolation works. A pool is scaffolding on top. Getting the per-instance Servo story right first means the pool has a solid base.
Cheap to add later. NimblePool already knows how to manage long-lived workers; wrapping Rover.Browser is ~50 lines of Elixir.

For 0.1, start one Rover.Browser per proxy config and reuse it, or use Rover.fetch/2 and pay the startup cost each time (~100–500ms on a warm binary).

Why Servo, not Chromium / WebKit / Firefox?

Brief answer: it's the only mature-ish browser engine that builds as a library. Chromium has CEF but it's a monster; WebKit requires GObject/GTK or Objective-C depending on where you build; Gecko has no embedder story. Servo was designed from day one to be embeddable, ships as a regular Rust crate, and its API surface (Servo, WebView, evaluate_javascript, take_screenshot, SiteDataManager) maps naturally onto what a browser automation library wants.

It also happens to be written in Rust, which means the path from Servo APIs to BEAM-safe values goes through serde and rmp-serde. No unsafe FFI boundary to worry about; no "what happens when Rust panics" question (the subprocess exits, we report it as :port_died, the supervisor deals).

The downside is Servo's browser feature set isn't Chrome's. Several web platform features are incomplete — Web Components is the big one. For the use cases Rover targets (scraping server-rendered sites, SPAs with standard React/Vue output, filling forms) this hasn't been a problem. For a pixel-perfect Chrome replacement, use a headless Chrome driver.

Public API surface

Rover
├─ fetch/2                       ─ one-shot
├─ start_link/1                  ─ long-lived (delegates to Rover.Browser)
├─ stop/2
├─ navigate/3, current_url/1, content/1, title/1
├─ wait_for/3
├─ evaluate/2, get_text/2, get_texts/2, get_attribute/3
├─ click/2, fill/3, hover/2, select_option/3
├─ screenshot/2
└─ get_cookies/1, set_cookie/2, clear_cookies/1

Rover.Browser                    ─ GenServer
Rover.Result                     ─ struct returned by fetch/2
Rover.Error                      ─ exception with typed :reason atoms
Rover.Protocol                   ─ internal — wire format (don't depend on it)
Rover.Runtime                    ─ internal — binary path resolution

Not yet

Rover.Pool — recyclable browsers keyed by proxy.
[:rover, :fetch, :*] telemetry events.
Download capture / file interception.
Multiple WebViews per engine — deliberately out of scope. One browser = one page. For concurrent pages, start multiple browsers.

License

MPL-2.0 — matches Servo.