NetRunner

Safe OS process execution for Elixir. Zero zombie processes, NIF-based backpressure, PTY support, and cgroup isolation.

NetRunner combines NIF-based async I/O (enif_select) with a persistent shepherd binary that watches over child processes. Three layers of cleanup guarantee zero zombies under every failure mode — BEAM crash, GenServer crash, or resource leak.

Installation

def deps do
  [
    {:net_runner, "~> 1.3"}
  ]
end

Requires a C compiler (gcc or clang) and make.

Quick Start

# Run a command, collect output
{output, 0} = NetRunner.run(~w(echo hello))
# => {"hello\n", 0}

# Pipe data through stdin
{output, 0} = NetRunner.run(~w(cat), input: "from elixir")
# => {"from elixir", 0}

# Nonzero exit status
{"", 1} = NetRunner.run(~w(false))

# Multi-word commands with arguments
{output, 0} = NetRunner.run(["sh", "-c", "echo $HOME"])

Streaming

Lazy, demand-driven streams with backpressure — data stays in the OS pipe buffer until you consume it, so you can stream gigabytes without OOM:

# Stream stdout chunks with backpressure
NetRunner.stream!(~w(cat /usr/share/dict/words))
|> Stream.filter(&String.starts_with?(&1, "elixir"))
|> Enum.to_list()

# Pipe input through a command
NetRunner.stream!(~w(tr a-z A-Z), input: "hello world")
|> Enum.join()
# => "HELLO WORLD"

# Sort lines
NetRunner.stream!(~w(sort), input: "cherry\napple\nbanana\n")
|> Enum.join()
# => "apple\nbanana\ncherry\n"

# Process large files without loading into memory
File.stream!("huge.csv")
|> NetRunner.stream!(~w(grep ERROR))
|> Stream.each(&IO.write/1)
|> Stream.run()

# Non-raising variant
{:ok, stream} = NetRunner.stream(~w(sort), input: "c\nb\na\n")
Enum.to_list(stream)
# => ["a\nb\nc\n"]

Timeouts and Limits

# Kill process after 500ms
{:error, :timeout} = NetRunner.run(~w(sleep 100), timeout: 500)

# Cap output size — kills process if exceeded
{:error, {:max_output_exceeded, _partial}} =
  NetRunner.run(["sh", "-c", "yes"], max_output_size: 1000)

# Custom kill escalation: SIGTERM → wait 2s → SIGKILL
NetRunner.run(~w(my_server), kill_timeout: 2000, timeout: 10_000)

Input Validation and Error Returns

run/2 and stream/2 return tagged errors for bad input instead of crashing. NUL bytes inside cmd or args are rejected early (they are undefined in argv on the C side).

# Empty executable
{:error, {:invalid_cmd, _}} = NetRunner.run([""])

# NUL byte in an argument
{:error, {:invalid_args, _}} = NetRunner.run(["echo", "he\0llo"])

# Same behaviour for streaming
{:error, {:invalid_args, _}} = NetRunner.stream(["echo", "he\0llo"])

# Unknown signal atoms come back as tagged errors, not raises
{:error, :unknown_signal} = NetRunner.Signal.resolve(:sigwhatever)
{:error, :unknown_signal} = NetRunner.Signal.resolve(99)

Working with Binary Output

stdout is delivered as a BEAM binary, not a String. It is safe to pass bytes containing NUL, high-bit, or anything else through the pipeline.

# NUL bytes round-trip unchanged
{out, 0} = NetRunner.run(["sh", "-c", ~S|printf 'a\0b\0c'|])
byte_size(out)          # => 5
out == "a\0b\0c"        # => true

# UTF-8 boundaries straddle chunks fine — just concatenate and then
# decode.
"héllo\n" =
  NetRunner.stream!(~w(echo héllo))
  |> Enum.join()

Process API

For fine-grained control over the OS process lifecycle:

alias NetRunner.Process, as: Proc

# Start a process
{:ok, pid} = Proc.start("cat", [])

# Write to stdin
:ok = Proc.write(pid, "hello world")
:ok = Proc.close_stdin(pid)

# Read from stdout (blocks until data available)
{:ok, "hello world"} = Proc.read(pid)
:eof = Proc.read(pid)

# Wait for exit
{:ok, 0} = Proc.await_exit(pid)

Incremental reads and writes

{:ok, pid} = Proc.start("cat", [])

# Write in chunks — useful for feeding large data
:ok = Proc.write(pid, "chunk 1 ")
:ok = Proc.write(pid, "chunk 2 ")
:ok = Proc.write(pid, "chunk 3")
:ok = Proc.close_stdin(pid)

# Read comes back in whatever chunks the OS delivers
{:ok, data} = Proc.read(pid)
# data => "chunk 1 chunk 2 chunk 3" (may come in multiple reads)

Signals and process groups

{:ok, pid} = Proc.start("sleep", ["100"])
:ok = Proc.kill(pid, :sigterm)
{:ok, 143} = Proc.await_exit(pid)  # 128 + SIGTERM(15)

# Signals kill the entire process group (catches grandchildren)
{:ok, pid} = Proc.start("sh", ["-c", "sleep 100 & sleep 100 & wait"])
:ok = Proc.kill(pid, :sigkill)
# All three processes (sh + both sleeps) are killed

Supported signals: :sigterm, :sigkill, :sigint, :sighup, :sigusr1, :sigusr2, :sigstop, :sigcont, :sigquit, :sigpipe.

Checking process state

{:ok, pid} = Proc.start("sleep", ["10"])

Proc.alive?(pid)   # => true
Proc.os_pid(pid)   # => 12345 (the actual OS PID)

Proc.kill(pid, :sigkill)
Proc.await_exit(pid)

Proc.alive?(pid)   # => false

Stats

Every process tracks I/O statistics automatically:

{:ok, pid} = Proc.start("cat", [])
Proc.write(pid, "hello")
Proc.close_stdin(pid)
Proc.read(pid)
Proc.await_exit(pid)

stats = Proc.stats(pid)
stats.bytes_in     # => 5       (bytes written to stdin)
stats.bytes_out    # => 5       (bytes read from stdout)
stats.bytes_err    # => 0       (bytes read from stderr, :consume mode)
stats.read_count   # => 1       (number of read calls)
stats.write_count  # => 1       (number of write calls)
stats.duration_ms  # => 3       (wall-clock time)
stats.exit_status  # => 0       (exit code)

Tying an OS process to an owner

If the calling process crashes, the OS process it launched should go with it. Pass :owner to have the Process GenServer monitor a pid; on :DOWN it SIGKILLs the child and stops cleanly. NetRunner.stream/2 does this automatically with self().

# Spawn a long-lived command tied to the caller
parent = self()

spawn(fn ->
  {:ok, pid} = Proc.start("sleep", ["30"], owner: self())
  send(parent, {:os_pid, Proc.os_pid(pid)})
  exit(:boom)   # caller dies → Process SIGKILLs sleep, stops itself
end)

Per-call kill timeout

Tune the SIGTERM→SIGKILL escalation window per-process. Useful when a command has its own graceful shutdown hook you want to honour, or when you need a fast hard-kill.

# Give my_server 10s to drain on SIGTERM before SIGKILL
{:ok, pid} = Proc.start("my_server", [], kill_timeout: 10_000)

# Or make it effectively immediate for tests
{:ok, pid} = Proc.start("sleep", ["100"], kill_timeout: 100)

PTY Mode

Run commands with a pseudo-terminal for programs that require a TTY. PTY mode is designed for interactive and long-running programs — shells, REPLs, curses apps.

# Programs see a real terminal
{:ok, pid} = Proc.start("python3", ["-c", "import sys; print(sys.stdout.isatty())"], pty: true)
{:ok, data} = Proc.read(pid)
# data =~ "True"

# Interactive REPL
{:ok, pid} = Proc.start("python3", ["-i"], pty: true)
Proc.write(pid, "print(1 + 2)\n")
{:ok, data} = Proc.read(pid)

# Resize the terminal window
Proc.set_window_size(pid, 40, 120)

# Clean up when done (PTY doesn't support independent stdin close)
Proc.kill(pid, :sigkill)
Proc.await_exit(pid)

PTY caveats

PTY mode differs from pipe mode in important ways:

• No independent stdin close — the PTY is a single bidirectional FD. Use kill/2 to terminate.
• Echo — the terminal echoes input back by default, so reads include what you wrote.
• Fast-exiting commands — if a command exits before you call read/1 (e.g., in iex), the PTY buffer may be lost. PTY mode is meant for long-running programs. For simple commands, use pipe mode (the default).
• Line buffering — the terminal line discipline buffers input until \n by default.

# DON'T: Use PTY for simple commands in iex (data may be lost)
{:ok, pid} = Proc.start("echo", ["hi"], pty: true)
# ... time passes while you type ...
Proc.read(pid)  # => :eof (too late, PTY torn down)

# DO: Use pipe mode (default) for simple commands
{:ok, pid} = Proc.start("echo", ["hi"])
{:ok, "hi\n"} = Proc.read(pid)

# DO: Use PTY for interactive programs that need a terminal
{:ok, pid} = Proc.start("bash", [], pty: true)
Proc.write(pid, "echo hello\n")
{:ok, data} = Proc.read(pid)  # works — bash stays alive

Daemon Mode

Run long-lived processes under a supervision tree. Automatically drains stdout/stderr to prevent pipe blocking:

# In your supervisor
children = [
  {NetRunner.Daemon,
   cmd: "redis-server",
   args: ["--port", "6380"],
   on_output: :log,
   name: MyApp.Redis}
]

Supervisor.start_link(children, strategy: :one_for_one)

# Interact with the daemon
NetRunner.Daemon.os_pid(MyApp.Redis)   # => 12345
NetRunner.Daemon.alive?(MyApp.Redis)   # => true
NetRunner.Daemon.write(MyApp.Redis, "PING\r\n")

Output handling options:

• :discard (default) — silently consume output to prevent pipe blocking
• :log — log each chunk via Logger.info
• fun/1 — custom callback for each chunk

# Custom output handler
{:ok, daemon} = NetRunner.Daemon.start_link(
  cmd: "tail",
  args: ["-f", "/var/log/system.log"],
  on_output: fn chunk -> MyApp.LogIngester.ingest(chunk) end
)

Graceful shutdown: on terminate/2, sends SIGTERM, waits 5 seconds, then SIGKILL.

cgroup Support (Linux)

Isolate child processes in a cgroup v2 hierarchy for resource control:

{:ok, pid} = Proc.start("my_worker", [],
  cgroup_path: "net_runner/job_123"
)

The shepherd creates the cgroup directory, moves the child into it before execvp, and cleans up on exit (kills all processes via cgroup.kill, then removes the directory). Spawns fail closed with {:error, {:shepherd_error, msg}} when the cgroup cannot be created, the leaf already exists, the migration is refused, or cgroup.kill (Linux ≥ 5.14) is unavailable. No-op on macOS.

Command DSL

Bundle an executable, default args, and default options into a reusable %NetRunner.Command{}. Both run/2 and stream/2 accept it, and call-site options override the defaults.

alias NetRunner.Command

# Inline construction
cmd = Command.new("curl", ["-sS"], timeout: 30_000)
{body, 0} = NetRunner.run(cmd, args: ["https://example.com"])

# Extend at call time (args append; opts merge with runtime winning)
listing = Command.new("ls", ["-la"])
{out, 0} = NetRunner.run(listing, args: ["/tmp"])

# `defcommand` in your own module captures a reusable template:
defmodule MyCmds do
  use NetRunner.Command

  defcommand :curl, "curl", ["-sS", "--max-time", "30"]
  defcommand :echo, "echo"
end

{out, 0} = NetRunner.run(MyCmds.echo(["hi"]))
{:ok, stream} = NetRunner.stream(MyCmds.curl(["https://example.com"]))

Error Handling Cheatsheet

case NetRunner.run(["my_tool", arg], timeout: 5_000) do
  {output, 0} ->
    {:ok, output}

  {_partial, status} when status != 0 ->
    {:error, {:nonzero_exit, status}}

  {:error, :timeout} ->
    {:error, :took_too_long}

  {:error, {:max_output_exceeded, partial}} ->
    {:error, {:too_much_output, byte_size(partial)}}

  {:error, {:invalid_cmd, msg}} ->
    {:error, {:bad_cmd, msg}}

  {:error, {:invalid_args, msg}} ->
    {:error, {:bad_args, msg}}
end

Parallel Execution

Every NetRunner process is fully independent — no shared state, no singleton bottleneck:

# Process files in parallel
files
|> Task.async_stream(fn file ->
  {out, 0} = NetRunner.run(["ffprobe", "-hide_banner", file])
  {file, out}
end, max_concurrency: System.schedulers_online())
|> Enum.to_list()

# Fan-out pattern
urls
|> Task.async_stream(fn url ->
  {body, 0} = NetRunner.run(["curl", "-s", url], timeout: 30_000)
  body
end, max_concurrency: 20)
|> Enum.to_list()

Why NetRunner?

System.cmd uses Erlang ports which tie stdin/stdout lifecycle together, have no backpressure (mailbox flooding), and leave zombies when programs ignore stdin EOF. This was filed as ERL-128 and marked Won't Fix.

Options Reference

NetRunner.run/2

Unknown options raise ArgumentError (Keyword.validate!/2) instead of being silently ignored. stream!/2 accepts the same options minus :timeout, :max_output_size, and :output, and raises NetRunner.Error (with the original reason in :reason) on spawn failure or a mid-stream read error.

NetRunner.Process.start/3

Accepts all options above except :input, :input_buffer, :timeout, :max_output_size, :output, and :stderr :capture (use :consume + NetRunner.Process.stderr_tail/1). NetRunner.Process.shutdown/3 owns the SIGTERM→await→SIGKILL escalation for callers that manage processes directly.

Architecture

BEAM Process (NetRunner.Process GenServer)
    |
    |-- Port.open("priv/shepherd", [:nouse_stdio, :exit_status])
    |     |
    |     v
    |   Shepherd Binary (stays alive for child's lifetime)
    |     |-- fork() → child process (execvp)
    |     |-- Passes pipe FDs to BEAM via UDS + SCM_RIGHTS
    |     |-- poll() loop: watches UDS + signal pipe (SIGCHLD)
    |     |-- BEAM dies (POLLHUP) → SIGTERM → SIGKILL child
    |     |-- Child dies (SIGCHLD) → notify BEAM, exit
    |
    |-- NIF (enif_select on raw O_NONBLOCK FDs, normal schedulers)
    |     |-- Demand-driven backpressure via OS pipe buffers
    |
    v
NetRunner.Watcher (belt-and-suspenders with shepherd)
    |-- Monitors GenServer, kills OS process if GenServer crashes

Three layers of zombie prevention:

1. Shepherd — detects BEAM death via POLLHUP on UDS, kills child process group
2. Watcher — detects GenServer crash via Process.monitor, kills child via NIF
3. NIF resource destructor — closes FDs on GC, child sees broken pipe

Performance

Medians on an idle Apple M1 Max (10 cores), OTP 29, default VM flags:

Spawn is a one-time cost and it is two execs by design — the shepherd's is the zero-zombie guarantee. One fork+exec on this host measures 2532 µs via System.cmd/2 and 2655 µs via a raw Port, so ~5.3 ms is the floor for this architecture. See Architecture.

I/O after that is limited by the pipe, not by NetRunner. Reads are sized at exactly one pipe buffer (65 536 bytes); see ADR-9 in Decisions before changing that constant.

Every NIF call is a bounded, non-blocking syscall on a normal scheduler. Readiness comes from enif_select, integrated with BEAM's epoll/kqueue, so an idle process costs nothing. NetRunner used to run these calls on dirty IO schedulers; each call then paid a scheduler handoff, which on a machine whose dirty schedulers are contended cost milliseconds rather than nanoseconds and capped throughput at single-digit MiB/s. See ADR-6 in Decisions.

Absolute numbers are host-specific — re-measure on your own hardware before relying on them.

Documentation

• API Reference (HexDocs)
• Architecture — component diagrams, spawn sequence, zombie prevention
• Protocol — shepherd ↔ BEAM byte protocol spec
• Backpressure — how demand-driven I/O works
• Decisions — architecture decision records
• Comparison — feature matrix vs alternatives

Requirements

• Elixir ~> 1.17
• Erlang/OTP 27+
• C compiler (gcc or clang)
• make
• Linux or macOS

License

Apache-2.0 — see LICENSE.