Petrex.Oban

Analyses Oban workflows as Petri nets, using Petrex. It takes a plain description of a workflow — jobs, their dependencies, and the queue limits they run under — and answers two questions before the workflow runs.

Will it finish? A dependency cycle is the one shape that never completes however long it runs. Oban accepts such a workflow and leaves those jobs pending; this names the jobs in each cycle.

Petrex.Oban.cycles(%{
  jobs: [
    %{id: :fetch, deps: [:index]},
    %{id: :parse, deps: [:fetch]},
    %{id: :index, deps: [:parse]}
  ]
})
#=> [[:fetch, :index, :parse]]

How much of each queue will it use? The highest number of jobs a queue ever runs at once, over every possible interleaving rather than one sampled run. A limit well above that figure is workers that will sit idle for this workflow; a queue that reaches its limit is where more capacity buys time.

Petrex.Oban.max_concurrency(%{
  jobs: [
    %{id: :fetch, deps: [], queue: :network},
    %{id: :thumbnail, deps: [:fetch], queue: :cpu},
    %{id: :transcode, deps: [:fetch], queue: :cpu}
  ],
  queues: %{network: 5, cpu: 8}
})
#=> {:ok, %{network: 1, cpu: 2}}

Eight CPU workers for a workflow that never runs more than two.

Both answers come from the analyser, so both hold for every interleaving, and both say when a state space was too large to finish: a {:partial, counts} figure is a lower bound, not a verdict. from_workflow/1 returns the net itself, for the rest of Petrex.Analysis.

The model

Each job becomes waiting, running and done places with start and finish transitions. A queue becomes a place holding as many tokens as its limit, taken while a job runs and returned when it finishes, plus a place counting the jobs it is running — the one whose bound max_concurrency/2 reports. A dependency is read rather than consumed, so any number of jobs can depend on the same one.

The nets this package generates are cross-checked against TINA and LoLA, like the core package's own.

What it does not model

A job holds a queue slot only while it runs, and a running job can always finish, so a positive queue limit slows a workflow down but cannot deadlock it. Deadlock means an unsatisfiable dependency or a queue whose limit is zero.

Nothing here covers what Oban does around the jobs: retries, timeouts, the difference between a job failing and a job finishing, or workers competing for a queue across nodes.

Oban

This package does not depend on Oban. Producing the description from an Oban Pro workflow is a few lines in an application that already has Oban; the module documentation shows one.

Installation

def deps do
  [{:petrex_oban, "~> 1.0"}]
end

It brings in petrex, which has no runtime dependencies of its own.