Beam Bots Logo

bb_camera_libcamera

CI License: Apache 2.0 Hex version badge Hexdocs badge REUSE status Ask DeepWiki

A Beam Bots camera driver for Raspberry Pi cameras — the CSI ribbon-cable kind — capturing MJPEG from libcamera through a supervised rpicam-vid process. Boards with more than one camera connector, such as the Pi 5, are supported: declare one sensor per camera.

It is a BB.Sensor and a BB.Camera, declared like any other sensor:

topology do
  link :head do
    sensor :eye, {BB.Camera.Libcamera, camera: 0, width: 1280, height: 720}
  end
end

Consumers go through bb_camera, which shares one capture between all of them. Two browser tabs previewing the same camera produce one rpicam-vid, not two.

Frames never travel on the robot's pubsub. That carries capture lifecycle only — BeginCapture and EndCapture. Frame bytes go over bb_camera's bounded media interface.

Installation

def deps do
  [
    {:bb, "~> 0.33"},
    {:bb_camera, "~> 0.1"},
    {:bb_camera_libcamera, "~> 0.1"}
  ]
end

Working on this driver from a bb_workspace checkout instead? BB_VERSION=local resolves bb, bb_camera and bb_camera_ffmpeg from the sibling checkouts rather than from Hex.

Requirements

Choosing a camera

:camera takes an index or a libcamera camera id. From IEx on the Pi:

BB.Camera.Libcamera.Devices.list()
#=> {:ok,
#=>  [
#=>    %{index: 0, model: "imx219", description: "3280x2464 10-bit RGGB",
#=>      id: "/base/axi/pcie@1000120000/rp1/i2c@88000/imx219@10"},
#=>    %{index: 1, model: "imx219", description: "3280x2464 10-bit RGGB",
#=>      id: "/base/axi/pcie@1000120000/rp1/i2c@80000/imx219@10"}
#=>  ]}

or rpicam-vid --list-cameras from a shell, which also prints each sensor's modes.

The id is the sensor's device-tree path, so it names the connector the camera is plugged into. That is the only thing telling two identical cameras apart, and it does not move. An index does: it is a position in that list, and a camera that fails to probe at boot shifts the ones after it. With one camera, camera: 0 is fine. With two, use ids:

link :head do
  sensor :left_eye, {BB.Camera.Libcamera,
    camera: "/base/axi/pcie@1000120000/rp1/i2c@88000/imx219@10",
    width: 1280, height: 720, buffer_count: 2}

  sensor :right_eye, {BB.Camera.Libcamera,
    camera: "/base/axi/pcie@1000120000/rp1/i2c@80000/imx219@10",
    width: 1280, height: 720, buffer_count: 2}
end

To find out which connector is which, cover one lens and look at the frames.

An id is resolved to an index by listing the cameras afresh at the start of every capture, which takes about a tenth of a second.

Two cameras and CMA

Each camera's frame buffers come from the kernel's contiguous memory pool (CMA). With too little of it, the second camera to start fails — rpicam-vid exits with status 255 and prints this on the console:

dmaHeap allocation failure for rpicam-apps5
ERROR: *** failed to allocate capture buffers for stream ***

Measured on a Pi 5 under Nerves with 64 MiB of CMA and two imx219s:

Each camera buffer_count Both capture?
640x480 default yes
1280x720 default no — the second fails
1280x720 2 yes, 30 fps each

buffer_count: 2 costs headroom: with fewer buffers, a frame the encoder is late collecting is a frame dropped. The other fix is more CMA — check CmaTotal in /proc/meminfo, and raise it with the cma= kernel argument or the cma- parameter of the vc4-kms-v3d overlay, whichever your system uses.

Smoke test on a Pi

From IEx on the Pi, with the package loaded:

defmodule PiCam do
  use BB

  settings do
    name :pi_cam
  end

  topology do
    link :head do
      sensor :eye, {BB.Camera.Libcamera, camera: 0, width: 1280, height: 720}
    end
  end
end

{:ok, _pid} = PiCam.start_link([])

BB.subscribe(PiCam, [:sensor, :head, :eye])

{:ok, session} = BB.Camera.Media.open(PiCam, :eye, format: :jpeg)
{:ok, frame} = BB.Camera.Media.next(session, 10_000)

{frame.width, frame.height, byte_size(frame.data)}
#=> {1280, 720, 17003}

File.write!("/tmp/camera.jpg", frame.data)

BB.Camera.Media.close(session)
flush()
#=> {:bb, [:sensor, :head, :eye], %BB.Message{payload: %BB.Message.Camera.BeginCapture{...}}}
#=> {:bb, [:sensor, :head, :eye], %BB.Message{payload: %BB.Message.Camera.EndCapture{reason: :no_subscribers, ...}}}

System.cmd("ps", []) while the session is open shows one rpicam-vid per capturing camera, however many consumers there are, and none once the last consumer has closed.

Options

Option Default
:camera 0 Index or libcamera id, as Devices.list/1 reports them
:width, :height 640, 480 Any positive size. libcamera picks a sensor mode and the ISP scales
:frame_rate 30 A hertz unit, a number, or {numerator, denominator}
:quality 50 JPEG quality, 1 to 100
:hflip, :vflip false Mirror the image. Both together for a camera mounted upside down
:buffer_count rpicam-vid's Frame buffers to allocate. Lower it to fit several cameras in CMA
:rpicam_vid $PATH Path to the rpicam-vid executable
:max_frame_bytes 4 MiB Largest frame accepted before the stream is resynchronised
:drop_queue_len 8 Shed frames while more reads than this are waiting
:stdio_window 256 KiB Bytes of output rpicam-vid may have in flight unacknowledged
:start_timeout_ms 10000 Fail capture when no frame has arrived by then
:stop_timeout_ms 2000 How long rpicam-vid gets after SIGTERM before SIGKILL

How it holds together

MuonTrap ──> rpicam-vid ──pipe──> Capture ──frames──> BB.Camera.Hub ──> consumers
                                                            │
                                                            └─ lifecycle ──> BB.PubSub

Measured on a Pi 5

Two imx219s under Nerves, both at 1280x720 and 30 fps with buffer_count: 2:

Limitations

Testing

BB_VERSION=local mix deps.get
BB_VERSION=local mix test                   # no camera, no rpicam-vid needed
BB_VERSION=local mix test --include rpicam  # on a Pi with a camera attached
BB_VERSION=local mix check --no-retry

The default suite drives the driver through bb_camera_ffmpeg's stub backend, and resolves camera ids against test/support/fake_rpicam_vid, which replays a real Pi 5's camera listing. No test sleeps.

Licence

Apache-2.0. See LICENSES/.