bb_camera_libcamera
A Beam Bots camera driver for Raspberry Pi
cameras — the CSI ribbon-cable kind — capturing MJPEG from libcamera through a
supervised rpicam-vid process. Boards with more than one camera connector,
such as the Pi 5, are supported: declare one sensor per camera.
It is a BB.Sensor and a BB.Camera, declared like any other sensor:
topology do
link :head do
sensor :eye, {BB.Camera.Libcamera, camera: 0, width: 1280, height: 720}
end
end
Consumers go through bb_camera,
which shares one capture between all of them. Two browser tabs previewing the
same camera produce one rpicam-vid, not two.
Frames never travel on the robot's pubsub. That carries capture lifecycle
only — BeginCapture and EndCapture. Frame bytes go over bb_camera's
bounded media interface.
Installation
def deps do
[
{:bb, "~> 0.33"},
{:bb_camera, "~> 0.1"},
{:bb_camera_libcamera, "~> 0.1"}
]
end
Working on this driver from a bb_workspace checkout instead? BB_VERSION=local
resolves bb, bb_camera and bb_camera_ffmpeg from the sibling checkouts
rather than from Hex.
Requirements
rpicam-vidon$PATH, fromrpicam-apps. Raspberry Pi OS ships it (older releases call itlibcamera-vid; pass that path as:rpicam_vid). A Nerves system needs libcamera and rpicam-apps in its Buildroot configuration.- A camera the Pi has detected at boot.
rpicam-vid --list-camerasis the check: if a camera is not listed there, nothing here can open it. - A C compiler at build time, for MuonTrap's wrapper program — see
bb_camera_ffmpeg. Nerves toolchains have one. - Not ffmpeg. This driver uses
bb_camera_ffmpegfor its capture machinery — MuonTrap, flow control, JPEG framing — but never runs ffmpeg.
Choosing a camera
:camera takes an index or a libcamera camera id. From IEx on the Pi:
BB.Camera.Libcamera.Devices.list()
#=> {:ok,
#=> [
#=> %{index: 0, model: "imx219", description: "3280x2464 10-bit RGGB",
#=> id: "/base/axi/pcie@1000120000/rp1/i2c@88000/imx219@10"},
#=> %{index: 1, model: "imx219", description: "3280x2464 10-bit RGGB",
#=> id: "/base/axi/pcie@1000120000/rp1/i2c@80000/imx219@10"}
#=> ]}
or rpicam-vid --list-cameras from a shell, which also prints each sensor's
modes.
The id is the sensor's device-tree path, so it names the connector the
camera is plugged into. That is the only thing telling two identical cameras
apart, and it does not move. An index does: it is a position in that list, and
a camera that fails to probe at boot shifts the ones after it. With one camera,
camera: 0 is fine. With two, use ids:
link :head do
sensor :left_eye, {BB.Camera.Libcamera,
camera: "/base/axi/pcie@1000120000/rp1/i2c@88000/imx219@10",
width: 1280, height: 720, buffer_count: 2}
sensor :right_eye, {BB.Camera.Libcamera,
camera: "/base/axi/pcie@1000120000/rp1/i2c@80000/imx219@10",
width: 1280, height: 720, buffer_count: 2}
end
To find out which connector is which, cover one lens and look at the frames.
An id is resolved to an index by listing the cameras afresh at the start of every capture, which takes about a tenth of a second.
Two cameras and CMA
Each camera's frame buffers come from the kernel's contiguous memory pool
(CMA). With too little of it, the second camera to start fails — rpicam-vid
exits with status 255 and prints this on the console:
dmaHeap allocation failure for rpicam-apps5
ERROR: *** failed to allocate capture buffers for stream ***
Measured on a Pi 5 under Nerves with 64 MiB of CMA and two imx219s:
| Each camera | buffer_count |
Both capture? |
|---|---|---|
| 640x480 | default | yes |
| 1280x720 | default | no — the second fails |
| 1280x720 | 2 |
yes, 30 fps each |
buffer_count: 2 costs headroom: with fewer buffers, a frame the encoder is
late collecting is a frame dropped. The other fix is more CMA — check
CmaTotal in /proc/meminfo, and raise it with the cma= kernel argument or
the cma- parameter of the vc4-kms-v3d overlay, whichever your system uses.
Smoke test on a Pi
From IEx on the Pi, with the package loaded:
defmodule PiCam do
use BB
settings do
name :pi_cam
end
topology do
link :head do
sensor :eye, {BB.Camera.Libcamera, camera: 0, width: 1280, height: 720}
end
end
end
{:ok, _pid} = PiCam.start_link([])
BB.subscribe(PiCam, [:sensor, :head, :eye])
{:ok, session} = BB.Camera.Media.open(PiCam, :eye, format: :jpeg)
{:ok, frame} = BB.Camera.Media.next(session, 10_000)
{frame.width, frame.height, byte_size(frame.data)}
#=> {1280, 720, 17003}
File.write!("/tmp/camera.jpg", frame.data)
BB.Camera.Media.close(session)
flush()
#=> {:bb, [:sensor, :head, :eye], %BB.Message{payload: %BB.Message.Camera.BeginCapture{...}}}
#=> {:bb, [:sensor, :head, :eye], %BB.Message{payload: %BB.Message.Camera.EndCapture{reason: :no_subscribers, ...}}}
System.cmd("ps", []) while the session is open shows one rpicam-vid per
capturing camera, however many consumers there are, and none once the last
consumer has closed.
Options
| Option | Default | |
|---|---|---|
:camera |
0 |
Index or libcamera id, as Devices.list/1 reports them |
:width, :height |
640, 480 |
Any positive size. libcamera picks a sensor mode and the ISP scales |
:frame_rate |
30 |
A hertz unit, a number, or {numerator, denominator} |
:quality |
50 |
JPEG quality, 1 to 100 |
:hflip, :vflip |
false |
Mirror the image. Both together for a camera mounted upside down |
:buffer_count |
rpicam-vid's |
Frame buffers to allocate. Lower it to fit several cameras in CMA |
:rpicam_vid |
$PATH |
Path to the rpicam-vid executable |
:max_frame_bytes |
4 MiB | Largest frame accepted before the stream is resynchronised |
:drop_queue_len |
8 |
Shed frames while more reads than this are waiting |
:stdio_window |
256 KiB | Bytes of output rpicam-vid may have in flight unacknowledged |
:start_timeout_ms |
10000 |
Fail capture when no frame has arrived by then |
:stop_timeout_ms |
2000 |
How long rpicam-vid gets after SIGTERM before SIGKILL |
How it holds together
MuonTrap ──> rpicam-vid ──pipe──> Capture ──frames──> BB.Camera.Hub ──> consumers
│
└─ lifecycle ──> BB.PubSub
BB.Camera.Libcamera.Commandbuilds therpicam-vidargument vector and validates it. No shell ever parses it.BB.Camera.Libcamera.Deviceslists cameras and resolves an id to the indexrpicam-vidwants.BB.Camera.Libcamera.Backendrunsrpicam-vidunder MuonTrap.bb_camera_ffmpeg'sCapturedoes the rest, as it does forbb_camera_v4landbb_camera_avf: flow control, JPEG framing, holding the first frame until the hub has been told capture started, and confirming teardown.
Measured on a Pi 5
Two imx219s under Nerves, both at 1280x720 and 30 fps with buffer_count: 2:
- first frame from both cameras within 0.8 s of the consumers opening;
- 30 fps sustained from each;
- about 17% of one core per
rpicam-vid, which is libjpeg encoding; rpicam-videxits within 10 ms ofSIGTERM, and nothing is left running after the last consumer closes.
Limitations
- Raspberry Pi only, in practice:
rpicam-vidis built for the Pi's camera stack. A USB webcam on a Pi wantsbb_camera_v4l. - Every frame is a software JPEG encode. The Pi's camera produces raw Bayer data, not JPEG, so there is no passthrough path, and CPU grows with frame size and rate.
- Timestamps are arrival times, not exposure times, and are not suitable for
synchronisation. The MJPEG stream on
rpicam-vid's stdout carries no timestamps; a frame is stamped when the read completing it reached the BEAM. Two cameras' frames cannot be paired by these stamps. - The frame rate is reported as requested. What the frames actually did is
measured by
BB.Camera.Huband published inBB.Message.Camera.CaptureState. - No camera controls beyond what is here: exposure, gain, white balance and the rest are left to libcamera's automatic algorithms.
- A failed capture is not retried. Reopening is the consumer's decision, taken by subscribing again.
- Changing an option that reaches
rpicam-vidrestarts it, withEndCaptureandBeginCapturearound the change. rpicam-vid's diagnostics go to the VM's console, not into the error. libcamera's own logging is held atWARN; seeBB.Camera.Libcamera.Backend.
Testing
BB_VERSION=local mix deps.get
BB_VERSION=local mix test # no camera, no rpicam-vid needed
BB_VERSION=local mix test --include rpicam # on a Pi with a camera attached
BB_VERSION=local mix check --no-retry
The default suite drives the driver through bb_camera_ffmpeg's stub backend,
and resolves camera ids against test/support/fake_rpicam_vid, which replays
a real Pi 5's camera listing. No test sleeps.
Licence
Apache-2.0. See LICENSES/.