Beam Bots Logo

bb_camera_v4l

CI License: Apache 2.0 Hex version badge Hexdocs badge REUSE status Ask DeepWiki

A Beam Bots camera driver for Linux V4L2 devices — webcams, capture cards, CSI cameras — capturing MJPEG through a supervised ffmpeg process.

It is a BB.Sensor and a BB.Camera, declared like any other sensor:

topology do
  link :head do
    sensor :eye, {BB.Camera.V4L, device: "/dev/video0", width: 1280, height: 720}
  end
end

Consumers go through bb_camera, which shares one capture between all of them. Two browser tabs previewing the same camera produce one ffmpeg, not two.

Frames never travel on the robot's pubsub. That carries capture lifecycle only — BeginCapture and EndCapture. Frame bytes go over bb_camera's bounded media interface.

Status

Not yet published. bb_camera_v4l and its bb_camera dependency are declared the way every package here declares bb — from Hex by default, and from the sibling checkout under bb_workspace when BB_VERSION=local is set.

Requirements

Smoke test on a Debian laptop

This walks from nothing to two consumers sharing one capture. The first half needs no camera at all, which is deliberate: if the synthetic source works and the real one does not, the problem is the camera or its permissions, and you have narrowed it down without guessing.

1. Install the tools

sudo apt update
sudo apt install ffmpeg v4l-utils

v4l-utils is not needed at runtime. It is how you ask the camera what it can do — which this driver deliberately does not guess at on your behalf.

2. Prove the pipeline works without a camera

From the package directory:

BB_VERSION=local mix deps.get
BB_VERSION=local iex -S mix

BB_VERSION=local is needed on both commands. It is read each time mix runs, not stored in the lockfile, so a shell that resolved bb from the sibling checkout to fetch will resolve it from Hex to compile unless you say so again.

Paste this robot into the session. It captures one of ffmpeg's own synthetic sources: real ffmpeg, real JPEG encoder, real pipe, real framing — and no hardware.

defmodule SynthCam do
  use BB

  settings do
    name :synth_cam
  end

  topology do
    link :head do
      sensor :eye, {BB.Camera.V4L, lavfi: "testsrc2", width: 320, height: 240, frame_rate: 10}
    end
  end
end

{:ok, _pid} = SynthCam.start_link([])

{:ok, session} = BB.Camera.Media.open(SynthCam, :eye, format: :jpeg)
{:ok, frame} = BB.Camera.Media.next(session, 10_000)

{frame.width, frame.height, byte_size(frame.data)}
#=> {320, 240, 8625}

File.write!("/tmp/frame.jpg", frame.data)
BB.Camera.Media.close(session)

Then, in another terminal, check it really is an image:

file /tmp/frame.jpg
#=> /tmp/frame.jpg: JPEG image data, JFIF standard 1.02, ... 320x240, components 3
xdg-open /tmp/frame.jpg

If that failed, nothing below will work, and the problem is ffmpeg rather than your camera.

3. Find the camera

v4l2-ctl --list-devices
Integrated Camera: Integrated C (usb-0000:00:14.0-8):
        /dev/video0
        /dev/video1
        /dev/media0

A single webcam usually claims two /dev/video* nodes. Only the first is the capture device; the second is a metadata node and will fail to produce frames. Confirm which is which:

v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'MJPG' (Motion-JPEG, compressed)
                Size: Discrete 1280x720
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 640x480
                        Interval: Discrete 0.033s (30.000 fps)
        [1]: 'YUYV' (YUYV 4:2:2)
                Size: Discrete 640x480
                        Interval: Discrete 0.033s (30.000 fps)

A node that prints nothing under Video Capture is not the one you want.

Note what it supports. If MJPG is listed, pick a size and rate from under it and the camera's own encoder does the work. If only YUYV is listed, pass input_format: :yuyv422 and ffmpeg encodes instead, which costs real CPU per frame.

This driver does not enumerate any of this for you, and does not pretend to know it. Ask for what you want; the frames report what you got.

4. Check permissions

Ask the only question that matters — can you read it?

test -r /dev/video0 && echo readable || echo "no access"

If that says readable, you are done; skip to the next step.

On a desktop Debian install it usually does, even though the node is owned by the video group and you are not in it:

ls -l /dev/video0
#=> crw-rw----+ 1 root video 81, 0 Nov 17 20:11 /dev/video0

The + is the giveaway: systemd-logind puts an ACL on the device granting the user of the active local seat access to it. getfacl /dev/video0 shows the entry. That is why adding yourself to video is often cited and often unnecessary.

It is necessary when there is no local seat — over SSH, from a service, in a container, or on a headless board:

sudo usermod -aG video "$USER"

Then log out and back in. A new shell is not enough; the group list is attached to your login session, and id -nG is the check.

A camera you have no permission to open fails the same way as one that is not there: ffmpeg prints Permission denied on the console and capture ends. That message is the fastest way to tell the two apart.

5. Capture from the real camera

Same shape as the synthetic one, pointed at the device you found. Defining it in the IEx session means nothing to recompile and nothing to edit:

defmodule MyCam do
  use BB

  settings do
    name :my_cam
  end

  topology do
    link :head do
      sensor :eye, {BB.Camera.V4L, device: "/dev/video0", width: 1280, height: 720, frame_rate: 30}
    end
  end
end

{:ok, _pid} = MyCam.start_link([])

Discovery works before anything is capturing:

BB.Camera.discover(MyCam)
#=> [%BB.Camera.Info{name: :eye, path: [:head, :eye], status: :running,
#=>   stream_info: %BB.Camera.StreamInfo{format: :jpeg, width: 1280, height: 720, ...}}]

Watch the lifecycle, then open a consumer:

BB.subscribe(MyCam, [:sensor, :head, :eye])

{:ok, first} = BB.Camera.Media.open(MyCam, :eye, format: :jpeg)
flush()
#=> {:bb, [:sensor, :head, :eye],
#=>   %BB.Message{payload: %BB.Message.Camera.BeginCapture{subscriber_count: 1, ...}}}

{:ok, frame} = BB.Camera.Media.next(first, 10_000)
{frame.width, frame.height}
File.write!("/tmp/camera.jpg", frame.data)

frame.width and frame.height are read out of the JPEG the camera actually produced. If you asked for a size it does not support and it quietly gave you another, this is where you find out:

BB.Camera.stream_info(MyCam, :eye)

6. Two consumers, one capture

Open a second one, and count the ffmpeg processes:

{:ok, second} = BB.Camera.Media.open(MyCam, :eye, format: :jpeg)
{:ok, _frame} = BB.Camera.Media.next(second, 10_000)
pgrep -ax ffmpeg
#=> 2193841 /usr/bin/ffmpeg -hide_banner -loglevel error -f v4l2 -input_format mjpeg ...

One line. Both consumers are reading the same capture.

Note pgrep -x, matching the process name exactly: pgrep -f ffmpeg also matches the shell command containing the word, which makes it look like there are several when there is one.

7. Shut it down

BB.Camera.Media.close(first)
BB.Camera.Media.close(second)
flush()
#=> {:bb, [:sensor, :head, :eye],
#=>   %BB.Message{payload: %BB.Message.Camera.EndCapture{reason: :no_subscribers, ...}}}
pgrep -ax ffmpeg

Nothing. Capture stops when the last consumer leaves, and the camera's light goes out.

This is worth checking rather than assuming. Closing an Erlang port does not kill the program behind it, and neither does killing the port's owner — both were measured, and in both cases the process survived and was reparented to init. A separate reaper signals it, which is why there is nothing left here.

A preview in a browser

bb_liveview mounts a dashboard that previews any camera a robot declares, this one included, as MJPEG. Declare the camera in your robot and point the dashboard at it; there is nothing driver-specific to wire up.

Triage it the same way as the smoke test above. Swap this driver for bb_camera's synthetic camera to take hardware and ffmpeg out of the picture, then lavfi: "testsrc2" to put ffmpeg back without the device, then the device itself. Working on synthetic but not on lavfi is a bb_camera or bb_liveview problem; working on lavfi but not on the device is the camera or its permissions, and this README is where that is worked through.

Options

Option Default
:device — V4L2 device node. Required unless :lavfi is given
:lavfi — An ffmpeg synthetic source such as "testsrc2", instead of a device
:width, :height 640, 480 Any positive size. Not checked against the device
:frame_rate 30 A hertz unit, a number, or {numerator, denominator} for an exact rational such as {30_000, 1001}
:input_format :mjpeg V4L2 pixel format to ask for. Anything else means ffmpeg must encode
:quality 5 ffmpeg's -q:v, 1 (best) to 31. Only used when encoding
:ffmpeg $PATH Path to the ffmpeg executable
:max_frame_bytes 4 MiB Largest frame accepted before the stream is resynchronised
:drop_queue_len 8 Shed frames above this capture mailbox depth
:fail_queue_len 256 Stop capture above this mailbox depth
:stop_timeout_ms 2000 How long ffmpeg gets to exit before it is signalled

How it holds together

ffmpeg ──pipe──> Capture ──frames──> BB.Camera.Hub ──> consumers
                    │                     │
                    │                     └─ lifecycle ──> BB.PubSub
                    └─ os pid ──> Reaper

Limitations

Testing

BB_VERSION=local mix deps.get
BB_VERSION=local mix test                   # no camera, no ffmpeg needed
BB_VERSION=local mix test --include ffmpeg  # also runs ffmpeg for real
BB_VERSION=local mix check --no-retry

BB_VERSION=local goes on every command, including deps.get.

The default suite drives capture through an injected backend, so every case — a frame split across reads, a truncated image, an unexpected exit status, a consumer falling behind — is a function call rather than something to wait for. No test sleeps.

The :ffmpeg tagged tests run the whole real path against a synthetic ffmpeg source. They need ffmpeg, but no camera.

Licence

Apache-2.0. See LICENSES/.