mob_midi
MIDI in + out for Mob apps, over
USB-MIDI and BLE-MIDI. Unlike classic Bluetooth, MIDI is first-class on both
platforms, so this is a real cross-platform surface: CoreMIDI on iOS,
android.media.midi on Android.
# Discover
MobMidi.list_devices(socket)
# => {:midi, :devices, [%{id: 7, name: "Oxygen 49", direction: :input}, ...]}
# Receive
MobMidi.open_input(socket, device_id)
# => {:midi, :raw, %{device: 7, bytes: <<0x90, 60, 100>>}}
# parse it: MobMidi.parse(bytes) -> [%{type: :note_on, channel: 0, note: 60, velocity: 100}]
# Send: open the output, then send. The caller gets
# {:midi, :opened, %{device: 7, direction: :output}} once the port is ready, or
# {:midi, :error, %{device: 7, op: :open_output, reason: :no_such_device, dropped: 0}}
MobMidi.open_output(socket, device_id)
def handle_info({:midi, :opened, %{device: id}}, socket) do
MobMidi.send_note_on(socket, id, 0, 60, 100)
MobMidi.send_cc(socket, id, 0, 7, 90)
{:noreply, socket}
end
Waiting for :opened is optional: Android opens the port asynchronously and
queues sends made before it opens (up to 256 64-byte chunks per port), writing them in order
the moment it does. If the open fails, the queue is discarded and the error's
dropped counts the lost messages (close/2 during the open reports reason: :closed the same way). iOS opens synchronously. send_* return socket when
the message was queued and {:error, reason} when it wasn't (:not_open,
:queue_full, :too_large for one send over 16 KiB, :no_listener,
:send_failed), so bind the result instead of piping it on as the socket.
Back-pressure. Sends join the port's queue and a thread of the
port's own writes them, so a listener that stops reading never blocks the
caller. When its queue is full, the send answers {:error, :queue_full} and
the port refuses every send until it has drained (the stuck write returned,
at most half the queue is left and the largest refused message fits). The
process that opened the port then gets
{:midi, :drained, %{device: id, port: n, handle: h, generation: g}}.
MobMidi.discard(id, port: n) drops what is still queued (stale notes the
listener would otherwise play late, or sends scheduled ahead that shouldn't
play) and returns {:ok, generation}; the next :drained carries it, or a
later one (Android: an own virtual output whose listeners all leave drops its
queue too). The usual reaction to :queue_full: discard, wait for
:drained, send note-offs or all-notes-off, then carry on. CoreMIDI never
makes a sender wait for a receiver, so on iOS only sends scheduled far ahead
fill a queue.
The NIF layer is deliberately thin (device enumeration + raw byte I/O); message
encode/parse lives in Elixir (MobMidi) where it's pure and unit-tested.
On-device self-test: MobMidi.SelfTest (declared as selftest: in the
manifest) lists MIDI devices through the NIF and, on iOS, loops a SysEx sent
20 ms ahead through an own virtual device ("mob_midi selftest") and checks
its bytes and timing; run it from a host app with mix mob.selftest
(mob_dev 0.7.17+). On Android no attached MIDI device is a
:needs_hardware skip.
Platforms
| Android | iOS | host (mix test) |
|
|---|---|---|---|
list_devices/1, hot-plug |
yes | yes | {:error, :unsupported} |
open_input/3, open_output/3, send_*, at: |
yes | yes | {:error, :unsupported} |
port: other than 0, port: :all |
yes | one port per endpoint | {:error, :unsupported} |
C ABI (mob_midi.h): deliver: :native | :both, output_handle/2, discard/2 |
yes | yes | {:error, :unsupported} |
:queue_full / :drained |
yes | yes (sends scheduled far ahead) | n/a |
open_virtual/3 |
declared in the app's manifest | made on first open | {:error, :unsupported} |
{:midi, :virtual_status, ...}, :no_listener |
yes | no (CoreMIDI doesn't tell) | n/a |
scan_bluetooth/2, connect_bluetooth/2 |
yes | system UI pairs BLE-MIDI | {:error, :unsupported} |
now_ns/0 |
CLOCK_MONOTONIC |
mach_absolute_time |
BEAM monotonic time |
Ports, timestamps, hot-plug
- Ports:
list_devices/1reports each device'sinput_ports/output_ports;open_input/3,open_output/3andsend_raw/4takeport:(default 0). CoreMIDI endpoints are single ports. - Timestamps: incoming
{:midi, :raw, ...}carries the sender'stimestampand when the bridgereceivedit (ns on theMobMidi.now_ns/0clock:CLOCK_MONOTONICon Android,mach_absolute_timeon iOS).send_raw(socket, id, bytes, at: ns)holds outgoing bytes in the port's queue until that time and hands them over stamped with it: MIDI clock without the BEAM's timer jitter. - Hot-plug: the last
list_devices/1caller gets{:midi, :device_added | :device_removed, nil}on both platforms; a removed device's open ports are closed first.
Virtual devices (the app as a MIDI device)
MobMidi.open_virtual(socket, name) subscribes to one of the app's own MIDI
devices: other apps on the phone see it, play into it (arriving as {:midi, :raw, ...}) and listen to what send_raw/4 sends to its id.
On iOS the first open_virtual/3 of a name creates a CoreMIDI source +
destination of that name (the device id is the source's); they stay until the
app exits, so other apps' connections survive a close/2 and reopen.
CoreMIDI doesn't say who is connected, so iOS sends no :virtual_status and
its sends never answer :no_listener. iOS creates virtual endpoints only for an app whose
Info.plist has audio in UIBackgroundModes (mob.new's template does);
otherwise open_virtual/3 answers reason: :not_permitted. On Android a
virtual device is declared in the app's manifest (Android reads its name and
ports from the app's own res/xml, so a plugin can't contribute a per-app
one). Generate the declaration, which uses the plugin's
io.mob.midi.MobMidiDeviceService:
mix mob_midi.gen.virtual_device --name "My Synth" --inputs 1 --outputs 1
# writes android/app/src/main/res/xml/mob_midi_device.xml and a fenced
# <service> block in android/app/src/main/AndroidManifest.xml;
# --check in CI, --remove to take it out; or config :mob_midi, :virtual_device
--inputs are ports other apps send into the app, --outputs ports it sends
out of (Humdinger: --inputs 0 --outputs 1; Magpie: --inputs 1 --outputs 1).
Then MobMidi.open_virtual(socket, "My Synth"). Android also reports who has
it open: {:midi, :virtual_status, %{device: id, senders: n, listeners: n}}.
A second device needs its own <service>: declare an empty subclass of
io.mob.midi.MobMidiDeviceService in the app. Android starts the service when
another app opens the device, possibly before the BEAM is up; bytes sent before
open_virtual/3 are dropped.
Native code: priv/include/mob_midi.h
An audio engine in another static NIF of the app can take MIDI without the
BEAM, on both platforms: open inputs with deliver: :native (or :both, per
port), register a sink with mob_midi_set_sink(fn, ctx) and get (source, generation, bytes, len, timestamp_ns) on the MIDI thread; pass an output's
handle (from :opened or MobMidi.output_handle/2) to it and call
mob_midi_send(handle, bytes, len, at_ns) from its own thread. Sends wait in
the port's queue until at_ns (the BEAM's send_raw(..., at:) shares that
queue); mob_midi_discard(handle) drops what is queued. Include it relative
to your source: #include "../deps/mob_midi/priv/include/mob_midi.h" (a
path dependency: the dependency's own path). The symbols are in
priv/native/android/mob_midi_core.c on Android and
priv/native/ios/mob_midi_nif.m on iOS.
The header documents thread and lifetime rules and the clock: on iOS every
timestamp is mach_absolute_time in ns (CoreMIDI's MIDITimeStamp,
AudioTimeStamp.mHostTime, clock_gettime(CLOCK_UPTIME_RAW)), not Darwin's
CLOCK_MONOTONIC, which keeps counting while the phone sleeps: take times
from mob_midi_now_ns(). iOS hands a due message to CoreMIDI from a
real-time thread, so it leaves within tens of microseconds of at_ns.
priv/include/mob_midi_parser.h is a matching header-only stream parser.
Bluetooth MIDI devices (Android)
MobMidi.scan_bluetooth(socket) finds BLE-MIDI devices ({:midi, :bluetooth_device, %{address: a, name: n, rssi: dbm}}), MobMidi.connect_bluetooth(socket, address) connects one, which then lists and opens like a USB device; close/2
disconnects it. Ask for the :midi_bluetooth permission first
(Mob.Permissions.request(socket, :midi_bluetooth)). The plugin declares the Bluetooth
and location permissions scanning needs; the request includes location on every
version (Android 12+ too, as mob_dev can't mark BLUETOOTH_SCAN
neverForLocation), and scans need location switched on (:location_off). iOS pairs BLE-MIDI devices in the system UI
(CABTMIDICentralViewController); they are then ordinary CoreMIDI devices.
Demo screens (tier 3)
Two screens the host auto-lists via Mob.Plugins.screens():
MobMidi.KeyboardScreen(/midi_keyboard) — a two-octave keyboard; tap a key to send a note to the selected output. Meant to be landscape; currently portrait-stubbed (white keys, two rows) untilMob.Device.lock_orientation/1lands in mob core (mob#49).MobMidi.InputScreen(/midi_input) — pick a source, watch incoming notes light up plus a decoded event log. Visual-first; an audible sine per note is a fast follow onceMob.Audiogrows a tone primitive.
Status: experimental
The Elixir surface (API, parser, screens) is unit-tested. The native layers (iOS
priv/native/ios/mob_midi_nif.m over CoreMIDI; Android
priv/native/android/MobMidiBridge.kt + priv/native/jni/mob_midi_nif.zig over
MidiManager) are experimental.
Verified for 0.1.2, on an Android emulator against virtual MIDI devices
(android.media.midi.MidiDeviceService loopbacks in a test host app):
list_devices/1, open_output/2 with its :opened / :error events, sends
queued before the port opened arriving first and in order, the 256-message queue
bound, send_* errors for unopened or closed outputs, open_input/2 receive, and
one device opened in both directions. On the iOS simulator against CoreMIDI
virtual endpoints (a MIDIDestinationCreate synth echoing to a
MIDISourceCreate source): list_devices/1, open_output/2 events, send_*
(including the :not_open error) and open_input/2 receive.
Ports, timestamps, hot-plug and virtual devices were then checked on an Android
emulator by a host app (FMutate) with a MobMidiDeviceService device played by,
and listening to, a second app; the iOS side of them is compile-checked only.
0.3 was checked on a moto e (Android 10, 32-bit armeabi-v7a)
and an Android 15 arm64 emulator with a throwaway host app, a static C NIF
and tools/MidiProbe.java as another process: delivery to the BEAM and the C
sink with the sender's stamps, scheduled sends, port: :all, a stalled
listener through :queue_full, discard/2 and :drained, open/close races,
hot-plug and an emoji device name; numbers in the CHANGELOG.
Not verified: USB-MIDI and BLE-MIDI hardware on either platform (including the
MobMidi.Ble peripheral and connect_bluetooth/2), and the native C path on
iOS (it has none yet: the mob_midi.h functions answer unsupported there).
Activating both mob_midi and mob_bluetooth: both declare
NSBluetoothAlwaysUsageDescription, and mob_dev (0.7.14) refuses the native
build when two activated plugins declare the same key. Set it in the host's
ios/Info.plist: the host's value wins and the plugins no longer conflict.