hexagon_tpu

Elixir bindings for the Qualcomm Hexagon NPU (HTP) via QNN (Qualcomm AI Engine Direct), targeting the Radxa Dragon Q6A (QCS6490, HTP v68) running Nerves with nerves_system_dragon_q6a.

Nx tensors in, Nx tensors out; quantization handled automatically from the model's own metadata.

{:ok, model} = HexagonTpu.load_model("/data/mobilenet_v2.bin")
{:ok, [scores]} = HexagonTpu.predict(model, [image_tensor])   # f32 in, f32 out

How it works

• Models are pre-compiled QNN context binaries (.bin) produced on an x86-64 host with the QAIRT SDK tools (see docs/models.md). On device we only deserialize (QnnContext_createFromBinary) and execute — no on-device graph prepare, no libQnnHtpPrepare.so.
• The NIF dlopen's libQnnHtp.so/libQnnSystem.so from the device rootfs (shipped by the system's qairt-runtime Buildroot package). Nothing proprietary is linked or bundled here; building only needs the SDK headers (QNN_SDK_ROOT).
• Tensor I/O metadata (names, shapes, dtypes, quant params) is introspected from the context binary via QnnSystem at load time.
• Layered API: HexagonTpu.Runtime (backend/device) → HexagonTpu.Context (loaded binary) → HexagonTpu.Graph (execute). Resources are freed by GC in dependency order; close/1 exists for deterministic teardown.

Requirements

On device (provided by nerves_system_dragon_q6a with BR2_PACKAGE_QAIRT_RUNTIME=y):

• fastrpc kernel driver + CDSP firmware + cdsprpcd
• /usr/lib/libQnnHtp.so, libQnnHtpV68Stub.so, libQnnSystem.so
• /usr/lib/dsp/libQnnHtpV68Skel.so (+ DSP_LIBRARY_PATH=/usr/lib/dsp)

Build host: QNN_SDK_ROOT pointing at an extracted QAIRT SDK (2.42.0.251225). The provided devenv.nix fetches it. Consumers of the published Hex package get a precompiled NIF and don't need the SDK.

Usage on host (development)

The QNN CPU backend (libQnnCpu.so, x86-64) allows exercising the runtime lifecycle and context introspection without hardware:

{:ok, rt} = HexagonTpu.Runtime.create(
  lib_path: Path.join(System.get_env("QNN_HOST_LIB_DIR"), "libQnnCpu.so"),
  system_lib_path: Path.join(System.get_env("QNN_HOST_LIB_DIR"), "libQnnSystem.so")
)

Run tests: mix test (pure), mix test --include qnn_host (needs devenv).

Quantization

QNN stores affine quant params as float = (q + offset) * scale; HexagonTpu.TensorInfo normalizes to the conventional zero_point = -offset. Graph.execute/3 quantizes f32 inputs and dequantizes outputs automatically (quantize: :none / dequantize: :none opt out for zero-overhead pre-quantized pipelines).

Observability & guards

• HexagonTpu.Stats.native/0 — NIF counters: live runtimes/contexts/graphs (leak guard: must return to baseline after GC), execute counts/errors, and pure QNN graphExecute time (execute_ns_total/execute_ns_last).
• HexagonTpu.Stats.dirty_utilization/1 — dirty CPU / dirty IO scheduler busy share via microstate accounting (inference runs on dirty schedulers).
• HexagonTpu.Stats.memory/0 — BEAM binary/total memory (output tensors are refc binaries).
• Telemetry: [:hexagon_tpu, :execute, :stop] per inference with %{duration: native_time} and %{graph:, status:} metadata.
• {HexagonTpu.Monitor, interval_ms: 30_000} in your supervision tree emits [:hexagon_tpu, :stats] periodically and logs warnings on sustained resource growth (leak suspicion) or dirty-scheduler saturation.

Leak-guard tests (test/stats_test.exs) assert alive counts return to baseline across create/close and GC-only cycles.

Releases

Pushing a v* tag runs .github/workflows/release-precompiled.yml: builds the dragon_q6a NIF with the Nerves toolchain, uploads the precompiled tarball + checksum.exs to the GitHub release, and publishes to Hex (requires the HEX_API_KEY secret).

License

Apache-2.0. The QAIRT SDK and its runtime libraries are Qualcomm proprietary — this repository neither vendors nor redistributes them.