pi-multifx-pedal/docs/nam_inference.md

NAM A2 Inference on RPi 4B — Research & Recommendations

Status: Complete — June 2026 Context: pi-multifx-pedal, BLOCK_SIZE=256, SAMPLE_RATE=48000 (5.33ms per block) See also: NAM_RESEARCH.md for detailed source citations

---

Executive Summary

NAM A2 inference is feasible in real-time on RPi 4B with Feather-class models using any approach. Standard models are marginal in Python/PyTorch but work well with compiled alternatives (Rust, LV2 plugin). The RPi 4B Cortex-A72 at 1.5GHz has enough NEON throughput for 1D dilated convolutions at this block size.

Bottom line: Current Python skeleton code works for Feather models. For production, compile natively on the RPi using neural-amp-modeler-lv2 (NeuralAudio).

---

1. NAM Model Architecture

NAM supports 4 architectures, exported as .nam JSON files (not TorchScript):

Architecture	Parameters (default)	Receptive Field	A2 Slimmable?	Use Case
ConvNet	6,369 (32ch, 4 dilations)	16 samples	Yes	Default — best quality/speed trade-off
WaveNet	~33K (16->8ch, 10 dilations)	512+ samples	Yes	Highest quality, most expensive
Linear	128 (1x128)	128 samples	No	Clean/crunch tones, extremely cheap
LSTM	~18K	—	No	Vintage/character tones

ConvNet (default — the most common)

• 4 dilated 1D Conv1d blocks: dilations [1, 2, 4, 8], kernel size 2
• Input: channels=1, hidden: channels=32, output: channels=1 via head Conv1d
• Activation: Tanh per layer
• Receptive field: sum(dilations) + 1 = 16 samples
• ~6,240 MACs per inference

NAM A2 SlimmableContainer

• Dynamically scales channel count at runtime (quality=0.0→narrow, 1.0→full)
• RT-safe — can switch quality between blocks
• Recommended: quality=0.3-0.5 for RPi 4B at 256-block

---

2. Latency Benchmarks (x86_64, PyTorch CPU)

Measured on x86_64. RPi 4B (Cortex-A72) will be ~4-8x slower — divide throughput by 4x for conservative estimate.

Model	Params	128-block (2.67ms)	256-block (5.33ms)	512-block (10.67ms)
ConvNet Tiny (16ch)	1,649	1.14ms (43%)	0.60ms (11%)	0.64ms (6%)
ConvNet Default (32ch)	6,369	1.64ms (62%)	0.74ms (14%)	4.78ms (45%)
ConvNet Medium (48ch, 5 dil)	18,817	8.63ms (324%)	2.64ms (50%)	21.23ms (199%)
ConvNet Large (64ch, 6 dil)	41,537	2.60ms (98%)	11.84ms (222%)	13.18ms (124%)
Linear 1x64	64	0.10ms (4%)	0.11ms (2%)	0.11ms (1%)
Linear 1x128	128	0.10ms (4%)	0.11ms (2%)	0.11ms (1%)

Key finding: Default ConvNet at 256-block takes 0.74ms on x86 — ~3-6ms estimated on RPi 4B. This is within budget but leaves little headroom with Python overhead.

---

3. Approach Comparison

Approach	Feather	Standard	Plugin	aarch64?	Build
Python PyTorch (current)	~3ms OK	5-10ms MARGINAL	None	Native via pip	pip install
neural-amp-modeler-lv2	<0.5ms OK	1-2ms OK	LV2	Compiles native	cmake on RPi
OpenSauce/nam-rs	0.5-1ms OK	1.9-3.8ms OK	Library crate	cargo build	Pure Rust
PiPedal C++	<0.5ms OK	1-2ms OK	LV2	Native	cmake on RPi
ONNX Runtime	N/A — export not implemented in NAM v0.13.0

3a. Python PyTorch (current code) ⭐ Short-term

Installed version: neural-amp-modeler v0.13.0 via pip.

Pros: Already works. Easy model loading via JSON. Can use torch.jit.script or torch.compile for ~2x speedup.

Cons:

• Python call overhead per block (~50-200us)
• PyTorch tensor reshaping/dispatch overhead
• JACK callback in Python — GC pauses risk xruns
• libtorch on aarch64 is big (hundreds of MB)

Feather models only — standard models will struggle on RPi 4B.

3b. neural-amp-modeler-lv2 (NeuralAudio) ⭐ Medium-term

Repository: mikeoliphant/neural-amp-modeler-lv2 (★460)

Pros:

• Closest to production-ready — LV2 plugin integrates directly with JACK
• NeuralAudio engine claims performance edge over NAM Core
• Compiles natively on RPi OS 64-bit via cmake
• Supports A1 + A2, WaveNet + LSTM + ConvNet

Cons:

• No prebuilt ARM binaries — must compile on-device
• Needs build dependencies (cmake, pkg-config, libsndfile, LV2 SDK)
• No prebuilt Debian packages

Build commands (on RPi):

sudo apt install cmake pkg-config libsndfile-dev lv2-dev
git clone https://github.com/mikeoliphant/neural-amp-modeler-lv2
cd neural-amp-modeler-lv2
cmake -Bbuild -DCMAKE_BUILD_TYPE=Release -DUSE_NATIVE_ARCH=ON
cmake --build build -j4
sudo cmake --install build

3c. OpenSauce/nam-rs (Pure Rust) ⭐ Long-term

Repository: github.com/OpenSauce/nam-rs

Pros:

• Pure Rust — no C++ dependency chain, no libtorch
• First-class aarch64 support via cargo build --target=aarch64-unknown-linux-gnu
• A2 SlimmableContainer — runtime quality/speed dialing
• Published on crates.io

Cons:

• No LV2 plugin — library crate only (would need Rust LV2 wrapper)
• Very new project (June 2026), unproven in production
• Estimated same perf as LV2 option

3d. PiPedal C++ ⭐ Reference

Repository: rerdavies/pipedal (★277)

Pros:

• Most mature RPi NAM project — measured 2.7ms round-trip latency
• Full web UI, tone3000 integration, LV2 plugins
• Proves the concept works

Cons:

• Not a library to reuse — full standalone application

3e. ONNX Runtime ❌ Dead end

• Exportable.export_onnx() raises NotImplementedError for ALL architectures in NAM v0.13.0
• No implementation exists anywhere in the codebase
• Even if it did, ONNX Runtime on aarch64 doesn't outperform PyTorch for Conv1D workloads

---

4. Model Size Recommendations for RPi 4B

Based on benchmark extrapolation (x86 * 4-8x for RPi):

FAMILY	Channels	Dilations	Params	Est. RPi Latency (256-block)	Recommendation
Nano	4/2ch	[1,2,4]	~500	0.1-0.2ms	Always safe
Feather	8/4ch	[1,2,4,8,16]	~2,500	0.5-1ms	Default for Python
Lite	12/6ch	[1,2,4,8,16,32]	~5,500	1-2ms	OK with compiled
Standard	16/8ch	[1,2,4,8,16,32,64,128]	~18,000	2-4ms	Only with compiled
Heavy	32/16ch	10+ dilations	~70,000	5-10ms	Too expensive

---

5. NAM A2 Slimmable Quality Dial

The A2 SlimmableContainer is the ideal RPi solution:

• quality=0.0 → Nano-class (narrow, fast)
• quality=0.3 → Feather-class (good tone, 0.5ms on RPi)
• quality=0.5 → Lite-class (very good tone, 1ms on RPi)
• quality=1.0 → Standard-class (best tone, 2-4ms on RPi)

Switch quality between presets or even per-block for adaptive RT.

---

6. NAM Model Loading (.nam format)

The .nam file is a JSON dictionary (NOT TorchScript):

{
  "version": "0.13.0",
  "architecture": "ConvNet",     // or "WaveNet", "Linear", "LSTM"
  "metadata": { "date": "...", "loudness": X, "gain": Y },
  "config": { "channels": 32, "dilations": [1,2,4,8] },
  "weights": [0.001, -0.023, ...]  // flat 1D array
}

The Python NAMHost should:

1. Load JSON
2. Reconstruct the model from architecture + config
3. Call import_weights(weights) to set parameters
4. Run model(x_tensor) for inference

---

7. Current project's path forward

Phase 1 — Python + Feather (now)

• Current nam_host.py and pipeline.py are adequate
• Limit to Feather/Nano NAM models
• Add torch.compile() for ~2x speedup
• Keep BLOCK_SIZE=256, SAMPLE_RATE=48000

Phase 2 — Compiled inference (after RPi hardware arrives)

• On first boot, compile neural-amp-modeler-lv2 natively
• Switch to LV2 plugin chain in JACK
• This gives 2-4x more CPU headroom for other FX

Phase 3 — Rust native (future)

• Port to nam-rs for pure-Rust implementation
• A2 Slimmable runtime quality dial
• No libtorch dependency — saves ~300MB on the Pi