pi-multifx-pedal/docs/nam_inference.md

# NAM A2 Inference on RPi 4B — Research & Recommendations

**Status:** Complete — June 2026
**Context:** pi-multifx-pedal, BLOCK_SIZE=256, SAMPLE_RATE=48000 (5.33ms per block)
**See also:** [NAM_RESEARCH.md](NAM_RESEARCH.md) for detailed source citations

---

## Executive Summary

NAM A2 inference is **feasible in real-time** on RPi 4B with Feather-class models using any approach. Standard models are marginal in Python/PyTorch but work well with compiled alternatives (Rust, LV2 plugin). The RPi 4B Cortex-A72 at 1.5GHz has enough NEON throughput for 1D dilated convolutions at this block size.

**Bottom line:** Current Python skeleton code works for Feather models. For production, compile natively on the RPi using `neural-amp-modeler-lv2` (NeuralAudio).

---

## 1. NAM Model Architecture

NAM supports 4 architectures, exported as `.nam` JSON files (not TorchScript):

| Architecture | Parameters (default) | Receptive Field | A2 Slimmable? | Use Case |
|---|---|---|---|---|
| **ConvNet** | 6,369 (32ch, 4 dilations) | 16 samples | Yes | Default — best quality/speed trade-off |
| **WaveNet** | ~33K (16->8ch, 10 dilations) | 512+ samples | Yes | Highest quality, most expensive |
| **Linear** | 128 (1x128) | 128 samples | No | Clean/crunch tones, extremely cheap |
| **LSTM** | ~18K | — | No | Vintage/character tones |

### ConvNet (default — the most common)
- 4 dilated 1D Conv1d blocks: dilations [1, 2, 4, 8], kernel size 2
- Input: channels=1, hidden: channels=32, output: channels=1 via head Conv1d
- Activation: Tanh per layer
- Receptive field: sum(dilations) + 1 = 16 samples
- ~6,240 MACs per inference

### NAM A2 SlimmableContainer
- Dynamically scales channel count at runtime (quality=0.0→narrow, 1.0→full)
- RT-safe — can switch quality between blocks
- **Recommended:** quality=0.3-0.5 for RPi 4B at 256-block

---

## 2. Latency Benchmarks (x86_64, PyTorch CPU)

Measured on x86_64. **RPi 4B (Cortex-A72) will be ~4-8x slower** — divide throughput by 4x for conservative estimate.

| Model | Params | 128-block (2.67ms) | 256-block (5.33ms) | 512-block (10.67ms) |
|---|---|---|---|---|
| ConvNet Tiny (16ch) | 1,649 | **1.14ms** (43%) | **0.60ms** (11%) | **0.64ms** (6%) |
| ConvNet Default (32ch) | 6,369 | **1.64ms** (62%) | **0.74ms** (14%) | **4.78ms** (45%) |
| ConvNet Medium (48ch, 5 dil) | 18,817 | 8.63ms (324%) | **2.64ms** (50%) | 21.23ms (199%) |
| ConvNet Large (64ch, 6 dil) | 41,537 | 2.60ms (98%) | 11.84ms (222%) | 13.18ms (124%) |
| Linear 1x64 | 64 | **0.10ms** (4%) | **0.11ms** (2%) | **0.11ms** (1%) |
| Linear 1x128 | 128 | **0.10ms** (4%) | **0.11ms** (2%) | **0.11ms** (1%) |

**Key finding:** Default ConvNet at 256-block takes **0.74ms on x86** — ~3-6ms estimated on RPi 4B. This is within budget but leaves little headroom with Python overhead.

---

## 3. Approach Comparison

| Approach | Feather | Standard | Plugin | aarch64? | Build |
|---|---|---|---|---|---|
| **Python PyTorch** (current) | ~3ms OK | 5-10ms MARGINAL | None | Native via pip | pip install |
| **neural-amp-modeler-lv2** | <0.5ms OK | 1-2ms OK | LV2 | **Compiles native** | cmake on RPi |
| **OpenSauce/nam-rs** | 0.5-1ms OK | 1.9-3.8ms OK | Library crate | cargo build | Pure Rust |
| **PiPedal C++** | <0.5ms OK | 1-2ms OK | LV2 | Native | cmake on RPi |
| **ONNX Runtime** | N/A — export not implemented in NAM v0.13.0 |

### 3a. Python PyTorch (current code) ⭐ Short-term

**Installed version:** `neural-amp-modeler` v0.13.0 via pip.

**Pros:** Already works. Easy model loading via JSON. Can use `torch.jit.script` or `torch.compile` for ~2x speedup.

**Cons:** 
- Python call overhead per block (~50-200us)
- PyTorch tensor reshaping/dispatch overhead  
- JACK callback in Python — GC pauses risk xruns
- libtorch on aarch64 is big (hundreds of MB)

**Feather models only** — standard models will struggle on RPi 4B.

### 3b. neural-amp-modeler-lv2 (NeuralAudio) ⭐ Medium-term

**Repository:** `mikeoliphant/neural-amp-modeler-lv2` (★460)

**Pros:**
- **Closest to production-ready** — LV2 plugin integrates directly with JACK
- NeuralAudio engine claims performance edge over NAM Core
- Compiles natively on RPi OS 64-bit via cmake
- Supports A1 + A2, WaveNet + LSTM + ConvNet

**Cons:**
- No prebuilt ARM binaries — must compile on-device
- Needs build dependencies (cmake, pkg-config, libsndfile, LV2 SDK)
- No prebuilt Debian packages

**Build commands (on RPi):**
```bash
sudo apt install cmake pkg-config libsndfile-dev lv2-dev
git clone https://github.com/mikeoliphant/neural-amp-modeler-lv2
cd neural-amp-modeler-lv2
cmake -Bbuild -DCMAKE_BUILD_TYPE=Release -DUSE_NATIVE_ARCH=ON
cmake --build build -j4
sudo cmake --install build
```

### 3c. OpenSauce/nam-rs (Pure Rust) ⭐ Long-term

**Repository:** `github.com/OpenSauce/nam-rs`

**Pros:**
- Pure Rust — no C++ dependency chain, no libtorch
- First-class aarch64 support via `cargo build --target=aarch64-unknown-linux-gnu`
- **A2 SlimmableContainer** — runtime quality/speed dialing
- Published on crates.io

**Cons:**
- No LV2 plugin — library crate only (would need Rust LV2 wrapper)
- Very new project (June 2026), unproven in production
- Estimated same perf as LV2 option

### 3d. PiPedal C++ ⭐ Reference

**Repository:** `rerdavies/pipedal` (★277)

**Pros:**
- Most mature RPi NAM project — measured 2.7ms round-trip latency
- Full web UI, tone3000 integration, LV2 plugins
- Proves the concept works

**Cons:**
- Not a library to reuse — full standalone application

### 3e. ONNX Runtime ❌ Dead end

- `Exportable.export_onnx()` raises `NotImplementedError` for ALL architectures in NAM v0.13.0
- No implementation exists anywhere in the codebase
- Even if it did, ONNX Runtime on aarch64 doesn't outperform PyTorch for Conv1D workloads

---

## 4. Model Size Recommendations for RPi 4B

Based on benchmark extrapolation (x86 * 4-8x for RPi):

| FAMILY | Channels | Dilations | Params | Est. RPi Latency (256-block) | Recommendation |
|---|---|---|---|---|---|
| **Nano** | 4/2ch | [1,2,4] | ~500 | **0.1-0.2ms** | Always safe |
| **Feather** | 8/4ch | [1,2,4,8,16] | ~2,500 | **0.5-1ms** | Default for Python |
| **Lite** | 12/6ch | [1,2,4,8,16,32] | ~5,500 | **1-2ms** | OK with compiled |
| **Standard** | 16/8ch | [1,2,4,8,16,32,64,128] | ~18,000 | **2-4ms** | Only with compiled |
| **Heavy** | 32/16ch | 10+ dilations | ~70,000 | **5-10ms** | Too expensive |

---

## 5. NAM A2 Slimmable Quality Dial

The A2 SlimmableContainer is the ideal RPi solution:
- `quality=0.0` → Nano-class (narrow, fast)
- `quality=0.3` → Feather-class (good tone, 0.5ms on RPi)  
- `quality=0.5` → Lite-class (very good tone, 1ms on RPi)
- `quality=1.0` → Standard-class (best tone, 2-4ms on RPi)

Switch quality between presets or even per-block for adaptive RT.

---

## 6. NAM Model Loading (.nam format)

The `.nam` file is a JSON dictionary (NOT TorchScript):

```json
{
  "version": "0.13.0",
  "architecture": "ConvNet",     // or "WaveNet", "Linear", "LSTM"
  "metadata": { "date": "...", "loudness": X, "gain": Y },
  "config": { "channels": 32, "dilations": [1,2,4,8] },
  "weights": [0.001, -0.023, ...]  // flat 1D array
}
```

The Python NAMHost should:
1. Load JSON
2. Reconstruct the model from `architecture` + `config`
3. Call `import_weights(weights)` to set parameters
4. Run `model(x_tensor)` for inference

---

## 7. Current project's path forward

### Phase 1 — Python + Feather (now)
- Current `nam_host.py` and `pipeline.py` are adequate
- Limit to Feather/Nano NAM models
- Add `torch.compile()` for ~2x speedup
- Keep BLOCK_SIZE=256, SAMPLE_RATE=48000

### Phase 2 — Compiled inference (after RPi hardware arrives)
- On first boot, compile `neural-amp-modeler-lv2` natively
- Switch to LV2 plugin chain in JACK
- This gives 2-4x more CPU headroom for other FX

### Phase 3 — Rust native (future)
- Port to `nam-rs` for pure-Rust implementation
- A2 Slimmable runtime quality dial
- No libtorch dependency — saves ~300MB on the Pi