Build IR convolution engine
- Full FFT overlap-add IR convolution in IRLoader (process(), set_mix(), toggle) - Lazy FFT computation — IR FFT padded to correct block+ir size on first process() - Wet/dry mix control, enabled/disabled toggle with tail clearing - Fixed pipeline._apply_ir_cab() to delegate to IRLoader.process() instead of poking internals (old code had array-size mismatch bug: IR FFT at ir_len vs block FFT at conv_size) - 46 tests: loading, convolution correctness, overlap-add state, mix, toggle, directory listing, performance budget (all <5ms even at 8192 taps), edge cases - scripts/download_irs.sh: free IR pack downloader (God's Cab, Seacow)
This commit is contained in:
@@ -0,0 +1,161 @@
|
||||
# NAM A2 / A1 Inference on RPi 4B — Structured Research Findings
|
||||
|
||||
**Date:** 2026-06-07
|
||||
**Project:** pi-multifx-pedal
|
||||
**Context:** BLOCK_SIZE=256, SAMPLE_RATE=48000, NAM v0.13.0 installed
|
||||
|
||||
---
|
||||
|
||||
## 1. NAM LV2 Plugin on aarch64
|
||||
|
||||
**Repository:** https://github.com/sdatkinson/NeuralAmpModelerPlugin (★2720, 223 forks)
|
||||
- Built with **iPlug2** framework (not JUCE)
|
||||
- Build scripts: NeuralAmpModeler/scripts/ — makedist-mac.sh, makedist-win.bat
|
||||
- README: "For Linux support, there is an LV2 plugin available at mikeoliphant/neural-amp-modeler-lv2"
|
||||
- **No Linux build CI exists** — GitHub Actions only build for macos-latest and windows-latest
|
||||
|
||||
### Prebuilt binaries
|
||||
- **None for ARM/Linux.** All releases (v0.7.6-v0.7.15) ship only macOS .dmg and Windows .zip. No Linux, no ARM.
|
||||
|
||||
### Issues mentioning ARM/aarch64
|
||||
| Issue | Title | State | Notes |
|
||||
|-------|-------|-------|-------|
|
||||
| #357 | [FEATURE] Arm64 support | **OPEN** (Aug 2023) | Windows ARM64 request. Maintainer asked for specifics. No resolution in 3+ years. |
|
||||
| #587 | NAM Setup — Windows 11 on ARM | CLOSED | Setup packaging issue, not cross-compile. |
|
||||
| #627 | fix: avoid audio-thread stalls (AU) | CLOSED | "arm" incidental in body (part of "harm"). |
|
||||
|
||||
**Conclusion: No official aarch64 support for the NAM plugin. Issue #357 has sat open for 3+ years. The iPlug2-based plugin is not designed for cross-compilation to aarch64.**
|
||||
|
||||
### The LV2 option (NeuralAudio-based)
|
||||
- https://github.com/mikeoliphant/neural-amp-modeler-lv2 (★460, 50 forks)
|
||||
- Uses **NeuralAudio** C++ engine (not iPlug2)
|
||||
- Supports NAM A1 + A2, WaveNet + LSTM + ConvNet
|
||||
- **NeuralAudio README explicitly claims**: "internal implementation outperforms NAM Core on all tested platforms (Windows x64, Linux x64/Arm64)"
|
||||
- Build: cmake .. -DCMAKE_BUILD_TYPE=Release && make -j4
|
||||
- CMake flags of interest: -DUSE_NATIVE_ARCH=ON, -DBUILD_INTERNAL_STATIC_WAVENET=ON, -DWAVENET_FRAMES=XX
|
||||
- **GitHub Actions only builds for amd64 Ubuntu + Windows** — no ARM in CI
|
||||
- **No prebuilt ARM binaries** released, but **should compile natively on RPi OS 64-bit**
|
||||
|
||||
---
|
||||
|
||||
## 2. NAMR (Rust NAM)
|
||||
|
||||
The URL `github.com/mikeoliphant/neural-amp-modeler-rs` **does not exist (404)**.
|
||||
|
||||
### a) OpenSauce/nam-rs (Pure Rust library crate)
|
||||
- **https://github.com/OpenSauce/nam-rs** (June 2026, very new, ★0)
|
||||
- Pure Rust, **no external C/C++ dependencies**
|
||||
- Supports: WaveNet (A1 + A2 single), LSTM, SlimmableContainer (NAM A2)
|
||||
- **aarch64:** Supported via Cargo, verified target aarch64-unknown-linux-gnu
|
||||
- Has **dedicated RPi performance analysis** in issues:
|
||||
- Issue #23: "Raspberry Pi / embedded support: run the A1 WaveNet presets real-time on aarch64"
|
||||
- Concluded: A2 SlimmableContainer is the CPU/quality dial, superseding bespoke specialization
|
||||
- **Performance (x86, standard WaveNet, 512 samples):** ~951us for process_buffer
|
||||
- **Estimated Pi 4 factor:** 4-8x slower
|
||||
- At 256 samples (our project): ~1.9-3.8ms standard; Feather ~0.5-1ms
|
||||
- Published on crates.io as `nam-rs`. MIT license.
|
||||
|
||||
### b) fabiohl/nam-rs (Standalone + CLAP plugin)
|
||||
- **https://github.com/fabiohl/nam-rs** (★4, April 2026, v1.7.0)
|
||||
- **x86-64-v3 only** (AVX2+FMA mandatory). **No aarch64 support.**
|
||||
- Standalone PipeWire + CLAP plugin. Explicitly no LV2.
|
||||
- Apache-2.0 license.
|
||||
|
||||
### Is Rust NAM faster than Python/PyTorch?
|
||||
**Yes.** Compiled Rust/C++ avoids Python interpreter overhead (~50-200us per call), PyTorch tensor dispatch, dynamic graph construction, and GC pauses. OpenSauce/nam-rs on x86: ~951us for 512-sample standard. Python equivalent via `nam` library is conservatively 3-5x slower.
|
||||
|
||||
---
|
||||
|
||||
## 3. ONNX Runtime Inference
|
||||
|
||||
### NAM ONNX Export Status (v0.13.0)
|
||||
- nam.models.exportable.Exportable has export_onnx() method
|
||||
- **ALL four model architectures (ConvNet, WaveNet, Linear, LSTM) raise NotImplementedError**
|
||||
- It is a stub only — no implementation exists in any class
|
||||
- The working export path is .nam JSON format via export() (not ONNX)
|
||||
- **ONNX export does not exist in NAM v0.13.0**
|
||||
|
||||
### Environment
|
||||
- PyTorch 2.12.0+cpu (installed)
|
||||
- ONNX Runtime 1.26.0 (installed but not usable without export pathway)
|
||||
- ONNX package not installed
|
||||
|
||||
### aarch64 ONNX Runtime
|
||||
Even if ONNX export were available: ONNX Runtime on aarch64 uses NEON SIMD + Arm Compute Library, but Conv1D workloads would be marginal vs PyTorch's oneDNN NEON kernels. **Not worth pursuing.**
|
||||
|
||||
---
|
||||
|
||||
## 4. RPi 4B Cortex-A72 NEON Performance
|
||||
|
||||
### From OpenSauce/nam-rs (#18)
|
||||
| Model | x86 512smp | Est. Pi4 4x 512smp | Est. Pi4 8x 512smp | Est. Pi4 256smp |
|
||||
|-------|:----------:|:------------------:|:------------------:|:---------------:|
|
||||
| Standard (16/8ch, 10 dil) | 951us | 3.8ms | 7.6ms | 1.9-3.8ms |
|
||||
| Lite (12/6ch) | ~520us | 2.1ms | 4.2ms | ~1-2ms |
|
||||
| Feather (8/4ch, 7 dil) | ~237us | 0.95ms | 1.9ms | 0.5-1ms |
|
||||
| Nano (4/2ch) | ~57us | 0.23ms | 0.46ms | ~0.1-0.2ms |
|
||||
|
||||
Cortex-A72 constraints: 2x NEON 128-bit (vs 256-bit AVX2), 1.5GHz, no FMA, lower bandwidth.
|
||||
|
||||
### From PiPedal (RPi OS + MOTU M2 USB) — round-trip latency
|
||||
| Buffer | 2 periods | 3 periods | 4 periods |
|
||||
|--------|:--------:|:--------:|:--------:|
|
||||
| 16 samples | Fails | 2.7ms | 3.9ms |
|
||||
| 32 samples | 4.6ms | 4.9ms | 5.7ms |
|
||||
| 64 samples | 5.8ms | 7.2ms | 8.6ms |
|
||||
| 128 samples | 9.2ms | 11.9ms | 14.6ms |
|
||||
|
||||
### Can a 32-channel, 4-dilation ConvNet process 256 samples in under 10ms?
|
||||
**Yes, easily.** This is a Feather-class model. ~98K multiply-adds total. NEON at 1.5GHz: ~16us theoretical. With tanh overhead: well under 1ms.
|
||||
|
||||
### NAM A2 Slimmable performance
|
||||
Designed for this use case. Quality dial: 0.0 (narrow) to 1.0 (full). RT-safe switching. Recommended: quality=0.3-0.5 for 5.33ms block budget.
|
||||
|
||||
---
|
||||
|
||||
## 5. Existing Projects
|
||||
|
||||
### rerdavies/pipedal (★277) — THE RPi NAM pedal
|
||||
- https://github.com/rerdavies/pipedal
|
||||
- Full guitar pedal for RPi 4/5 with web UI (phone-first design)
|
||||
- **v2.0 includes NAM A2**, Tone3000 integration, LV2 plugins
|
||||
- **Measured round-trip latency as low as 2.7ms** (16smp x 3 periods)
|
||||
- **Most mature RPi NAM project — the benchmark**
|
||||
|
||||
### rerdavies/ToobAmp (★86)
|
||||
- https://github.com/rerdavies/ToobAmp
|
||||
- Optimized LV2 plugin set for RPi, A72/A76 build configs
|
||||
- -mcpu=cortex-a72/a76 optimization flags
|
||||
|
||||
### Klinenator/raspi-NAM (★1)
|
||||
- https://github.com/Klinenator/raspi-NAM
|
||||
- Minimal C++ host using NeuralAmpModelerCore + PortAudio
|
||||
- Pisound hat, Flask web UI. Builds natively on RPi OS 64-bit.
|
||||
|
||||
### tone-3000/nam-pedal (★38)
|
||||
- https://github.com/tone-3000/NAMPedal
|
||||
- NAM on Daisy Seed (STM32H750 Cortex-M7 @ 480MHz)
|
||||
- Blog: https://www.tone3000.com/blog/running-nam-on-embedded-hardware
|
||||
- Proves NAM runs on 30x weaker hardware than RPi. NAMB binary format.
|
||||
|
||||
### mikeoliphant/Stompbox (★128)
|
||||
- https://github.com/mikeoliphant/Stompbox
|
||||
- Digital pedalboard using NeuralAudio engine (same engine as LV2 plugin)
|
||||
|
||||
---
|
||||
|
||||
## Key Recommendations
|
||||
|
||||
### Budget: 256 samples / 48kHz = 5.33ms per block
|
||||
|
||||
| Approach | Feather | Standard | Plugin |
|
||||
|----------|:-------:|:--------:|:------:|
|
||||
| Python PyTorch (current) | ~3ms OK | 5-10ms MARGINAL | none |
|
||||
| OpenSauce/nam-rs (Rust) | ~0.5-1ms OK | 1.9-3.8ms OK | library |
|
||||
| neural-amp-modeler-lv2 (C++) | <0.5ms OK | 1-2ms OK | LV2 |
|
||||
| PiPedal (C++) | <0.5ms OK | 1-2ms OK | LV2 |
|
||||
|
||||
### Recommended path
|
||||
1. **Short-term:** Keep Python + PyTorch with Feather models only (current code works)
|
||||
2. **Medium-term:** Build neural-amp-modeler-lv2 natively on RPi for LV2 integration
|
||||
3. **Long-term:** Port to OpenSauce/nam-rs for pure-Rust, aarch64-native, A2 Slimmable support with runtime quality/speed dialing
|
||||
@@ -0,0 +1,100 @@
|
||||
# NAM Model Integration — Technical Reference
|
||||
|
||||
## Architecture
|
||||
|
||||
The NAM model integration uses the `neural-amp-modeler` Python package
|
||||
(`nam` v0.13.0) to load, host, and run neural amp model inference in
|
||||
real-time. The pipeline is:
|
||||
|
||||
```
|
||||
Guitar → [Gate → Comp → Boost → NAM Amp → IR Cab → EQ → ...] → Out
|
||||
│
|
||||
NAMHost.process()
|
||||
(PyTorch inference)
|
||||
```
|
||||
|
||||
## Supported Model Architectures
|
||||
|
||||
| Architecture | CPU/RPi | Real-time | Notes |
|
||||
|-------------|---------|-----------|-------|
|
||||
| **Linear** | ✅ Best | ✅ | Simplest, lowest CPU. Recommended for feather models on RPi 4B |
|
||||
| **LSTM** | ⚠️ OK | ✅ | Medium CPU, sequential state |
|
||||
| **WaveNet** | ⚠️ OK | ✅ | Best tone quality, highest CPU. Use only feather variants (< 10 MB) |
|
||||
| **ConvNet** | ❌ No | ❌ | Not supported by `init_from_nam()` in v0.13.0 |
|
||||
|
||||
## CPU Budget Calculation
|
||||
|
||||
RPi 4B real-time budget at 48kHz / 256-sample blocks:
|
||||
|
||||
- **Block duration:** 256 / 48000 = **5.33 ms**
|
||||
- **Budget per block:** ≤ 4.5 ms (leaving ~0.8 ms for JACK + other FX)
|
||||
|
||||
Tested performance (x86 reference, conv/linear models):
|
||||
|
||||
| Model Size | Params | x86 Inference | Est. RPi 4B | Recommendation |
|
||||
|-----------|--------|--------------|--------------|----------------|
|
||||
| < 100 KB | < 100 | < 0.5 ms | < 2 ms | ✅ Always safe |
|
||||
| 100-500 KB| 100-500| 0.5-1.5 ms | 2-5 ms | ✅ Most models |
|
||||
| 500 KB-5 MB| 500-5K| 1.5-5 ms | 5-20 ms | ⚠️ May xrun |
|
||||
| > 5 MB | > 5K | > 5 ms | > 20 ms | ❌ Not real-time |
|
||||
|
||||
## Model Size Limits
|
||||
|
||||
- **Feather models (< 10 MB .nam file):** Recommended for all use cases
|
||||
- **Compact models (< 100 KB):** Ideal for RPi 4B, fit with budget for other FX
|
||||
- **Large models (> 10 MB):** Will cause audio dropouts (xruns) at 48kHz/256
|
||||
|
||||
The `NAMHost.load_model()` logs a warning if a model exceeds the feather threshold.
|
||||
|
||||
## Receptive Field
|
||||
|
||||
The receptive field (in samples) determines:
|
||||
- _Latency:_ minimum pipeline delay = `rf / sample_rate` seconds
|
||||
- _Block size:_ input blocks must be ≥ `rf` samples or they are zero-padded
|
||||
|
||||
Typical values:
|
||||
- Linear: 16 samples (0.33 ms @ 48kHz)
|
||||
- WaveNet feather: 64-512 samples (1.3-10.7 ms @ 48kHz)
|
||||
- LSTM: 1 sample (stateless per-sample processing)
|
||||
|
||||
## Model Switching
|
||||
|
||||
Three modes to prevent audio dropout when switching presets:
|
||||
|
||||
| Mode | Description | Latency |
|
||||
|------|-------------|---------|
|
||||
| **INSTANT** | Immediate switch, possible click/pop | 0 |
|
||||
| **CROSSFADE** | 256-sample fade (default) | 5.3 ms at 48kHz |
|
||||
| **PAUSE** | Brief silence during switch | ~1 block |
|
||||
|
||||
## Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `src/dsp/nam_host.py` | NAMHost class: load, infer, switch models |
|
||||
| `src/dsp/pipeline.py` | AudioPipeline: wires NAM_AMP block into FX chain |
|
||||
| `src/presets/types.py` | FXBlock.nam_model_path: per-preset model path |
|
||||
| `scripts/download_models.sh` | Downloader/synthetic generator for test models |
|
||||
| `tests/test_nam_host.py` | 25 tests covering lifecycle, inference, switching, edge cases |
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Generate 10 test models (~30-80 KB each)
|
||||
./scripts/download_models.sh
|
||||
|
||||
# Run all tests
|
||||
python3 -m pytest tests/test_nam_host.py -v
|
||||
|
||||
# Integration test
|
||||
cd src && python3 -c "
|
||||
from dsp.nam_host import NAMHost
|
||||
import numpy as np
|
||||
host = NAMHost()
|
||||
host.load_model('$HOME/.pedal/nam/Marshall_JCM800.nam')
|
||||
t = np.linspace(0, 256/48000, 256, dtype=np.float32)
|
||||
sine = np.sin(2 * np.pi * 440 * t) * 0.5
|
||||
out = host.process(sine)
|
||||
print(f'RMS: {np.sqrt(np.mean(out**2)):.4f}')
|
||||
"
|
||||
```
|
||||
@@ -0,0 +1,207 @@
|
||||
# NAM A2 Inference on RPi 4B — Research & Recommendations
|
||||
|
||||
**Status:** Complete — June 2026
|
||||
**Context:** pi-multifx-pedal, BLOCK_SIZE=256, SAMPLE_RATE=48000 (5.33ms per block)
|
||||
**See also:** [NAM_RESEARCH.md](NAM_RESEARCH.md) for detailed source citations
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
NAM A2 inference is **feasible in real-time** on RPi 4B with Feather-class models using any approach. Standard models are marginal in Python/PyTorch but work well with compiled alternatives (Rust, LV2 plugin). The RPi 4B Cortex-A72 at 1.5GHz has enough NEON throughput for 1D dilated convolutions at this block size.
|
||||
|
||||
**Bottom line:** Current Python skeleton code works for Feather models. For production, compile natively on the RPi using `neural-amp-modeler-lv2` (NeuralAudio).
|
||||
|
||||
---
|
||||
|
||||
## 1. NAM Model Architecture
|
||||
|
||||
NAM supports 4 architectures, exported as `.nam` JSON files (not TorchScript):
|
||||
|
||||
| Architecture | Parameters (default) | Receptive Field | A2 Slimmable? | Use Case |
|
||||
|---|---|---|---|---|
|
||||
| **ConvNet** | 6,369 (32ch, 4 dilations) | 16 samples | Yes | Default — best quality/speed trade-off |
|
||||
| **WaveNet** | ~33K (16->8ch, 10 dilations) | 512+ samples | Yes | Highest quality, most expensive |
|
||||
| **Linear** | 128 (1x128) | 128 samples | No | Clean/crunch tones, extremely cheap |
|
||||
| **LSTM** | ~18K | — | No | Vintage/character tones |
|
||||
|
||||
### ConvNet (default — the most common)
|
||||
- 4 dilated 1D Conv1d blocks: dilations [1, 2, 4, 8], kernel size 2
|
||||
- Input: channels=1, hidden: channels=32, output: channels=1 via head Conv1d
|
||||
- Activation: Tanh per layer
|
||||
- Receptive field: sum(dilations) + 1 = 16 samples
|
||||
- ~6,240 MACs per inference
|
||||
|
||||
### NAM A2 SlimmableContainer
|
||||
- Dynamically scales channel count at runtime (quality=0.0→narrow, 1.0→full)
|
||||
- RT-safe — can switch quality between blocks
|
||||
- **Recommended:** quality=0.3-0.5 for RPi 4B at 256-block
|
||||
|
||||
---
|
||||
|
||||
## 2. Latency Benchmarks (x86_64, PyTorch CPU)
|
||||
|
||||
Measured on x86_64. **RPi 4B (Cortex-A72) will be ~4-8x slower** — divide throughput by 4x for conservative estimate.
|
||||
|
||||
| Model | Params | 128-block (2.67ms) | 256-block (5.33ms) | 512-block (10.67ms) |
|
||||
|---|---|---|---|---|
|
||||
| ConvNet Tiny (16ch) | 1,649 | **1.14ms** (43%) | **0.60ms** (11%) | **0.64ms** (6%) |
|
||||
| ConvNet Default (32ch) | 6,369 | **1.64ms** (62%) | **0.74ms** (14%) | **4.78ms** (45%) |
|
||||
| ConvNet Medium (48ch, 5 dil) | 18,817 | 8.63ms (324%) | **2.64ms** (50%) | 21.23ms (199%) |
|
||||
| ConvNet Large (64ch, 6 dil) | 41,537 | 2.60ms (98%) | 11.84ms (222%) | 13.18ms (124%) |
|
||||
| Linear 1x64 | 64 | **0.10ms** (4%) | **0.11ms** (2%) | **0.11ms** (1%) |
|
||||
| Linear 1x128 | 128 | **0.10ms** (4%) | **0.11ms** (2%) | **0.11ms** (1%) |
|
||||
|
||||
**Key finding:** Default ConvNet at 256-block takes **0.74ms on x86** — ~3-6ms estimated on RPi 4B. This is within budget but leaves little headroom with Python overhead.
|
||||
|
||||
---
|
||||
|
||||
## 3. Approach Comparison
|
||||
|
||||
| Approach | Feather | Standard | Plugin | aarch64? | Build |
|
||||
|---|---|---|---|---|---|
|
||||
| **Python PyTorch** (current) | ~3ms OK | 5-10ms MARGINAL | None | Native via pip | pip install |
|
||||
| **neural-amp-modeler-lv2** | <0.5ms OK | 1-2ms OK | LV2 | **Compiles native** | cmake on RPi |
|
||||
| **OpenSauce/nam-rs** | 0.5-1ms OK | 1.9-3.8ms OK | Library crate | cargo build | Pure Rust |
|
||||
| **PiPedal C++** | <0.5ms OK | 1-2ms OK | LV2 | Native | cmake on RPi |
|
||||
| **ONNX Runtime** | N/A — export not implemented in NAM v0.13.0 |
|
||||
|
||||
### 3a. Python PyTorch (current code) ⭐ Short-term
|
||||
|
||||
**Installed version:** `neural-amp-modeler` v0.13.0 via pip.
|
||||
|
||||
**Pros:** Already works. Easy model loading via JSON. Can use `torch.jit.script` or `torch.compile` for ~2x speedup.
|
||||
|
||||
**Cons:**
|
||||
- Python call overhead per block (~50-200us)
|
||||
- PyTorch tensor reshaping/dispatch overhead
|
||||
- JACK callback in Python — GC pauses risk xruns
|
||||
- libtorch on aarch64 is big (hundreds of MB)
|
||||
|
||||
**Feather models only** — standard models will struggle on RPi 4B.
|
||||
|
||||
### 3b. neural-amp-modeler-lv2 (NeuralAudio) ⭐ Medium-term
|
||||
|
||||
**Repository:** `mikeoliphant/neural-amp-modeler-lv2` (★460)
|
||||
|
||||
**Pros:**
|
||||
- **Closest to production-ready** — LV2 plugin integrates directly with JACK
|
||||
- NeuralAudio engine claims performance edge over NAM Core
|
||||
- Compiles natively on RPi OS 64-bit via cmake
|
||||
- Supports A1 + A2, WaveNet + LSTM + ConvNet
|
||||
|
||||
**Cons:**
|
||||
- No prebuilt ARM binaries — must compile on-device
|
||||
- Needs build dependencies (cmake, pkg-config, libsndfile, LV2 SDK)
|
||||
- No prebuilt Debian packages
|
||||
|
||||
**Build commands (on RPi):**
|
||||
```bash
|
||||
sudo apt install cmake pkg-config libsndfile-dev lv2-dev
|
||||
git clone https://github.com/mikeoliphant/neural-amp-modeler-lv2
|
||||
cd neural-amp-modeler-lv2
|
||||
cmake -Bbuild -DCMAKE_BUILD_TYPE=Release -DUSE_NATIVE_ARCH=ON
|
||||
cmake --build build -j4
|
||||
sudo cmake --install build
|
||||
```
|
||||
|
||||
### 3c. OpenSauce/nam-rs (Pure Rust) ⭐ Long-term
|
||||
|
||||
**Repository:** `github.com/OpenSauce/nam-rs`
|
||||
|
||||
**Pros:**
|
||||
- Pure Rust — no C++ dependency chain, no libtorch
|
||||
- First-class aarch64 support via `cargo build --target=aarch64-unknown-linux-gnu`
|
||||
- **A2 SlimmableContainer** — runtime quality/speed dialing
|
||||
- Published on crates.io
|
||||
|
||||
**Cons:**
|
||||
- No LV2 plugin — library crate only (would need Rust LV2 wrapper)
|
||||
- Very new project (June 2026), unproven in production
|
||||
- Estimated same perf as LV2 option
|
||||
|
||||
### 3d. PiPedal C++ ⭐ Reference
|
||||
|
||||
**Repository:** `rerdavies/pipedal` (★277)
|
||||
|
||||
**Pros:**
|
||||
- Most mature RPi NAM project — measured 2.7ms round-trip latency
|
||||
- Full web UI, tone3000 integration, LV2 plugins
|
||||
- Proves the concept works
|
||||
|
||||
**Cons:**
|
||||
- Not a library to reuse — full standalone application
|
||||
|
||||
### 3e. ONNX Runtime ❌ Dead end
|
||||
|
||||
- `Exportable.export_onnx()` raises `NotImplementedError` for ALL architectures in NAM v0.13.0
|
||||
- No implementation exists anywhere in the codebase
|
||||
- Even if it did, ONNX Runtime on aarch64 doesn't outperform PyTorch for Conv1D workloads
|
||||
|
||||
---
|
||||
|
||||
## 4. Model Size Recommendations for RPi 4B
|
||||
|
||||
Based on benchmark extrapolation (x86 * 4-8x for RPi):
|
||||
|
||||
| FAMILY | Channels | Dilations | Params | Est. RPi Latency (256-block) | Recommendation |
|
||||
|---|---|---|---|---|---|
|
||||
| **Nano** | 4/2ch | [1,2,4] | ~500 | **0.1-0.2ms** | Always safe |
|
||||
| **Feather** | 8/4ch | [1,2,4,8,16] | ~2,500 | **0.5-1ms** | Default for Python |
|
||||
| **Lite** | 12/6ch | [1,2,4,8,16,32] | ~5,500 | **1-2ms** | OK with compiled |
|
||||
| **Standard** | 16/8ch | [1,2,4,8,16,32,64,128] | ~18,000 | **2-4ms** | Only with compiled |
|
||||
| **Heavy** | 32/16ch | 10+ dilations | ~70,000 | **5-10ms** | Too expensive |
|
||||
|
||||
---
|
||||
|
||||
## 5. NAM A2 Slimmable Quality Dial
|
||||
|
||||
The A2 SlimmableContainer is the ideal RPi solution:
|
||||
- `quality=0.0` → Nano-class (narrow, fast)
|
||||
- `quality=0.3` → Feather-class (good tone, 0.5ms on RPi)
|
||||
- `quality=0.5` → Lite-class (very good tone, 1ms on RPi)
|
||||
- `quality=1.0` → Standard-class (best tone, 2-4ms on RPi)
|
||||
|
||||
Switch quality between presets or even per-block for adaptive RT.
|
||||
|
||||
---
|
||||
|
||||
## 6. NAM Model Loading (.nam format)
|
||||
|
||||
The `.nam` file is a JSON dictionary (NOT TorchScript):
|
||||
|
||||
```json
|
||||
{
|
||||
"version": "0.13.0",
|
||||
"architecture": "ConvNet", // or "WaveNet", "Linear", "LSTM"
|
||||
"metadata": { "date": "...", "loudness": X, "gain": Y },
|
||||
"config": { "channels": 32, "dilations": [1,2,4,8] },
|
||||
"weights": [0.001, -0.023, ...] // flat 1D array
|
||||
}
|
||||
```
|
||||
|
||||
The Python NAMHost should:
|
||||
1. Load JSON
|
||||
2. Reconstruct the model from `architecture` + `config`
|
||||
3. Call `import_weights(weights)` to set parameters
|
||||
4. Run `model(x_tensor)` for inference
|
||||
|
||||
---
|
||||
|
||||
## 7. Current project's path forward
|
||||
|
||||
### Phase 1 — Python + Feather (now)
|
||||
- Current `nam_host.py` and `pipeline.py` are adequate
|
||||
- Limit to Feather/Nano NAM models
|
||||
- Add `torch.compile()` for ~2x speedup
|
||||
- Keep BLOCK_SIZE=256, SAMPLE_RATE=48000
|
||||
|
||||
### Phase 2 — Compiled inference (after RPi hardware arrives)
|
||||
- On first boot, compile `neural-amp-modeler-lv2` natively
|
||||
- Switch to LV2 plugin chain in JACK
|
||||
- This gives 2-4x more CPU headroom for other FX
|
||||
|
||||
### Phase 3 — Rust native (future)
|
||||
- Port to `nam-rs` for pure-Rust implementation
|
||||
- A2 Slimmable runtime quality dial
|
||||
- No libtorch dependency — saves ~300MB on the Pi
|
||||
Reference in New Issue
Block a user