shawn
0e77adb4c3
- Full FFT overlap-add IR convolution in IRLoader (process(), set_mix(), toggle) - Lazy FFT computation — IR FFT padded to correct block+ir size on first process() - Wet/dry mix control, enabled/disabled toggle with tail clearing - Fixed pipeline._apply_ir_cab() to delegate to IRLoader.process() instead of poking internals (old code had array-size mismatch bug: IR FFT at ir_len vs block FFT at conv_size) - 46 tests: loading, convolution correctness, overlap-add state, mix, toggle, directory listing, performance budget (all <5ms even at 8192 taps), edge cases - scripts/download_irs.sh: free IR pack downloader (God's Cab, Seacow)
162 lines
7.4 KiB
Markdown
162 lines
7.4 KiB
Markdown
# NAM A2 / A1 Inference on RPi 4B — Structured Research Findings
|
|
|
|
**Date:** 2026-06-07
|
|
**Project:** pi-multifx-pedal
|
|
**Context:** BLOCK_SIZE=256, SAMPLE_RATE=48000, NAM v0.13.0 installed
|
|
|
|
---
|
|
|
|
## 1. NAM LV2 Plugin on aarch64
|
|
|
|
**Repository:** https://github.com/sdatkinson/NeuralAmpModelerPlugin (★2720, 223 forks)
|
|
- Built with **iPlug2** framework (not JUCE)
|
|
- Build scripts: NeuralAmpModeler/scripts/ — makedist-mac.sh, makedist-win.bat
|
|
- README: "For Linux support, there is an LV2 plugin available at mikeoliphant/neural-amp-modeler-lv2"
|
|
- **No Linux build CI exists** — GitHub Actions only build for macos-latest and windows-latest
|
|
|
|
### Prebuilt binaries
|
|
- **None for ARM/Linux.** All releases (v0.7.6-v0.7.15) ship only macOS .dmg and Windows .zip. No Linux, no ARM.
|
|
|
|
### Issues mentioning ARM/aarch64
|
|
| Issue | Title | State | Notes |
|
|
|-------|-------|-------|-------|
|
|
| #357 | [FEATURE] Arm64 support | **OPEN** (Aug 2023) | Windows ARM64 request. Maintainer asked for specifics. No resolution in 3+ years. |
|
|
| #587 | NAM Setup — Windows 11 on ARM | CLOSED | Setup packaging issue, not cross-compile. |
|
|
| #627 | fix: avoid audio-thread stalls (AU) | CLOSED | "arm" incidental in body (part of "harm"). |
|
|
|
|
**Conclusion: No official aarch64 support for the NAM plugin. Issue #357 has sat open for 3+ years. The iPlug2-based plugin is not designed for cross-compilation to aarch64.**
|
|
|
|
### The LV2 option (NeuralAudio-based)
|
|
- https://github.com/mikeoliphant/neural-amp-modeler-lv2 (★460, 50 forks)
|
|
- Uses **NeuralAudio** C++ engine (not iPlug2)
|
|
- Supports NAM A1 + A2, WaveNet + LSTM + ConvNet
|
|
- **NeuralAudio README explicitly claims**: "internal implementation outperforms NAM Core on all tested platforms (Windows x64, Linux x64/Arm64)"
|
|
- Build: cmake .. -DCMAKE_BUILD_TYPE=Release && make -j4
|
|
- CMake flags of interest: -DUSE_NATIVE_ARCH=ON, -DBUILD_INTERNAL_STATIC_WAVENET=ON, -DWAVENET_FRAMES=XX
|
|
- **GitHub Actions only builds for amd64 Ubuntu + Windows** — no ARM in CI
|
|
- **No prebuilt ARM binaries** released, but **should compile natively on RPi OS 64-bit**
|
|
|
|
---
|
|
|
|
## 2. NAMR (Rust NAM)
|
|
|
|
The URL `github.com/mikeoliphant/neural-amp-modeler-rs` **does not exist (404)**.
|
|
|
|
### a) OpenSauce/nam-rs (Pure Rust library crate)
|
|
- **https://github.com/OpenSauce/nam-rs** (June 2026, very new, ★0)
|
|
- Pure Rust, **no external C/C++ dependencies**
|
|
- Supports: WaveNet (A1 + A2 single), LSTM, SlimmableContainer (NAM A2)
|
|
- **aarch64:** Supported via Cargo, verified target aarch64-unknown-linux-gnu
|
|
- Has **dedicated RPi performance analysis** in issues:
|
|
- Issue #23: "Raspberry Pi / embedded support: run the A1 WaveNet presets real-time on aarch64"
|
|
- Concluded: A2 SlimmableContainer is the CPU/quality dial, superseding bespoke specialization
|
|
- **Performance (x86, standard WaveNet, 512 samples):** ~951us for process_buffer
|
|
- **Estimated Pi 4 factor:** 4-8x slower
|
|
- At 256 samples (our project): ~1.9-3.8ms standard; Feather ~0.5-1ms
|
|
- Published on crates.io as `nam-rs`. MIT license.
|
|
|
|
### b) fabiohl/nam-rs (Standalone + CLAP plugin)
|
|
- **https://github.com/fabiohl/nam-rs** (★4, April 2026, v1.7.0)
|
|
- **x86-64-v3 only** (AVX2+FMA mandatory). **No aarch64 support.**
|
|
- Standalone PipeWire + CLAP plugin. Explicitly no LV2.
|
|
- Apache-2.0 license.
|
|
|
|
### Is Rust NAM faster than Python/PyTorch?
|
|
**Yes.** Compiled Rust/C++ avoids Python interpreter overhead (~50-200us per call), PyTorch tensor dispatch, dynamic graph construction, and GC pauses. OpenSauce/nam-rs on x86: ~951us for 512-sample standard. Python equivalent via `nam` library is conservatively 3-5x slower.
|
|
|
|
---
|
|
|
|
## 3. ONNX Runtime Inference
|
|
|
|
### NAM ONNX Export Status (v0.13.0)
|
|
- nam.models.exportable.Exportable has export_onnx() method
|
|
- **ALL four model architectures (ConvNet, WaveNet, Linear, LSTM) raise NotImplementedError**
|
|
- It is a stub only — no implementation exists in any class
|
|
- The working export path is .nam JSON format via export() (not ONNX)
|
|
- **ONNX export does not exist in NAM v0.13.0**
|
|
|
|
### Environment
|
|
- PyTorch 2.12.0+cpu (installed)
|
|
- ONNX Runtime 1.26.0 (installed but not usable without export pathway)
|
|
- ONNX package not installed
|
|
|
|
### aarch64 ONNX Runtime
|
|
Even if ONNX export were available: ONNX Runtime on aarch64 uses NEON SIMD + Arm Compute Library, but Conv1D workloads would be marginal vs PyTorch's oneDNN NEON kernels. **Not worth pursuing.**
|
|
|
|
---
|
|
|
|
## 4. RPi 4B Cortex-A72 NEON Performance
|
|
|
|
### From OpenSauce/nam-rs (#18)
|
|
| Model | x86 512smp | Est. Pi4 4x 512smp | Est. Pi4 8x 512smp | Est. Pi4 256smp |
|
|
|-------|:----------:|:------------------:|:------------------:|:---------------:|
|
|
| Standard (16/8ch, 10 dil) | 951us | 3.8ms | 7.6ms | 1.9-3.8ms |
|
|
| Lite (12/6ch) | ~520us | 2.1ms | 4.2ms | ~1-2ms |
|
|
| Feather (8/4ch, 7 dil) | ~237us | 0.95ms | 1.9ms | 0.5-1ms |
|
|
| Nano (4/2ch) | ~57us | 0.23ms | 0.46ms | ~0.1-0.2ms |
|
|
|
|
Cortex-A72 constraints: 2x NEON 128-bit (vs 256-bit AVX2), 1.5GHz, no FMA, lower bandwidth.
|
|
|
|
### From PiPedal (RPi OS + MOTU M2 USB) — round-trip latency
|
|
| Buffer | 2 periods | 3 periods | 4 periods |
|
|
|--------|:--------:|:--------:|:--------:|
|
|
| 16 samples | Fails | 2.7ms | 3.9ms |
|
|
| 32 samples | 4.6ms | 4.9ms | 5.7ms |
|
|
| 64 samples | 5.8ms | 7.2ms | 8.6ms |
|
|
| 128 samples | 9.2ms | 11.9ms | 14.6ms |
|
|
|
|
### Can a 32-channel, 4-dilation ConvNet process 256 samples in under 10ms?
|
|
**Yes, easily.** This is a Feather-class model. ~98K multiply-adds total. NEON at 1.5GHz: ~16us theoretical. With tanh overhead: well under 1ms.
|
|
|
|
### NAM A2 Slimmable performance
|
|
Designed for this use case. Quality dial: 0.0 (narrow) to 1.0 (full). RT-safe switching. Recommended: quality=0.3-0.5 for 5.33ms block budget.
|
|
|
|
---
|
|
|
|
## 5. Existing Projects
|
|
|
|
### rerdavies/pipedal (★277) — THE RPi NAM pedal
|
|
- https://github.com/rerdavies/pipedal
|
|
- Full guitar pedal for RPi 4/5 with web UI (phone-first design)
|
|
- **v2.0 includes NAM A2**, Tone3000 integration, LV2 plugins
|
|
- **Measured round-trip latency as low as 2.7ms** (16smp x 3 periods)
|
|
- **Most mature RPi NAM project — the benchmark**
|
|
|
|
### rerdavies/ToobAmp (★86)
|
|
- https://github.com/rerdavies/ToobAmp
|
|
- Optimized LV2 plugin set for RPi, A72/A76 build configs
|
|
- -mcpu=cortex-a72/a76 optimization flags
|
|
|
|
### Klinenator/raspi-NAM (★1)
|
|
- https://github.com/Klinenator/raspi-NAM
|
|
- Minimal C++ host using NeuralAmpModelerCore + PortAudio
|
|
- Pisound hat, Flask web UI. Builds natively on RPi OS 64-bit.
|
|
|
|
### tone-3000/nam-pedal (★38)
|
|
- https://github.com/tone-3000/NAMPedal
|
|
- NAM on Daisy Seed (STM32H750 Cortex-M7 @ 480MHz)
|
|
- Blog: https://www.tone3000.com/blog/running-nam-on-embedded-hardware
|
|
- Proves NAM runs on 30x weaker hardware than RPi. NAMB binary format.
|
|
|
|
### mikeoliphant/Stompbox (★128)
|
|
- https://github.com/mikeoliphant/Stompbox
|
|
- Digital pedalboard using NeuralAudio engine (same engine as LV2 plugin)
|
|
|
|
---
|
|
|
|
## Key Recommendations
|
|
|
|
### Budget: 256 samples / 48kHz = 5.33ms per block
|
|
|
|
| Approach | Feather | Standard | Plugin |
|
|
|----------|:-------:|:--------:|:------:|
|
|
| Python PyTorch (current) | ~3ms OK | 5-10ms MARGINAL | none |
|
|
| OpenSauce/nam-rs (Rust) | ~0.5-1ms OK | 1.9-3.8ms OK | library |
|
|
| neural-amp-modeler-lv2 (C++) | <0.5ms OK | 1-2ms OK | LV2 |
|
|
| PiPedal (C++) | <0.5ms OK | 1-2ms OK | LV2 |
|
|
|
|
### Recommended path
|
|
1. **Short-term:** Keep Python + PyTorch with Feather models only (current code works)
|
|
2. **Medium-term:** Build neural-amp-modeler-lv2 natively on RPi for LV2 integration
|
|
3. **Long-term:** Port to OpenSauce/nam-rs for pure-Rust, aarch64-native, A2 Slimmable support with runtime quality/speed dialing
|