shawn
0e77adb4c3
- Full FFT overlap-add IR convolution in IRLoader (process(), set_mix(), toggle) - Lazy FFT computation — IR FFT padded to correct block+ir size on first process() - Wet/dry mix control, enabled/disabled toggle with tail clearing - Fixed pipeline._apply_ir_cab() to delegate to IRLoader.process() instead of poking internals (old code had array-size mismatch bug: IR FFT at ir_len vs block FFT at conv_size) - 46 tests: loading, convolution correctness, overlap-add state, mix, toggle, directory listing, performance budget (all <5ms even at 8192 taps), edge cases - scripts/download_irs.sh: free IR pack downloader (God's Cab, Seacow)
100 lines
3.5 KiB
Markdown
100 lines
3.5 KiB
Markdown
# NAM Model Integration — Technical Reference
|
|
|
|
## Architecture
|
|
|
|
The NAM model integration uses the `neural-amp-modeler` Python package
|
|
(`nam` v0.13.0) to load, host, and run neural amp model inference in
|
|
real-time. The pipeline is:
|
|
|
|
```
|
|
Guitar → [Gate → Comp → Boost → NAM Amp → IR Cab → EQ → ...] → Out
|
|
│
|
|
NAMHost.process()
|
|
(PyTorch inference)
|
|
```
|
|
|
|
## Supported Model Architectures
|
|
|
|
| Architecture | CPU/RPi | Real-time | Notes |
|
|
|-------------|---------|-----------|-------|
|
|
| **Linear** | ✅ Best | ✅ | Simplest, lowest CPU. Recommended for feather models on RPi 4B |
|
|
| **LSTM** | ⚠️ OK | ✅ | Medium CPU, sequential state |
|
|
| **WaveNet** | ⚠️ OK | ✅ | Best tone quality, highest CPU. Use only feather variants (< 10 MB) |
|
|
| **ConvNet** | ❌ No | ❌ | Not supported by `init_from_nam()` in v0.13.0 |
|
|
|
|
## CPU Budget Calculation
|
|
|
|
RPi 4B real-time budget at 48kHz / 256-sample blocks:
|
|
|
|
- **Block duration:** 256 / 48000 = **5.33 ms**
|
|
- **Budget per block:** ≤ 4.5 ms (leaving ~0.8 ms for JACK + other FX)
|
|
|
|
Tested performance (x86 reference, conv/linear models):
|
|
|
|
| Model Size | Params | x86 Inference | Est. RPi 4B | Recommendation |
|
|
|-----------|--------|--------------|--------------|----------------|
|
|
| < 100 KB | < 100 | < 0.5 ms | < 2 ms | ✅ Always safe |
|
|
| 100-500 KB| 100-500| 0.5-1.5 ms | 2-5 ms | ✅ Most models |
|
|
| 500 KB-5 MB| 500-5K| 1.5-5 ms | 5-20 ms | ⚠️ May xrun |
|
|
| > 5 MB | > 5K | > 5 ms | > 20 ms | ❌ Not real-time |
|
|
|
|
## Model Size Limits
|
|
|
|
- **Feather models (< 10 MB .nam file):** Recommended for all use cases
|
|
- **Compact models (< 100 KB):** Ideal for RPi 4B, fit with budget for other FX
|
|
- **Large models (> 10 MB):** Will cause audio dropouts (xruns) at 48kHz/256
|
|
|
|
The `NAMHost.load_model()` logs a warning if a model exceeds the feather threshold.
|
|
|
|
## Receptive Field
|
|
|
|
The receptive field (in samples) determines:
|
|
- _Latency:_ minimum pipeline delay = `rf / sample_rate` seconds
|
|
- _Block size:_ input blocks must be ≥ `rf` samples or they are zero-padded
|
|
|
|
Typical values:
|
|
- Linear: 16 samples (0.33 ms @ 48kHz)
|
|
- WaveNet feather: 64-512 samples (1.3-10.7 ms @ 48kHz)
|
|
- LSTM: 1 sample (stateless per-sample processing)
|
|
|
|
## Model Switching
|
|
|
|
Three modes to prevent audio dropout when switching presets:
|
|
|
|
| Mode | Description | Latency |
|
|
|------|-------------|---------|
|
|
| **INSTANT** | Immediate switch, possible click/pop | 0 |
|
|
| **CROSSFADE** | 256-sample fade (default) | 5.3 ms at 48kHz |
|
|
| **PAUSE** | Brief silence during switch | ~1 block |
|
|
|
|
## Files
|
|
|
|
| File | Purpose |
|
|
|------|---------|
|
|
| `src/dsp/nam_host.py` | NAMHost class: load, infer, switch models |
|
|
| `src/dsp/pipeline.py` | AudioPipeline: wires NAM_AMP block into FX chain |
|
|
| `src/presets/types.py` | FXBlock.nam_model_path: per-preset model path |
|
|
| `scripts/download_models.sh` | Downloader/synthetic generator for test models |
|
|
| `tests/test_nam_host.py` | 25 tests covering lifecycle, inference, switching, edge cases |
|
|
|
|
## Quick Start
|
|
|
|
```bash
|
|
# Generate 10 test models (~30-80 KB each)
|
|
./scripts/download_models.sh
|
|
|
|
# Run all tests
|
|
python3 -m pytest tests/test_nam_host.py -v
|
|
|
|
# Integration test
|
|
cd src && python3 -c "
|
|
from dsp.nam_host import NAMHost
|
|
import numpy as np
|
|
host = NAMHost()
|
|
host.load_model('$HOME/.pedal/nam/Marshall_JCM800.nam')
|
|
t = np.linspace(0, 256/48000, 256, dtype=np.float32)
|
|
sine = np.sin(2 * np.pi * 440 * t) * 0.5
|
|
out = host.process(sine)
|
|
print(f'RMS: {np.sqrt(np.mean(out**2)):.4f}')
|
|
"
|
|
``` |