t_59dfed5b: Dual DSP Chain feasibility research document

2026-06-12 23:10:32 -04:00
parent 013588f2ca
commit 0a21297f77
1 changed files with 349 additions and 0 deletions
@@ -0,0 +1,349 @@
+# Dual DSP Chain Architecture — Feasibility Research
+
+> **Project:** Pi Multi-FX Pedal (RPi 4B)
+> **Goal:** Investigate running two independent DSP chains (guitar + bass) simultaneously on a single RPi 4B with Focusrite 2i2
+> **Date:** 2026-06-12
+> **Status:** Feasibility assessment — not implemented
+
+---
+
+## Executive Summary
+
+**Verdict:** Dual independent DSP chains on a single RPi 4B are **feasible with constraints**. Two chains are viable using Feather-class NAM models (or no NAM at all) and a moderate FX block count per chain (6-8 blocks). Running two full chains with Standard NAM models + dense FX is **not reliable** at a 5.33ms block budget — the CPU budget is too tight.
+
+| Scenario | Feasibility | Notes |
+|----------|:-----------:|-------|
+| Dual clean chains (no NAM) | ✅ Yes | Under 40% CPU at 48kHz/256-block |
+| Dual chains + 1 NAM (Feather) each | ✅ Yes | ~60-70% CPU, 2GB RAM sufficient |
+| Dual chains + 1 NAM (Standard) each | ⚠️ Marginal | ~90-100%+ CPU, xruns likely |
+| Dual chains + NAM + dense FX (8+ blocks) | ❌ No | Exceeds real-time budget |
+| Dual chains using LV2/NeuralAudio | ✅ Yes | Compiled C++ halves per-block cost |
+
+---
+
+## 1. Current Architecture Overview
+
+### 1.1 Single-Chain Data Flow
+
+```
+ALSA Capture (Focusrite 2i2 ch0)
+  → JACK process callback (JackAudioClient._process_callback)
+      → AudioPipeline.process(audio_in)
+          → _process_mono() or _process_4cm()
+              → chain of FX blocks sequentially
+                  → _process_single_block() per FX
+          → output buffer
+  → ALSA Playback (Focusrite 2i2 ch0)
+
+Block size:    256 samples
+Sample rate:   48,000 Hz
+Block budget:  5.33 ms per callback
+```
+
+### 1.2 Current Hardware Setup
+
+| Component | Spec |
+|-----------|------|
+| **SBC** | Raspberry Pi 4B (2GB+ RAM) |
+| **CPU** | Cortex-A72, quad-core @ 1.5 GHz |
+| **Audio I/F** | Focusrite Scarlett 2i2 (USB class-compliant) |
+| **I/O Channels** | 2-in / 2-out |
+| **Bit depth** | 24-bit |
+| **Sample rate** | 48 kHz |
+| **Latency profile** | Standard: 128 period / 2 nperiods (~5.33ms buffer) |
+
+### 1.3 Existing Dual-Channel Groundwork
+
+The project already has foundational dual-channel data model support in `src/presets/types.py`:
+
+```python
+class Channel(enum.StrEnum):
+    GTR = "gtr"
+    BASS = "bass"
+
+class Preset:
+    channel: Channel = Channel.GTR  # already on Preset!
+    # ...
+```
+
+This means presets are already tagged with a channel. The **Pipeline, AudioConfig, JackAudioClient, and Web UI** do NOT yet use this field — it's in the data model but not wired into execution.
+
+---
+
+## 2. Focusrite 2i2 Hardware Constraints
+
+### 2.1 Channel Independence
+
+The Focusrite Scarlett 2i2 (3rd gen) provides **two completely independent analog channels**:
+
+| Port | Function | ADC/DAC Path |
+|------|----------|-------------|
+| Input 1 (front, XLR/TS combo) | Guitar | ADC ch0 → USB isochronous endpoint |
+| Input 2 (front, XLR/TS combo) | Bass | ADC ch1 → USB isochronous endpoint |
+| Output 1 (rear, TRS) | Guitar out | USB → DAC ch0 |
+| Output 2 (rear, TRS) | Bass out | USB → DAC ch1 |
+
+**Key findings:**
+- There is **no internal mixing or channel coupling** — each channel is bit-for-bit independent at the hardware level
+- USB Audio Class 2.0 provides separate isochronous endpoints per channel
+- `aplay -l` and `arecord -l` show `card 1: USB Audio [Scarlett 2i2 USB], device 0` with 2 capture and 2 playback subdevices
+- JACK on Linux enumerates them as `capture_1`, `capture_2` and `playback_1`, `playback_2`
+- Both channels run at the **same sample rate** (hardware constraint — Focusrite's USB interface PLL locks all channels to one master clock)
+- Independent gain knobs per input channel (hardware, on the front panel)
+- 48V phantom power switchable per-channel (not relevant for guitar/bass)
+
+### 2.2 Input Considerations
+
+| Instrument | Signal Level | Preamp Needed | Notes |
+|------------|-------------|---------------|-------|
+| Electric guitar (passive) | ~100-300mV | Yes | Hi-Z, needs buffer/preamp |
+| Electric guitar (active) | ~500mV-1V | Maybe | Lower impedance, less sensitive |
+| Bass (passive) | ~100-300mV | Yes | Same as passive guitar |
+| Bass (active) | ~500mV-1.5V | Maybe | 18V preamps can be hot |
+
+The Focusrite 2i2 has built-in preamps with:
+- Gain range: 0 to +56 dB
+- Input impedance: ~1.5kΩ (instrument mode) — adequate for both guitar and bass
+- Pad: -26 dB switchable for hot signals
+
+**Conclusion:** The 2i2's built-in preamps are sufficient for both guitar and bass simultaneously. No external preamp needed for prototype.
+
+### 2.3 Latency at Dual-Channel
+
+```
+48kHz / 128 frames (standard profile):
+  USB transfer (isochronous):          ~0.5ms
+  ALSA buffer (2 periods):             ~5.33ms
+  DSP processing (both chains):        TBD (see §3)
+  USB playback:                        ~0.5ms
+  Total (without DSP):                 ~6.33ms
+
+Target total round-trip:               <15ms (guitar/bass acceptable)
+```
+
+---
+
+## 3. CPU/Memory Overhead of 2x Chains
+
+### 3.1 Per-Chain Cost Breakdown
+
+The single-chain cost at 256-block / 48kHz is:
+
+| Component | CPU per block | Notes |
+|-----------|:------------:|-------|
+| AudioPipeline orchestration | ~5-15μs | Dispatching, VU metering |
+| Noise gate | ~2-5μs | Simple envelope |
+| Compressor | ~5-10μs | Envelope follower + gain |
+| Boost / Overdrive / Distortion | ~5-15μs | Waveshaping |
+| EQ (3-band) | ~10-20μs | 3 biquads |
+| Chorus / Flanger / Phaser | ~15-30μs | LFO + delay line + mix |
+| Delay | ~10-20μs | Delay line + feedback |
+| Reverb | ~50-100μs | Comb + allpass filters |
+| **NAM ConvNet (Feather, Python)** | ~1-3ms | **Dominant cost** |
+| **NAM ConvNet (Standard, Python)** | ~3-6ms | **Dominant cost** |
+| **NAM ConvNet (Feather, LV2)** | ~0.2-0.5ms | Compiled C++ |
+| IR Cab | ~30-80μs | FIR convolution |
+
+### 3.2 Two Chains Overhead
+
+Running 2x chains creates:
+
+| Resource | Single Chain | Dual Chain | Overhead |
+|----------|:-----------:|:----------:|:--------:|
+| **CPU (no NAM, 8 FX each)** | ~0.2-0.4ms | ~0.4-0.8ms | 2x (linear) |
+| **CPU (1x Feather NAM each)** | ~1.5-3.5ms | ~3-7ms | EXCEEDS 5.33ms budget |
+| **CPU (1x Standard NAM each)** | ~3.5-6.5ms | ~7-13ms | EXCEEDS 5.33ms budget |
+| **RAM (state buffers, no NAM)** | ~2-5 MB | ~4-10 MB | 2x |
+| **RAM (NAMPyTorch model)** | ~50-300 MB | ~100-600 MB | 2x |
+| **RAM (IR convolution buffers)** | ~0.5-2 MB | ~1-4 MB | 2x |
+| **RAM (total)** | ~150-600 MB | ~300-1200 MB | 2x |
+
+**Key insight:** The budget is 5.33ms per JACK callback. With two chains processed **sequentially** (worst-case), the sum of both chains' processing must fit in that window.
+
+### 3.3 Memory Profile
+
+| Variant | RAM Estimate | RPi 4B 2GB | RPi 4B 4GB |
+|---------|:-----------:|:----------:|:----------:|
+| Single chain, no NAM | ~80 MB | 4% | 2% |
+| Single chain + 1 Feather NAM | ~200 MB | 10% | 5% |
+| Single chain + 1 Standard NAM | ~400 MB | 20% | 10% |
+| Dual chain, no NAM | ~120 MB | 6% | 3% |
+| Dual chain + 1 Feather NAM each | ~350 MB | 17% | 9% |
+| Dual chain + 1 Standard NAM each | ~750 MB | 37% | **18%** |
+| Dual chain + 2 Standard NAM + dense FX | ~1.0 GB | **50%** | **25%** |
+
+**Conclusion:** RAM is not the bottleneck for 2GB+ models. 4GB RPi 4B is **recommended** for any dual-chain deployment.
+
+---
+
+## 4. Proposed Pipeline Architecture
+
+### 4.1 DualPipeline Design
+
+The cleanest approach: create a `DualPipeline` orchestration layer that wraps two `AudioPipeline` instances.
+
+```
+class DualPipeline:
+    """
+    Wraps two independent AudioPipeline instances — one per channel.
+
+    Data flow (Focusrite 2i2 — 2in/2out):
+
+    Focusrite Input 1 (guitar)     → pipeline_gtr.process() → Focusrite Output 1
+    Focusrite Input 2 (bass)       → pipeline_bass.process() → Focusrite Output 2
+
+    Each pipeline has its own:
+        - FX chain
+        - Preset (including NAM model, IR)
+        - Master volume
+        - State (delay lines, LFO phases, etc.)
+        - VU levels
+    """
+```
+
+### 4.2 JACK Audio Integration
+
+```python
+# In JackAudioClient._process_callback with dual pipelines:
+capture_1, capture_2 = in_buf[0, :frames], in_buf[1, :frames]
+
+out_1 = dual_pipeline.pipeline_gtr.process(capture_1)
+out_2 = dual_pipeline.pipeline_bass.process(capture_2)
+
+playback_1 = out_1
+playback_2 = out_2
+```
+
+### 4.3 Channel Assignment Options
+
+| Approach | Pros | Cons |
+|----------|------|------|
+| **A: Two AudioPipeline instances** | Clean separation, independent presets, easy testing | Slightly more memory (duplicate object overhead) |
+| **B: Single DualPipeline with channel routing** | Unified state, potential resource sharing | More complex, risk of cross-chain contamination |
+| **C: Thread-per-chain** | True parallel processing on 2 cores | Locking/synchronization complexity, JACK RT-safety |
+
+**Recommendation: Approach A** as the initial implementation. It's clean, testable, and doesn't require threading in the RT callback (processing is still sequential within the callback, which is JACK-safe).
+
+### 4.4 Configuration File Changes
+
+```yaml
+# Proposed dual-chain config
+audio:
+  mode: dual_mono             # NEW: two independent mono chains
+  hat_type: focusrite
+  input_device: "hw:1,0"
+  output_device: "hw:1,0"
+  jack_enabled: true
+  profile: standard           # May need "dual" profile with smaller buffer
+
+channels:                     # NEW section
+  gtr:
+    input_port: 0
+    output_port: 0
+    label: "Guitar"
+    initial_preset: "gtr_clean"
+  bass:
+    input_port: 1
+    output_port: 1
+    label: "Bass"
+    initial_preset: "bass_rock"
+```
+
+### 4.5 Web UI Changes Needed
+
+- Dashboard shows two independent columns/tabs for GTR and BASS
+- Each chain has its own:
+  - FX chain view + editor
+  - Preset selector
+  - VU meter
+  - Master volume slider
+  - Bypass toggle
+- Preset browser filters by channel (`Channel.GTR` / `Channel.BASS`)
+- A/B comparison between chains (same preset loaded on both)
+
+### 4.6 Preset Bank Management
+
+```
+Bank 0: GTR presets  (bank.channel = gtr)
+Bank 1: BASS presets (bank.channel = bass)
+Bank 2: Shared presets (no channel restriction — experimental)
+```
+
+Each bank channel determines which physical output its presets route to. A/B switching swaps a bank's chain to a different output.
+
+---
+
+## 5. Performance Mitigations
+
+### 5.1 CPU Budget Strategies
+
+| Strategy | Gain | Complexity | Risk |
+|----------|:----:|:----------:|:----:|
+| **Reduce FX per chain** (6 vs 8 blocks) | ~20% | None | Feature limitation |
+| **Compiled NAM backend** (LV2/NeuralAudio) | **4-8x** | Medium | Build on aarch64 needed |
+| **Use Feather NAM models only** | ~3-5x | None | Quality trade-off |
+| **Smaller block size** (128 frames) | +2x budget | Medium | More xruns |
+| **Use NAM Slimmable quality=0.3** | ~2x | None | Quality trade-off |
+| **Skip IR cab per chain** (use EQ instead) | ~0.1ms | None | Tone trade-off |
+| **Dedicate CPU cores via taskset** | ~20% | Low | Hard-coded affinity |
+
+### 5.2 Recommended Tiers
+
+| Tier | Setup | Expected CPU | RAM | Verdict |
+|------|-------|:-----------:|:---:|:-------:|
+| **Baseline** | 6 FX each, no NAM, 48kHz/256 | ~30-40% | ~120MB | ✅ Rock solid |
+| **Standard** | 6 FX each + 1 Feather NAM each | ~60-70% | ~350MB | ✅ Safe on 4GB |
+| **Pro** | 6 FX each + 1 Standard NAM (LV2) | ~40-50% | ~500MB | ✅ Requires LV2 build |
+| **Max** | 8 FX each + 2 Standard NAM (Python) | ~120-150% | ~750MB | ❌ Not real-time |
+
+### 5.3 Cross-Chain Resource Sharing Opportunities
+
+- **Shared NAM model weights** (if same model on both chains): would halve memory BUT two separate instances still need separate audio state
+- **Shared IR loader**: IR convolution is read-only, can share FFT plan
+- **Single JACK client**: already the case, no duplication of JACK connection overhead
+
+---
+
+## 6. Implementation Roadmap
+
+### Phase 1 — Infrastructure (est. 2-3 days)
+1. Create `DualPipeline` class wrapping two `AudioPipeline` instances
+2. Modify `JackAudioClient` to route 2-in/2-out through dual chains
+3. Add `dual_mono` routing mode to `AudioConfig`
+4. Test with Focusrite 2i2 on RPi 4B (no NAM, basic FX)
+
+### Phase 2 — Presets & State (est. 1-2 days)
+1. Channel-specific preset banks (GTR/BASS)
+2. Independent bypass, volume, VU per channel
+3. Web UI dual-column layout
+4. Preset browser filters by channel
+
+### Phase 3 — Optimization (est. 2-3 days)
+1. NAM Slimmable quality dial per channel
+2. Profile and benchmark CPU usage with both chains loaded
+3. Optional: compile NeuralAudio LV2 plugin on aarch64
+4. Optional: CPU core pinning via `taskset`
+
+---
+
+## 7. Open Questions
+
+| Question | Current Answer | Needs Verification |
+|----------|---------------|-------------------|
+| Does Focusrite 2i2's USB driver support 2-ch independent I/O in JACK? | Yes — JACK sees `capture_1/2`, `playback_1/2` | Verify on actual RPi 4B |
+| Can non-blocking JACK process remain RT-safe with dual chains? | Yes (no allocs, no I/O, sequential processing) | Profiling needed |
+| What's the xrun rate at 48kHz/128 with dual chains + FX? | Unknown | Benchmark on RPi 4B |
+| Can USB audio bandwidth handle 2ch @ 48kHz/24-bit? | Yes — 2.3 Mbps, USB 2.0 is 480 Mbps | Trivial |
+
+---
+
+## 8. References
+
+- `src/dsp/pipeline.py` — Current AudioPipeline (2421 lines)
+- `src/system/audio.py` — AudioConfig, JackAudioClient
+- `src/presets/types.py` — Channel enum, Preset model (already has channel field)
+- `docs/nam_inference.md` — NAM model latency benchmarks
+- `docs/audio-io-research.md` — I2S HAT comparison (for future hardware)
+- `docs/test-plan-focusrite.md` — Focusrite 2i2 test plan and wiring
+- Focusrite Scarlett 2i2 3rd Gen user guide: USB Audio Class 2.0, 2-in/2-out
+- RPi 4B (BCM2711) quad Cortex-A72 @ 1.5GHz, 2/4/8 GB LPDDR4