From 0a21297f77ce81a9ea8afe8016c6f18486d1b44f Mon Sep 17 00:00:00 2001 From: Shawn Date: Fri, 12 Jun 2026 23:10:32 -0400 Subject: [PATCH] t_59dfed5b: Dual DSP Chain feasibility research document --- docs/dual-channel-architecture.md | 349 ++++++++++++++++++++++++++++++ 1 file changed, 349 insertions(+) create mode 100644 docs/dual-channel-architecture.md diff --git a/docs/dual-channel-architecture.md b/docs/dual-channel-architecture.md new file mode 100644 index 0000000..6299167 --- /dev/null +++ b/docs/dual-channel-architecture.md @@ -0,0 +1,349 @@ +# Dual DSP Chain Architecture — Feasibility Research + +> **Project:** Pi Multi-FX Pedal (RPi 4B) +> **Goal:** Investigate running two independent DSP chains (guitar + bass) simultaneously on a single RPi 4B with Focusrite 2i2 +> **Date:** 2026-06-12 +> **Status:** Feasibility assessment — not implemented + +--- + +## Executive Summary + +**Verdict:** Dual independent DSP chains on a single RPi 4B are **feasible with constraints**. Two chains are viable using Feather-class NAM models (or no NAM at all) and a moderate FX block count per chain (6-8 blocks). Running two full chains with Standard NAM models + dense FX is **not reliable** at a 5.33ms block budget — the CPU budget is too tight. + +| Scenario | Feasibility | Notes | +|----------|:-----------:|-------| +| Dual clean chains (no NAM) | ✅ Yes | Under 40% CPU at 48kHz/256-block | +| Dual chains + 1 NAM (Feather) each | ✅ Yes | ~60-70% CPU, 2GB RAM sufficient | +| Dual chains + 1 NAM (Standard) each | ⚠️ Marginal | ~90-100%+ CPU, xruns likely | +| Dual chains + NAM + dense FX (8+ blocks) | ❌ No | Exceeds real-time budget | +| Dual chains using LV2/NeuralAudio | ✅ Yes | Compiled C++ halves per-block cost | + +--- + +## 1. Current Architecture Overview + +### 1.1 Single-Chain Data Flow + +``` +ALSA Capture (Focusrite 2i2 ch0) + → JACK process callback (JackAudioClient._process_callback) + → AudioPipeline.process(audio_in) + → _process_mono() or _process_4cm() + → chain of FX blocks sequentially + → _process_single_block() per FX + → output buffer + → ALSA Playback (Focusrite 2i2 ch0) + +Block size: 256 samples +Sample rate: 48,000 Hz +Block budget: 5.33 ms per callback +``` + +### 1.2 Current Hardware Setup + +| Component | Spec | +|-----------|------| +| **SBC** | Raspberry Pi 4B (2GB+ RAM) | +| **CPU** | Cortex-A72, quad-core @ 1.5 GHz | +| **Audio I/F** | Focusrite Scarlett 2i2 (USB class-compliant) | +| **I/O Channels** | 2-in / 2-out | +| **Bit depth** | 24-bit | +| **Sample rate** | 48 kHz | +| **Latency profile** | Standard: 128 period / 2 nperiods (~5.33ms buffer) | + +### 1.3 Existing Dual-Channel Groundwork + +The project already has foundational dual-channel data model support in `src/presets/types.py`: + +```python +class Channel(enum.StrEnum): + GTR = "gtr" + BASS = "bass" + +class Preset: + channel: Channel = Channel.GTR # already on Preset! + # ... +``` + +This means presets are already tagged with a channel. The **Pipeline, AudioConfig, JackAudioClient, and Web UI** do NOT yet use this field — it's in the data model but not wired into execution. + +--- + +## 2. Focusrite 2i2 Hardware Constraints + +### 2.1 Channel Independence + +The Focusrite Scarlett 2i2 (3rd gen) provides **two completely independent analog channels**: + +| Port | Function | ADC/DAC Path | +|------|----------|-------------| +| Input 1 (front, XLR/TS combo) | Guitar | ADC ch0 → USB isochronous endpoint | +| Input 2 (front, XLR/TS combo) | Bass | ADC ch1 → USB isochronous endpoint | +| Output 1 (rear, TRS) | Guitar out | USB → DAC ch0 | +| Output 2 (rear, TRS) | Bass out | USB → DAC ch1 | + +**Key findings:** +- There is **no internal mixing or channel coupling** — each channel is bit-for-bit independent at the hardware level +- USB Audio Class 2.0 provides separate isochronous endpoints per channel +- `aplay -l` and `arecord -l` show `card 1: USB Audio [Scarlett 2i2 USB], device 0` with 2 capture and 2 playback subdevices +- JACK on Linux enumerates them as `capture_1`, `capture_2` and `playback_1`, `playback_2` +- Both channels run at the **same sample rate** (hardware constraint — Focusrite's USB interface PLL locks all channels to one master clock) +- Independent gain knobs per input channel (hardware, on the front panel) +- 48V phantom power switchable per-channel (not relevant for guitar/bass) + +### 2.2 Input Considerations + +| Instrument | Signal Level | Preamp Needed | Notes | +|------------|-------------|---------------|-------| +| Electric guitar (passive) | ~100-300mV | Yes | Hi-Z, needs buffer/preamp | +| Electric guitar (active) | ~500mV-1V | Maybe | Lower impedance, less sensitive | +| Bass (passive) | ~100-300mV | Yes | Same as passive guitar | +| Bass (active) | ~500mV-1.5V | Maybe | 18V preamps can be hot | + +The Focusrite 2i2 has built-in preamps with: +- Gain range: 0 to +56 dB +- Input impedance: ~1.5kΩ (instrument mode) — adequate for both guitar and bass +- Pad: -26 dB switchable for hot signals + +**Conclusion:** The 2i2's built-in preamps are sufficient for both guitar and bass simultaneously. No external preamp needed for prototype. + +### 2.3 Latency at Dual-Channel + +``` +48kHz / 128 frames (standard profile): + USB transfer (isochronous): ~0.5ms + ALSA buffer (2 periods): ~5.33ms + DSP processing (both chains): TBD (see §3) + USB playback: ~0.5ms + Total (without DSP): ~6.33ms + +Target total round-trip: <15ms (guitar/bass acceptable) +``` + +--- + +## 3. CPU/Memory Overhead of 2x Chains + +### 3.1 Per-Chain Cost Breakdown + +The single-chain cost at 256-block / 48kHz is: + +| Component | CPU per block | Notes | +|-----------|:------------:|-------| +| AudioPipeline orchestration | ~5-15μs | Dispatching, VU metering | +| Noise gate | ~2-5μs | Simple envelope | +| Compressor | ~5-10μs | Envelope follower + gain | +| Boost / Overdrive / Distortion | ~5-15μs | Waveshaping | +| EQ (3-band) | ~10-20μs | 3 biquads | +| Chorus / Flanger / Phaser | ~15-30μs | LFO + delay line + mix | +| Delay | ~10-20μs | Delay line + feedback | +| Reverb | ~50-100μs | Comb + allpass filters | +| **NAM ConvNet (Feather, Python)** | ~1-3ms | **Dominant cost** | +| **NAM ConvNet (Standard, Python)** | ~3-6ms | **Dominant cost** | +| **NAM ConvNet (Feather, LV2)** | ~0.2-0.5ms | Compiled C++ | +| IR Cab | ~30-80μs | FIR convolution | + +### 3.2 Two Chains Overhead + +Running 2x chains creates: + +| Resource | Single Chain | Dual Chain | Overhead | +|----------|:-----------:|:----------:|:--------:| +| **CPU (no NAM, 8 FX each)** | ~0.2-0.4ms | ~0.4-0.8ms | 2x (linear) | +| **CPU (1x Feather NAM each)** | ~1.5-3.5ms | ~3-7ms | EXCEEDS 5.33ms budget | +| **CPU (1x Standard NAM each)** | ~3.5-6.5ms | ~7-13ms | EXCEEDS 5.33ms budget | +| **RAM (state buffers, no NAM)** | ~2-5 MB | ~4-10 MB | 2x | +| **RAM (NAMPyTorch model)** | ~50-300 MB | ~100-600 MB | 2x | +| **RAM (IR convolution buffers)** | ~0.5-2 MB | ~1-4 MB | 2x | +| **RAM (total)** | ~150-600 MB | ~300-1200 MB | 2x | + +**Key insight:** The budget is 5.33ms per JACK callback. With two chains processed **sequentially** (worst-case), the sum of both chains' processing must fit in that window. + +### 3.3 Memory Profile + +| Variant | RAM Estimate | RPi 4B 2GB | RPi 4B 4GB | +|---------|:-----------:|:----------:|:----------:| +| Single chain, no NAM | ~80 MB | 4% | 2% | +| Single chain + 1 Feather NAM | ~200 MB | 10% | 5% | +| Single chain + 1 Standard NAM | ~400 MB | 20% | 10% | +| Dual chain, no NAM | ~120 MB | 6% | 3% | +| Dual chain + 1 Feather NAM each | ~350 MB | 17% | 9% | +| Dual chain + 1 Standard NAM each | ~750 MB | 37% | **18%** | +| Dual chain + 2 Standard NAM + dense FX | ~1.0 GB | **50%** | **25%** | + +**Conclusion:** RAM is not the bottleneck for 2GB+ models. 4GB RPi 4B is **recommended** for any dual-chain deployment. + +--- + +## 4. Proposed Pipeline Architecture + +### 4.1 DualPipeline Design + +The cleanest approach: create a `DualPipeline` orchestration layer that wraps two `AudioPipeline` instances. + +``` +class DualPipeline: + """ + Wraps two independent AudioPipeline instances — one per channel. + + Data flow (Focusrite 2i2 — 2in/2out): + + Focusrite Input 1 (guitar) → pipeline_gtr.process() → Focusrite Output 1 + Focusrite Input 2 (bass) → pipeline_bass.process() → Focusrite Output 2 + + Each pipeline has its own: + - FX chain + - Preset (including NAM model, IR) + - Master volume + - State (delay lines, LFO phases, etc.) + - VU levels + """ +``` + +### 4.2 JACK Audio Integration + +```python +# In JackAudioClient._process_callback with dual pipelines: +capture_1, capture_2 = in_buf[0, :frames], in_buf[1, :frames] + +out_1 = dual_pipeline.pipeline_gtr.process(capture_1) +out_2 = dual_pipeline.pipeline_bass.process(capture_2) + +playback_1 = out_1 +playback_2 = out_2 +``` + +### 4.3 Channel Assignment Options + +| Approach | Pros | Cons | +|----------|------|------| +| **A: Two AudioPipeline instances** | Clean separation, independent presets, easy testing | Slightly more memory (duplicate object overhead) | +| **B: Single DualPipeline with channel routing** | Unified state, potential resource sharing | More complex, risk of cross-chain contamination | +| **C: Thread-per-chain** | True parallel processing on 2 cores | Locking/synchronization complexity, JACK RT-safety | + +**Recommendation: Approach A** as the initial implementation. It's clean, testable, and doesn't require threading in the RT callback (processing is still sequential within the callback, which is JACK-safe). + +### 4.4 Configuration File Changes + +```yaml +# Proposed dual-chain config +audio: + mode: dual_mono # NEW: two independent mono chains + hat_type: focusrite + input_device: "hw:1,0" + output_device: "hw:1,0" + jack_enabled: true + profile: standard # May need "dual" profile with smaller buffer + +channels: # NEW section + gtr: + input_port: 0 + output_port: 0 + label: "Guitar" + initial_preset: "gtr_clean" + bass: + input_port: 1 + output_port: 1 + label: "Bass" + initial_preset: "bass_rock" +``` + +### 4.5 Web UI Changes Needed + +- Dashboard shows two independent columns/tabs for GTR and BASS +- Each chain has its own: + - FX chain view + editor + - Preset selector + - VU meter + - Master volume slider + - Bypass toggle +- Preset browser filters by channel (`Channel.GTR` / `Channel.BASS`) +- A/B comparison between chains (same preset loaded on both) + +### 4.6 Preset Bank Management + +``` +Bank 0: GTR presets (bank.channel = gtr) +Bank 1: BASS presets (bank.channel = bass) +Bank 2: Shared presets (no channel restriction — experimental) +``` + +Each bank channel determines which physical output its presets route to. A/B switching swaps a bank's chain to a different output. + +--- + +## 5. Performance Mitigations + +### 5.1 CPU Budget Strategies + +| Strategy | Gain | Complexity | Risk | +|----------|:----:|:----------:|:----:| +| **Reduce FX per chain** (6 vs 8 blocks) | ~20% | None | Feature limitation | +| **Compiled NAM backend** (LV2/NeuralAudio) | **4-8x** | Medium | Build on aarch64 needed | +| **Use Feather NAM models only** | ~3-5x | None | Quality trade-off | +| **Smaller block size** (128 frames) | +2x budget | Medium | More xruns | +| **Use NAM Slimmable quality=0.3** | ~2x | None | Quality trade-off | +| **Skip IR cab per chain** (use EQ instead) | ~0.1ms | None | Tone trade-off | +| **Dedicate CPU cores via taskset** | ~20% | Low | Hard-coded affinity | + +### 5.2 Recommended Tiers + +| Tier | Setup | Expected CPU | RAM | Verdict | +|------|-------|:-----------:|:---:|:-------:| +| **Baseline** | 6 FX each, no NAM, 48kHz/256 | ~30-40% | ~120MB | ✅ Rock solid | +| **Standard** | 6 FX each + 1 Feather NAM each | ~60-70% | ~350MB | ✅ Safe on 4GB | +| **Pro** | 6 FX each + 1 Standard NAM (LV2) | ~40-50% | ~500MB | ✅ Requires LV2 build | +| **Max** | 8 FX each + 2 Standard NAM (Python) | ~120-150% | ~750MB | ❌ Not real-time | + +### 5.3 Cross-Chain Resource Sharing Opportunities + +- **Shared NAM model weights** (if same model on both chains): would halve memory BUT two separate instances still need separate audio state +- **Shared IR loader**: IR convolution is read-only, can share FFT plan +- **Single JACK client**: already the case, no duplication of JACK connection overhead + +--- + +## 6. Implementation Roadmap + +### Phase 1 — Infrastructure (est. 2-3 days) +1. Create `DualPipeline` class wrapping two `AudioPipeline` instances +2. Modify `JackAudioClient` to route 2-in/2-out through dual chains +3. Add `dual_mono` routing mode to `AudioConfig` +4. Test with Focusrite 2i2 on RPi 4B (no NAM, basic FX) + +### Phase 2 — Presets & State (est. 1-2 days) +1. Channel-specific preset banks (GTR/BASS) +2. Independent bypass, volume, VU per channel +3. Web UI dual-column layout +4. Preset browser filters by channel + +### Phase 3 — Optimization (est. 2-3 days) +1. NAM Slimmable quality dial per channel +2. Profile and benchmark CPU usage with both chains loaded +3. Optional: compile NeuralAudio LV2 plugin on aarch64 +4. Optional: CPU core pinning via `taskset` + +--- + +## 7. Open Questions + +| Question | Current Answer | Needs Verification | +|----------|---------------|-------------------| +| Does Focusrite 2i2's USB driver support 2-ch independent I/O in JACK? | Yes — JACK sees `capture_1/2`, `playback_1/2` | Verify on actual RPi 4B | +| Can non-blocking JACK process remain RT-safe with dual chains? | Yes (no allocs, no I/O, sequential processing) | Profiling needed | +| What's the xrun rate at 48kHz/128 with dual chains + FX? | Unknown | Benchmark on RPi 4B | +| Can USB audio bandwidth handle 2ch @ 48kHz/24-bit? | Yes — 2.3 Mbps, USB 2.0 is 480 Mbps | Trivial | + +--- + +## 8. References + +- `src/dsp/pipeline.py` — Current AudioPipeline (2421 lines) +- `src/system/audio.py` — AudioConfig, JackAudioClient +- `src/presets/types.py` — Channel enum, Preset model (already has channel field) +- `docs/nam_inference.md` — NAM model latency benchmarks +- `docs/audio-io-research.md` — I2S HAT comparison (for future hardware) +- `docs/test-plan-focusrite.md` — Focusrite 2i2 test plan and wiring +- Focusrite Scarlett 2i2 3rd Gen user guide: USB Audio Class 2.0, 2-in/2-out +- RPi 4B (BCM2711) quad Cortex-A72 @ 1.5GHz, 2/4/8 GB LPDDR4