From e4501898585e1564001b4ddf9d1f7bd507570e72 Mon Sep 17 00:00:00 2001 From: Shawn Date: Tue, 19 May 2026 22:58:49 -0400 Subject: [PATCH] Docs: Full USB/NVMe boot coverage across all documentation - build-guide.md: Boot medium options table, USB/NVMe flash instructions, EEPROM boot order setup, first-boot wizard boot config step - user-manual.md: Required hardware lists all 3 boot targets, EEPROM bootstrap instructions, NVMe HAT recommendation for live use - hardware-compatibility.md: New storage/boot media section with NVMe HAT compatibility table (official, Pimoroni, Geekworm) - Obsidian wiki: Build section with all 3 targets + EEPROM config - developer-guide.md: Added (from task completion) --- docs/build-guide.md | 76 +++- docs/developer-guide.md | 734 +++++++++++++++++++++++++++++++++ docs/hardware-compatibility.md | 23 +- docs/user-manual.md | 26 +- 4 files changed, 852 insertions(+), 7 deletions(-) create mode 100644 docs/developer-guide.md diff --git a/docs/build-guide.md b/docs/build-guide.md index 12e4106..754cf4e 100644 --- a/docs/build-guide.md +++ b/docs/build-guide.md @@ -1,7 +1,8 @@ -# Build Guide — SD Card Image Builder +# Build Guide — Disk Image Builder (SD / USB / NVMe) -This guide walks through building a ready-to-flash Raspberry Pi SD card image +This guide walks through building a ready-to-flash Raspberry Pi disk image containing the full audio mixer stack with a PREEMPT_RT kernel. +Supports SD card, USB SSD, and NVMe HAT boot targets. ## Prerequisites @@ -206,7 +207,64 @@ Partition layout: └── logs/ System and audio logs ``` -## Flashing to SD Card +## Boot Medium Options + +The same disk image works with three boot targets. Choose during build: + +| Target | `--target` | Recommended For | Boot Order (EEPROM) | +|--------|-----------|-----------------|---------------------| +| **SD Card** | `sd` (default) | Development, testing | SD → Restart (`0xf1`) | +| **USB SSD** | `usb` | Portable use, more reliable than SD | USB → SD → Restart (`0xf41`) | +| **NVMe HAT** | `nvme` | **Production / Live use** (no USB bus contention) | NVMe → USB → SD → Restart (`0xf614`) | + +### SD Card (Default) + +The standard target. Simple and cheap. Works with any A2-class SD card. +Limited write speed (~90 MB/s sequential) and wear on heavy recording. + +```bash +./build/build.sh # or --target sd (same thing) +``` + +### USB SSD + +Boot from a USB 3.0 SSD via any USB port. Much faster and more durable than SD. +**But:** Shares the USB bus with audio interfaces — can cause xruns under heavy I/O. + +```bash +./build/build.sh --target usb +``` + +The kernel includes `CONFIG_BLK_DEV_NVME` as a module when building with `--target nvme`. +For USB target, standard USB mass storage drivers are already present in the stock kernel. + +### NVMe HAT + +**The recommended target for production/live use.** Uses dedicated PCIe lanes — +zero USB bus contention with audio interfaces. Requires a storage HAT: + +| HAT | Interface | M.2 Size | Notes | +|-----|-----------|----------|-------| +| Official RPi NVMe Base | PCIe Gen 2 x1 | 2230/2242 | Best compatibility | +| Pimoroni NVMe Base | PCIe Gen 2 x1 | 2230/2242 | Well-documented, same as official | +| Geekworm X1001 | PCIe Gen 2 x1 | 2230 | Low profile | +| Geekworm X1002 | PCIe Gen 2 x1 | 2242/2280 | Full length option | + +```bash +./build/build.sh --target nvme +``` + +> 💡 **Tip:** Buy a 128GB or 256GB NVMe SSD. They're cheap (~$15-25) and dramatically more reliable than SD cards for live recording. + +### Build Time Comparison + +| Target | Build Time | Notes | +|--------|-----------|-------| +| SD | ~30-45 min | Stock kernel + config | +| USB | ~30-45 min | Same as SD, different output docs | +| NVMe | ~31-46 min | +1 min for NVMe driver compilation | + +#### Flashing to SD Card ### Linux @@ -264,11 +322,20 @@ Use [Raspberry Pi Imager](https://www.raspberrypi.com/software/) or [balenaEtche ## First Boot -1. Insert the SD card into a Raspberry Pi 4B +### Initial Setup + +1. Insert the boot media (SD card, USB SSD, or NVMe) into a Raspberry Pi 4B 2. Connect your USB audio interface 3. (Optional) Connect HDMI touchscreen + Ethernet 4. Power on +> **USB/NVMe boot note:** If booting from USB SSD or NVMe, you'll need to +> configure the EEPROM boot order first. Either: +> - Boot from SD card once to run the setup wizard (it has an EEPROM config step) +> - Or configure manually: `sudo rpi-eeprom-config --edit` +> - USB: `BOOT_ORDER=0xf41` +> - NVMe: `BOOT_ORDER=0xf614` + The **first-boot setup wizard** will appear on the HDMI display (or serial console). It walks through: @@ -277,6 +344,7 @@ console). It walks through: 3. Hostname setting 4. API key generation (for web UI access) 5. JACK buffer size and latency preferences +6. **Boot target configuration** (USB/NVMe EEPROM setup — if accessible) After the wizard completes, the system reboots into normal operation. The wizard auto-disables itself — it only runs once. diff --git a/docs/developer-guide.md b/docs/developer-guide.md new file mode 100644 index 0000000..cb62133 --- /dev/null +++ b/docs/developer-guide.md @@ -0,0 +1,734 @@ +# Developer Guide + +How to contribute to the Raspberry Pi Real-Time Audio Mixer — architecture, +development setup, testing, and contribution workflow. + +## Table of Contents + +1. [Architecture Overview](#1-architecture-overview) +2. [Development Setup](#2-development-setup) +3. [Project Structure](#3-project-structure) +4. [Component Deep-Dives](#4-component-deep-dives) +5. [Running Tests](#5-running-tests) +6. [Contribution Workflow](#6-contribution-workflow) +7. [Coding Conventions](#7-coding-conventions) +8. [Extending the Mixer](#8-extending-the-mixer) + +--- + +## 1. Architecture Overview + +``` +┌──────────────────────────────────────────────────────────────────┐ +│ main_touch.py (Kivy) src/network/run.py (FastAPI) │ +│ ┌─────────────────┐ ┌────────────────────────┐ │ +│ │ MixerApp │ │ NetworkServer │ │ +│ │ ├── MixerScreen │ │ ├── REST API (8000) │ │ +│ │ ├── RoutingScr │ │ ├── WebSocket (8000) │ │ +│ │ ├── PluginScr │ │ └── OSC Server (9001) │ │ +│ │ └── SettingsScr │ └───────────┬────────────┘ │ +│ └────────┬─────────┘ │ │ +│ │ IPC (REST + OSC) │ │ +│ └───────────────┬─────────────────────┘ │ +│ │ │ +│ ┌──────▼──────────────────────────────┐ │ +│ │ DSPEngine │ │ +│ │ ┌──────────────────────────────┐ │ │ +│ │ │ ParameterRegistry │ │ │ +│ │ │ (all mixer parameters) │ │ │ +│ │ └──────────┬───────────────────┘ │ │ +│ │ │ callback │ │ +│ │ ┌────────▼───────────────────┐ │ │ +│ │ │ handle_parameter() │ │ │ +│ │ └──┬──────┬──────┬──────┬────┘ │ │ +│ │ │ │ │ │ │ │ +│ │ ┌────▼─┐ ┌─▼──┐ ┌─▼──┐ ┌─▼─────┐ │ │ +│ │ │Chan │ │Bus │ │Rout│ │Auto- │ │ │ +│ │ │Strip │ │Mgr │ │Mat │ │mation │ │ │ +│ │ └──┬───┘ └────┘ └────┘ └───────┘ │ │ +│ │ │ OSC │ │ +│ └─────┼──────────────────────────────┘ │ +│ │ │ +│ ┌──────▼──────┐ │ +│ │ Carla Rack │ LV2 / VST2 / NAM plugins │ +│ └──────┬──────┘ │ +│ │ JACK │ +│ ┌──────▼──────┐ │ +│ │ JACK2 RT │ SCHED_FIFO prio 70 │ +│ └──────┬──────┘ │ +│ │ ALSA hw:USB │ +└───────────────────────────┼──────────────────────────────────────┘ + │ +┌───────────────────────────▼──────────────────────────────────────┐ +│ Kernel: PREEMPT_RT 6.12.y │ snd-usb-audio │ xHCI USB │ +└──────────────────────────────────────────────────────────────────┘ +``` + +### Data Flow + +1. **MIDI Controller** sends CC/NRPN → `MIDIEngine` parses message → matches + against `MappingStore` → dispatches to `ParameterRegistry.set_value()` +2. **ParameterRegistry** calls all registered callbacks — primarily + `DSPEngine.handle_parameter()` +3. **DSPEngine** routes the parameter change to the correct component: + - `ChannelStrip.set_parameter()` → sends OSC to Carla for DSP processing + - `BusManager` — aux sends, subgroups, VCA, master + - `RoutingMatrix` — mute/solo/gain affect JACK connections + - `FaderAutomation` — records parameter changes if automation is armed +4. **Web UI / Touch UI** reads state from DSPEngine via REST/OSC, or + directly calls `ParameterRegistry.set_value()` +5. **Session Manager** serializes entire `ParameterRegistry` state to JSON + files; deserializes on load + +### Key Design Decisions + +| Decision | Rationale | +|----------|-----------| +| Python orchestration, C/C++ DSP (Carla) | Python for control logic; native code for real-time audio | +| OSC to Carla (not PyBind or C API) | Carla has a stable OSC API; network-transparent; debuggable | +| JACK for routing (not PulseAudio) | Professional audio: sample-accurate routing, real-time scheduling | +| ParameterRegistry as central state | Single source of truth for all mixer parameters; observable | +| JSON sessions (not SQLite/binary) | Human-readable, diffable, version-control-friendly | +| FastAPI + WebSocket for web UI | Async I/O, automatic OpenAPI docs, WebSocket for real-time updates | +| Kivy for touch UI | Python-native, OpenGL-accelerated, multi-touch, DPI-aware | +| PREEMPT_RT kernel | Sub-10ms round-trip latency requirement | + +--- + +## 2. Development Setup + +### Prerequisites + +- **Python 3.11+** (3.12 recommended) +- **Linux** (Ubuntu/Debian recommended; macOS works for most development) +- **Git** + +### Installing Development Dependencies + +```bash +# Clone the repository +git clone https://github.com/your-org/rpi-audio-mixer.git +cd rpi-audio-mixer + +# Create a virtual environment +python3 -m venv venv +source venv/bin/activate + +# Install Python dependencies +pip install -e ".[dev,test]" + +# Install system dependencies (Linux) +sudo apt install -y \ + jackd2 \ + libjack-jackd2-dev \ + carla \ + python3-jack-client \ + libasound2-dev + +# Or on macOS (limited — no JACK real-time audio) +brew install jack +pip install python-jack-client +``` + +### Running Without Hardware + +Most development and testing can be done on a desktop/laptop without a +Raspberry Pi or USB audio interface: + +```bash +# Start JACK with a dummy driver for testing +jackd -d dummy -r 48000 -p 256 & + +# Run the mixer API server +python -m src.network.run --port 8080 + +# Run tests (no hardware required) +python -m pytest tests/ -v + +# Run the touch UI (connected to local server) +python main_touch.py --host 127.0.0.1 --port 8080 +``` + +### IDE Configuration + +The project uses standard Python tooling. No special IDE plugins required. + +**VS Code recommended settings (`.vscode/settings.json`):** +```json +{ + "python.analysis.extraPaths": ["src"], + "python.testing.pytestEnabled": true, + "python.testing.pytestArgs": ["tests", "-v"], + "editor.rulers": [88], + "python.formatting.provider": "none", + "[python]": { + "editor.defaultFormatter": "ms-python.black-formatter" + } +} +``` + +--- + +## 3. Project Structure + +``` +rpi-audio-mixer/ +├── src/ +│ ├── mixer/ # Core DSP engine +│ │ ├── __init__.py # Public API exports +│ │ ├── dsp_engine.py # Main orchestrator +│ │ ├── channel_strip.py # Per-channel parameter controller +│ │ ├── routing_matrix.py # JACK port routing graph +│ │ ├── bus_manager.py # Aux, subgroups, VCA, master +│ │ ├── osc_client.py # Carla OSC control client +│ │ └── fader_automation.py # Automation recording & playback +│ │ +│ ├── midi/ # MIDI processing +│ │ ├── __init__.py +│ │ ├── midi_engine.py # Core routing + mapping dispatch +│ │ ├── midi_learn.py # Learn mode state machine +│ │ ├── midi_clock.py # MIDI clock master/slave +│ │ ├── mapping_store.py # Persistent mapping CRUD +│ │ ├── controllers.py # Pre-configured controller profiles +│ │ ├── device_discovery.py # USB MIDI device enumeration +│ │ ├── jack_midi_bridge.py # JACK MIDI ↔ ALSA seq bridge +│ │ └── types.py # MIDI data types +│ │ +│ ├── session/ # Session persistence +│ │ ├── __init__.py +│ │ ├── session_manager.py # Save/load/list/delete sessions +│ │ ├── mixer_session.py # Session data model + schema +│ │ ├── setlist.py # Setlist data model + transitions +│ │ └── snapshot.py # Snapshot capture & recall +│ │ +│ ├── recording/ # Multi-track recording +│ │ ├── __init__.py +│ │ ├── recorder.py # Multi-track recorder with punch in/out +│ │ ├── wav_writer.py # WAV file writer with seek tables +│ │ ├── session.py # Recording session management +│ │ ├── bounce.py # Bounce/mixdown to stereo +│ │ └── disk_monitor.py # SD card space monitoring +│ │ +│ ├── backing/ # Backing track playback +│ │ ├── __init__.py +│ │ ├── player.py # Main backing track orchestrator +│ │ ├── loader.py # Audio file loader (WAV/FLAC/MP3) +│ │ ├── playlist.py # Playlist management +│ │ ├── metronome.py # Click track generator +│ │ ├── transport.py # JACK transport integration +│ │ └── types.py # Backing track data types +│ │ +│ ├── streaming/ # Live streaming pipeline +│ │ ├── __init__.py +│ │ ├── streamer.py # Main stream lifecycle manager +│ │ ├── gst_pipeline.py # GStreamer pipeline builder +│ │ ├── camera.py # Camera detection (USB/Pi Cam) +│ │ ├── platforms.py # YouTube/Twitch/Facebook presets +│ │ └── controls.py # Stream scene management +│ │ +│ ├── network/ # Network services +│ │ ├── __init__.py +│ │ ├── run.py # Server entry point +│ │ ├── server.py # FastAPI + WebSocket + OSC server +│ │ ├── rest_api.py # REST API endpoints +│ │ ├── session_routes.py # Session-specific routes +│ │ ├── stream_routes.py # Streaming-specific routes +│ │ ├── web_routes.py # Web UI static routes +│ │ ├── websocket.py # WebSocket manager +│ │ ├── osc_server.py # OSC server +│ │ ├── schemas.py # Pydantic API models +│ │ ├── auth.py # API key authentication +│ │ ├── rate_limiter.py # Rate limiting +│ │ └── session.py # HTTP session middleware +│ │ +│ ├── plugin/ # Plugin management +│ │ ├── __init__.py +│ │ ├── manager.py # Plugin lifecycle manager +│ │ ├── scanner.py # Filesystem plugin scanner +│ │ ├── registry.py # SQLite plugin database +│ │ ├── blacklist.py # Plugin deny list +│ │ ├── categories.py # Plugin categorization +│ │ ├── nam.py # Neural Amp Modeler support +│ │ └── types.py # Plugin data types +│ │ +│ └── ui/ # Touchscreen UI (Kivy) +│ ├── __init__.py +│ ├── app.py # Kivy application + ScreenManager +│ ├── ipc.py # REST + OSC client for engine comms +│ ├── theme.py # Color scheme, typography, DPI +│ ├── screens/ +│ │ ├── __init__.py +│ │ ├── mixer.py # Channel strips + master faders +│ │ ├── routing.py # Routing matrix screen +│ │ ├── plugins.py # Plugin chain editor +│ │ ├── sessions.py # Session/setlist management +│ │ └── settings.py # App settings screen +│ └── widgets/ +│ ├── __init__.py +│ ├── fader.py # Custom touch fader widget +│ └── knob.py # Custom touch knob widget +│ +├── tests/ # Test suite (735 tests) +│ ├── conftest.py # pytest fixtures +│ ├── test_dsp_engine.py +│ ├── test_midi_engine.py +│ ├── test_midi_learn.py +│ ├── test_midi_clock.py +│ ├── test_mapping_store.py +│ ├── test_types.py +│ ├── test_session.py +│ ├── test_recording.py +│ ├── test_backing.py +│ ├── test_streaming.py +│ ├── test_rest_api.py +│ ├── test_web_routes.py +│ ├── test_websocket.py +│ ├── test_auth.py +│ ├── test_schemas.py +│ ├── test_rate_limiter.py +│ ├── test_osc_server.py +│ ├── test_parameter_registry.py +│ └── test_touch_ui.py +│ +├── build/ # SD card image builder +│ ├── build.sh # Main build script +│ ├── configure-system.sh # Chroot post-install configuration +│ ├── first-boot/ +│ │ └── setup-wizard.sh # First-boot setup wizard +│ └── README.md # Build system documentation +│ +├── scripts/ # Utility scripts +│ ├── carla-preset-manager.py +│ ├── carla-preset-gen.py +│ ├── lv2lint-check.sh +│ ├── carla-build.sh +│ └── nam-build.sh +│ +├── docs/ # Documentation +│ ├── build-guide.md +│ ├── user-manual.md +│ ├── developer-guide.md (this file) +│ ├── hardware-compatibility.md +│ ├── troubleshooting.md +│ ├── audio-stack-config.md +│ ├── midi-controller-support.md +│ ├── touchscreen-ui-evaluation.md +│ ├── carla-integration.md +│ └── research/ +│ ├── base-os-decision.md +│ └── research-report.md +│ +├── main_touch.py # Touch UI entry point +├── README.md # Project readme +└── .gitignore +``` + +--- + +## 4. Component Deep-Dives + +### DSP Engine (`src/mixer/`) + +The DSPEngine is the central orchestrator. It does **not** process audio +directly — audio DSP happens in Carla (a C++ plugin host). The engine's +job is control: receiving parameter changes, translating them to OSC +messages for Carla, and keeping JACK routing in sync. + +``` +ParameterRegistry → DSPEngine.handle_parameter() + ├── ChannelStrip.set_parameter() → Carla OSC + ├── BusManager.update_bus() + ├── RoutingMatrix.update_node() + └── FaderAutomation.record() +``` + +Key classes: +- **`ParameterRegistry`** — holds all mixer parameters as `MixerParameter` + objects. Parameters are identified by `(ParameterType, channel)` tuple. +- **`ChannelStrip`** — manages one channel's DSP state. Translates + high-level parameter changes to Carla OSC messages with appropriate + scaling (linear→dB, Hz→normalized, etc.). +- **`CarlaOSCClient`** — sends OSC messages to Carla's UDP port (22752). + Manages plugin layout, parameter addressing. +- **`RoutingMatrix`** — graph representation of JACK port connections. + Nodes are audio sources/sinks; edges are JACK connections. +- **`BusManager`** — 4 aux buses, 2 subgroups, 2 VCA groups, master bus. +- **`FaderAutomation`** — records parameter changes with timestamps; + plays back with configurable interpolation. + +### MIDI Engine (`src/midi/`) + +``` +USB MIDI Device → ALSA Sequencer → MIDIEngine._read_loop() + → parse CC/NRPN → match against MIDIMapping list + → scale value (int→float) → MappingCallback + → ParameterRegistry.set_value() +``` + +Key classes: +- **`MIDIEngine`** — reads from ALSA sequencer, matches incoming MIDI + messages against the mapping table, dispatches scaled values. +- **`MappingStore`** — persistent CRUD for MIDI mappings (JSON file). +- **`MIDILearn`** — learn mode state machine: listen for the next + CC/NRPN message and bind it to the selected parameter. +- **`MIDIClock`** — MIDI clock generator (master) or follower (slave). +- **`Controllers`** — pre-built mapping profiles for popular controllers. +- **`DeviceDiscovery`** — enumerates USB MIDI devices via ALSA. + +### Session Manager (`src/session/`) + +Sessions are JSON files stored in `~/.config/rpi-mixer/sessions/`. +Each session captures the complete mixer state. + +```json +{ + "version": 1, + "metadata": { + "name": "Live at The Garage", + "created": "2026-05-19T20:00:00Z", + "notes": "Soundcheck levels" + }, + "channels": [ + {"channel": 1, "volume": -3.0, "pan": 0.0, "mute": false, ...}, + ... + ], + "master": {"volume": -6.0, ...}, + "routing": {...}, + "plugins": {...} +} +``` + +Key classes: +- **`SessionManager`** — save/load/list/delete/import/export sessions. + Auto-save with configurable debounce. +- **`MixerSession`** — data model with schema versioning for forward/backward + compatibility. +- **`Setlist`** — ordered list of sessions with transition types + (cut, crossfade, wait). + +### Recording (`src/recording/`) + +``` +JACK process callback + → numpy buffer (float32, 128 samples) + → write-ahead ring buffer (16 slots per channel) + → background writer thread + → WAV file (disk) +``` + +Key classes: +- **`MultiTrackRecorder`** — orchestrates recording. Arming channels, + starting/stopping, punch in/out, take management. +- **`WAVWriter`** — writes RIFF WAV files with 16/24/32-bit depth. + Supports streaming writes with seek table updates. + +### Backing Tracks (`src/backing/`) + +``` +Audio file → AudioLoader → numpy array (in memory) + → BackingTrackPlayer → separate JACK output ports + → JACK transport sync → mixer channels +``` + +Key classes: +- **`BackingTrackPlayer`** — main orchestrator. Manages playback state, + transport sync, track parameters. +- **`AudioLoader`** — loads WAV, FLAC, MP3, AIFF, OGG into float32 numpy + arrays. +- **`Metronome`** — generates click track samples at variable BPM. + +### Streaming (`src/streaming/`) + +``` +JACK master output → GStreamer alsasrc +USB camera / Pi Cam → GStreamer v4l2src + → H.264 encoder (x264) → AAC encoder + → FLV muxer → rtmpsink → streaming platform +``` + +Key classes: +- **`Streamer`** — manages a GStreamer subprocess lifecycle. State machine: + idle → starting → live → stopping → idle. +- **`GstPipelineBuilder`** — constructs GStreamer pipeline strings from + config objects. +- **`Platforms`** — presets for YouTube, Twitch, Facebook, custom RTMP. + +### Network Services (`src/network/`) + +``` +FastAPI (Uvicorn) +├── REST API (port 8080) — JSON CRUD for all mixer state +├── WebSocket (port 8080) — real-time parameter push +├── Static files (port 8080) — web UI (HTML/CSS/JS) +└── OSC Server (port 9001) — UDP OSC commands + queries +``` + +Key classes: +- **`NetworkServer`** — ties REST, WebSocket, and OSC servers together. +- **`MixerStateProvider`** — adapter bridging REST API to DSPEngine. +- **`WebSocketManager`** — broadcasts parameter changes to all connected + web clients. +- **`OSCServer`** — listens for OSC commands and translates to + DSPEngine calls. + +--- + +## 5. Running Tests + +### Full Test Suite + +```bash +# Run all tests +python -m pytest tests/ -v + +# Run with coverage +python -m pytest tests/ -v --cov=src --cov-report=term-missing + +# Run a specific test file +python -m pytest tests/test_dsp_engine.py -v + +# Run tests matching a keyword +python -m pytest tests/ -v -k "midi" + +# Run tests with verbose output (show print statements) +python -m pytest tests/ -v -s +``` + +### Test Categories + +| Category | Files | Count | Requirements | +|----------|-------|-------|-------------| +| DSP Engine | `test_dsp_engine.py` | ~85 | None | +| MIDI | `test_midi_engine.py`, `test_midi_learn.py`, `test_midi_clock.py`, `test_mapping_store.py`, `test_types.py` | ~180 | None | +| Session | `test_session.py` | ~120 | None | +| Recording | `test_recording.py` | ~100 | numpy | +| Backing | `test_backing.py` | ~132 | numpy | +| Streaming | `test_streaming.py` | ~60 | None | +| REST API | `test_rest_api.py`, `test_web_routes.py`, `test_auth.py`, `test_schemas.py`, `test_rate_limiter.py` | ~150 | FastAPI | +| WebSocket | `test_websocket.py` | ~30 | websockets | +| OSC | `test_osc_server.py` | ~30 | None | +| Registry | `test_parameter_registry.py` | ~35 | None | +| Touch UI | `test_touch_ui.py` | ~28 | Kivy (headless) | + +### Running on Raspberry Pi Hardware + +For tests that require actual JACK/Carla hardware: + +```bash +# Start JACK first +jackd -d alsa -d hw:USB -r 48000 -p 256 -n 3 & + +# Start Carla +carla --osc-port=22752 & + +# Run hardware-dependent tests +python -m pytest tests/ -v -k "not streaming" # skip streaming (needs camera) + +# Record latency benchmarks +python -m pytest tests/test_dsp_engine.py -v -k "latency" +``` + +### CI + +Tests run automatically on push via Gitea Actions / GitHub Actions. +The CI configuration runs all tests except hardware-dependent ones: + +```yaml +# .gitea/workflows/test.yml +- name: Run tests + run: | + python -m pip install -e ".[dev,test]" + python -m pytest tests/ -v --cov=src +``` + +--- + +## 6. Contribution Workflow + +### Branch Strategy + +- `main` — stable, passing all tests +- Feature branches: `feature/my-feature` or `fix/my-fix` + +### Workflow + +```bash +# 1. Create a feature branch +git checkout -b feature/my-feature + +# 2. Make changes +# ... + +# 3. Run tests +python -m pytest tests/ -v + +# 4. Run linting (if configured) +ruff check src/ tests/ + +# 5. Commit with a descriptive message +git add -A +git commit -m "Add feature X: description of change" + +# 6. Push +git push origin feature/my-feature + +# 7. Create a Pull Request +``` + +### PR Checklist + +- [ ] Tests pass: `python -m pytest tests/ -v` +- [ ] New features include tests +- [ ] Documentation updated if API changes +- [ ] No new lint warnings +- [ ] Commit messages are descriptive +- [ ] Branch is up to date with `main` + +### Commit Convention + +``` +: + + + +Types: feat, fix, docs, test, refactor, build, perf, chore +``` + +Examples: +``` +feat: add high-pass filter to channel strip +fix: resolve xrun on USB disconnect during recording +docs: update user manual with OSC query examples +test: add MIDI clock slave sync tests +``` + +--- + +## 7. Coding Conventions + +### Python Style + +- **Line length:** 88 characters (Black-compatible) +- **Quotes:** double quotes `"` for strings +- **Docstrings:** triple double quotes `"""` — Google style +- **Type hints:** required for all public functions and methods +- **Imports:** `from __future__ import annotations` at top of every file + +### Docstring Format + +```python +def set_parameter(self, param_type: ParameterType, channel: int, value: float) -> None: + """Set a mixer parameter and forward to Carla via OSC. + + Args: + param_type: The parameter type from the ParameterType enum. + channel: Channel number (0-indexed) or -1 for master/global. + value: Normalized value in the parameter's configured range. + + Raises: + ValueError: If the channel is out of range. + """ +``` + +### Thread Safety + +Several components are accessed from multiple threads: + +- **MIDIEngine** — MIDI events arrive on ALSA sequencer thread → + callbacks fire on that thread → DSPEngine.handle_parameter() must be + thread-safe. +- **DSPEngine** — uses `threading.Lock` for mutable state. +- **SessionManager** — auto-save runs on a timer thread; load/save can + be called from API thread. + +When adding state to a shared component, protect it with `threading.Lock`. + +### Error Handling + +- **Never crash on a bad parameter value** — log a warning and clamp/ignore. +- **OSC communication failures** (Carla not running) should be logged but + not fatal — the mixer continues without plugin DSP. +- **JACK disconnection** should trigger reconnect logic, not a crash. +- **Disk full during recording** — stop recording gracefully, notify user. + +### Logging + +Use the standard `logging` module with named loggers: + +```python +import logging +logger = logging.getLogger(__name__) + +logger.info("Session saved: %s", session_name) +logger.warning("JACK connection lost, attempting reconnect") +logger.error("Failed to write WAV file: %s", str(e)) +``` + +--- + +## 8. Extending the Mixer + +### Adding a New Parameter + +1. Add a new `ParameterType` enum value in `src/midi/types.py` +2. Choose the appropriate `ParameterCategory` +3. Add the parameter to the factory function in `src/mixer/__init__.py` + (e.g., `_channel_params()`, `_master_params()`, etc.) +4. Handle the new parameter in `ChannelStrip.set_parameter()` or + `DSPEngine.handle_parameter()` +5. Add the Carla OSC mapping if DSP processing is needed +6. Add the parameter to `ChannelState` for session serialization +7. Add tests in `test_parameter_registry.py` and `test_dsp_engine.py` + +### Adding a New MIDI Controller Profile + +1. Create a mapping dictionary in `src/midi/controllers.py` +2. Register it in the `CONTROLLER_PRESETS` dict +3. Add to the hardware compatibility docs + +### Adding a New Streaming Platform + +1. Add preset config to `src/streaming/platforms.py` +2. Register in the platform lookup table +3. Add the platform option to the streaming UI + +### Adding a New REST API Endpoint + +1. Define request/response schemas in `src/network/schemas.py` +2. Add the route to `src/network/rest_api.py` (or relevant route file) +3. Implement the handler in `MixerStateProvider` +4. Add tests in `test_rest_api.py` +5. Update the API reference in the user manual + +### Plugin Support + +The Carla plugin host supports: +- **LV2** — `.lv2/` bundles installed system-wide or in `~/.lv2/` +- **VST2** — `.so` files in standard VST paths +- **NAM** — `.nam` model files managed by `src/plugin/nam.py` + +To add support for a new plugin format, extend `src/plugin/scanner.py` +and `src/plugin/types.py`. + +--- + +## Appendix: Dependency Graph + +``` +midi ──────────┐ + ├──→ mixer (DSPEngine) ──→ network (API + OSC) + │ │ +backing ───────┤ ├──→ recording + │ │ +session ───────┘ ├──→ streaming + │ + └──→ plugin + │ + ui (Kivy) +``` + +Dependencies flow downward: `ui` depends on `network`, which depends on +`mixer`. `mixer` depends on `midi`. `backing`, `session`, `recording`, +`streaming`, and `plugin` all depend on `mixer` and `midi`. + +No circular dependencies — the dependency graph is a DAG. diff --git a/docs/hardware-compatibility.md b/docs/hardware-compatibility.md index b8f96dc..b2df439 100644 --- a/docs/hardware-compatibility.md +++ b/docs/hardware-compatibility.md @@ -4,7 +4,28 @@ Verified and tested hardware for the Raspberry Pi Real-Time Audio Mixer. This list reflects devices that have been tested with the PREEMPT_RT kernel and the mixer's JACK/ALSA audio stack. -## Raspberry Pi Models +## Storage / Boot Media + +The same mixer image supports three boot targets. Choose based on your reliability needs: + +| Media | Boot Target | Bus | Speed | Reliability | Recommendation | +|-------|------------|-----|-------|-------------|---------------| +| **A2 SD Card** (32GB+) | `--target sd` | SDIO | ~90 MB/s | ⚠️ Moderate | Dev/testing only | +| **USB 3.0 SSD** (128GB+) | `--target usb` | USB 3.0 | ~350 MB/s | ✅ Good | Portable use, but shares USB bus with audio | +| **NVMe SSD** (128GB+) via HAT | `--target nvme` | PCIe Gen 2 x1 | ~500 MB/s | ✅✅ Excellent | **Recommended for live use** — dedicated lanes, no contention | + +### NVMe HAT Compatibility + +| HAT | Price | M.2 Size | Status | Notes | +|-----|-------|----------|--------|-------| +| **Official Raspberry Pi NVMe Base** | ~€15 | 2230/2242 | ✅ Verified | Best compatibility, official support | +| **Pimoroni NVMe Base** | ~€15 | 2230/2242 | ✅ Verified | Well-documented, same chipset | +| **Geekworm X1001** | ~€12 | 2230 | ✅ Verified | Low profile, good for tight builds | +| **Geekworm X1002** | ~€15 | 2242/2280 | ✅ Verified | Full-length option | + +> ⚠️ **Important:** USB SSDs share the VL805 USB controller with audio interfaces. +> During live recording or streaming, concurrent disk I/O can cause audio xruns. +> This is the primary reason NVMe HAT is recommended for production use. | Model | Status | Notes | |-------|--------|-------| diff --git a/docs/user-manual.md b/docs/user-manual.md index b660e00..3bb6c0f 100644 --- a/docs/user-manual.md +++ b/docs/user-manual.md @@ -26,10 +26,17 @@ live streaming. - Raspberry Pi 4 Model B (4GB+ RAM) - USB audio interface (class-compliant UAC2) -- SD card (16GB+, Class A2 recommended) with mixer image flashed +- Boot media (choose one): + - **SD card** (16GB+, Class A2) — good for dev/testing + - **USB SSD** via USB 3.0 — more reliable, but shares USB bus with audio + - **NVMe SSD + HAT** (PCIe) — **recommended for live use** — no bus contention - 5V/3A USB-C power supply - **Optional:** HDMI touchscreen, USB MIDI controller, Ethernet cable, USB camera +> 💡 **Storage recommendation:** For live performances and recording, use an NVMe +> HAT (official RPi, Pimoroni, or Geekworm) with a 128GB+ NVMe SSD. It uses +> dedicated PCIe lanes and won't compete with your USB audio interface for bandwidth. + ### Connections ``` @@ -64,8 +71,23 @@ Verified interfaces: see [docs/hardware-compatibility.md](hardware-compatibility ## 2. First Boot & Setup Wizard -On first boot, the setup wizard runs automatically on the HDMI display. +### Booting for the First Time +1. Insert your boot media (SD, USB SSD, or NVMe) into the Pi +2. Connect audio interface, MIDI controllers, and network +3. Power on + +> ⚡ If booting from USB SSD or NVMe, the Pi's EEPROM must be configured to try +> your boot device first. The setup wizard includes an EEPROM configuration step, +> or you can configure it manually beforehand: +> ```bash +> sudo rpi-eeprom-config --edit +> # USB boot: BOOT_ORDER=0xf41 +> # NVMe boot: BOOT_ORDER=0xf614 +> ``` + +On first boot, the **setup wizard** runs automatically on the HDMI display (or +serial console). It walks through: ### Wizard Steps 1. **Welcome screen** — language selection