# Developer Guide How to contribute to the Raspberry Pi Real-Time Audio Mixer — architecture, development setup, testing, and contribution workflow. ## Table of Contents 1. [Architecture Overview](#1-architecture-overview) 2. [Development Setup](#2-development-setup) 3. [Project Structure](#3-project-structure) 4. [Component Deep-Dives](#4-component-deep-dives) 5. [Running Tests](#5-running-tests) 6. [Contribution Workflow](#6-contribution-workflow) 7. [Coding Conventions](#7-coding-conventions) 8. [Extending the Mixer](#8-extending-the-mixer) --- ## 1. Architecture Overview ``` ┌──────────────────────────────────────────────────────────────────┐ │ main_touch.py (Kivy) src/network/run.py (FastAPI) │ │ ┌─────────────────┐ ┌────────────────────────┐ │ │ │ MixerApp │ │ NetworkServer │ │ │ │ ├── MixerScreen │ │ ├── REST API (8000) │ │ │ │ ├── RoutingScr │ │ ├── WebSocket (8000) │ │ │ │ ├── PluginScr │ │ └── OSC Server (9001) │ │ │ │ └── SettingsScr │ └───────────┬────────────┘ │ │ └────────┬─────────┘ │ │ │ │ IPC (REST + OSC) │ │ │ └───────────────┬─────────────────────┘ │ │ │ │ │ ┌──────▼──────────────────────────────┐ │ │ │ DSPEngine │ │ │ │ ┌──────────────────────────────┐ │ │ │ │ │ ParameterRegistry │ │ │ │ │ │ (all mixer parameters) │ │ │ │ │ └──────────┬───────────────────┘ │ │ │ │ │ callback │ │ │ │ ┌────────▼───────────────────┐ │ │ │ │ │ handle_parameter() │ │ │ │ │ └──┬──────┬──────┬──────┬────┘ │ │ │ │ │ │ │ │ │ │ │ │ ┌────▼─┐ ┌─▼──┐ ┌─▼──┐ ┌─▼─────┐ │ │ │ │ │Chan │ │Bus │ │Rout│ │Auto- │ │ │ │ │ │Strip │ │Mgr │ │Mat │ │mation │ │ │ │ │ └──┬───┘ └────┘ └────┘ └───────┘ │ │ │ │ │ OSC │ │ │ └─────┼──────────────────────────────┘ │ │ │ │ │ ┌──────▼──────┐ │ │ │ Carla Rack │ LV2 / VST2 / NAM plugins │ │ └──────┬──────┘ │ │ │ JACK │ │ ┌──────▼──────┐ │ │ │ JACK2 RT │ SCHED_FIFO prio 70 │ │ └──────┬──────┘ │ │ │ ALSA hw:USB │ └───────────────────────────┼──────────────────────────────────────┘ │ ┌───────────────────────────▼──────────────────────────────────────┐ │ Kernel: PREEMPT_RT 6.12.y │ snd-usb-audio │ xHCI USB │ └──────────────────────────────────────────────────────────────────┘ ``` ### Data Flow 1. **MIDI Controller** sends CC/NRPN → `MIDIEngine` parses message → matches against `MappingStore` → dispatches to `ParameterRegistry.set_value()` 2. **ParameterRegistry** calls all registered callbacks — primarily `DSPEngine.handle_parameter()` 3. **DSPEngine** routes the parameter change to the correct component: - `ChannelStrip.set_parameter()` → sends OSC to Carla for DSP processing - `BusManager` — aux sends, subgroups, VCA, master - `RoutingMatrix` — mute/solo/gain affect JACK connections - `FaderAutomation` — records parameter changes if automation is armed 4. **Web UI / Touch UI** reads state from DSPEngine via REST/OSC, or directly calls `ParameterRegistry.set_value()` 5. **Session Manager** serializes entire `ParameterRegistry` state to JSON files; deserializes on load ### Key Design Decisions | Decision | Rationale | |----------|-----------| | Python orchestration, C/C++ DSP (Carla) | Python for control logic; native code for real-time audio | | OSC to Carla (not PyBind or C API) | Carla has a stable OSC API; network-transparent; debuggable | | JACK for routing (not PulseAudio) | Professional audio: sample-accurate routing, real-time scheduling | | ParameterRegistry as central state | Single source of truth for all mixer parameters; observable | | JSON sessions (not SQLite/binary) | Human-readable, diffable, version-control-friendly | | FastAPI + WebSocket for web UI | Async I/O, automatic OpenAPI docs, WebSocket for real-time updates | | Kivy for touch UI | Python-native, OpenGL-accelerated, multi-touch, DPI-aware | | PREEMPT_RT kernel | Sub-10ms round-trip latency requirement | --- ## 2. Development Setup ### Prerequisites - **Python 3.11+** (3.12 recommended) - **Linux** (Ubuntu/Debian recommended; macOS works for most development) - **Git** ### Installing Development Dependencies ```bash # Clone the repository git clone https://github.com/your-org/rpi-audio-mixer.git cd rpi-audio-mixer # Create a virtual environment python3 -m venv venv source venv/bin/activate # Install Python dependencies pip install -e ".[dev,test]" # Install system dependencies (Linux) sudo apt install -y \ jackd2 \ libjack-jackd2-dev \ carla \ python3-jack-client \ libasound2-dev # Or on macOS (limited — no JACK real-time audio) brew install jack pip install python-jack-client ``` ### Running Without Hardware Most development and testing can be done on a desktop/laptop without a Raspberry Pi or USB audio interface: ```bash # Start JACK with a dummy driver for testing jackd -d dummy -r 48000 -p 256 & # Run the mixer API server python -m src.network.run --port 8080 # Run tests (no hardware required) python -m pytest tests/ -v # Run the touch UI (connected to local server) python main_touch.py --host 127.0.0.1 --port 8080 ``` ### IDE Configuration The project uses standard Python tooling. No special IDE plugins required. **VS Code recommended settings (`.vscode/settings.json`):** ```json { "python.analysis.extraPaths": ["src"], "python.testing.pytestEnabled": true, "python.testing.pytestArgs": ["tests", "-v"], "editor.rulers": [88], "python.formatting.provider": "none", "[python]": { "editor.defaultFormatter": "ms-python.black-formatter" } } ``` --- ## 3. Project Structure ``` rpi-audio-mixer/ ├── src/ │ ├── mixer/ # Core DSP engine │ │ ├── __init__.py # Public API exports │ │ ├── dsp_engine.py # Main orchestrator │ │ ├── channel_strip.py # Per-channel parameter controller │ │ ├── routing_matrix.py # JACK port routing graph │ │ ├── bus_manager.py # Aux, subgroups, VCA, master │ │ ├── osc_client.py # Carla OSC control client │ │ └── fader_automation.py # Automation recording & playback │ │ │ ├── midi/ # MIDI processing │ │ ├── __init__.py │ │ ├── midi_engine.py # Core routing + mapping dispatch │ │ ├── midi_learn.py # Learn mode state machine │ │ ├── midi_clock.py # MIDI clock master/slave │ │ ├── mapping_store.py # Persistent mapping CRUD │ │ ├── controllers.py # Pre-configured controller profiles │ │ ├── device_discovery.py # USB MIDI device enumeration │ │ ├── jack_midi_bridge.py # JACK MIDI ↔ ALSA seq bridge │ │ └── types.py # MIDI data types │ │ │ ├── session/ # Session persistence │ │ ├── __init__.py │ │ ├── session_manager.py # Save/load/list/delete sessions │ │ ├── mixer_session.py # Session data model + schema │ │ ├── setlist.py # Setlist data model + transitions │ │ └── snapshot.py # Snapshot capture & recall │ │ │ ├── recording/ # Multi-track recording │ │ ├── __init__.py │ │ ├── recorder.py # Multi-track recorder with punch in/out │ │ ├── wav_writer.py # WAV file writer with seek tables │ │ ├── session.py # Recording session management │ │ ├── bounce.py # Bounce/mixdown to stereo │ │ └── disk_monitor.py # SD card space monitoring │ │ │ ├── backing/ # Backing track playback │ │ ├── __init__.py │ │ ├── player.py # Main backing track orchestrator │ │ ├── loader.py # Audio file loader (WAV/FLAC/MP3) │ │ ├── playlist.py # Playlist management │ │ ├── metronome.py # Click track generator │ │ ├── transport.py # JACK transport integration │ │ └── types.py # Backing track data types │ │ │ ├── streaming/ # Live streaming pipeline │ │ ├── __init__.py │ │ ├── streamer.py # Main stream lifecycle manager │ │ ├── gst_pipeline.py # GStreamer pipeline builder │ │ ├── camera.py # Camera detection (USB/Pi Cam) │ │ ├── platforms.py # YouTube/Twitch/Facebook presets │ │ └── controls.py # Stream scene management │ │ │ ├── network/ # Network services │ │ ├── __init__.py │ │ ├── run.py # Server entry point │ │ ├── server.py # FastAPI + WebSocket + OSC server │ │ ├── rest_api.py # REST API endpoints │ │ ├── session_routes.py # Session-specific routes │ │ ├── stream_routes.py # Streaming-specific routes │ │ ├── web_routes.py # Web UI static routes │ │ ├── websocket.py # WebSocket manager │ │ ├── osc_server.py # OSC server │ │ ├── schemas.py # Pydantic API models │ │ ├── auth.py # API key authentication │ │ ├── rate_limiter.py # Rate limiting │ │ └── session.py # HTTP session middleware │ │ │ ├── plugin/ # Plugin management │ │ ├── __init__.py │ │ ├── manager.py # Plugin lifecycle manager │ │ ├── scanner.py # Filesystem plugin scanner │ │ ├── registry.py # SQLite plugin database │ │ ├── blacklist.py # Plugin deny list │ │ ├── categories.py # Plugin categorization │ │ ├── nam.py # Neural Amp Modeler support │ │ └── types.py # Plugin data types │ │ │ └── ui/ # Touchscreen UI (Kivy) │ ├── __init__.py │ ├── app.py # Kivy application + ScreenManager │ ├── ipc.py # REST + OSC client for engine comms │ ├── theme.py # Color scheme, typography, DPI │ ├── screens/ │ │ ├── __init__.py │ │ ├── mixer.py # Channel strips + master faders │ │ ├── routing.py # Routing matrix screen │ │ ├── plugins.py # Plugin chain editor │ │ ├── sessions.py # Session/setlist management │ │ └── settings.py # App settings screen │ └── widgets/ │ ├── __init__.py │ ├── fader.py # Custom touch fader widget │ └── knob.py # Custom touch knob widget │ ├── tests/ # Test suite (735 tests) │ ├── conftest.py # pytest fixtures │ ├── test_dsp_engine.py │ ├── test_midi_engine.py │ ├── test_midi_learn.py │ ├── test_midi_clock.py │ ├── test_mapping_store.py │ ├── test_types.py │ ├── test_session.py │ ├── test_recording.py │ ├── test_backing.py │ ├── test_streaming.py │ ├── test_rest_api.py │ ├── test_web_routes.py │ ├── test_websocket.py │ ├── test_auth.py │ ├── test_schemas.py │ ├── test_rate_limiter.py │ ├── test_osc_server.py │ ├── test_parameter_registry.py │ └── test_touch_ui.py │ ├── build/ # SD card image builder │ ├── build.sh # Main build script │ ├── configure-system.sh # Chroot post-install configuration │ ├── first-boot/ │ │ └── setup-wizard.sh # First-boot setup wizard │ └── README.md # Build system documentation │ ├── scripts/ # Utility scripts │ ├── carla-preset-manager.py │ ├── carla-preset-gen.py │ ├── lv2lint-check.sh │ ├── carla-build.sh │ └── nam-build.sh │ ├── docs/ # Documentation │ ├── build-guide.md │ ├── user-manual.md │ ├── developer-guide.md (this file) │ ├── hardware-compatibility.md │ ├── troubleshooting.md │ ├── audio-stack-config.md │ ├── midi-controller-support.md │ ├── touchscreen-ui-evaluation.md │ ├── carla-integration.md │ └── research/ │ ├── base-os-decision.md │ └── research-report.md │ ├── main_touch.py # Touch UI entry point ├── README.md # Project readme └── .gitignore ``` --- ## 4. Component Deep-Dives ### DSP Engine (`src/mixer/`) The DSPEngine is the central orchestrator. It does **not** process audio directly — audio DSP happens in Carla (a C++ plugin host). The engine's job is control: receiving parameter changes, translating them to OSC messages for Carla, and keeping JACK routing in sync. ``` ParameterRegistry → DSPEngine.handle_parameter() ├── ChannelStrip.set_parameter() → Carla OSC ├── BusManager.update_bus() ├── RoutingMatrix.update_node() └── FaderAutomation.record() ``` Key classes: - **`ParameterRegistry`** — holds all mixer parameters as `MixerParameter` objects. Parameters are identified by `(ParameterType, channel)` tuple. - **`ChannelStrip`** — manages one channel's DSP state. Translates high-level parameter changes to Carla OSC messages with appropriate scaling (linear→dB, Hz→normalized, etc.). - **`CarlaOSCClient`** — sends OSC messages to Carla's UDP port (22752). Manages plugin layout, parameter addressing. - **`RoutingMatrix`** — graph representation of JACK port connections. Nodes are audio sources/sinks; edges are JACK connections. - **`BusManager`** — 4 aux buses, 2 subgroups, 2 VCA groups, master bus. - **`FaderAutomation`** — records parameter changes with timestamps; plays back with configurable interpolation. ### MIDI Engine (`src/midi/`) ``` USB MIDI Device → ALSA Sequencer → MIDIEngine._read_loop() → parse CC/NRPN → match against MIDIMapping list → scale value (int→float) → MappingCallback → ParameterRegistry.set_value() ``` Key classes: - **`MIDIEngine`** — reads from ALSA sequencer, matches incoming MIDI messages against the mapping table, dispatches scaled values. - **`MappingStore`** — persistent CRUD for MIDI mappings (JSON file). - **`MIDILearn`** — learn mode state machine: listen for the next CC/NRPN message and bind it to the selected parameter. - **`MIDIClock`** — MIDI clock generator (master) or follower (slave). - **`Controllers`** — pre-built mapping profiles for popular controllers. - **`DeviceDiscovery`** — enumerates USB MIDI devices via ALSA. ### Session Manager (`src/session/`) Sessions are JSON files stored in `~/.config/rpi-mixer/sessions/`. Each session captures the complete mixer state. ```json { "version": 1, "metadata": { "name": "Live at The Garage", "created": "2026-05-19T20:00:00Z", "notes": "Soundcheck levels" }, "channels": [ {"channel": 1, "volume": -3.0, "pan": 0.0, "mute": false, ...}, ... ], "master": {"volume": -6.0, ...}, "routing": {...}, "plugins": {...} } ``` Key classes: - **`SessionManager`** — save/load/list/delete/import/export sessions. Auto-save with configurable debounce. - **`MixerSession`** — data model with schema versioning for forward/backward compatibility. - **`Setlist`** — ordered list of sessions with transition types (cut, crossfade, wait). ### Recording (`src/recording/`) ``` JACK process callback → numpy buffer (float32, 128 samples) → write-ahead ring buffer (16 slots per channel) → background writer thread → WAV file (disk) ``` Key classes: - **`MultiTrackRecorder`** — orchestrates recording. Arming channels, starting/stopping, punch in/out, take management. - **`WAVWriter`** — writes RIFF WAV files with 16/24/32-bit depth. Supports streaming writes with seek table updates. ### Backing Tracks (`src/backing/`) ``` Audio file → AudioLoader → numpy array (in memory) → BackingTrackPlayer → separate JACK output ports → JACK transport sync → mixer channels ``` Key classes: - **`BackingTrackPlayer`** — main orchestrator. Manages playback state, transport sync, track parameters. - **`AudioLoader`** — loads WAV, FLAC, MP3, AIFF, OGG into float32 numpy arrays. - **`Metronome`** — generates click track samples at variable BPM. ### Streaming (`src/streaming/`) ``` JACK master output → GStreamer alsasrc USB camera / Pi Cam → GStreamer v4l2src → H.264 encoder (x264) → AAC encoder → FLV muxer → rtmpsink → streaming platform ``` Key classes: - **`Streamer`** — manages a GStreamer subprocess lifecycle. State machine: idle → starting → live → stopping → idle. - **`GstPipelineBuilder`** — constructs GStreamer pipeline strings from config objects. - **`Platforms`** — presets for YouTube, Twitch, Facebook, custom RTMP. ### Network Services (`src/network/`) ``` FastAPI (Uvicorn) ├── REST API (port 8080) — JSON CRUD for all mixer state ├── WebSocket (port 8080) — real-time parameter push ├── Static files (port 8080) — web UI (HTML/CSS/JS) └── OSC Server (port 9001) — UDP OSC commands + queries ``` Key classes: - **`NetworkServer`** — ties REST, WebSocket, and OSC servers together. - **`MixerStateProvider`** — adapter bridging REST API to DSPEngine. - **`WebSocketManager`** — broadcasts parameter changes to all connected web clients. - **`OSCServer`** — listens for OSC commands and translates to DSPEngine calls. --- ## 5. Running Tests ### Full Test Suite ```bash # Run all tests python -m pytest tests/ -v # Run with coverage python -m pytest tests/ -v --cov=src --cov-report=term-missing # Run a specific test file python -m pytest tests/test_dsp_engine.py -v # Run tests matching a keyword python -m pytest tests/ -v -k "midi" # Run tests with verbose output (show print statements) python -m pytest tests/ -v -s ``` ### Test Categories | Category | Files | Count | Requirements | |----------|-------|-------|-------------| | DSP Engine | `test_dsp_engine.py` | ~85 | None | | MIDI | `test_midi_engine.py`, `test_midi_learn.py`, `test_midi_clock.py`, `test_mapping_store.py`, `test_types.py` | ~180 | None | | Session | `test_session.py` | ~120 | None | | Recording | `test_recording.py` | ~100 | numpy | | Backing | `test_backing.py` | ~132 | numpy | | Streaming | `test_streaming.py` | ~60 | None | | REST API | `test_rest_api.py`, `test_web_routes.py`, `test_auth.py`, `test_schemas.py`, `test_rate_limiter.py` | ~150 | FastAPI | | WebSocket | `test_websocket.py` | ~30 | websockets | | OSC | `test_osc_server.py` | ~30 | None | | Registry | `test_parameter_registry.py` | ~35 | None | | Touch UI | `test_touch_ui.py` | ~28 | Kivy (headless) | ### Running on Raspberry Pi Hardware For tests that require actual JACK/Carla hardware: ```bash # Start JACK first jackd -d alsa -d hw:USB -r 48000 -p 256 -n 3 & # Start Carla carla --osc-port=22752 & # Run hardware-dependent tests python -m pytest tests/ -v -k "not streaming" # skip streaming (needs camera) # Record latency benchmarks python -m pytest tests/test_dsp_engine.py -v -k "latency" ``` ### CI Tests run automatically on push via Gitea Actions / GitHub Actions. The CI configuration runs all tests except hardware-dependent ones: ```yaml # .gitea/workflows/test.yml - name: Run tests run: | python -m pip install -e ".[dev,test]" python -m pytest tests/ -v --cov=src ``` --- ## 6. Contribution Workflow ### Branch Strategy - `main` — stable, passing all tests - Feature branches: `feature/my-feature` or `fix/my-fix` ### Workflow ```bash # 1. Create a feature branch git checkout -b feature/my-feature # 2. Make changes # ... # 3. Run tests python -m pytest tests/ -v # 4. Run linting (if configured) ruff check src/ tests/ # 5. Commit with a descriptive message git add -A git commit -m "Add feature X: description of change" # 6. Push git push origin feature/my-feature # 7. Create a Pull Request ``` ### PR Checklist - [ ] Tests pass: `python -m pytest tests/ -v` - [ ] New features include tests - [ ] Documentation updated if API changes - [ ] No new lint warnings - [ ] Commit messages are descriptive - [ ] Branch is up to date with `main` ### Commit Convention ``` : Types: feat, fix, docs, test, refactor, build, perf, chore ``` Examples: ``` feat: add high-pass filter to channel strip fix: resolve xrun on USB disconnect during recording docs: update user manual with OSC query examples test: add MIDI clock slave sync tests ``` --- ## 7. Coding Conventions ### Python Style - **Line length:** 88 characters (Black-compatible) - **Quotes:** double quotes `"` for strings - **Docstrings:** triple double quotes `"""` — Google style - **Type hints:** required for all public functions and methods - **Imports:** `from __future__ import annotations` at top of every file ### Docstring Format ```python def set_parameter(self, param_type: ParameterType, channel: int, value: float) -> None: """Set a mixer parameter and forward to Carla via OSC. Args: param_type: The parameter type from the ParameterType enum. channel: Channel number (0-indexed) or -1 for master/global. value: Normalized value in the parameter's configured range. Raises: ValueError: If the channel is out of range. """ ``` ### Thread Safety Several components are accessed from multiple threads: - **MIDIEngine** — MIDI events arrive on ALSA sequencer thread → callbacks fire on that thread → DSPEngine.handle_parameter() must be thread-safe. - **DSPEngine** — uses `threading.Lock` for mutable state. - **SessionManager** — auto-save runs on a timer thread; load/save can be called from API thread. When adding state to a shared component, protect it with `threading.Lock`. ### Error Handling - **Never crash on a bad parameter value** — log a warning and clamp/ignore. - **OSC communication failures** (Carla not running) should be logged but not fatal — the mixer continues without plugin DSP. - **JACK disconnection** should trigger reconnect logic, not a crash. - **Disk full during recording** — stop recording gracefully, notify user. ### Logging Use the standard `logging` module with named loggers: ```python import logging logger = logging.getLogger(__name__) logger.info("Session saved: %s", session_name) logger.warning("JACK connection lost, attempting reconnect") logger.error("Failed to write WAV file: %s", str(e)) ``` --- ## 8. Extending the Mixer ### Adding a New Parameter 1. Add a new `ParameterType` enum value in `src/midi/types.py` 2. Choose the appropriate `ParameterCategory` 3. Add the parameter to the factory function in `src/mixer/__init__.py` (e.g., `_channel_params()`, `_master_params()`, etc.) 4. Handle the new parameter in `ChannelStrip.set_parameter()` or `DSPEngine.handle_parameter()` 5. Add the Carla OSC mapping if DSP processing is needed 6. Add the parameter to `ChannelState` for session serialization 7. Add tests in `test_parameter_registry.py` and `test_dsp_engine.py` ### Adding a New MIDI Controller Profile 1. Create a mapping dictionary in `src/midi/controllers.py` 2. Register it in the `CONTROLLER_PRESETS` dict 3. Add to the hardware compatibility docs ### Adding a New Streaming Platform 1. Add preset config to `src/streaming/platforms.py` 2. Register in the platform lookup table 3. Add the platform option to the streaming UI ### Adding a New REST API Endpoint 1. Define request/response schemas in `src/network/schemas.py` 2. Add the route to `src/network/rest_api.py` (or relevant route file) 3. Implement the handler in `MixerStateProvider` 4. Add tests in `test_rest_api.py` 5. Update the API reference in the user manual ### Plugin Support The Carla plugin host supports: - **LV2** — `.lv2/` bundles installed system-wide or in `~/.lv2/` - **VST2** — `.so` files in standard VST paths - **NAM** — `.nam` model files managed by `src/plugin/nam.py` To add support for a new plugin format, extend `src/plugin/scanner.py` and `src/plugin/types.py`. --- ## Appendix: Dependency Graph ``` midi ──────────┐ ├──→ mixer (DSPEngine) ──→ network (API + OSC) │ │ backing ───────┤ ├──→ recording │ │ session ───────┘ ├──→ streaming │ └──→ plugin │ ui (Kivy) ``` Dependencies flow downward: `ui` depends on `network`, which depends on `mixer`. `mixer` depends on `midi`. `backing`, `session`, `recording`, `streaming`, and `plugin` all depend on `mixer` and `midi`. No circular dependencies — the dependency graph is a DAG.