raspberry-pi-mixer/docs/developer-guide.md

Developer Guide

How to contribute to the Raspberry Pi Real-Time Audio Mixer — architecture, development setup, testing, and contribution workflow.

Table of Contents

1. Architecture Overview
2. Development Setup
3. Project Structure
4. Component Deep-Dives
5. Running Tests
6. Contribution Workflow
7. Coding Conventions
8. Extending the Mixer

---

1. Architecture Overview

┌──────────────────────────────────────────────────────────────────┐
│  main_touch.py (Kivy)              src/network/run.py (FastAPI)  │
│  ┌─────────────────┐               ┌────────────────────────┐    │
│  │ MixerApp         │               │ NetworkServer           │    │
│  │  ├── MixerScreen │               │  ├── REST API (8000)    │    │
│  │  ├── RoutingScr  │               │  ├── WebSocket (8000)   │    │
│  │  ├── PluginScr   │               │  └── OSC Server (9001)  │    │
│  │  └── SettingsScr │               └───────────┬────────────┘    │
│  └────────┬─────────┘                           │                 │
│           │ IPC (REST + OSC)                    │                 │
│           └───────────────┬─────────────────────┘                 │
│                           │                                       │
│                    ┌──────▼──────────────────────────────┐        │
│                    │         DSPEngine                    │        │
│                    │  ┌──────────────────────────────┐   │        │
│                    │  │    ParameterRegistry         │   │        │
│                    │  │  (all mixer parameters)      │   │        │
│                    │  └──────────┬───────────────────┘   │        │
│                    │             │ callback               │        │
│                    │    ┌────────▼───────────────────┐   │        │
│                    │    │    handle_parameter()       │   │        │
│                    │    └──┬──────┬──────┬──────┬────┘   │        │
│                    │       │      │      │      │        │        │
│                    │  ┌────▼─┐ ┌─▼──┐ ┌─▼──┐ ┌─▼─────┐ │        │
│                    │  │Chan  │ │Bus │ │Rout│ │Auto-  │ │        │
│                    │  │Strip │ │Mgr │ │Mat │ │mation │ │        │
│                    │  └──┬───┘ └────┘ └────┘ └───────┘ │        │
│                    │     │ OSC                          │        │
│                    └─────┼──────────────────────────────┘        │
│                          │                                       │
│                    ┌──────▼──────┐                                │
│                    │ Carla Rack  │  LV2 / VST2 / NAM plugins     │
│                    └──────┬──────┘                                │
│                           │ JACK                                  │
│                    ┌──────▼──────┐                                │
│                    │  JACK2 RT   │  SCHED_FIFO prio 70            │
│                    └──────┬──────┘                                │
│                           │ ALSA hw:USB                           │
└───────────────────────────┼──────────────────────────────────────┘
                            │
┌───────────────────────────▼──────────────────────────────────────┐
│  Kernel: PREEMPT_RT 6.12.y  │  snd-usb-audio  │  xHCI USB       │
└──────────────────────────────────────────────────────────────────┘

Data Flow

1. MIDI Controller sends CC/NRPN → MIDIEngine parses message → matches against MappingStore → dispatches to ParameterRegistry.set_value()
2. ParameterRegistry calls all registered callbacks — primarily DSPEngine.handle_parameter()
3. DSPEngine routes the parameter change to the correct component:
   • ChannelStrip.set_parameter() → sends OSC to Carla for DSP processing
   • BusManager — aux sends, subgroups, VCA, master
   • RoutingMatrix — mute/solo/gain affect JACK connections
   • FaderAutomation — records parameter changes if automation is armed
4. Web UI / Touch UI reads state from DSPEngine via REST/OSC, or directly calls ParameterRegistry.set_value()
5. Session Manager serializes entire ParameterRegistry state to JSON files; deserializes on load

Key Design Decisions

Decision	Rationale
Python orchestration, C/C++ DSP (Carla)	Python for control logic; native code for real-time audio
OSC to Carla (not PyBind or C API)	Carla has a stable OSC API; network-transparent; debuggable
JACK for routing (not PulseAudio)	Professional audio: sample-accurate routing, real-time scheduling
ParameterRegistry as central state	Single source of truth for all mixer parameters; observable
JSON sessions (not SQLite/binary)	Human-readable, diffable, version-control-friendly
FastAPI + WebSocket for web UI	Async I/O, automatic OpenAPI docs, WebSocket for real-time updates
Kivy for touch UI	Python-native, OpenGL-accelerated, multi-touch, DPI-aware
PREEMPT_RT kernel	Sub-10ms round-trip latency requirement

---

2. Development Setup

Prerequisites

• Python 3.11+ (3.12 recommended)
• Linux (Ubuntu/Debian recommended; macOS works for most development)
• Git

Installing Development Dependencies

# Clone the repository
git clone https://github.com/your-org/rpi-audio-mixer.git
cd rpi-audio-mixer

# Create a virtual environment
python3 -m venv venv
source venv/bin/activate

# Install Python dependencies
pip install -e ".[dev,test]"

# Install system dependencies (Linux)
sudo apt install -y \
    jackd2 \
    libjack-jackd2-dev \
    carla \
    python3-jack-client \
    libasound2-dev

# Or on macOS (limited — no JACK real-time audio)
brew install jack
pip install python-jack-client

Running Without Hardware

Most development and testing can be done on a desktop/laptop without a Raspberry Pi or USB audio interface:

# Start JACK with a dummy driver for testing
jackd -d dummy -r 48000 -p 256 &

# Run the mixer API server
python -m src.network.run --port 8080

# Run tests (no hardware required)
python -m pytest tests/ -v

# Run the touch UI (connected to local server)
python main_touch.py --host 127.0.0.1 --port 8080

IDE Configuration

The project uses standard Python tooling. No special IDE plugins required.

VS Code recommended settings (.vscode/settings.json):

{
    "python.analysis.extraPaths": ["src"],
    "python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": ["tests", "-v"],
    "editor.rulers": [88],
    "python.formatting.provider": "none",
    "[python]": {
        "editor.defaultFormatter": "ms-python.black-formatter"
    }
}

---

3. Project Structure

rpi-audio-mixer/
├── src/
│   ├── mixer/                  # Core DSP engine
│   │   ├── __init__.py         # Public API exports
│   │   ├── dsp_engine.py       # Main orchestrator
│   │   ├── channel_strip.py    # Per-channel parameter controller
│   │   ├── routing_matrix.py   # JACK port routing graph
│   │   ├── bus_manager.py      # Aux, subgroups, VCA, master
│   │   ├── osc_client.py       # Carla OSC control client
│   │   └── fader_automation.py # Automation recording & playback
│   │
│   ├── midi/                   # MIDI processing
│   │   ├── __init__.py
│   │   ├── midi_engine.py      # Core routing + mapping dispatch
│   │   ├── midi_learn.py       # Learn mode state machine
│   │   ├── midi_clock.py       # MIDI clock master/slave
│   │   ├── mapping_store.py    # Persistent mapping CRUD
│   │   ├── controllers.py      # Pre-configured controller profiles
│   │   ├── device_discovery.py # USB MIDI device enumeration
│   │   ├── jack_midi_bridge.py # JACK MIDI ↔ ALSA seq bridge
│   │   └── types.py            # MIDI data types
│   │
│   ├── session/                # Session persistence
│   │   ├── __init__.py
│   │   ├── session_manager.py  # Save/load/list/delete sessions
│   │   ├── mixer_session.py    # Session data model + schema
│   │   ├── setlist.py          # Setlist data model + transitions
│   │   └── snapshot.py         # Snapshot capture & recall
│   │
│   ├── recording/              # Multi-track recording
│   │   ├── __init__.py
│   │   ├── recorder.py         # Multi-track recorder with punch in/out
│   │   ├── wav_writer.py       # WAV file writer with seek tables
│   │   ├── session.py          # Recording session management
│   │   ├── bounce.py           # Bounce/mixdown to stereo
│   │   └── disk_monitor.py     # SD card space monitoring
│   │
│   ├── backing/                # Backing track playback
│   │   ├── __init__.py
│   │   ├── player.py           # Main backing track orchestrator
│   │   ├── loader.py           # Audio file loader (WAV/FLAC/MP3)
│   │   ├── playlist.py         # Playlist management
│   │   ├── metronome.py        # Click track generator
│   │   ├── transport.py        # JACK transport integration
│   │   └── types.py            # Backing track data types
│   │
│   ├── streaming/              # Live streaming pipeline
│   │   ├── __init__.py
│   │   ├── streamer.py         # Main stream lifecycle manager
│   │   ├── gst_pipeline.py     # GStreamer pipeline builder
│   │   ├── camera.py           # Camera detection (USB/Pi Cam)
│   │   ├── platforms.py        # YouTube/Twitch/Facebook presets
│   │   └── controls.py         # Stream scene management
│   │
│   ├── network/                # Network services
│   │   ├── __init__.py
│   │   ├── run.py              # Server entry point
│   │   ├── server.py           # FastAPI + WebSocket + OSC server
│   │   ├── rest_api.py         # REST API endpoints
│   │   ├── session_routes.py   # Session-specific routes
│   │   ├── stream_routes.py    # Streaming-specific routes
│   │   ├── web_routes.py       # Web UI static routes
│   │   ├── websocket.py        # WebSocket manager
│   │   ├── osc_server.py       # OSC server
│   │   ├── schemas.py          # Pydantic API models
│   │   ├── auth.py             # API key authentication
│   │   ├── rate_limiter.py     # Rate limiting
│   │   └── session.py          # HTTP session middleware
│   │
│   ├── plugin/                 # Plugin management
│   │   ├── __init__.py
│   │   ├── manager.py          # Plugin lifecycle manager
│   │   ├── scanner.py          # Filesystem plugin scanner
│   │   ├── registry.py         # SQLite plugin database
│   │   ├── blacklist.py        # Plugin deny list
│   │   ├── categories.py       # Plugin categorization
│   │   ├── nam.py              # Neural Amp Modeler support
│   │   └── types.py            # Plugin data types
│   │
│   └── ui/                     # Touchscreen UI (Kivy)
│       ├── __init__.py
│       ├── app.py              # Kivy application + ScreenManager
│       ├── ipc.py              # REST + OSC client for engine comms
│       ├── theme.py            # Color scheme, typography, DPI
│       ├── screens/
│       │   ├── __init__.py
│       │   ├── mixer.py        # Channel strips + master faders
│       │   ├── routing.py      # Routing matrix screen
│       │   ├── plugins.py      # Plugin chain editor
│       │   ├── sessions.py     # Session/setlist management
│       │   └── settings.py     # App settings screen
│       └── widgets/
│           ├── __init__.py
│           ├── fader.py        # Custom touch fader widget
│           └── knob.py         # Custom touch knob widget
│
├── tests/                      # Test suite (735 tests)
│   ├── conftest.py             # pytest fixtures
│   ├── test_dsp_engine.py
│   ├── test_midi_engine.py
│   ├── test_midi_learn.py
│   ├── test_midi_clock.py
│   ├── test_mapping_store.py
│   ├── test_types.py
│   ├── test_session.py
│   ├── test_recording.py
│   ├── test_backing.py
│   ├── test_streaming.py
│   ├── test_rest_api.py
│   ├── test_web_routes.py
│   ├── test_websocket.py
│   ├── test_auth.py
│   ├── test_schemas.py
│   ├── test_rate_limiter.py
│   ├── test_osc_server.py
│   ├── test_parameter_registry.py
│   └── test_touch_ui.py
│
├── build/                      # SD card image builder
│   ├── build.sh                # Main build script
│   ├── configure-system.sh     # Chroot post-install configuration
│   ├── first-boot/
│   │   └── setup-wizard.sh     # First-boot setup wizard
│   └── README.md               # Build system documentation
│
├── scripts/                    # Utility scripts
│   ├── carla-preset-manager.py
│   ├── carla-preset-gen.py
│   ├── lv2lint-check.sh
│   ├── carla-build.sh
│   └── nam-build.sh
│
├── docs/                       # Documentation
│   ├── build-guide.md
│   ├── user-manual.md
│   ├── developer-guide.md      (this file)
│   ├── hardware-compatibility.md
│   ├── troubleshooting.md
│   ├── audio-stack-config.md
│   ├── midi-controller-support.md
│   ├── touchscreen-ui-evaluation.md
│   ├── carla-integration.md
│   └── research/
│       ├── base-os-decision.md
│       └── research-report.md
│
├── main_touch.py               # Touch UI entry point
├── README.md                   # Project readme
└── .gitignore

---

4. Component Deep-Dives

DSP Engine (src/mixer/)

The DSPEngine is the central orchestrator. It does not process audio directly — audio DSP happens in Carla (a C++ plugin host). The engine's job is control: receiving parameter changes, translating them to OSC messages for Carla, and keeping JACK routing in sync.

ParameterRegistry → DSPEngine.handle_parameter()
                        ├── ChannelStrip.set_parameter() → Carla OSC
                        ├── BusManager.update_bus()
                        ├── RoutingMatrix.update_node()
                        └── FaderAutomation.record()

Key classes:

• ParameterRegistry — holds all mixer parameters as MixerParameter objects. Parameters are identified by (ParameterType, channel) tuple.
• ChannelStrip — manages one channel's DSP state. Translates high-level parameter changes to Carla OSC messages with appropriate scaling (linear→dB, Hz→normalized, etc.).
• CarlaOSCClient — sends OSC messages to Carla's UDP port (22752). Manages plugin layout, parameter addressing.
• RoutingMatrix — graph representation of JACK port connections. Nodes are audio sources/sinks; edges are JACK connections.
• BusManager — 4 aux buses, 2 subgroups, 2 VCA groups, master bus.
• FaderAutomation — records parameter changes with timestamps; plays back with configurable interpolation.

MIDI Engine (src/midi/)

USB MIDI Device → ALSA Sequencer → MIDIEngine._read_loop()
    → parse CC/NRPN → match against MIDIMapping list
    → scale value (int→float) → MappingCallback
    → ParameterRegistry.set_value()

Key classes:

• MIDIEngine — reads from ALSA sequencer, matches incoming MIDI messages against the mapping table, dispatches scaled values.
• MappingStore — persistent CRUD for MIDI mappings (JSON file).
• MIDILearn — learn mode state machine: listen for the next CC/NRPN message and bind it to the selected parameter.
• MIDIClock — MIDI clock generator (master) or follower (slave).
• Controllers — pre-built mapping profiles for popular controllers.
• DeviceDiscovery — enumerates USB MIDI devices via ALSA.

Session Manager (src/session/)

Sessions are JSON files stored in ~/.config/rpi-mixer/sessions/. Each session captures the complete mixer state.

{
  "version": 1,
  "metadata": {
    "name": "Live at The Garage",
    "created": "2026-05-19T20:00:00Z",
    "notes": "Soundcheck levels"
  },
  "channels": [
    {"channel": 1, "volume": -3.0, "pan": 0.0, "mute": false, ...},
    ...
  ],
  "master": {"volume": -6.0, ...},
  "routing": {...},
  "plugins": {...}
}

Key classes:

• SessionManager — save/load/list/delete/import/export sessions. Auto-save with configurable debounce.
• MixerSession — data model with schema versioning for forward/backward compatibility.
• Setlist — ordered list of sessions with transition types (cut, crossfade, wait).

Recording (src/recording/)

JACK process callback
    → numpy buffer (float32, 128 samples)
    → write-ahead ring buffer (16 slots per channel)
    → background writer thread
    → WAV file (disk)

Key classes:

• MultiTrackRecorder — orchestrates recording. Arming channels, starting/stopping, punch in/out, take management.
• WAVWriter — writes RIFF WAV files with 16/24/32-bit depth. Supports streaming writes with seek table updates.

Backing Tracks (src/backing/)

Audio file → AudioLoader → numpy array (in memory)
    → BackingTrackPlayer → separate JACK output ports
    → JACK transport sync → mixer channels

Key classes:

• BackingTrackPlayer — main orchestrator. Manages playback state, transport sync, track parameters.
• AudioLoader — loads WAV, FLAC, MP3, AIFF, OGG into float32 numpy arrays.
• Metronome — generates click track samples at variable BPM.

Streaming (src/streaming/)

JACK master output → GStreamer alsasrc
USB camera / Pi Cam → GStreamer v4l2src
    → H.264 encoder (x264) → AAC encoder
    → FLV muxer → rtmpsink → streaming platform

Key classes:

• Streamer — manages a GStreamer subprocess lifecycle. State machine: idle → starting → live → stopping → idle.
• GstPipelineBuilder — constructs GStreamer pipeline strings from config objects.
• Platforms — presets for YouTube, Twitch, Facebook, custom RTMP.

Network Services (src/network/)

FastAPI (Uvicorn)
├── REST API (port 8080)     — JSON CRUD for all mixer state
├── WebSocket (port 8080)    — real-time parameter push
├── Static files (port 8080) — web UI (HTML/CSS/JS)
└── OSC Server (port 9001)   — UDP OSC commands + queries

Key classes:

• NetworkServer — ties REST, WebSocket, and OSC servers together.
• MixerStateProvider — adapter bridging REST API to DSPEngine.
• WebSocketManager — broadcasts parameter changes to all connected web clients.
• OSCServer — listens for OSC commands and translates to DSPEngine calls.

---

5. Running Tests

Full Test Suite

# Run all tests
python -m pytest tests/ -v

# Run with coverage
python -m pytest tests/ -v --cov=src --cov-report=term-missing

# Run a specific test file
python -m pytest tests/test_dsp_engine.py -v

# Run tests matching a keyword
python -m pytest tests/ -v -k "midi"

# Run tests with verbose output (show print statements)
python -m pytest tests/ -v -s

Test Categories

Category	Files	Count	Requirements
DSP Engine	test_dsp_engine.py	~85	None
MIDI	test_midi_engine.py, test_midi_learn.py, test_midi_clock.py, test_mapping_store.py, test_types.py	~180	None
Session	test_session.py	~120	None
Recording	test_recording.py	~100	numpy
Backing	test_backing.py	~132	numpy
Streaming	test_streaming.py	~60	None
REST API	test_rest_api.py, test_web_routes.py, test_auth.py, test_schemas.py, test_rate_limiter.py	~150	FastAPI
WebSocket	test_websocket.py	~30	websockets
OSC	test_osc_server.py	~30	None
Registry	test_parameter_registry.py	~35	None
Touch UI	test_touch_ui.py	~28	Kivy (headless)

Running on Raspberry Pi Hardware

For tests that require actual JACK/Carla hardware:

# Start JACK first
jackd -d alsa -d hw:USB -r 48000 -p 256 -n 3 &

# Start Carla
carla --osc-port=22752 &

# Run hardware-dependent tests
python -m pytest tests/ -v -k "not streaming"  # skip streaming (needs camera)

# Record latency benchmarks
python -m pytest tests/test_dsp_engine.py -v -k "latency"

CI

Tests run automatically on push via Gitea Actions / GitHub Actions. The CI configuration runs all tests except hardware-dependent ones:

# .gitea/workflows/test.yml
- name: Run tests
  run: |
    python -m pip install -e ".[dev,test]"
    python -m pytest tests/ -v --cov=src

---

6. Contribution Workflow

Branch Strategy

• main — stable, passing all tests
• Feature branches: feature/my-feature or fix/my-fix

Workflow

# 1. Create a feature branch
git checkout -b feature/my-feature

# 2. Make changes
# ...

# 3. Run tests
python -m pytest tests/ -v

# 4. Run linting (if configured)
ruff check src/ tests/

# 5. Commit with a descriptive message
git add -A
git commit -m "Add feature X: description of change"

# 6. Push
git push origin feature/my-feature

# 7. Create a Pull Request

PR Checklist

• Tests pass: python -m pytest tests/ -v
• New features include tests
• Documentation updated if API changes
• No new lint warnings
• Commit messages are descriptive
• Branch is up to date with main

Commit Convention

<type>: <short description>

<optional body with details>

Types: feat, fix, docs, test, refactor, build, perf, chore

Examples:

feat: add high-pass filter to channel strip
fix: resolve xrun on USB disconnect during recording
docs: update user manual with OSC query examples
test: add MIDI clock slave sync tests

---

7. Coding Conventions

Python Style

• Line length: 88 characters (Black-compatible)
• Quotes: double quotes " for strings
• Docstrings: triple double quotes """ — Google style
• Type hints: required for all public functions and methods
• Imports: from __future__ import annotations at top of every file

Docstring Format

def set_parameter(self, param_type: ParameterType, channel: int, value: float) -> None:
    """Set a mixer parameter and forward to Carla via OSC.

    Args:
        param_type: The parameter type from the ParameterType enum.
        channel: Channel number (0-indexed) or -1 for master/global.
        value: Normalized value in the parameter's configured range.

    Raises:
        ValueError: If the channel is out of range.
    """

Thread Safety

Several components are accessed from multiple threads:

• MIDIEngine — MIDI events arrive on ALSA sequencer thread → callbacks fire on that thread → DSPEngine.handle_parameter() must be thread-safe.
• DSPEngine — uses threading.Lock for mutable state.
• SessionManager — auto-save runs on a timer thread; load/save can be called from API thread.

When adding state to a shared component, protect it with threading.Lock.

Error Handling

• Never crash on a bad parameter value — log a warning and clamp/ignore.
• OSC communication failures (Carla not running) should be logged but not fatal — the mixer continues without plugin DSP.
• JACK disconnection should trigger reconnect logic, not a crash.
• Disk full during recording — stop recording gracefully, notify user.

Logging

Use the standard logging module with named loggers:

import logging
logger = logging.getLogger(__name__)

logger.info("Session saved: %s", session_name)
logger.warning("JACK connection lost, attempting reconnect")
logger.error("Failed to write WAV file: %s", str(e))

---

8. Extending the Mixer

Adding a New Parameter

1. Add a new ParameterType enum value in src/midi/types.py
2. Choose the appropriate ParameterCategory
3. Add the parameter to the factory function in src/mixer/__init__.py (e.g., _channel_params(), _master_params(), etc.)
4. Handle the new parameter in ChannelStrip.set_parameter() or DSPEngine.handle_parameter()
5. Add the Carla OSC mapping if DSP processing is needed
6. Add the parameter to ChannelState for session serialization
7. Add tests in test_parameter_registry.py and test_dsp_engine.py

Adding a New MIDI Controller Profile

1. Create a mapping dictionary in src/midi/controllers.py
2. Register it in the CONTROLLER_PRESETS dict
3. Add to the hardware compatibility docs

Adding a New Streaming Platform

1. Add preset config to src/streaming/platforms.py
2. Register in the platform lookup table
3. Add the platform option to the streaming UI

Adding a New REST API Endpoint

1. Define request/response schemas in src/network/schemas.py
2. Add the route to src/network/rest_api.py (or relevant route file)
3. Implement the handler in MixerStateProvider
4. Add tests in test_rest_api.py
5. Update the API reference in the user manual

Plugin Support

The Carla plugin host supports:

• LV2 — .lv2/ bundles installed system-wide or in ~/.lv2/
• VST2 — .so files in standard VST paths
• NAM — .nam model files managed by src/plugin/nam.py

To add support for a new plugin format, extend src/plugin/scanner.py and src/plugin/types.py.

---

Appendix: Dependency Graph

midi ──────────┐
               ├──→ mixer (DSPEngine) ──→ network (API + OSC)
               │         │
backing ───────┤         ├──→ recording
               │         │
session ───────┘         ├──→ streaming
                         │
                         └──→ plugin
                              │
                         ui (Kivy)

Dependencies flow downward: ui depends on network, which depends on mixer. mixer depends on midi. backing, session, recording, streaming, and plugin all depend on mixer and midi.

No circular dependencies — the dependency graph is a DAG.