672 lines
30 KiB
Markdown
672 lines
30 KiB
Markdown
# Pi Multi-FX Pedal — Full Test Plan
|
||
|
||
**Scope:** Every endpoint, every UI element, the entire DSP/N audio chain,
|
||
hardware interfaces (MIDI, Bluetooth, WiFi), and system stability.
|
||
|
||
**Test types:**
|
||
| Icon | Meaning | When to run |
|
||
|------|---------|-------------|
|
||
| 🤖 | Fully automated (API + UI script) | Every commit |
|
||
| 🧪 | Semi-automated (scriptable with signal present) | Per-release |
|
||
| 🧑 | Manual (ears + hands) | Per-release, during tuning |
|
||
| 💀 | Destructive (service restart, reboot) | Per-major-release |
|
||
|
||
**Setup needed for manual tests:**
|
||
- Guitar plugged into DI box → interface input
|
||
- Patch cable from interface output → interface input (loopback for latency)
|
||
- Headphones or monitors on interface output
|
||
- MIDI controller (optional)
|
||
- Bluetooth device (optional)
|
||
|
||
---
|
||
|
||
## 1. Automated API Tests 🤖
|
||
|
||
Every endpoint must return correct HTTP status, correct content-type,
|
||
and valid JSON matching its documented schema.
|
||
|
||
### 1.1 Alive / Connection
|
||
|
||
| # | Test | Method | Expect |
|
||
|---|------|--------|--------|
|
||
| 1.1.1 | Root | `GET /` | 200, HTML page |
|
||
| 1.1.2 | State (guitar) | `GET /api/state` | 200, `connected: true`, `blocks[]` |
|
||
| 1.1.3 | State (combined) | `GET /api/state?combined=true` | 200, `channels` with 5 keys |
|
||
| 1.1.4 | State (bass) | `GET /api/state?channel=bass` | 200, `channel: "bass"` |
|
||
| 1.1.5 | State (keys) | `GET /api/state?channel=keys` | 200 |
|
||
| 1.1.6 | State (vocals) | `GET /api/state?channel=vocals` | 200 |
|
||
| 1.1.7 | State (backing_tracks) | `GET /api/state?channel=backing_tracks` | 200 |
|
||
|
||
### 1.2 State Fields
|
||
|
||
| # | Test | Expect |
|
||
|---|------|--------|
|
||
| 1.2.1 | `connected` is bool | true |
|
||
| 1.2.2 | `current_preset` has name/bank/program | all present |
|
||
| 1.2.3 | `bypass` is bool | false initially |
|
||
| 1.2.4 | `tuner_enabled` is bool | false initially |
|
||
| 1.2.5 | `master_volume` is float 0.0–1.0 | 0.8 initially |
|
||
| 1.2.6 | `routing_mode` is string | "mono" or "4cm" |
|
||
| 1.2.7 | `nam_loaded` matches model list | bool |
|
||
| 1.2.8 | `nam_cpu` is int 0–100 | > 0 when NAM loaded |
|
||
| 1.2.9 | `blocks` is array | ≥ 1 block |
|
||
| 1.2.10 | Each block has `block_id` | unique, 12-char hex |
|
||
| 1.2.11 | Each block has `fx_type`, `enabled`, `bypass` | all present |
|
||
| 1.2.12 | `cpu_percent` is int 0–100 | present |
|
||
| 1.2.13 | `sample_rate` matches config | 48000 |
|
||
| 1.2.14 | `needs_reboot` is bool | false |
|
||
|
||
### 1.3 Block API
|
||
|
||
| # | Test | Method | Expect |
|
||
|---|------|--------|--------|
|
||
| 1.3.1 | PATCH toggle block off | `PATCH /api/blocks {block_id, enabled:false}` | 200, `enabled: false` |
|
||
| 1.3.2 | State reflects block disabled | `GET /api/state` | block `enabled: false` |
|
||
| 1.3.3 | PATCH toggle block on | `PATCH /api/blocks {block_id, enabled:true}` | 200, `enabled: true` |
|
||
| 1.3.4 | PATCH non-existent block_id | `PATCH /api/blocks {block_id:"bogus", enabled:true}` | 404 |
|
||
| 1.3.5 | PATCH missing block_id | `PATCH /api/blocks {enabled:true}` | 400 |
|
||
| 1.3.6 | PATCH missing enabled | `PATCH /api/blocks {block_id:"x"}` | 400 |
|
||
| 1.3.7 | PATCH block params | `PATCH /api/block {block_id, param:val}` | 200 |
|
||
| 1.3.8 | Get block params schema | `GET /api/block-params/nam_amp` | 200, params object |
|
||
| 1.3.9 | Get block params (invalid type) | `GET /api/block-params/bogus` | 200 or 404 |
|
||
|
||
### 1.4 Bypass
|
||
|
||
| # | Test | Method | Expect |
|
||
|---|------|--------|--------|
|
||
| 1.4.1 | Toggle bypass ON | `POST /api/bypass/toggle` | 200, `bypass: true` |
|
||
| 1.4.2 | State reflects bypass | `GET /api/state` | `bypass: true` |
|
||
| 1.4.3 | Toggle bypass OFF | `POST /api/bypass/toggle` | 200, `bypass: false` |
|
||
| 1.4.4 | Set bypass directly | `POST /api/bypass {"bypass":true}` | 200 |
|
||
| 1.4.5 | Per-channel bypass | `POST /api/channel/bass/bypass` | 200 |
|
||
|
||
### 1.5 Volume
|
||
|
||
| # | Test | Method | Expect |
|
||
|---|------|--------|--------|
|
||
| 1.5.1 | Set volume 50% | `POST /api/volume {"volume":0.5}` | 200 |
|
||
| 1.5.2 | State reflects volume | `GET /api/state` | `master_volume: 0.5` |
|
||
| 1.5.3 | Set volume min | `POST /api/volume {"volume":0.0}` | 200, `volume: 0.0` |
|
||
| 1.5.4 | Set volume max | `POST /api/volume {"volume":1.0}` | 200, `volume: 1.0` |
|
||
| 1.5.5 | Set volume clamped below 0 | `POST /api/volume {"volume":-0.5}` | 200, `volume: 0.0` |
|
||
| 1.5.6 | Set volume clamped above 1 | `POST /api/volume {"volume":1.5}` | 200, `volume: 1.0` |
|
||
| 1.5.7 | Per-channel volume | `PUT /api/channel/bass/volume {"volume":0.3}` | 200 |
|
||
| 1.5.8 | Restore | `POST /api/volume {"volume":0.8}` | 200 |
|
||
| 1.5.9 | Volumes per channel are independent | set guitar 0.8, bass 0.3, check both | different |
|
||
|
||
### 1.6 Presets
|
||
|
||
| # | Test | Method | Expect |
|
||
|---|------|--------|--------|
|
||
| 1.6.1 | List presets | `GET /api/presets` | 200, `banks[]` |
|
||
| 1.6.2 | Get specific preset | `GET /api/presets/{bank}/{program}` | 200, full preset |
|
||
| 1.6.3 | Save preset | `PUT /api/presets/{bank}/{program} { chain, name }` | 200 |
|
||
| 1.6.4 | Save + reload matches | save → GET → compare chain | block_ids preserved |
|
||
| 1.6.5 | Delete preset | `DELETE /api/presets/{bank}/{program}` | 200 |
|
||
| 1.6.6 | Get deleted preset | `GET /api/presets/{bank}/{program}` | 404 |
|
||
| 1.6.7 | Activate preset | `POST /api/presets/{bank}/{program}/activate` | 200 |
|
||
| 1.6.8 | State reflects active preset | `GET /api/state` | `current_preset` matches |
|
||
| 1.6.9 | Per-channel presets | `GET /api/channel/bass/presets` | 200 |
|
||
| 1.6.10 | Per-channel preset save | `PUT /api/channel/bass/presets/0/0` | 200 |
|
||
| 1.6.11 | Save with block_id → reload preserves it | PUT → GET → block_id same | block_id stable |
|
||
| 1.6.12 | Save without block_id → generated | PUT without block_id → GET has block_id | generated |
|
||
|
||
### 1.7 Snapshots
|
||
|
||
| # | Test | Method | Expect |
|
||
|---|------|--------|--------|
|
||
| 1.7.1 | Save snapshot | `POST /api/snapshots/{slot}/save` | 200 |
|
||
| 1.7.2 | Save named snapshot | `POST /api/snapshots/{slot}/save {"name":"test"}` | 200 |
|
||
| 1.7.3 | List snapshots | `GET /api/snapshots` | 200, `snapshots` dict |
|
||
| 1.7.4 | Get snapshot | `GET /api/snapshots/{slot}` | 200 |
|
||
| 1.7.5 | Recall snapshot | `POST /api/snapshots/{slot}/recall` | 200 |
|
||
| 1.7.6 | Recall restores block states | save state A → change → recall → check state | restored to A |
|
||
| 1.7.7 | Delete snapshot | `DELETE /api/snapshots/{slot}` | 200 |
|
||
| 1.7.8 | Recall empty slot | `POST /api/snapshots/{slot}/recall` on empty | 404 |
|
||
| 1.7.9 | Save slot 1–8 | all 8 slots | each 200 |
|
||
| 1.7.10 | Save slot 0 (invalid) | `POST /api/snapshots/0/save` | 400 |
|
||
| 1.7.11 | Save slot 9 (invalid) | `POST /api/snapshots/9/save` | 400 |
|
||
|
||
### 1.8 Tuner
|
||
|
||
| # | Test | Method | Expect |
|
||
|---|------|--------|--------|
|
||
| 1.8.1 | Enable tuner | `POST /api/tuner {"enabled":true}` | 200 |
|
||
| 1.8.2 | State reflects tuner ON | `GET /api/state` | `tuner_enabled: true` |
|
||
| 1.8.3 | Get tuner pitch | `GET /api/tuner/pitch` | 200, `frequency`, `note`, `cents` |
|
||
| 1.8.4 | Disable tuner | `POST /api/tuner {"enabled":false}` | 200 |
|
||
| 1.8.5 | Per-channel tuner | `POST /api/channel/bass/tuner {"enabled":true}` | 200 |
|
||
| 1.8.6 | Tuner output rates | poll `GET /api/tuner/pitch` 5×/sec for 3s | different readings |
|
||
| 1.8.7 | Double-tuner (enable when already on) | `POST /api/tuner {"enabled":true}` ×2 | still enabled |
|
||
|
||
### 1.9 NAM / Models
|
||
|
||
| # | Test | Method | Expect |
|
||
|---|------|--------|--------|
|
||
| 1.9.1 | List models | `GET /api/models` | 200, `models[]` |
|
||
| 1.9.2 | Per-channel models | `GET /api/channel/guitar/models` | 200 |
|
||
| 1.9.3 | Load model by path | `POST /api/models/load {"path":"..."}` | 200 |
|
||
| 1.9.4 | State reflects loaded model | `GET /api/state` | `nam_model` matches |
|
||
| 1.9.5 | Load model (invalid path) | `POST /api/models/load {"path":"/nonexistent.nam"}` | 422 |
|
||
| 1.9.6 | Load model (missing path) | `POST /api/models/load {}` | 400 |
|
||
| 1.9.7 | Unload model | `POST /api/models/unload` | 200 |
|
||
| 1.9.8 | State after unload | `GET /api/state` | `nam_loaded: false` |
|
||
| 1.9.9 | Reload original model | `POST /api/models/load {"path":"..."}` | 200 |
|
||
| 1.9.10 | NAM CPU > 0 after load | `GET /api/state` | `nam_cpu` > 5 |
|
||
| 1.9.11 | Cycle through all models | load each model → verify state | each 200 |
|
||
| 1.9.12 | Duplicate model paths | check `/api/models` for duplicates | no duplicates |
|
||
| 1.9.13 | Get NAM engine info | `GET /api/nam/engine` | 200, engine mode |
|
||
| 1.9.14 | Switch NAM engine (if available) | `POST /api/nam/engine {"engine_mode":"pytorch"}` | 200 or 400 if unsupported |
|
||
| 1.9.15 | Switch back to cpp | `POST /api/nam/engine {"engine_mode":"cpp"}` | 200 |
|
||
|
||
### 1.10 IR
|
||
|
||
| # | Test | Method | Expect |
|
||
|---|------|--------|--------|
|
||
| 1.10.1 | List IRs | `GET /api/irs` | 200 |
|
||
| 1.10.2 | Load IR by path | `POST /api/irs/load {"path":"..."}` | 200 |
|
||
| 1.10.3 | State reflects loaded IR | `GET /api/state` | `ir_loaded: true`, `ir_name` |
|
||
| 1.10.4 | Unload IR | `POST /api/irs/unload` | 200 |
|
||
| 1.10.5 | Load invalid path | `POST /api/irs/load {"path":"bogus.wav"}` | 422 |
|
||
|
||
### 1.11 Routing
|
||
|
||
| # | Test | Method | Expect |
|
||
|---|------|--------|--------|
|
||
| 1.11.1 | Get routing | `GET /api/routing` | 200 |
|
||
| 1.11.2 | Set routing mono | `POST /api/routing {"routing_mode":"mono"}` | 200 |
|
||
| 1.11.3 | State reflects routing | `GET /api/state` | `routing_mode: "mono"` |
|
||
| 1.11.4 | Set routing 4CM | `POST /api/routing {"routing_mode":"4cm","routing_breakpoint":5}` | 200 |
|
||
| 1.11.5 | Get channel routing | `GET /api/channel/guitar/routing` | 200 |
|
||
| 1.11.6 | Set channel routing | `POST /api/channel/guitar/routing {"mode":"mono"}` | 200 |
|
||
|
||
### 1.12 Channel Mode & Output
|
||
|
||
| # | Test | Method | Expect |
|
||
|---|------|--------|--------|
|
||
| 1.12.1 | Get channel mode | `GET /api/channel-mode` | 200 |
|
||
| 1.12.2 | Set channel mode | `POST /api/channel-mode {"mode":"dual-mono"}` | 200 |
|
||
| 1.12.3 | Get channel output | `GET /api/channel-output` | 200 |
|
||
| 1.12.4 | Set channel output | `POST /api/channel-output {"ch1_port":"...","ch2_port":"..."}` | 200 |
|
||
|
||
### 1.13 Backing Source
|
||
|
||
| # | Test | Method | Expect |
|
||
|---|------|--------|--------|
|
||
| 1.13.1 | Get backing source | `GET /api/backing-source` | 200 |
|
||
| 1.13.2 | Set backing source | `POST /api/backing-source {"type":"line_in","port":"...","gain":0.5}` | 200 |
|
||
|
||
### 1.14 Audio Profile
|
||
|
||
| # | Test | Method | Expect |
|
||
|---|------|--------|--------|
|
||
| 1.14.1 | List profiles | `GET /api/audio/profiles` | 200 |
|
||
| 1.14.2 | Get current profile | `GET /api/audio/profile` | 200 |
|
||
| 1.14.3 | Set profile | `POST /api/audio/profile {"profile":"standard"}` | 200 |
|
||
| 1.14.4 | Get audio devices | `GET /api/audio/devices` | 200 |
|
||
| 1.14.5 | State reflects new profile | `GET /api/state` | stable after switch |
|
||
|
||
### 1.15 Settings
|
||
|
||
| # | Test | Method | Expect |
|
||
|---|------|--------|--------|
|
||
| 1.15.1 | Get settings | `GET /api/settings` | 200, config dict |
|
||
| 1.15.2 | Save setting | `POST /api/settings {"input_impedance":"1m"}` | 200, `saved` ≥ 1 |
|
||
| 1.15.3 | Save unknown setting | `POST /api/settings {"bogus_key":123}` | 200, `saved` = 0 |
|
||
| 1.15.4 | Save brightness | `POST /api/settings {"brightness_percent":75}` | 200 |
|
||
| 1.15.5 | Settings persist after restart | change → restart → GET | same value |
|
||
|
||
### 1.16 System
|
||
|
||
| # | Test | Method | Expect |
|
||
|---|------|--------|--------|
|
||
| 1.16.1 | Get system info | `GET /api/system` | 200, hostname/kernel/uptime |
|
||
| 1.16.2 | `cpu_temp_c` < 80 | sanity check | < 80°C |
|
||
| 1.16.3 | `memory.free` > 100MB | sanity check | > 100 |
|
||
| 1.16.4 | `disk.avail` > 100MB | sanity check | > 100M |
|
||
| 1.16.5 | Get logs | `GET /api/system/logs?lines=20` | 200 |
|
||
| 1.16.6 | Set hostname | `POST /api/system/hostname {"hostname":"testpedal"}` | 200 |
|
||
| 1.16.7 | Restore hostname | `POST /api/system/hostname {"hostname":"pedal"}` | 200 |
|
||
|
||
### 1.17 Diagnostics
|
||
|
||
| # | Test | Method | Expect |
|
||
|---|------|--------|--------|
|
||
| 1.17.1 | Get diagnostics | `GET /api/diagnostics` | 200, jack/nam/midi/wifi/bt sections |
|
||
|
||
### 1.18 Audio Capture
|
||
|
||
| # | Test | Method | Expect |
|
||
|---|------|--------|--------|
|
||
| 1.18.1 | Start capture | `POST /api/capture/start` | 200 |
|
||
| 1.18.2 | Stop capture | `POST /api/capture/stop` | 200 |
|
||
| 1.18.3 | List captures | `GET /api/captures` | 200 |
|
||
| 1.18.4 | Get capture file | `GET /api/captures/{filename}` | 200, audio file |
|
||
|
||
### 1.19 Levels
|
||
|
||
| # | Test | Method | Expect |
|
||
|---|------|--------|--------|
|
||
| 1.19.1 | Get levels with signal | `GET /api/levels` | all zero when no signal |
|
||
| 1.19.2 | Levels bounded 0–100 | sanity check | never negative, never > 100 |
|
||
|
||
### 1.20 MIDI
|
||
|
||
| # | Test | Method | Expect |
|
||
|---|------|--------|--------|
|
||
| 1.20.1 | Get MIDI mappings | `GET /api/midi/mappings` | 200 |
|
||
| 1.20.2 | Start MIDI learn | `POST /api/midi/learn {"param_key":"delay.feedback"}` | 200 |
|
||
|
||
### 1.21 Network / WiFi
|
||
|
||
| # | Test | Method | Expect |
|
||
|---|------|--------|--------|
|
||
| 1.21.1 | Get network status | `GET /api/network` | 200 |
|
||
| 1.21.2 | Get WiFi status | `GET /api/wifi/status` | 200 |
|
||
| 1.21.3 | Scan WiFi | `GET /api/wifi/scan` | 200 |
|
||
| 1.21.4 | Saved networks | `GET /api/wifi/saved` | 200 |
|
||
| 1.21.5 | Hotspot status | `GET /api/wifi/hotspot` | 200 |
|
||
| 1.21.6 | DHCP config | `POST /api/network/dhcp` | 200 |
|
||
| 1.21.7 | Static IP | `POST /api/network/static {"ip":"192.168.4.100","mask":"24","gateway":"192.168.4.1"}` | 200 |
|
||
|
||
### 1.22 Backup / Restore
|
||
|
||
| # | Test | Method | Expect |
|
||
|---|------|--------|--------|
|
||
| 1.22.1 | Create backup | `POST /api/backup` | 200 |
|
||
| 1.22.2 | List backups | `GET /api/backup/list` | 200 |
|
||
| 1.22.3 | Restore backup | `POST /api/restore {"filename":"backup_xxx.zip"}` | 200 |
|
||
| 1.22.4 | Restore invalid backup | `POST /api/restore {"filename":"bogus.zip"}` | 404 or 500 |
|
||
|
||
---
|
||
|
||
## 2. Automated UI Tests 🤖
|
||
|
||
Run with cloakbrowser. Verify all UI elements render and interact.
|
||
|
||
### 2.1 Page Load
|
||
|
||
| # | Test | Expect |
|
||
|---|------|--------|
|
||
| 2.1.1 | Page loads without JS errors | console empty |
|
||
| 2.1.2 | Title shows "Pi Multi-FX" | `document.title` contains "FX" |
|
||
| 2.1.3 | No broken images | all img tags have src that resolves |
|
||
|
||
### 2.2 Preset Display
|
||
|
||
| # | Test | Expect |
|
||
|---|------|--------|
|
||
| 2.2.1 | Current preset name visible | preset name from state rendered |
|
||
| 2.2.2 | Bank/program shown | "Bank 0 · Program 1" or similar |
|
||
| 2.2.3 | Block chain renders blocks | one block element per `blocks[]` |
|
||
|
||
### 2.3 Block Controls
|
||
|
||
| # | Test | Expect |
|
||
|---|------|--------|
|
||
| 2.3.1 | Block has enable/disable toggle | click toggles block off → state updates |
|
||
| 2.3.2 | Block shows name | name from state rendered |
|
||
| 2.3.3 | Block shows params (if any) | parameter labels/values visible |
|
||
|
||
### 2.4 VU Meters / Stats
|
||
|
||
| # | Test | Expect |
|
||
|---|------|--------|
|
||
| 2.4.1 | Input VU meter visible | element with VU class/id |
|
||
| 2.4.2 | Output VU meter visible | element with VU class/id |
|
||
| 2.4.3 | CPU percentage shown | number matching `cpu_percent` |
|
||
| 2.4.4 | NAM CPU shown | number matching `nam_cpu` |
|
||
|
||
### 2.5 Bypass Button
|
||
|
||
| # | Test | Expect |
|
||
|---|------|--------|
|
||
| 2.5.1 | Bypass button exists | clickable element |
|
||
| 2.5.2 | Click toggles bypass | state change reflected in class/icon |
|
||
|
||
### 2.6 Volume Slider
|
||
|
||
| # | Test | Expect |
|
||
|---|------|--------|
|
||
| 2.6.1 | Volume slider exists | input[type=range] |
|
||
| 2.6.2 | Slider value matches state | initial value = master_volume |
|
||
| 2.6.3 | Slider change updates state | drag to 0.5, state changes |
|
||
|
||
### 2.7 Tuner
|
||
|
||
| # | Test | Expect |
|
||
|---|------|--------|
|
||
| 2.7.1 | Tuner button exists | clickable element |
|
||
| 2.7.2 | Click opens tuner display | tuner overlay or panel appears |
|
||
| 2.7.3 | Tuner shows note/frequency | label elements present |
|
||
| 2.7.4 | Close tuner | click close/disable, tuner off |
|
||
|
||
### 2.8 Overlays
|
||
|
||
| # | Test | Expect |
|
||
|---|------|--------|
|
||
| 2.8.1 | Preset browser opens | click preset name → overlay |
|
||
| 2.8.2 | Settings opens | gear icon → settings overlay |
|
||
| 2.8.3 | Downloads opens | downloads button → overlay |
|
||
| 2.8.4 | Snapshots panel opens | snapshot button → overlay |
|
||
| 2.8.5 | Each overlay can close | close button or backdrop click |
|
||
|
||
---
|
||
|
||
## 3. Audio Chain — Functional 🧪🧑
|
||
|
||
**Setup:** Guitar → DI → Interface input. Headphones on interface output.
|
||
|
||
### 3.1 Basic Signal Flow
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 3.1.1 | Clean input passes through | Play open low E, no blocks enabled | Hear clean DI signal |
|
||
| 3.1.2 | No digital artifacts at idle | Stop playing, listen 10s | No hiss, hum, clicks, pops |
|
||
| 3.1.3 | Input level visible | Play hard, watch `/api/levels` | `input_rms` > 0 |
|
||
| 3.1.4 | Output level visible | Play, watch `/api/levels` | `output_rms` > 0 |
|
||
| 3.1.5 | Output follows input dynamically | Palm mute → open chord | Levels change in real time |
|
||
|
||
### 3.2 Bypass (Relay)
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 3.2.1 | Bypass ON signal | Bypass on, play | Signal is 1:1 with input (guitar straight to out) |
|
||
| 3.2.2 | Bypass OFF with NAM | Load NAM, bypass off | NAM sound (amp/cab sim) |
|
||
| 3.2.3 | Bypass toggle pop test | Toggle bypass 10× while playing | No pops, clicks, or dropouts |
|
||
| 3.2.4 | Bypass volume parity | Bypass on vs bypass off (no NAM) | Same loudness ±1dB |
|
||
|
||
### 3.3 NAM Processing
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 3.3.1 | Clean NAM model | Load `fender_deluxe_reverb_clean` | Clean amp sound, not DI |
|
||
| 3.3.2 | High-gain NAM model | Load `marshall_jcm900_cha_full_rig` | Distorted/high-gain sound |
|
||
| 3.3.3 | Model switch no-drop | Switch between 2 models while playing | < 50ms gap, no crash |
|
||
| 3.3.4 | NAM CPU within limits | Load heaviest model, play hard | `nam_cpu` < 85% |
|
||
| 3.3.5 | NAM responds to dynamics | Soft pick → hard pick | Noticeable gain increase |
|
||
| 3.3.6 | NAM + clean comparison | Bypass toggle: A/B clean ↔ NAM | Audibly different (amp sim) |
|
||
|
||
### 3.4 IR Cabinet
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 3.4.1 | Load IR with NAM | Load NAM + IR | Cabinet simulation audible |
|
||
| 3.4.2 | IR vs no-IR diff | Toggle IR on/off | IR adds room/cabinet character |
|
||
| 3.4.3 | IR unload (clean NAM) | Unload IR, keep NAM | NAM still works, IR removed |
|
||
| 3.4.4 | IR switch while playing | Load IR A → play → load IR B | Switches clean, no crash |
|
||
|
||
### 3.5 Volume & Dynamics
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 3.5.1 | Volume 0 = silence | Set volume 0.0, play | No output (or silence) |
|
||
| 3.5.2 | Volume taper | Sweep 0.0 → 1.0 in 0.1 steps | Smooth, no jumps |
|
||
| 3.5.3 | Volume max clarity | Volume 1.0 | No clipping if input clean |
|
||
| 3.5.4 | Volume change while playing | Set vol 0.5 → 0.8 while playing | Smooth transition, no click |
|
||
|
||
### 3.6 Hum / Noise
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 3.6.1 | 60Hz hum level (with humbuckers) | Guitar plugged, not touching strings | -60dB or better vs signal |
|
||
| 3.6.2 | Noise floor (no cable) | Unplug input cable | No audible hiss/hum |
|
||
| 3.6.3 | Noise with NAM + IR | Load both, no playing | Noise floor doesn't mask quiet playing |
|
||
| 3.6.4 | Noise gate effect | Enable noise gate block | Noise suppressed when not playing |
|
||
|
||
### 3.7 Latency (subjective)
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 3.7.1 | Subjective feel | Strum chords, fast picking | Feels "instant" — < 10ms |
|
||
| 3.7.2 | Latency vs buffer | Test 64, 128, 256, 512 at 48kHz | Each step increases latency audibly |
|
||
| 3.7.3 | Latency with NAM | NAM loaded, play fast passages | No flamming or doubling |
|
||
|
||
### 3.8 Latency (loopback measurement)
|
||
|
||
**Loopback:** Patch cable from interface OUT → interface IN (second channel).
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 3.8.1 | Round-trip latency | Record click → measure time to loopback | < 15ms RTL at 512/48k |
|
||
| 3.8.2 | NAM adds latency | Same test with NAM loaded | < +2ms over bypass |
|
||
| 3.8.3 | Jitter | 10 loopback measurements | Max-min < 2ms |
|
||
|
||
---
|
||
|
||
## 4. Buffer / Sample Rate 🧪🧑
|
||
|
||
### 4.1 Buffer Size
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 4.1.1 | 64 @ 48kHz | Set period 64, rate 48000 (high xrun risk) | JACK starts, no xrun burst within 5s |
|
||
| 4.1.2 | 128 @ 48kHz | Set period 128 | Stable, NAM CPU < 85% |
|
||
| 4.1.3 | 256 @ 48kHz | Set period 256 (default high-perf) | Stable, NAM CPU < 65% |
|
||
| 4.1.4 | 512 @ 48kHz | Set period 512 (default stable) | Stable, NAM CPU < 45% |
|
||
| 4.1.5 | 1024 @ 48kHz | Set period 1024 | Stable, high latency, low CPU |
|
||
| 4.1.6 | Rapid change 64→512→64 while playing | Change both ends quickly | No crash, no sustained xruns |
|
||
| 4.1.7 | State reflects period | Check `/api/settings` after change | `audio.period` matches |
|
||
|
||
### 4.2 Sample Rate
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 4.2.1 | 44100 Hz | Set rate 44100 | JACK starts, audio plays |
|
||
| 4.2.2 | 48000 Hz | Set rate 48000 | Audio plays correctly |
|
||
| 4.2.3 | 96000 Hz | Set rate 96000 | JACK starts (may fail on USB 1.1) |
|
||
| 4.2.4 | Rate change while playing | 48k → 44.1k while strumming | Brief gap, then resumes correctly |
|
||
| 4.2.5 | Rate change preserves other state | Volume, routing, bypass unchanged after rate switch | All preserved |
|
||
|
||
### 4.3 Combined Changes (stability)
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 4.3.1 | Rapid cycle 10× | 512/48k → 128/44k → 512/48k → 256/96k → 512/48k | No crash, no unrecoverable state |
|
||
| 4.3.2 | 30s soak at each | Play for 30s at each (64,128,256,512,1024) × (44.1k,48k,96k) | No xruns reported in logs |
|
||
|
||
---
|
||
|
||
## 5. Routing Modes 🧪🧑
|
||
|
||
**Setup:** Interface with 2 output channels → 2 input channels (loopback).
|
||
|
||
### 5.1 Mono
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 5.1.1 | Mono routing | Set routing=mono, single NAM block | Audio out both channels (mono) |
|
||
| 5.1.2 | Breakpoint in mono | Set breakpoint, verify | No effect in mono (skip) |
|
||
|
||
### 5.2 4-Cable Method
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 5.2.1 | 4CM engages | Set routing=4cm | Two output sends active |
|
||
| 5.2.2 | Pre/post split audible | Set breakpoint mid-chain, A/B | Blocks before breakpoint vs after are routed separately |
|
||
| 5.2.3 | Switch mono↔4cm while playing | Toggle while strumming | No crash, transitions cleanly |
|
||
| 5.2.4 | 4CM with single block (no split) | 1 block, breakpoint at 7 | Works, no error |
|
||
|
||
---
|
||
|
||
## 6. Preset System (Functional) 🧪
|
||
|
||
### 6.1 Preset Loading
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 6.1.1 | Preset recall restores NAM | Save preset with NAM → load different → reload saved | NAM model restored |
|
||
| 6.1.2 | Preset recall restores blocks | Blocks enabled/bypass states restored exactly | Each block toggled to saved state |
|
||
| 6.1.3 | Preset recall restores volume | Volume restored | Exactly as saved |
|
||
| 6.1.4 | Preset recall restores routing | Routing mode/breakpoint restored | Exactly as saved |
|
||
| 6.1.5 | Preset A/B switching while playing | Switch between 2 presets rapidly | No crash, no stuck notes |
|
||
| 6.1.6 | Preset persistence across restart | Save → restart → recall | All state intact |
|
||
|
||
### 6.2 Snapshot
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 6.2.1 | Snapshot captures block state | Toggle some blocks → save snapshot → change blocks → recall | Blocks restored to saved state |
|
||
| 6.2.2 | Snapshot captures volume | Volume 0.3 → save snap → vol 1.0 → recall | Volume restored to 0.3 |
|
||
| 6.2.3 | Snapshot recall different preset | Load preset A, save snap → load preset B → recall snap | Blocks from A's snap applied to B |
|
||
| 6.2.4 | All 8 slots work | Save + recall each slot | Each returns correct state |
|
||
|
||
---
|
||
|
||
## 7. MIDI (Manual) 🧑
|
||
|
||
**Setup:** MIDI controller (e.g., USB pedal board) connected to Pi.
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 7.1 | MIDI Program Change | Send PC from controller | Preset changes to matching bank/program |
|
||
| 7.2 | MIDI CC mapping | Assign CC to parameter → send CC | Parameter changes |
|
||
| 7.3 | MIDI learn | Use learn mode | CC mapping saved |
|
||
| 7.4 | MIDI clock sync | Start external clock | Tempo-based FX (delay) sync |
|
||
| 7.5 | USB MIDI hotplug | Plug/unplug controller | No crash, reconnects |
|
||
|
||
---
|
||
|
||
## 8. Bluetooth (Manual) 🧑
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 8.1 | BT pairing | Pair with phone | Paired device shown in `GET /api/bluetooth/paired` |
|
||
| 8.2 | BT audio streaming | Stream backing track from phone | Audio routed through FX chain |
|
||
| 8.3 | BT MIDI | Pair MIDI controller over BT | MIDI messages received |
|
||
| 8.4 | BT scan | Trigger scan | Nearby devices listed |
|
||
| 8.5 | BT disconnect + reconnect | Disconnect → reconnect | Stable connection |
|
||
|
||
---
|
||
|
||
## 9. WiFi (Manual) 🧑
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 9.1 | WiFi scan | `GET /api/wifi/scan` | Networks listed |
|
||
| 9.2 | Connect to network | `POST /api/wifi/connect {"ssid":"...","password":"..."}` | Connected |
|
||
| 9.3 | Hotspot enable | Enable hotspot | Pi becomes AP, clients can connect |
|
||
| 9.4 | Hotspot disable | Disable hotspot | Pi leaves AP mode |
|
||
| 9.5 | Static IP | Configure static | IP persists, reachable |
|
||
| 9.6 | DHCP fallback | Switch to DHCP | IP from DHCP assigned |
|
||
| 9.7 | Audio over WiFi (capture) | Capture audio via network download | Capture file downloads correctly |
|
||
|
||
---
|
||
|
||
## 10. System / Stability 💀🧪
|
||
|
||
### 10.1 Resource Monitor
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 10.1.1 | CPU < 70% at 512/48k | Play continuously 1 min | `cpu_percent` < 70 |
|
||
| 10.1.2 | NAM CPU < 85% under load | Heaviest model + IR | `nam_cpu` < 85 |
|
||
| 10.1.3 | Memory stable over 1hr | Poll `/api/system` every 60s | `memory.used` doesn't grow > 50MB |
|
||
| 10.1.4 | CPU temp < 75°C under load | Continuous playing 10 min | `cpu_temp_c` < 75 |
|
||
|
||
### 10.2 Service Restart
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 10.2.1 | Restore last preset after restart | Play → restart → wait → check state | Current preset unchanged, NAM loaded |
|
||
| 10.2.2 | Restart with JACK already running | Kill JACK → restart pedal | Pedal cleans up and starts fresh |
|
||
| 10.2.3 | Restart with active audio | Play while restarting | Brief gap, no crash |
|
||
|
||
### 10.3 Reboot
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 10.3.1 | Reboot recovery | Reboot Pi → wait 60s → check /api/state | Service running, preset restored |
|
||
| 10.3.2 | Reboot with audio interface unplugged | Unplug interface → reboot → plug in | JACK restarts when device appears |
|
||
|
||
### 10.4 Long Soak
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 10.4.1 | 30-min continuous play | Play guitar for 30 min at 512/48k | No xruns, no crash, stable CPU/mem |
|
||
| 10.4.2 | 30-min idle | No signal for 30 min | No crash, no runaway processes |
|
||
| 10.4.3 | 60-min mixed | 20 min play + 10 idle × 2 cycles | Same |
|
||
|
||
### 10.5 Stress
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 10.5.1 | Rapid block toggles | Toggle all blocks 50× in 10s | No crash, no desync |
|
||
| 10.5.2 | Rapid preset switching | Switch presets 30× in 15s | No crash, each loads correctly |
|
||
| 10.5.3 | API hammer | 100 concurrent `GET /api/state` | All 200 OK, no crash |
|
||
| 10.5.4 | Browser + API simultaneous | Load UI + hammer API | Both work, WS stays connected |
|
||
|
||
---
|
||
|
||
## 11. Edge Cases & Error Handling 🤖🧪
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 11.1 | Empty body to POST endpoint | `POST /api/bypass` with no body | 422 or 400, not crash |
|
||
| 11.2 | Invalid JSON body | `POST /api/bypass` with `{bad json}` | 422, not crash |
|
||
| 11.3 | Missing required fields | `POST /api/wifi/connect {}` | 400 or 422 |
|
||
| 11.4 | Negative bank/program | `GET /api/presets/-1/-1` | 200 (empty slot) or 404 |
|
||
| 11.5 | Bank/program overshoot | `GET /api/presets/999/999` | 200 (empty slot) or 404 |
|
||
| 11.6 | XSS in preset name | Save preset with `<script>alert(1)</script>` | Rendered as text, not executed |
|
||
| 11.7 | Long preset name | Save with 500 char name | Truncated at UI but doesn't crash |
|
||
| 11.8 | Unicode preset name | Save with "Jazz 🎸 Tone" | Stored and returned correctly |
|
||
| 11.9 | Zero blocks | Save preset with `chain: []` | Returns 200, edge case |
|
||
| 11.10 | Duplicate block IDs | Manually assign same ID to 2 blocks | Second one still matches (no crash) |
|
||
|
||
---
|
||
|
||
## 12. Audio-Capture Recordings 🧪
|
||
|
||
Use the capture endpoint to record audio for analysis.
|
||
|
||
| # | Test | Procedure | Pass criteria |
|
||
|---|------|-----------|--------------|
|
||
| 12.1 | Record clean bypass | Start capture → play → stop | WAV downloads, plays back clean |
|
||
| 12.2 | Record with NAM | Capture with NAM loaded | WAV has NAM processing |
|
||
| 12.3 | Record duration | Record 10s | File length ≈ 10s |
|
||
| 12.4 | Record cycle | Start → stop → start → stop | Multiple files, no orphaned state |
|
||
| 12.5 | Record + block toggle in progress | Toggle while recording | Capture file has toggle point (practical test) |
|
||
|
||
---
|
||
|
||
## Test Execution Sheets
|
||
|
||
Use these sheets to track results per run.
|
||
|
||
### Run Info
|
||
|
||
```
|
||
Date: _______________
|
||
Tester: _______________
|
||
Git commit: _______________
|
||
Config: ____/_____ (period/rate, e.g. 512/48000)
|
||
NAM model: _______________
|
||
IR loaded: _______________
|
||
Interface: _______________
|
||
```
|
||
|
||
### Pass / Fail Summary
|
||
|
||
| Section | 🤖 Automated | 🧪 Manual | 🧑 Ears | 💀 Destructive |
|
||
|---------|:---:|:---:|:---:|:---:|
|
||
| 1. API | ___/___ | — | — | — |
|
||
| 2. UI | ___/___ | — | — | — |
|
||
| 3. Audio Chain | — | ___/___ | ___/___ | — |
|
||
| 4. Buffer/SR | — | ___/___ | ___/___ | — |
|
||
| 5. Routing | — | ___/___ | ___/___ | — |
|
||
| 6. Presets | ___/___ | ___/___ | — | — |
|
||
| 7. MIDI | — | — | ___/___ | — |
|
||
| 8. Bluetooth | — | — | ___/___ | — |
|
||
| 9. WiFi | — | — | ___/___ | — |
|
||
| 10. Stability | — | — | — | ___/___ |
|
||
| 11. Edge Cases | ___/___ | ___/___ | — | — |
|
||
| 12. Captures | — | ___/___ | — | — |
|
||
| **Total** | **___/___** | **___/___** | **___/___** | **___/___** |
|
||
|
||
### Known Issues / Notes
|
||
|
||
```
|
||
______________________________________________________________________
|
||
______________________________________________________________________
|
||
```
|