shawn/pi-multifx-pedal
1
0
Fork 0
Code Issues 5 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: comprehensive manual test plan for mobile QA
CI / test (push) Has been cancelled
Details

Browse Source
This commit is contained in:
Shawn
2026-06-18 18:37:04 -04:00
parent bb9c835f98
commit 6f24a940ba
1 changed files with 335 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
TEST_PLAN.md
+335
View File
@@ -0,0 +1,335 @@
# Pi Multi-FX Pedal — Manual Test Plan
> **Device:** Pixel 9a, Firefox
> **URL:** `http://pedal.local`
> **Pre-req:** Guitar plugged into Focusrite input 1, speaker/monitors connected to Focusrite outputs
---
## 1. Initial State Check
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 1.1 | Open `http://pedal.local` in Firefox | Page loads, dark theme, status bar at top |
| 1.2 | Status bar shows green dot (connected) | Green dot, not red |
| 1.3 | Status bar shows channel name (e.g. `GUITAR`) | Amber text on dark badge |
| 1.4 | Status bar shows preset name | Truncated with ellipsis if long |
| 1.5 | Status bar shows VU meter bars (I / O) | Green bar (I), Amber bar (O) |
| 1.6 | Status bar shows NAM CPU % | e.g. `42%` |
| 1.7 | Play guitar — I bar moves | Green VU fills in response to playing |
| 1.8 | Play guitar — O bar moves | Amber VU fills in response |
| 1.9 | Play guitar — NAM CPU % changes | Should move between ~30-50% |
---
## 2. Home Screen — NAM Hero
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 2.1 | NAM badge shows current model name | Large text in center of hero card |
| 2.2 | Green LED dot top-right is ON | Model is loaded and active |
| 2.3 | Tap NAM hero | Opens Downloads overlay (local NAM models) |
| 2.4 | Close Downloads overlay | Returns to home screen |
---
## 3. Home Screen — Pedal Row (FX Blocks)
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 3.1 | Pedal row shows FX blocks as stompbox icons | Horizontal scrollable row of pedals |
| 3.2 | Each pedal shows LED (green=on, red=off) | Colored dot at top of each pedal |
| 3.3 | Each pedal shows icon + label | Text below icon |
| 3.4 | Each pedal shows toggle switch | Green when enabled, grey when bypassed |
| 3.5 | Tap a pedal's toggle switch | Pedal toggles on/off, LED changes |
| 3.6 | **AUDIO:** Play guitar while toggling — no dropout | No pop/click, audio continues |
| 3.7 | "+ Add Block" pedal at end (dashed border) | Last item in row, shows + icon |
| 3.8 | Tap "+ Add Block" | Opens Downloads overlay |
---
## 4. Home Screen — Volume Slider
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 4.1 | Volume label reads `VOL` | Text at left of slider |
| 4.2 | Drag volume slider up/down | Amber fill bar moves, percentage updates |
| 4.3 | **AUDIO:** Volume changes smoothly | No pops when dragging |
| 4.4 | Drag to 0% | Audio silent |
| 4.5 | Drag back to 80% | Audio returns at normal level |
---
## 5. Home Screen — Footswitch
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 5.1 | Footswitch shows green when bypass OFF | Green gradient, `BYPASS` text |
| 5.2 | **AUDIO:** Tap footswitch | Bypass toggles — audio goes from processed to DI |
| 5.3 | Footswitch turns red when bypass ON | Red gradient, `BYPASS` text |
| 5.4 | **AUDIO:** Tap again | Audio returns to processed (NAM engaged) |
| 5.5 | **AUDIO:** No pop/click on toggle | Smooth transition |
---
## 6. Home Screen — Tab Bar
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 6.1 | Bottom tab bar visible | 5 tabs: Presets, Snapshots, Downloads, Settings, 🔄 |
| 6.2 | Tap each tab | Opens corresponding overlay |
| 6.3 | Tap same tab again | Closes overlay (or overlay stays open) |
| 6.4 | Tap close (X) button on overlay | Returns to home screen |
---
## 7. Presets Overlay
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 7.1 | Open Presets overlay | Grid of bank rows with preset names |
| 7.2 | Tap a preset | Preset loads, overlay closes |
| 7.3 | **AUDIO:** Sound changes to match preset | NAM model, FX chain, and routing load |
| 7.4 | **AUDIO:** No dropout during preset change | Smooth transition |
| 7.5 | Status bar preset name updates | Shows newly loaded preset name |
| 7.6 | Tap another preset in same bank | Loads that preset |
| 7.7 | Tap a preset in different bank | Loads preset from other bank |
| 7.8 | Close and reopen Presets | Last loaded preset highlighted |
---
## 8. Snapshots Overlay
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 8.1 | Open Snapshots overlay | Grid of numbered snapshot slots |
| 8.2 | Tap a snapshot slot | Recalls that snapshot |
| 8.3 | **AUDIO:** No dropout recalling snapshot | Smooth |
| 8.4 | Tap "Save Current" button | Saves current state to snapshot |
| 8.5 | Change a setting, then recall saved snapshot | Reverts to saved state |
| 8.6 | **AUDIO:** No dropout saving | Sound unaffected |
---
## 9. Downloads Overlay — Local Tab
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 9.1 | Open Downloads overlay | Opens to Local tab by default |
| 9.2 | Local tab shows installed `.nam` files | List of local models |
| 9.3 | Each model card shows name, size, architecture | Text info on card |
| 9.4 | Currently-loaded model has green border + ✅ | Visual indicator |
| 9.5 | Tap an unloaded model | Model loads, green border + ✅ appears |
| 9.6 | **AUDIO:** Sound changes to new model | New NAM model processes audio |
| 9.7 | **AUDIO:** No dropout during model switch | Smooth transition |
| 9.8 | status bar model name updates | NAM hero shows new name |
---
## 10. Downloads Overlay — NAM Tab
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 10.1 | Tap NAM tab (🎛) | Shows Tone3000 search |
| 10.2 | Search bar and filters visible | Search input, sort, gear, arch filters |
| 10.3 | Results load automatically (trending) | Grid of model cards appears |
| 10.4 | Type in search bar + tap Search button | Results filter by query |
| 10.5 | Change sort dropdown | Results re-sort |
| 10.6 | Tap gear filter (Amp, Pedal, Full Rig, etc.) | Results filtered by gear type |
| 10.7 | Scroll results + tap Next → | Pagination loads next page |
| 10.8 | Tap ← Prev | Goes back a page |
| 10.9 | Tap a model card | Opens detail panel |
| 10.10 | Detail shows name, author, size, description | Metadata visible |
| 10.11 | Tap "Get" / Install | Downloads model |
| 10.12 | After download, model appears in Local tab | Switch to Local to verify |
---
## 11. Downloads Overlay — IR Tab
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 11.1 | Tap IR tab (📦) | Shows Tone3000 IR search |
| 11.2 | Search loads trending IRs | Grid of IR cards |
| 11.3 | Search, filter, pagination (same as NAM tab) | Works like NAM tab |
| 11.4 | Install an IR | Downloads, appears in IR list |
---
## 12. Settings Overlay — Navigation
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 12.1 | Open Settings overlay | Sections: Network, Audio, Audio Interface, Bluetooth, System |
| 12.2 | Nav strip pinned below header | Doesn't scroll away |
| 12.3 | Tap each nav button | Scrolls to that section |
---
## 13. Settings — Audio Section
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 13.1 | Sample Rate dropdown shows current value | e.g. "48000" |
| 13.2 | **AUDIO:** Change Sample Rate (e.g. 44100) | POST succeeds, JACK restarts |
| 13.3 | **AUDIO:** NAM survives the change | NAM CPU > 0% after restart |
| 13.4 | Buffer Size dropdown shows current value | e.g. "512" |
| 13.5 | **AUDIO:** Change Buffer Size (e.g. 256) | POST succeeds, JACK restarts |
| 13.6 | **AUDIO:** NAM survives (CPU > 0%) | No DI signal |
| 13.7 | **AUDIO:** Change both rate + buffer together | Both apply, NAM survives |
| 13.8 | **AUDIO:** Change to unsupported rate (192000) | May fail gracefully or work |
| 13.9 | **AUDIO:** Change buffer to 1024 | Larger buffer, lower CPU, higher latency |
| 13.10 | **AUDIO:** Change back to 512 | Restores original |
| 13.11 | **AUDIO:** Verify no persistent "peaking/poping every second" | Clean audio through NAM |
---
## 14. Settings — Audio Interface Section
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 14.1 | Input Device dropdown lists ALSA devices | Focusrite Scarlett 2i2 shown |
| 14.2 | Output Device dropdown lists ALSA devices | Same device |
| 14.3 | Change Input Device | JACK restarts with new input |
| 14.4 | **AUDIO:** No dropout after device change | NAM CPU > 0% |
| 14.5 | Change back to original device | Restores |
| 14.6 | CH 1 Output dropdown | playback_1 / playback_2 / both |
| 14.7 | **AUDIO:** Change CH 1 Output | Output routing changes |
| 14.8 | CH 2 Output dropdown | playback_1 / playback_2 / both |
| 14.9 | **AUDIO:** Change CH 2 Output | Output routing changes |
| 14.10 | Backing Track Source dropdown | None / Bluetooth Audio / other JACK ports |
---
## 15. Settings — Routing Section
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 15.1 | Routing Mode shows "Mono" or "4CM" | Current mode displayed |
| 15.2 | **AUDIO:** Tap to toggle Mono ↔ 4CM | Routing changes |
| 15.3 | **AUDIO:** No dropout on toggle | NAM CPU > 0% after toggle |
| 15.4 | Toggle back | Restores |
| 15.5 | Channel Mode shows "Dual-Mono" or "Stereo" | Current mode displayed |
| 15.6 | Tap to toggle | Mode switches |
| 15.7 | **AUDIO:** No dropout on channel mode toggle | NAM CPU > 0% |
---
## 16. Settings — Channel Cards (CH 1 / CH 2)
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 16.1 | CH 1 card shows name, instrument, power toggle | Card visible in Audio Interface section |
| 16.2 | CH 2 card shows same | Second card visible |
| 16.3 | Instrument selector shows options | Guitar / Bass / Keys / Vocals |
| 16.4 | Change instrument | Updates |
| 16.5 | Power toggle (mute channel) | Channel mutes/unmutes |
---
## 17. Settings — Network Section
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 17.1 | Hostname displayed | e.g. "pedal" |
| 17.2 | Tap hostname → edit | Text input appears |
| 17.3 | Change hostname + save | Updates, may need reboot |
| 17.4 | Network mode shows DHCP or Static | Current mode |
| 17.5 | Tap to toggle DHCP/Static | Switches mode |
| 17.6 | IP, Gateway, DNS shown (if static) | Fields visible |
| 17.7 | Wi-Fi status shows connected or hotspot | Current state |
| 17.8 | Tap Wi-Fi scan | Lists available networks |
| 17.9 | Tap a network | Connects |
| 17.10 | **AUDIO:** Wi-Fi connect doesn't drop audio | NAM continues |
| 17.11 | Hotspot toggle | Enables/disables hotspot |
---
## 18. Settings — Bluetooth Section
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 18.1 | Bluetooth status shows Available/Powered | Current state |
| 18.2 | Discoverable toggle | Toggles discoverability |
| 18.3 | MIDI over BT toggle | Enables/disables BT MIDI |
| 18.4 | BT MIDI status shows running or not | After enable |
| 18.5 | **AUDIO:** BT toggle doesn't drop audio | NAM continues |
---
## 19. Settings — System Section
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 19.1 | NAM Engine field shows "C++" or "PyTorch" | Current engine mode |
| 19.2 | Tap to switch engine | Reloads model in new engine |
| 19.3 | **AUDIO:** NAM survives engine switch | CPU > 0% |
| 19.4 | CPU readout shows NAM + system | e.g. `42%` with `sys 30%` dimmed |
| 19.5 | Tap "Restart Service" | Pedal restarts, comes back up |
| 19.6 | **AUDIO:** After restart, NAM returns | CPU > 0%, model loaded |
| 19.7 | Tap "Reset Defaults" | Resets config |
| 19.8 | Tap "Reboot" | System reboots |
---
## 20. Detail Overlay (FX Block Controls)
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 20.1 | Tap any FX block in pedal row | Opens detail overlay |
| 20.2 | Block name and type shown at top | Header text |
| 20.3 | Parameter controls visible (varies by block type) | Knobs, sliders, toggles |
| 20.4 | Adjust a parameter | Value updates |
| 20.5 | **AUDIO:** Parameter changes take effect | Sound changes accordingly |
| 20.6 | **AUDIO:** No dropout adjusting parameters | Smooth |
| 20.7 | Close detail overlay | Returns to home screen |
---
## 21. Tuner
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 21.1 | Open Tuner (tap tuner icon if available) | Tuner screen opens |
| 21.2 | Play open string | Note name + cents displayed |
| 21.3 | Tune to reference | Visual indicator moves to center |
| 21.4 | Close tuner | Returns to home screen |
| 21.5 | **AUDIO:** Muted when tuner active | No output (mute for tuning) |
---
## 22. Stress Tests
| # | Test | Expected Result | Pass/Fail |
|---|------|-----------------|-----------|
| 22.1 | Rapidly tap footswitch 10x | Toggles each time, no crash |
| 22.2 | Rapidly change buffer size 5x | Each change works, NAM survives |
| 22.3 | Load 3 different NAM models in succession | Each loads cleanly |
| 22.4 | Change routing mode 5x in a row | No crash, CPU stable |
| 22.5 | Open/close all 5 overlays rapidly | No UI freezes |
| 22.6 | Leave pedal running 30 min with audio | No audio dropouts or xruns |
| 22.7 | Hard-refresh browser (Ctrl+Shift+R) | UI reloads cleanly, state intact |
---
## 23. Expected Reference Values
| Setting | Typical Value | CPU Range |
|---------|---------------|-----------|
| Buffer 128 @ 48k | 2.67ms window | 70-90% (may xrun) |
| Buffer 256 @ 48k | 5.33ms window | 60-80% (may xrun occasionally) |
| Buffer 512 @ 48k | 10.67ms window | 35-50% (stable) |
| Buffer 1024 @ 48k | 21.33ms window | 18-25% (very stable) |
| 4CM routing mode | Both channels processed | CPU ~1.5x mono |
| Model unloaded | No NAM processing | 0% NAM CPU |
| Bypass ON | DI signal | 0% NAM CPU |
---
## Known Quirks
- **After model unload:** NAM CPU drops to 0% (no model to process). Reload a model to restore.
- **Engine switch (C++ ↔ PyTorch):** May take 5-8 seconds. Model reloads in new engine.
- **Profile changes:** ~6-10s JACK restart pause. NAM CPU should return >0% after restart.
- **Browser cache:** Hard refresh (Ctrl+Shift+R) if UI seems stale after an update.
- **"Peaking/poping every second":** If this returns, increase buffer to 1024. Should be fixed by GC disable + 512 default.