From 6f24a940bad0fa22f8cf22989158aefe051074e6 Mon Sep 17 00:00:00 2001 From: Shawn Date: Thu, 18 Jun 2026 18:37:04 -0400 Subject: [PATCH] docs: comprehensive manual test plan for mobile QA --- TEST_PLAN.md | 335 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 335 insertions(+) create mode 100644 TEST_PLAN.md diff --git a/TEST_PLAN.md b/TEST_PLAN.md new file mode 100644 index 0000000..d4b0d0e --- /dev/null +++ b/TEST_PLAN.md @@ -0,0 +1,335 @@ +# Pi Multi-FX Pedal — Manual Test Plan + +> **Device:** Pixel 9a, Firefox +> **URL:** `http://pedal.local` +> **Pre-req:** Guitar plugged into Focusrite input 1, speaker/monitors connected to Focusrite outputs + +--- + +## 1. Initial State Check + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 1.1 | Open `http://pedal.local` in Firefox | Page loads, dark theme, status bar at top | +| 1.2 | Status bar shows green dot (connected) | Green dot, not red | +| 1.3 | Status bar shows channel name (e.g. `GUITAR`) | Amber text on dark badge | +| 1.4 | Status bar shows preset name | Truncated with ellipsis if long | +| 1.5 | Status bar shows VU meter bars (I / O) | Green bar (I), Amber bar (O) | +| 1.6 | Status bar shows NAM CPU % | e.g. `42%` | +| 1.7 | Play guitar — I bar moves | Green VU fills in response to playing | +| 1.8 | Play guitar — O bar moves | Amber VU fills in response | +| 1.9 | Play guitar — NAM CPU % changes | Should move between ~30-50% | + +--- + +## 2. Home Screen — NAM Hero + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 2.1 | NAM badge shows current model name | Large text in center of hero card | +| 2.2 | Green LED dot top-right is ON | Model is loaded and active | +| 2.3 | Tap NAM hero | Opens Downloads overlay (local NAM models) | +| 2.4 | Close Downloads overlay | Returns to home screen | + +--- + +## 3. Home Screen — Pedal Row (FX Blocks) + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 3.1 | Pedal row shows FX blocks as stompbox icons | Horizontal scrollable row of pedals | +| 3.2 | Each pedal shows LED (green=on, red=off) | Colored dot at top of each pedal | +| 3.3 | Each pedal shows icon + label | Text below icon | +| 3.4 | Each pedal shows toggle switch | Green when enabled, grey when bypassed | +| 3.5 | Tap a pedal's toggle switch | Pedal toggles on/off, LED changes | +| 3.6 | **AUDIO:** Play guitar while toggling — no dropout | No pop/click, audio continues | +| 3.7 | "+ Add Block" pedal at end (dashed border) | Last item in row, shows + icon | +| 3.8 | Tap "+ Add Block" | Opens Downloads overlay | + +--- + +## 4. Home Screen — Volume Slider + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 4.1 | Volume label reads `VOL` | Text at left of slider | +| 4.2 | Drag volume slider up/down | Amber fill bar moves, percentage updates | +| 4.3 | **AUDIO:** Volume changes smoothly | No pops when dragging | +| 4.4 | Drag to 0% | Audio silent | +| 4.5 | Drag back to 80% | Audio returns at normal level | + +--- + +## 5. Home Screen — Footswitch + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 5.1 | Footswitch shows green when bypass OFF | Green gradient, `BYPASS` text | +| 5.2 | **AUDIO:** Tap footswitch | Bypass toggles — audio goes from processed to DI | +| 5.3 | Footswitch turns red when bypass ON | Red gradient, `BYPASS` text | +| 5.4 | **AUDIO:** Tap again | Audio returns to processed (NAM engaged) | +| 5.5 | **AUDIO:** No pop/click on toggle | Smooth transition | + +--- + +## 6. Home Screen — Tab Bar + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 6.1 | Bottom tab bar visible | 5 tabs: Presets, Snapshots, Downloads, Settings, 🔄 | +| 6.2 | Tap each tab | Opens corresponding overlay | +| 6.3 | Tap same tab again | Closes overlay (or overlay stays open) | +| 6.4 | Tap close (X) button on overlay | Returns to home screen | + +--- + +## 7. Presets Overlay + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 7.1 | Open Presets overlay | Grid of bank rows with preset names | +| 7.2 | Tap a preset | Preset loads, overlay closes | +| 7.3 | **AUDIO:** Sound changes to match preset | NAM model, FX chain, and routing load | +| 7.4 | **AUDIO:** No dropout during preset change | Smooth transition | +| 7.5 | Status bar preset name updates | Shows newly loaded preset name | +| 7.6 | Tap another preset in same bank | Loads that preset | +| 7.7 | Tap a preset in different bank | Loads preset from other bank | +| 7.8 | Close and reopen Presets | Last loaded preset highlighted | + +--- + +## 8. Snapshots Overlay + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 8.1 | Open Snapshots overlay | Grid of numbered snapshot slots | +| 8.2 | Tap a snapshot slot | Recalls that snapshot | +| 8.3 | **AUDIO:** No dropout recalling snapshot | Smooth | +| 8.4 | Tap "Save Current" button | Saves current state to snapshot | +| 8.5 | Change a setting, then recall saved snapshot | Reverts to saved state | +| 8.6 | **AUDIO:** No dropout saving | Sound unaffected | + +--- + +## 9. Downloads Overlay — Local Tab + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 9.1 | Open Downloads overlay | Opens to Local tab by default | +| 9.2 | Local tab shows installed `.nam` files | List of local models | +| 9.3 | Each model card shows name, size, architecture | Text info on card | +| 9.4 | Currently-loaded model has green border + ✅ | Visual indicator | +| 9.5 | Tap an unloaded model | Model loads, green border + ✅ appears | +| 9.6 | **AUDIO:** Sound changes to new model | New NAM model processes audio | +| 9.7 | **AUDIO:** No dropout during model switch | Smooth transition | +| 9.8 | status bar model name updates | NAM hero shows new name | + +--- + +## 10. Downloads Overlay — NAM Tab + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 10.1 | Tap NAM tab (🎛) | Shows Tone3000 search | +| 10.2 | Search bar and filters visible | Search input, sort, gear, arch filters | +| 10.3 | Results load automatically (trending) | Grid of model cards appears | +| 10.4 | Type in search bar + tap Search button | Results filter by query | +| 10.5 | Change sort dropdown | Results re-sort | +| 10.6 | Tap gear filter (Amp, Pedal, Full Rig, etc.) | Results filtered by gear type | +| 10.7 | Scroll results + tap Next → | Pagination loads next page | +| 10.8 | Tap ← Prev | Goes back a page | +| 10.9 | Tap a model card | Opens detail panel | +| 10.10 | Detail shows name, author, size, description | Metadata visible | +| 10.11 | Tap "Get" / Install | Downloads model | +| 10.12 | After download, model appears in Local tab | Switch to Local to verify | + +--- + +## 11. Downloads Overlay — IR Tab + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 11.1 | Tap IR tab (📦) | Shows Tone3000 IR search | +| 11.2 | Search loads trending IRs | Grid of IR cards | +| 11.3 | Search, filter, pagination (same as NAM tab) | Works like NAM tab | +| 11.4 | Install an IR | Downloads, appears in IR list | + +--- + +## 12. Settings Overlay — Navigation + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 12.1 | Open Settings overlay | Sections: Network, Audio, Audio Interface, Bluetooth, System | +| 12.2 | Nav strip pinned below header | Doesn't scroll away | +| 12.3 | Tap each nav button | Scrolls to that section | + +--- + +## 13. Settings — Audio Section + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 13.1 | Sample Rate dropdown shows current value | e.g. "48000" | +| 13.2 | **AUDIO:** Change Sample Rate (e.g. 44100) | POST succeeds, JACK restarts | +| 13.3 | **AUDIO:** NAM survives the change | NAM CPU > 0% after restart | +| 13.4 | Buffer Size dropdown shows current value | e.g. "512" | +| 13.5 | **AUDIO:** Change Buffer Size (e.g. 256) | POST succeeds, JACK restarts | +| 13.6 | **AUDIO:** NAM survives (CPU > 0%) | No DI signal | +| 13.7 | **AUDIO:** Change both rate + buffer together | Both apply, NAM survives | +| 13.8 | **AUDIO:** Change to unsupported rate (192000) | May fail gracefully or work | +| 13.9 | **AUDIO:** Change buffer to 1024 | Larger buffer, lower CPU, higher latency | +| 13.10 | **AUDIO:** Change back to 512 | Restores original | +| 13.11 | **AUDIO:** Verify no persistent "peaking/poping every second" | Clean audio through NAM | + +--- + +## 14. Settings — Audio Interface Section + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 14.1 | Input Device dropdown lists ALSA devices | Focusrite Scarlett 2i2 shown | +| 14.2 | Output Device dropdown lists ALSA devices | Same device | +| 14.3 | Change Input Device | JACK restarts with new input | +| 14.4 | **AUDIO:** No dropout after device change | NAM CPU > 0% | +| 14.5 | Change back to original device | Restores | +| 14.6 | CH 1 Output dropdown | playback_1 / playback_2 / both | +| 14.7 | **AUDIO:** Change CH 1 Output | Output routing changes | +| 14.8 | CH 2 Output dropdown | playback_1 / playback_2 / both | +| 14.9 | **AUDIO:** Change CH 2 Output | Output routing changes | +| 14.10 | Backing Track Source dropdown | None / Bluetooth Audio / other JACK ports | + +--- + +## 15. Settings — Routing Section + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 15.1 | Routing Mode shows "Mono" or "4CM" | Current mode displayed | +| 15.2 | **AUDIO:** Tap to toggle Mono ↔ 4CM | Routing changes | +| 15.3 | **AUDIO:** No dropout on toggle | NAM CPU > 0% after toggle | +| 15.4 | Toggle back | Restores | +| 15.5 | Channel Mode shows "Dual-Mono" or "Stereo" | Current mode displayed | +| 15.6 | Tap to toggle | Mode switches | +| 15.7 | **AUDIO:** No dropout on channel mode toggle | NAM CPU > 0% | + +--- + +## 16. Settings — Channel Cards (CH 1 / CH 2) + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 16.1 | CH 1 card shows name, instrument, power toggle | Card visible in Audio Interface section | +| 16.2 | CH 2 card shows same | Second card visible | +| 16.3 | Instrument selector shows options | Guitar / Bass / Keys / Vocals | +| 16.4 | Change instrument | Updates | +| 16.5 | Power toggle (mute channel) | Channel mutes/unmutes | + +--- + +## 17. Settings — Network Section + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 17.1 | Hostname displayed | e.g. "pedal" | +| 17.2 | Tap hostname → edit | Text input appears | +| 17.3 | Change hostname + save | Updates, may need reboot | +| 17.4 | Network mode shows DHCP or Static | Current mode | +| 17.5 | Tap to toggle DHCP/Static | Switches mode | +| 17.6 | IP, Gateway, DNS shown (if static) | Fields visible | +| 17.7 | Wi-Fi status shows connected or hotspot | Current state | +| 17.8 | Tap Wi-Fi scan | Lists available networks | +| 17.9 | Tap a network | Connects | +| 17.10 | **AUDIO:** Wi-Fi connect doesn't drop audio | NAM continues | +| 17.11 | Hotspot toggle | Enables/disables hotspot | + +--- + +## 18. Settings — Bluetooth Section + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 18.1 | Bluetooth status shows Available/Powered | Current state | +| 18.2 | Discoverable toggle | Toggles discoverability | +| 18.3 | MIDI over BT toggle | Enables/disables BT MIDI | +| 18.4 | BT MIDI status shows running or not | After enable | +| 18.5 | **AUDIO:** BT toggle doesn't drop audio | NAM continues | + +--- + +## 19. Settings — System Section + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 19.1 | NAM Engine field shows "C++" or "PyTorch" | Current engine mode | +| 19.2 | Tap to switch engine | Reloads model in new engine | +| 19.3 | **AUDIO:** NAM survives engine switch | CPU > 0% | +| 19.4 | CPU readout shows NAM + system | e.g. `42%` with `sys 30%` dimmed | +| 19.5 | Tap "Restart Service" | Pedal restarts, comes back up | +| 19.6 | **AUDIO:** After restart, NAM returns | CPU > 0%, model loaded | +| 19.7 | Tap "Reset Defaults" | Resets config | +| 19.8 | Tap "Reboot" | System reboots | + +--- + +## 20. Detail Overlay (FX Block Controls) + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 20.1 | Tap any FX block in pedal row | Opens detail overlay | +| 20.2 | Block name and type shown at top | Header text | +| 20.3 | Parameter controls visible (varies by block type) | Knobs, sliders, toggles | +| 20.4 | Adjust a parameter | Value updates | +| 20.5 | **AUDIO:** Parameter changes take effect | Sound changes accordingly | +| 20.6 | **AUDIO:** No dropout adjusting parameters | Smooth | +| 20.7 | Close detail overlay | Returns to home screen | + +--- + +## 21. Tuner + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 21.1 | Open Tuner (tap tuner icon if available) | Tuner screen opens | +| 21.2 | Play open string | Note name + cents displayed | +| 21.3 | Tune to reference | Visual indicator moves to center | +| 21.4 | Close tuner | Returns to home screen | +| 21.5 | **AUDIO:** Muted when tuner active | No output (mute for tuning) | + +--- + +## 22. Stress Tests + +| # | Test | Expected Result | Pass/Fail | +|---|------|-----------------|-----------| +| 22.1 | Rapidly tap footswitch 10x | Toggles each time, no crash | +| 22.2 | Rapidly change buffer size 5x | Each change works, NAM survives | +| 22.3 | Load 3 different NAM models in succession | Each loads cleanly | +| 22.4 | Change routing mode 5x in a row | No crash, CPU stable | +| 22.5 | Open/close all 5 overlays rapidly | No UI freezes | +| 22.6 | Leave pedal running 30 min with audio | No audio dropouts or xruns | +| 22.7 | Hard-refresh browser (Ctrl+Shift+R) | UI reloads cleanly, state intact | + +--- + +## 23. Expected Reference Values + +| Setting | Typical Value | CPU Range | +|---------|---------------|-----------| +| Buffer 128 @ 48k | 2.67ms window | 70-90% (may xrun) | +| Buffer 256 @ 48k | 5.33ms window | 60-80% (may xrun occasionally) | +| Buffer 512 @ 48k | 10.67ms window | 35-50% (stable) | +| Buffer 1024 @ 48k | 21.33ms window | 18-25% (very stable) | +| 4CM routing mode | Both channels processed | CPU ~1.5x mono | +| Model unloaded | No NAM processing | 0% NAM CPU | +| Bypass ON | DI signal | 0% NAM CPU | + +--- + +## Known Quirks + +- **After model unload:** NAM CPU drops to 0% (no model to process). Reload a model to restore. +- **Engine switch (C++ ↔ PyTorch):** May take 5-8 seconds. Model reloads in new engine. +- **Profile changes:** ~6-10s JACK restart pause. NAM CPU should return >0% after restart. +- **Browser cache:** Hard refresh (Ctrl+Shift+R) if UI seems stale after an update. +- **"Peaking/poping every second":** If this returns, increase buffer to 1024. Should be fixed by GC disable + 512 default.