- Add USB audio IRQ affinity pinning to core 3 in main.py
- Add enable_xrun_tracking() to AudioSystem for kernel-level diagnostics
- Wrap Python process with chrt -f 80 in systemd service template
- Add LimitSIGPENDING=128 for signal queue depth
- Create scripts/rt-tune.sh — comprehensive RT tuning startup script
(IRQ affinity, CPU governor, C-states, ALSA limits, xrun_debug)
- Create docs/rt-performance-tuning.md — reference doc with all
tuning knobs, measurement tools, and systematic procedure
Targets: <12ms RT latency (8ms ideal), zero xruns, CPU <40% at 512/48k
- Add first-order DC blocker (R=0.999) after NAM processing to kill subsonic offset
- Add 80Hz Butterworth HPF after NAM to catch residual 60/120Hz hum
- Recompute HPF coefficients on sample rate change in set_audio_profile()
- Warm-before-kill: spawn new C++ subprocess before stopping old one (no gap)
- Add background reader thread for non-blocking stdout consumption
- Reuse last known output frame if engine is slow (keeps stream aligned)
- Pass sample_rate to NAMEngineProcess and FastNAMHost constructors
- Forward sample_rate in server.py profile change and main.py init
- Read actual architecture from .nam files instead of hardcoding 'LSTM'
- Add threading.Lock to FastNAMHost for safe engine ref swaps
- Fix#5: Preset state lost on service restart — call restore_state()
in PresetManager.__init__() so last active preset is tracked after boot
- Fix#14: Block bypass/toggle WS sync — add WebSocket connection
in SPA with block_toggled handler that calls loadState()
- Fix#15: IR file naming — add display_name property to IRFile
dataclass that strips common IR prefixes/suffixes
- Fix: block_id regeneration on every preset load — pass block_id
from stored JSON data to FXBlock constructor in _preset_from_dict
(both preset chain and snapshot chain)
_jack_is_operational(): jack_lsp is unreliable on JACK 1.9.22 (returns 0
exit even when dead, and socket path varies). Instead, try creating a
real JACK.Client('_healthcheck') which is the definitive test — exactly
what jack_client.start() does. If the client connects, JACK is truly
operational.
Standard profile: period changed from 256 to 512. The NAM engine takes
2-5ms per inference at 256/48k (5.33ms window = 93% CPU). Any jitter
causes xruns (audible pops at ~1Hz). At 512/48k (10.67ms window), the
same inference takes 40-50% CPU with 5ms headroom — far more stable.
When routing mode is '4cm' but JACK provides mono audio (1D array),
the _process_4cm method crashes with IndexError on audio_in[0,:].
Treat mono input as guitar channel with silent return channel.
This happens when JACK is configured with 1 capture port (mono mode)
but routing_mode is set to '4cm' — e.g. after a settings sweep that
toggles routing_mode without changing JACK port config.
Default GC threshold (700 allocs) triggers ~every 1.4s in the
real-time audio callback due to ~500 numpy allocs/sec. Each GC
pause (10-50ms) exceeds the 5.3ms callback window at 256/48k,
producing audible pops at ~1Hz intervals.
Reference counting handles 99% of object cleanup — GC only handles
cyclic garbage. Disabling it is safe for a process that exits cleanly
(os reclaims everything).
Two-part fix for the rogue jackd race:
1. stop_jack() now kills alsa_in (from bt-a2dp bridge). When JACK
dies, alsa_in auto-spawns a rogue jackd -T -ndefault -d alsa
that satisfies _jack_is_operational() but is the wrong server.
2. _jack_is_operational() now paired with _jack_matches_config()
which verifies the running jackd's /proc/cmdline contains the
expected ALSA device string (e.g. hw:USB,0). A rogue jackd
without this won't pass the readiness check.
Also added _jack_matches_config() helper that scans /proc/*/cmdline.
JACK can briefly become operational (jack_lsp returns ports) during
startup then crash when the ALSA device fails. The initial readiness
check catches this window and returns True prematurely. Add a 0.5s
stabilization window after first success — if JACK drops, continue
the wait loop instead of returning.
Without this, profile changes succeed in the API but leave JACK in a
zombie state: process exists with SHM infrastructure but can't serve
clients.
jack_lsp on JACK 1.9.22 always exits 0 even when JACK is dead —
it prints error to stderr but returns success. Check for non-empty
stdout (port names) instead, which only appears when JACK is truly
operational.
start_jack() was checking for /dev/shm/jack_default_0_0 which never
exists on JACK 1.9.22 (used on RPi/DietPi). The check made start_jack()
always timeout and return False. Boot sequence ignored the return value
(luck), but the profile change handler checked it via 'if not ok: rollback',
causing all buffer/rate changes to fail silently every other time.
Replace with _jack_is_operational() using jack_lsp which tests actual
JACK IPC and works correctly on all JACK versions.
The bt-a2dp-jack systemd unit runs an independent JACK instance for
Bluetooth audio. When stop_jack() kills jackd, bt-a2dp-jack restarts
it within 5 seconds with default parameters. By the time start_jack()
runs in the profile handler, _jack_is_running() already returns True
for the bt-a2dp JACK, causing start_jack() to return immediately
without configuring our period/rate. The pedal's JACK client then
fails with server_error (0x21) because the running JACK doesn't match.
Fix:
- stop_jack() temporarily stops bt-a2dp-jack before killing jackd
- Profile handler restarts bt-a2dp-jack after JACK is configured
- Rollback path also restarts bt-a2dp-jack
The profile change rollback only restored audio_sys.config.profile to
old_key, but my earlier fix had started syncing period/rate to AudioConfig
BEFORE start_jack(). This meant the rollback's start_jack() still used
the NEW period/rate (via latency_profile) because self.period was never
reverted — the same failing config was retried and failed again.
Fix: capture old_period, old_rate, and old LATENCY_PROFILES['custom']
entry before applying the new profile, then restore all of them in the
rollback path.
Root causes of 'changing bit rate / sample rate kills NAM':
1. Zombie jackd from stale second Python process — when stop_jack()
kills jackd, the other process's auto-restart respawns jackd with
OLD config between stop and start. start_jack() sees a running
jackd and returns True immediately without verifying it's actually
accepting connections.
2. Zombie Python process from incomplete service restart — two
main.py instances running, each with its own AudioSystem and
JackAudioClient. Profile change kills one JACK but the other
respawns it. jack_client.start() then fails with server_error
because the stale SHM conflicts.
3. start_jack() returned True for a dead jackd — process exists but
JackServer::Open failed with -1 (bad period/rate for device).
No SHM socket was created, but the process check passed.
Fixes:
- _ensure_singleton(): kill any other main.py process at boot
- start_jack(): verify jack_default_0_0 SHM socket exists before
returning True; kill zombie process if socket is missing
- Profile handler: clean SHM right before jack_client.start()
- JackAudioClient.start(): retry once with SHM cleanup on
server_error (0x21)
start_jack() only checked _jack_is_running() (pgrep -x jackd) which
returns True even when jackd exists but failed to initialize the ALSA
device — the process starts but hangs without creating SHM segments,
making jack_lsp/jack_wait fail. The profile handler then continues as
if JACK is running, jack_client.start() fails with server_error, the
pi-multifx ports never get created, and NAM CPU drops to 0%.
Fix: also verify /dev/shm/jack_default_0_0 (the JACK control socket)
exists before declaring JACK started. This socket is only created after
the ALSA device is successfully initialized, so a zombie jackd with a
bad period/rate combination won't pass the check.
jack.Client('pi-multifx') fails with server_error (0x21) when stale
jack_sem.0_default_pi-multifx segments remain in /dev/shm from a
previous client instance. This commonly happens after a profile change
(buffer+rate) because:
1. stop_jack() kills jackd and cleans SHM
2. start_jack() starts new JACK server
3. Old process's JackAudioClient.__del__ (garbage collection) recreates
the pi-multifx semaphore in /dev/shm
4. jack_client.start() → jack.Client('pi-multifx') finds stale SHM
and fails with server_error → no pi-multifx ports → NAM never
called → 0% nam_cpu → 'nam is dying'
Fix:
- Clean /dev/shm/jack* in profile handler right before jack_client.start()
- JackAudioClient.start() also retries once with SHM cleanup on
server_error, so other callers are also protected.
Auth was blocking all POST/PUT operations with 401.
Disabled per user request. PIN generation kept (harmless).
Frontend reverted to no-auth API helpers.
- Frontend now reads auth_pin from /api/state and includes X-Pedal-Auth on all non-GET requests
- Fixed test imports after removing module-level BLOCK_SIZE/SAMPLE_RATE constants
- Removed rogue test_issue_2_3.py from tracking
Root cause: auth middleware (1603bfe) blocked all non-GET /api/ endpoints
requiring X-Pedal-Auth header, but the PIN was never served to the frontend.
Fix:
- Added auth_pin to /api/state response (safe GET endpoint, no auth required)
- Frontend stores pin from state on initial load
- apiPost/apiPut/apiDelete/apiUpload now include X-Pedal-Auth header
_gather_channel_state() was accessing pl.sample_rate but AudioPipeline
only had private _sample_rate. Added public properties so the /api/state
endpoint doesn't crash with AttributeError.
- _ensure_auth_pin() generates random 6-digit PIN on first boot
- PIN stored in config.yaml under web.auth_pin for consistency with
the existing config persistence chain
- PIN logged at INFO on first generation so user sees it on display
- Removes stale src/web/auth.py (superseded by config.yaml approach)
- **4.1:** Consolidated duplicate PUT/POST /api/volume and POST /api/bypass/toggle
into single handlers via stacked FastAPI decorators
- **4.2:** Renamed 'channel_mode' → 'audio_routing_mode' in settings KEY_MAP
and runtime apply, decoupling it from instrument routing
- **4.4:** Changed brightness API from ambiguous 'brightness' (1-10/0-1 heuristic)
to explicit 'brightness_percent' (0-100, unambiguous)
- **4.5:** Added config snapshot/rollback on JACK device switch failure.
Restores original devices + persists restored config. Sets needs_reboot
flag if even the rollback restart fails. Surfaces needs_reboot in state.
Wrap long-running synchronous calls in run_in_executor to prevent
the web UI from freezing during:
- wifi_scan (~20s scan)
- hotspot_enable / hotspot_disable (~60s each)
- bluetooth_scan (~12s scan)
- start_jack (~15s JACK restart) in set_audio_profile and save_settings
Each blocking call now runs in a thread executor, keeping the FastAPI
event loop responsive for other requests.
Added: functools.partial import, asyncio.get_event_loop() in two handlers.
The CPU label in the UI was showing psutil.cpu_percent(interval=None)
which returns total system CPU — barely moves and doesn't change when
the NAM model is bypassed.
Fix:
- Added _calc_nam_cpu() that computes NAM engine CPU from the engine's
own per-block inference timing (avg_inference_ms) vs the JACK callback
interval. When the NAM block is bypassed in the chain, returns 0.
- Added 'nam_cpu' field to /api/state response
- Status bar shows NAM CPU % next to VU meters (drops to 0 on bypass)
- Settings CPU row shows NAM CPU with system CPU in dim text beside it
- Changed psutil.cpu_percent(interval=None) → interval=0.2 for a real
system CPU reading
Shows input (green) and output (amber) level bars in the status bar,
updated every 2s from /api/state. Scales factor 300x to make small
signals visible, capped at 100%.
When JACK is killed forcefully (killall -9), the /dev/shm/jack_sem.*
shared memory segments and /dev/shm/jack_db-0/ directory are left
behind. The next start_jack() call finds these stale segments and
fails silently — no audio output, JACK process may appear briefly
then crash.
Fix:
- AudioSystem.stop_jack(): after killing jackd, also rm -rf
/dev/shm/jack* to clean up stale SHM.
- main.py boot sequence: clean SHM before first start_jack(), so
the pedal recovers from crashes that happened before this fix.
jack.Client.deactivate() is a synchronous protocol call that hangs
indefinitely on a broken server socket. In the audio profile change
handler, stop_jack() kills the jackd process first, then tries
jack_client.stop() which calls deactivate() on the now-dead socket.
Fix: check if jackd is still running via pidof before calling
deactivate(). If the server is already gone, skip directly to
close() which just closes the local file descriptor (safe and fast).
This was causing the POST /api/audio/profile endpoint to hang for
30+ seconds on every buffer change.
Two-part fix for NAM audio dying after buffer change:
1. NAM block size sync moved BEFORE jack_client.start() so the NAM
subprocess is reloaded before the process callback fires.
2. AudioConfig.period/rate must be updated BEFORE start_jack() reads
latency_profile. Otherwise the stale boot-time override period
stomps on the LATENCY_PROFILES entry set by the POST handler,
causing JACK to start at the wrong period while NAM is at the
new period — permanent JACK/NAM block size mismatch.
Both bugs together: user changes buffer, JACK restarts with wrong
period due to stale override, NAM subprocess gets mismatched pipe
bytes, returns DI (direct input) for every subsequent callback.
The profile change handler was calling nam_host.set_block_size() AFTER
jack_client.start(), creating a race where the JACK callback fired with
the new buffer size while the old NAM subprocess was still running at
the old block_size. Pipe byte counts desynced => NAM returned passthrough.
+ also: auto_connect disable during restart, connect_fx_ports reorder,
Tone3000 gear/sort/pagination params, NAM engine switch API endpoints
The setAudioParams call restarts JACK which takes up to 10 seconds.
The default API timeout was 6s, causing the request to abort and
leaving JACK in a broken state.
- Changed setAudioParams to use API._fetch directly with timeout:15000
- Also calls loadAudioParams() after success to refresh dropdown values
- NAM/IR tab toggle in Downloads overlay
- Architecture filter (All/Nano/Feather/Standard) - filters via
Supabase architecture_version column
- Model detail overlay: tap a search result to see full description,
gear info, author, pack, architecture, thumbnail before downloading
- IR search + install with .meta.json metadata saving
- Backend: search_models/get_trending_models accept arch param
- Backend: tone_gear field added to model search response
- UI now sends full Tone3000 metadata on install (id, author,
architecture, pack_name, description, thumbnail, etc.)
- Search results show size, architecture, and pack name
- Install endpoint saves a .meta.json file next to each .nam with:
author, architecture, description, pack, thumbnail, download date
- Enables offline browsing of model provenance
- Audio devices endpoint now includes Bluetooth Audio (bluealsa) and
other software PCM devices from aplay -L
- Added GET/POST /api/backing-source endpoints with JACK port routing
- Backing Track Source dropdown in Audio section (None, Bluetooth
Audio, or any JACK capture port found)
- Selecting a source routes it into fx_in:input_2 via jack_connect
- Also supports bt_audio:capture_1/2 from existing JACK bridge
- Added Output Device dropdown alongside Input Device
- loadAudioDevices() now populates both dropdowns from the API and
selects current values
- Separated setAudioInputDevice / setAudioOutputDevice functions
(each calls API with only its respective field)
- Added input_device / output_device to GET /api/audio/profile response
- Removed the old combined setAudioDevice function
shawn