pi-multifx-pedal/docs/rt-performance-tuning.md

RT Performance Tuning — Pi Multi-FX Pedal

Reference doc for real-time audio performance on Raspberry Pi 4B. Targets: <12ms round-trip latency (ideally <8ms), zero xruns, CPU <40%.

Overview

The Pi Multi-FX Pedal runs a JACK audio server with an ALSA backend on a Raspberry Pi 4B. The signal path is:

Guitar → Focusrite 2i2 → ALSA → JACK → Python pipeline (NAM + FX) → JACK → ALSA → Output

Each stage adds latency. The total round-trip latency is dominated by:

1. ALSA period size (-p): The buffer size in frames JACK exchanges with the audio hardware. This is the #1 tuning knob.
2. Number of periods (-n): ALSA ring buffer depth. More periods = more tolerance for scheduling jitter but higher latency.
3. Sample rate (-r): Higher rate = lower per-frame latency but more CPU. 48kHz is the sweet spot for USB audio interfaces.
4. NAM inference time: The C++ subprocess takes 2-5ms per block on Pi 4B. This is the bottleneck that sets the minimum viable buffer size.

Recommended Settings (Pi 4B)

Setting	Standard	Low Latency	Ultra Low	Unit
Period (buffer)	512	256	128	frames
Sample rate	48000	48000	48000	Hz
Periods (nperiods)	2	2	3
RT priority	70	75	80
Expected latency	~10.7ms	~5.3ms	~2.7ms
Expected NAM CPU	35-50%	60-93%	80-100%+
Xrun stability	✅ Stable	⚠️ Possible	❌ Likely

Default recommendation: 512/48k (standard)

The standard profile (512 frames, 48kHz, 2 periods) is the recommended default for the Pi 4B. This provides:

• 10.67ms callback window (more than enough for 2-5ms NAM inference)
• 35-50% CPU load with LSTM NAM models
• Zero xruns in normal playing
• Enough headroom for the FX chain (filters, modulation, reverb)

Even at 512 frames, the round-trip latency (capture → process → playback) is typically 6-10ms with a USB audio interface — well under the <12ms target. The round-trip includes two ALSA transfers (capture + playback), which is why it's lower than the raw period calculation.

Tuning Knobs

1. JACK buffer size (--period / -p)

The JACK period is the number of frames per audio block. Lower = lower latency but more CPU and more xrun risk.

# Current: 512 frames at 48kHz = 10.67ms
jackd -p 512 -r 48000 ...

# Aggressive: 128 frames at 48kHz = 2.67ms
jackd -p 128 -r 48000 ...

# Conservative: 1024 frames at 48kHz = 21.33ms (safe, higher latency)
jackd -p 1024 -r 48000 ...

Measurement: When you change the period in the UI, the server:

1. Updates AudioConfig.period
2. Updates LATENCY_PROFILES["custom"]
3. Stops JACK (with bt-a2dp dance)
4. Updates NAM block size (set_block_size())
5. Updates pipeline DSP (set_audio_profile())
6. Restarts JACK with new period
7. Reconnects FX ports
8. Restarts bt-a2dp

Timeout caveat: The UI's POST must use timeout: 15000 (15s) because JACK restart takes 6-10s on Pi 4B.

2. Number of periods (--nperiods / -n)

The ALSA period count controls the ring buffer depth:

• nperiods=2 (default): Lower latency, less tolerance for scheduling jitter. Good for stable USB audio interfaces.
• nperiods=3: More tolerance for scheduling jitter at the cost of ~50% more ALSA buffer latency. Recommended when pushing below 256 frames where every microsecond counts.

The nperiods trade-off: at 128/48k, nperiods=3 adds 128×3/48000 = 8ms of ALSA buffer vs 128×2/48000 = 5.3ms for nperiods=2. The extra 2.7ms can prevent xruns when the CPU governor ramps or a system timer fires.

3. RT priority (-P / rt_priority)

JACK uses -P to set SCHED_FIFO priority. On RPi 4B:

Priority	Effect
60	Default — works but shares CPU with other RT tasks
70	Standard profile — good balance
80	Low latency profile — less scheduling jitter
90-95	Aggressive — use if xruns persist at 256/48k

The Python pedal process should also run with RT scheduling:

chrt -f 80 python3 main.py

The systemd service (pi-multifx-pedal.service) now wraps this automatically via ExecStart=/usr/bin/chrt -f 80 python3 main.py.

Requirements:

• LimitRTPRIO=95 in the systemd unit (already present)
• @audio - rtprio 95 in /etc/security/limits.d/99-audio.conf
• Process must run as root or with CAP_SYS_NICE

4. CPU governor → performance

The RPi 4B's CPU governor defaults to ondemand or powersave, which keeps the CPU at 600MHz idle and ramps up under load. The ramp-up takes 1-2ms — significant at 256/48k (5.33ms callback window).

Applied in main.py at boot:

for c in range(os.cpu_count() or 1):
    with open(f"/sys/devices/system/cpu/cpu{c}/cpufreq/scaling_governor", "w") as f:
        f.write("performance")

Verify:

cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
# All should show "performance"

Trade-off: ~0.5W extra power draw (CPU stays at 1.5GHz).

5. IRQ affinity — pin USB audio to a dedicated core

On RPi 4B, interrupts are distributed across all 4 cores by default. Pinning the USB audio IRQ to core 3 isolates it from kernel housekeeping on cores 0-2.

Applied in main.py at boot:

# Find xhci-hcd or dwc_otg IRQ → pin to core 3 (mask 0x8)
echo "8" > /proc/irq/<IRQ>/smp_affinity
echo "3" > /proc/irq/<IRQ>/smp_affinity_list
# All other IRQs moved to cores 0-2 (mask 0x7)

Verify:

./scripts/rt-tune.sh --status
# or manually:
cat /proc/irq/*/smp_affinity | sort | uniq -c

6. mlockall() — lock process memory

Prevents page faults in the RT callback by locking all process pages in RAM.

Applied in main.py at boot:

import ctypes
libc = ctypes.CDLL('libc.so.6')
libc.mlockall(3)  # MCL_CURRENT | MCL_FUTURE

Requires: LimitMEMLOCK=infinity in the systemd unit (already present).

Verify:

grep -i lock /proc/$(pidof python3)/status | head -5
# VmLck should be non-zero

7. GC disable — prevent Python GC pauses

Python's default GC (threshold=700) triggers every ~1.4s in the audio pipeline due to ~500 numpy allocations/second. Each 10-50ms GC pause causes audible pops.

Applied in main.py:

import gc
gc.disable()
gc.collect()  # one final sweep

Periodic GC on the HTTP thread (never in RT callback):

import gc
gc.collect()  # in get_state() handler, ~2s poll

Trade-off: Reference counting handles 99% of cleanup. The OS reclaims all memory on process exit. Safe for a long-running daemon.

Measurement Tools

Round-trip latency (jack_iodelay)

Requires a physical loopback cable (output → input).

# Quick measurement
jack_iodelay

# Automated (8 samples)
python3 -c "
from src.system.audio import AudioSystem
AudioSystem.measure_roundtrip_latency(samples=8)
"

Skip if: no loopback cable. The UI round-trip latency is approximately 2 × period / rate × 1000ms (capture + playback):

Period	Rate	Calc. RT latency	Real RT latency
512	48k	21.33ms	~6-10ms (USB interface)
256	48k	10.67ms	~4-6ms
128	48k	5.33ms	~2-4ms

The real RT latency is lower than the formula because the USB interface and ALSA driver pipeline the transfers.

XRun monitoring

# Enable kernel tracking
echo 3 | sudo tee /proc/asound/card*/xrun_debug

# Quick check
jack_showtime -c | grep xruns

# Automated monitor (5 min)
python3 -c "
from src.system.audio import AudioSystem
asys = AudioSystem()
result = asys.monitor_xruns(duration=300, interval=10)
print(result)
"

XRun debug bits:

• Bit 0 (1): Log xruns to kernel ring buffer (dmesg | grep xrun)
• Bit 1 (2): Show stack backtrace
• Bit 2 (4): Inhibit xruns (test mode — disables recovery)

The pedal enables bit 0+1 (value 3) at boot for diagnostics.

NAM CPU load

The state API (GET /api/state) now includes nam_cpu — the percentage of the callback window spent in NAM inference:

curl -s http://pedal.local/api/state | python3 -c "
import json,sys
s = json.load(sys.stdin)
print(f'NAM CPU: {s[\"nam_cpu\"]:.1f}%')
print(f'System CPU: {s[\"cpu_percent\"]:.1f}%')
print(f'Input level: {s[\"input_level\"]:.3f}')
"

Expected values (Pi 4B, LSTM NAM model):

Buffer	NAM CPU	Notes
64	180-200%	xruns guaranteed
128	80-100%	xruns likely
256	60-93%	xruns possible
512	35-50%	stable
1024	15-25%	safe, higher latency

Systematic Tuning Procedure

Step 1: Establish baseline

With the current settings, run:

# 1. Check current profile
curl -s http://pedal.local/api/audio/profile | python3 -m json.tool

# 2. Monitor xruns for 5 minutes
timeout 300 bash -c '
  while true; do
    xruns=$(jack_showtime -c 2>/dev/null | grep xruns)
    echo "$(date +%H:%M:%S) $xruns"
    sleep 10
  done
'

# 3. Measure NAM CPU
curl -s http://pedal.local/api/state | python3 -c "
import json,sys
s = json.load(sys.stdin)
print(f'nam_cpu={s[\"nam_cpu\"]}%  sys_cpu={s[\"cpu_percent\"]}%  '
      f'input={s[\"input_level\"]:.3f}  output={s[\"output_level\"]:.3f}')
"

Step 2: Sweep buffer sizes

For each size (512 → 256 → 128 → 64), test for 5 minutes:

for period in 512 256 128 64; do
  echo "=== Testing period=$period ==="
  curl -s -X POST -d "{\"period\":$period}" http://pedal.local/api/audio/profile
  sleep 15  # wait for JACK restart + stabilization

  # Check NAM CPU
  curl -s http://pedal.local/api/state | python3 -c "
import json,sys; s=json.load(sys.stdin); print(f'  nam_cpu={s[\"nam_cpu\"]}%')"

  # Monitor 5 min
  python3 -c "
from src.system.audio import AudioSystem
r = AudioSystem().monitor_xruns(300, 10)
print(f'  xruns={r[\"xrun_total\"]} rate={r[\"xrun_rate_per_min\"]}/min stable={r[\"stable\"]}')
"
done

Step 3: Evaluate nperiods sweep

For the best buffer candidates, test nperiods=2 vs nperiods=3:

for period in 128 256; do
  for nperiods in 2 3; do
    echo "=== p=$period n=$nperiods ==="
    # Manually restart JACK with -n $nperiods
    ssh pedal "sudo killall jackd; sleep 1; \
      jackd -P 70 -d alsa -d hw:0,0 -r 48000 -p $period -n $nperiods -i 2 -o 2 &"
    sleep 5
    # Test...
  done
done

Step 4: Select optimal

Choose the lowest period that achieves zero xruns over a 30-minute test with active playing. Save to config:

curl -s -X POST -d '{"period":512,"rate":48000}' \
  http://pedal.local/api/audio/profile

Config File Reference

Key fields in ~/.pedal/config.yaml:

audio:
  profile: custom           # or "standard", "low", "stable"
  period: 512               # frames (64-2048, powers of 2)
  rate: 48000              # Hz (44100, 48000, 96000, 192000)
  input_device: hw:0,0     # ALSA device for capture
  output_device: hw:0,0    # ALSA device for playback
  mode: mono               # or "stereo_4cm"
  jack_enabled: true
  auto_connect: true        # re-connect JACK ports on restart
  hat_type: audioinjector   # or "focusrite_2i2_3gen"

notes:
  - RT tuning March 2025: standard=512/48k stable on Pi 4B with LSTM NAM
  - Changing period/rate in UI saves to these fields automatically
  - On restart, period/rate overrides the profile defaults

Common Issues

Pops/crackle at 256 frames

Cause: NAM inference takes 2-5ms on Pi 4B. At 256/48k (5.33ms window), there's only 0.33-3.33ms headroom. Any scheduling jitter causes xrun.

Fixes (in order of effectiveness):

1. Increase to 512 frames (10.67ms window)
2. Set CPU governor to performance (already done at boot)
3. Pin USB audio IRQ to dedicated core (already done at boot)
4. Use nperiods=3 for more ALSA buffer tolerance
5. Disable Wi-Fi/BT if not needed (both share the USB bus on Pi 4B)
6. Use a lighter NAM model (Feather/Nano instead of LSTM)

Audio drops out after profile change

Check the startup order:

1. NAM block size must be set BEFORE jack_client.start()
2. Pipeline DSP must be updated BEFORE jack_client.start()
3. SHM cleanup must not delete running JACK server files
4. bt-a2dp must be stopped before killing jackd

Verify:

# Check what JACK actually started with
ps aux | grep jackd | grep -v grep
# Expected: jackd -P 70 -d alsa -d hw:0,0 -r 48000 -p 512 -n 2 -i 2 -o 2

Settings revert after restart

Check:

1. Does config.yaml contain period: and rate:?
2. Does AudioConfig load them from config.yaml?
3. Is the latency_profile property applying the overrides?

grep -E "period:|rate:|profile:" ~/.pedal/config.yaml

References

• JACK documentation: https://jackaudio.org/faq/
• RPi 4B audio latency: https://wiki.linuxaudio.org/wiki/raspberrypi
• ALSA xrun_debug: https://www.alsa-project.org/wiki/XRUN_Debug
• Pi 4B CPU freq scaling: /sys/devices/system/cpu/cpu*/cpufreq/