raspberry-pi-mixer/docs/research/base-os-decision.md

# Base OS Selection — Buildroot/Yocto/RPiOS Lite + RT Kernel

**Date:** 2026-05-19
**Status:** Approved
**Parent:** P1-R1 (System Research Report)
**Next:** P2 (Kernel Build)

---

## Executive Summary

**Decision: Raspberry Pi OS Lite (64-bit, Bookworm) + custom PREEMPT_RT kernel.**

RPiOS Lite is selected over Buildroot and Yocto for this project. It provides the
Debian/bookworm ecosystem for rapid audio-stack iteration, active Foundation
maintenance, and straightforward custom kernel integration. Buildroot and Yocto
are evaluated below and rejected for reasons of development velocity (Buildroot
requires full-image rebuild per package change) and disproportionate complexity
(Yocto's ~50GB build footprint and steep learning curve are unjustified for a
single-purpose audio appliance with one developer).

---

## 1. Build System Comparison

### 1.1 Buildroot

| Aspect | Assessment |
|--------|-----------|
| **Version** | 2024.02+ (rolling releases every 3 months) |
| **RPi4 64-bit support** | First-class: `raspberrypi4_64_defconfig` |
| **Build time (clean)** | ~30-45 min on modern x86 (8-core) |
| **Build output** | ~120MB EXT4 rootfs image + FAT32 boot partition |
| **Init system** | BusyBox init (default) or systemd (optional) |
| **Cross-compilation** | Bootlin external toolchain (aarch64 glibc stable) |
| **Package format** | None at runtime — every change requires image rebuild |
| **Kernel config** | `BR2_LINUX_KERNEL_CUSTOM_TARBALL` → RPi kernel tarball |
| **PREEMPT_RT support** | Via kernel config fragment overlay (`BR2_LINUX_KERNEL_CONFIG_FRAGMENT_FILES`) |
| **Read-only rootfs** | SquashFS built-in (`BR2_TARGET_ROOTFS_SQUASHFS`), initramfs/OverlayFS support |

**Verdict for this project: Rejected.**

Buildroot excels at producing minimal, reproducible appliance images. However,
the "rebuild image to change a package" model is a poor fit for audio development
where you need to iterate on JACK configs, test different DSP toolchains, and
install debugging tools rapidly. The 30-min rebuild cycle per package tweak
would severely hamper velocity. Buildroot is best for the "final appliance image"
stage — it could be revisited for production deployment after development
stabilizes on RPiOS Lite.

### 1.2 Yocto Project / OpenEmbedded

| Aspect | Assessment |
|--------|-----------|
| **Version** | Scarthgap 5.0 (LTS, April 2024) or Styhead 5.1 |
| **RPi4 64-bit support** | First-class via `meta-raspberrypi` layer (maintained by Andrei Gherzan) |
| **Build time (clean)** | ~2-4 hours first build; ~15-30 min incremental |
| **Build footprint** | ~50 GB disk for build directory |
| **Init system** | systemd (default in poky) |
| **Cross-compilation** | Built-in toolchain generation (cross + native + nativesdk) |
| **Package format** | opkg, rpm, or deb at runtime |
| **Kernel config** | `linux-raspberrypi` recipe in meta-raspberrypi; config fragments via `.bbappend` |
| **PREEMPT_RT support** | Via kernel config fragment in recipe append |
| **Read-only rootfs** | `IMAGE_FEATURES += "read-only-rootfs"` built-in |

**Verdict for this project: Rejected.**

Yocto is the industry standard for complex embedded Linux products where
reproducibility, long-term maintenance, and multi-target builds matter. However,
for a single-developer audio project targeting one board (RPi4B), the overhead is
disproportionate: 50GB build disk, 2-4 hour clean build, steep BitBake/recipe
learning curve, and meta-layer dependency management. The project has no need for
Yocto's multi-machine, multi-distro, SDK-generation capabilities. Revisit Yocto
only if this project scales to a product line with multiple hardware targets.

### 1.3 Raspberry Pi OS Lite (64-bit, Bookworm)

| Aspect | Assessment |
|--------|-----------|
| **Version** | Bookworm (Debian 12), kernel 6.6 or 6.12 (rpi-update) |
| **Architecture** | aarch64 (64-bit ARMv8-A) |
| **Install size** | ~3 GB (Lite, no desktop) |
| **RAM usage (idle)** | ~150 MB |
| **Init system** | systemd |
| **Package format** | .deb via apt — full Debian repositories |
| **Kernel source** | `github.com/raspberrypi/linux` — `rpi-6.12.y` branch |
| **PREEMPT_RT support** | Not in stock kernel; build custom kernel with `CONFIG_PREEMPT_RT=y` |
| **Read-only rootfs** | Manual setup via OverlayFS or fstab `ro` + tmpfs mounts |
| **Cross-compilation** | `aarch64-linux-gnu-` toolchain (available via apt on Ubuntu/Debian host) |
| **Maintenance** | Active — Raspberry Pi Foundation + Debian upstream |

**Verdict for this project: Selected.**

RPiOS Lite offers the right balance:
- **Development speed**: `apt install` for JACK, ALSA tools, debugging — minutes, not rebuilds
- **Ecosystem**: Full Debian Bookworm repository with up-to-date audio packages
- **Familiarity**: Standard Linux systemd-based system — no proprietary init or build system
- **Custom kernel**: Build once, drop kernel8.img onto /boot — simple and well-documented
- **Production path**: Can later repackage the working RPiOS setup into a Buildroot image if appliance-style deployment is desired

The 3GB install size and 150MB idle RAM are acceptable for an 8GB RPi4B. If
size reduction becomes important later, the system can be trimmed (remove
unnecessary packages, switch to read-only squashfs).

---

## 2. PREEMPT_RT Kernel Integration

### 2.1 Source Tree

The Raspberry Pi Foundation maintains a downstream kernel at:

```
https://github.com/raspberrypi/linux
Branch: rpi-6.12.y (64-bit, kernel8 build)
Defconfig: bcm2711_defconfig
```

As of May 2026, the 6.12.y RT patchset is mostly upstream. Only
`CONFIG_PREEMPT_RT=y` needs to be enabled on top of `bcm2711_defconfig`.

### 2.2 Build Method: Cross-Compilation on x86 Host

**Recommended approach** — build the kernel on a fast x86 machine, then copy
the resulting kernel image and DTBs to the Pi's /boot partition.

```bash
# Toolchain (Ubuntu 26.04)
sudo apt install gcc-aarch64-linux-gnu binutils-aarch64-linux-gnu

# Clone RPi kernel
git clone --depth=1 -b rpi-6.12.y \
  https://github.com/raspberrypi/linux.git linux-rpi
cd linux-rpi

# Apply config
KERNEL=kernel8
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bcm2711_defconfig

# Enable PREEMPT_RT
scripts/config -e CONFIG_PREEMPT_RT
scripts/config -d CONFIG_PREEMPT
scripts/config -e CONFIG_HIGH_RES_TIMERS
scripts/config -e CONFIG_HZ_1000
scripts/config -e CONFIG_NO_HZ_FULL
scripts/config -e CONFIG_RCU_NOCB_CPU
scripts/config -e CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE
scripts/config --set-val CONFIG_HZ 1000
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- olddefconfig

# Build (adjust -j for your CPU count)
make -j$(nproc) ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
  Image modules dtbs
```

### 2.3 Kernel Installation on RPi

```bash
# Copy kernel
scp arch/arm64/boot/Image root@<pi-ip>:/boot/kernel8-rt.img

# Copy DTBs
scp arch/arm64/boot/dts/broadcom/*.dtb root@<pi-ip>:/boot/

# Install modules
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
  INSTALL_MOD_PATH=./modules modules_install
scp -r modules/lib/modules/* root@<pi-ip>:/lib/modules/

# On the Pi, update config.txt:
# kernel=kernel8-rt.img
# Add cmdline.txt: threadirqs nohz_full=1-3 rcu_nocbs=1-3 isolcpus=1-3
```

### 2.4 Kernel Configuration Checklist

| Config | Value | Rationale |
|--------|-------|-----------|
| `CONFIG_PREEMPT_RT` | y | Full real-time preemption |
| `CONFIG_PREEMPT` | n | Disable voluntary-only preemption |
| `CONFIG_HZ` | 1000 | 1ms timer tick for lower scheduling jitter |
| `CONFIG_HZ_1000` | y | Required for HZ=1000 |
| `CONFIG_HIGH_RES_TIMERS` | y | High-resolution timer subsystem |
| `CONFIG_NO_HZ_FULL` | y | Tickless operation on isolated CPUs |
| `CONFIG_RCU_NOCB_CPU` | y | Offload RCU callbacks from RT CPUs |
| `CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE` | y | Lock CPU at max frequency |
| `CONFIG_CPU_FREQ_GOV_PERFORMANCE` | y | Performance governor available |
| `CONFIG_IRQ_FORCED_THREADING` | y | Force threaded IRQ handlers |
| `CONFIG_PREEMPTIRQ_DELAY_TEST` | n | Debug — disable for production |
| `CONFIG_USB_XHCI_HCD` | y | USB 3.0 host controller (required) |
| `CONFIG_SND_USB_AUDIO` | y | USB Audio Class driver (build as module preferred) |

### 2.5 Alternative: Native Build on RPi4B

Building natively on the RPi4B itself is possible but slow (~2-3 hours for a
full kernel build). Cross-compilation on a modern x86 machine completes in
~10-15 minutes. **Recommendation:** Cross-compile for development iterations,
fall back to native build on the Pi only if cross-compilation toolchain issues
arise.

### 2.6 Kernel Verification

```bash
# Verify RT preemption is active
uname -v | grep PREEMPT_RT
cat /sys/kernel/realtime          # should return "1"

# Measure worst-case latency
sudo cyclictest -t 4 -p 99 -i 200 -n -l 100000 -q
# Target: max latency < 100µs on idle system
```

---

## 3. Boot Time Optimization

### 3.1 Current Baseline

A stock RPiOS Lite 64-bit boots in approximately:
- **Firmware/Bootloader:** ~3-5 seconds
- **Kernel + initrd:** ~5-8 seconds
- **Userspace (systemd):** ~10-15 seconds
- **Total:** ~18-28 seconds to login prompt

Measured with `systemd-analyze`:
```bash
systemd-analyze          # overall boot time
systemd-analyze blame    # per-unit timing
systemd-analyze critical-chain  # dependency chain
systemd-analyze plot > boot.svg  # visual timeline
```

### 3.2 Optimization Targets

| Target | Approach | Expected Saving |
|--------|----------|-----------------|
| **Disable WiFi/BT** | `dtoverlay=disable-wifi`, `dtoverlay=disable-bt` in config.txt | ~2-3s |
| **Reduce kernel modules** | Disable unused drivers (HDMI audio, camera, codecs) | ~1-2s |
| **Minimal initramfs** | Build custom initramfs with only essential modules (usb, ext4, xhci) | ~2-4s |
| **Disable unnecessary services** | `systemctl disable` bluetooth, avahi, triggerhappy, etc. | ~3-5s |
| **Quiet boot** | `quiet loglevel=3` on kernel cmdline | ~1-2s (fewer console writes) |
| **Disable swap wait** | Remove swapfile or reduce swap timeout | ~0-5s |
| **Fast boot in config.txt** | `boot_delay=0`, `disable_splash=1` | ~0.5-1s |

**Target: Sub-15-second boot** (from power-on to JACK running).

### 3.3 Services to Disable

```bash
# Audio appliance — disable non-essential services
sudo systemctl disable bluetooth
sudo systemctl disable hciuart
sudo systemctl disable avahi-daemon
sudo systemctl disable triggerhappy
sudo systemctl disable ModemManager
sudo systemctl disable polkit
sudo systemctl mask systemd-random-seed.service  # if using read-only root
```

### 3.4 Kernel Command Line

```
console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 rootwait
quiet loglevel=3
threadirqs nohz_full=1-3 rcu_nocbs=1-3 isolcpus=1-3
usbcore.autosuspend=-1
fsck.mode=skip
```

| Parameter | Purpose |
|-----------|---------|
| `quiet loglevel=3` | Suppress verbose kernel messages |
| `threadirqs` | Force threaded IRQ handlers (RT audio) |
| `nohz_full=1-3` | Disable timer ticks on CPU cores 1-3 |
| `rcu_nocbs=1-3` | Offload RCU callbacks from audio cores |
| `isolcpus=1-3` | Isolate cores 1-3 from scheduler (reserve for JACK/DSP) |
| `usbcore.autosuspend=-1` | Disable USB autosuspend (prevents audio dropouts) |
| `fsck.mode=skip` | Skip filesystem check on boot (for speed; use only on reliable storage) |

### 3.5 CPU Core Allocation

With 4 Cortex-A72 cores:
- **Core 0:** System + IRQ handling (including xHCI USB interrupts)
- **Cores 1-3:** JACK real-time threads + DSP processing

This is enforced via:
- `isolcpus=1-3` on kernel cmdline
- `taskset -c 1-3` for JACK server
- IRQ affinity: pin xHCI to CPU 0 only

---

## 4. Read-Only Root Filesystem Options

### 4.1 Why Read-Only Rootfs

For an embedded audio appliance:
- **Reliability:** Prevents SD card corruption from unexpected power loss
- **Consistency:** System state is known and reproducible
- **Longevity:** Reduces SD card wear (no unexpected writes)
- **Safety:** User cannot accidentally break the system

### 4.2 Approach Comparison

| Approach | Complexity | SD Card Wear | Runtime Writes | Boot Impact |
|----------|------------|-------------|----------------|-------------|
| **OverlayFS (tmpfs upper)** | Low | Minimal | Lost on reboot | +1-2s |
| **OverlayFS (persistent upper on separate partition)** | Medium | Low | Persisted | +1-2s |
| **squashfs root + tmpfs overlay** | Medium | None | Lost on reboot | +0s (faster read) |
| **ext4 `ro` + tmpfs for /var and /tmp** | Low | Minimal | Lost on reboot | +0s |
| **Full read-write ext4** | None | Full | Normal | Baseline |

### 4.3 Recommended: OverlayFS with Persistent /data

```
┌─────────────────────────────────────────┐
│  SD Card / NVMe / USB SSD               │
├──────────┬──────────┬───────────────────┤
│ /boot    │ /        │ /data             │
│ FAT32    │ ext4 ro  │ ext4 rw           │
│ 256MB    │ 4GB      │ remaining         │
│ (kernel, │ (base OS)│ (configs, presets,│
│  firmware│          │  logs, user data) │
└──────────┴──────────┴───────────────────┘
```

**Boot process:**
1. Kernel mounts `/dev/mmcblk0p2` as read-only ext4
2. initramfs sets up OverlayFS:
   - Lower: `/dev/mmcblk0p2` (read-only root)
   - Upper: tmpfs (writable layer, lost on reboot)
   - Workdir: tmpfs
3. `/data` partition mounted read-write for persistent state
4. `/var` and `/tmp` are tmpfs

**To remount root read-write temporarily (for system updates):**
```bash
sudo mount -o remount,rw /
# ... make changes ...
sudo mount -o remount,ro /
sudo sync
```

### 4.4 OverlayFS initramfs Setup

In `/etc/initramfs-tools/scripts/init-bottom/overlay`:

```bash
#!/bin/sh
# Setup OverlayFS root
mount -t tmpfs tmpfs /overlay
mkdir -p /overlay/upper /overlay/work
mount -t overlay overlay \
  -o lowerdir=${ROOT},upperdir=/overlay/upper,workdir=/overlay/work \
  ${rootmnt}
```

### 4.5 Alternative: Buildroot SquashFS Image

If an appliance-style read-only image is desired for production deployment,
Buildroot can produce a squashfs rootfs (~120MB compressed) that boots with
initramfs. The squashfs approach eliminates SD card writes entirely for the
system partition and reduces image size. This can be explored as a Phase 2
optimization after the audio stack is validated on RPiOS Lite.

---

## 5. Partition Layout

### 5.1 Recommended Layout (RPiOS Lite)

| Partition | Type | Size | Filesystem | Mount | Contents |
|-----------|------|------|------------|-------|----------|
| 1 | Primary | 256 MB | FAT32 | /boot | GPU firmware, kernel8.img, config.txt, cmdline.txt, DTBs |
| 2 | Primary | 8 GB | ext4 (ro) | / | Root filesystem (RPiOS Lite + audio stack) |
| 3 | Primary | 20+ GB | ext4 (rw) | /data | JACK configs, DSP presets, session data, logs |

### 5.2 Minimum vs Recommended Sizes

| Component | Minimum | Recommended | Notes |
|-----------|---------|-------------|-------|
| /boot | 128 MB | 256 MB | Multiple kernel images + firmware backups |
| / (root) | 4 GB | 8 GB | RPiOS Lite base ~3GB + audio packages (~500MB-1GB) + headroom |
| /data | 2 GB | 20+ GB | Session recordings, presets, logs (use remaining SD card space) |

### 5.3 Storage Medium

| Medium | Pros | Cons | Verdict |
|--------|------|------|---------|
| **SD Card** (32GB A2) | Cheap, no USB bus contention | Wear, slower I/O, shared bus | Acceptable for development |
| **USB 3.0 SSD** | Fast, durable, large capacity | **Shares USB bus with audio interfaces!** | ⚠️ Risk of xrun interference |
| **NVMe SSD** (via HAT) | Fast, no USB bus contention | Requires HAT, adds cost | **Best for production** |

**Recommendation:** SD card for development; NVMe HAT for production if
latency-sensitive I/O is needed during operation. Avoid USB SSD as primary
storage — it competes with audio interface isochronous bandwidth on the VL805
controller.

### 5.5 USB Boot — Ditch the SD Card

The RPi4B supports native USB boot (USB mass storage device boot mode in EEPROM).
This is **recommended** for this project — USB SSDs and NVMe adapters are far more
reliable than SD cards under sustained audio write loads.

#### NVMe HAT (Best Option)

Uses dedicated PCIe lanes — does **not** share the USB bus with audio interfaces.

| Option | Pros | Cons | Cost |
|--------|------|------|------|
| **NVMe Base HAT** (official) | Uses PCIe, no bus contention, fast | Adds height, needs spacer | ~€15 |
| **Pimoroni NVMe Base** | Same, well-documented | Same | ~€15 |
| **Geekworm X1001 / X1002** | Low profile, M.2 2230/2242 | Some need EEPROM update | ~€15-25 |

#### USB 3.0 SSD Boot

Booting from USB SSD is possible but **contraindicated** during audio operation —
it competes with audio interfaces on the shared VL805 USB controller.

| Aspect | Detail |
|--------|--------|
| **Boot EEPROM setting** | `sudo rpi-eeprom-config --edit` → `BOOT_ORDER=0xf41` |
| **Performance** | ~350 MB/s sequential vs ~90 MB/s on SD |
| **Runtime conflict** | I/O contention with USB audio on same bus — risk of xruns |
| **Mitigation** | Boot from USB SSD, then minimize writes during audio (log to tmpfs, use OverlayFS) |

#### Recommended Config

- **Development:** SD card (cheap, easy to flash)
- **Production / Live use:** NVMe HAT + SSD via PCIe (best reliability, no USB conflict)
- **Fallback:** USB SSD boot if NVMe HAT unavailable — with OverlayFS + tmpfs for runtime paths

#### EEPROM USB Boot Enable

```bash
# Check current boot order
sudo rpi-eeprom-config | grep BOOT_ORDER

# Set boot order: USB → SD → Restart
echo 'BOOT_ORDER=0xf41' | sudo tee -a /boot/firmware/config.txt

# Or via rpi-eeprom-config
sudo rpi-eeprom-config --edit
# Change BOOT_ORDER to 0xf241 for SD → USB → Restart
# Change BOOT_ORDER to 0xf41 for USB → SD → Restart

# Reboot to apply
sudo reboot
```

**Boot order codes:**
| Mode | Value |
|------|-------|
| SD card | 0x1 |
| USB mass storage | 0x4 |
| NVMe (if available) | 0x6 |
| Restart | 0xf |
| SD → USB → Restart | 0xf241 |
| USB → SD → Restart | 0xf41 |
| NVMe → USB → SD → Restart | 0xf614 |

Also ensure P1-R3 (build script task) includes an NVMe HAT HOWTO section.

### 5.4 Image Size Constraints

For SD card deployment:
- **Minimum card:** 16 GB (Class A2 recommended for random I/O)
- **Image size:** ~4 GB compressed (RPiOS Lite + audio stack)
- **Expanded on first boot:** Auto-expand to fill SD card

For Buildroot appliance image (future):
- **Compressed image:** ~50-80 MB (kernel + squashfs root)
- **Install size:** ~200-300 MB on disk

---

## 6. Cross-Compilation Toolchain

### 6.1 Toolchain Selection

**Recommended:** `gcc-aarch64-linux-gnu` from Ubuntu 26.04 repositories.

```bash
sudo apt install \
  gcc-aarch64-linux-gnu \
  g++-aarch64-linux-gnu \
  binutils-aarch64-linux-gnu \
  libc6-dev-arm64-cross
```

### 6.2 Toolchain Details

| Component | Package | Version (Ubuntu 26.04) |
|-----------|---------|------------------------|
| C compiler | `gcc-aarch64-linux-gnu` | 14.2.0 / 15.x |
| C++ compiler | `g++-aarch64-linux-gnu` | 14.2.0 / 15.x |
| Binutils | `binutils-aarch64-linux-gnu` | 2.43 |
| C library | `libc6-dev-arm64-cross` | glibc 2.40 |
| Target triple | `aarch64-linux-gnu` | — |
| Architecture | ARMv8-A (Cortex-A72) | `-march=armv8-a+crc+simd` |

### 6.3 Environment Setup

```bash
export ARCH=arm64
export CROSS_COMPILE=aarch64-linux-gnu-
export KERNEL=kernel8

# Verify
aarch64-linux-gnu-gcc --version
# aarch64-linux-gnu-gcc (Ubuntu 14.2.0-4ubuntu2) 14.2.0
```

### 6.4 Kernel Build Commands

```bash
# Configure
make bcm2711_defconfig
make menuconfig   # enable PREEMPT_RT and tuning options

# Build
make -j$(nproc) Image modules dtbs

# Output files:
#   arch/arm64/boot/Image           → kernel8-rt.img (on Pi /boot)
#   arch/arm64/boot/dts/broadcom/   → *.dtb files (on Pi /boot)
#   modules (INSTALL_MOD_PATH)      → /lib/modules/<version>/ (on Pi)
```

### 6.5 Alternative: Bootlin Toolchains

Buildroot uses Bootlin pre-built toolchains. These are available standalone:

```
https://toolchains.bootlin.com/
→ aarch64--glibc--stable-2024.02-1.tar.bz2
```

Bootlin toolchains include the full sysroot (headers, libraries) needed for
building userspace as well as the kernel. This is useful if building
userspace audio tools from source (e.g., a custom JACK build), but for
kernel-only cross-compilation, the distro `aarch64-linux-gnu-` package is
sufficient.

### 6.6 Native Compilation Fallback

If cross-compilation proves problematic (32-bit host, missing headers), the
RPi4B 8GB can build its own kernel natively in ~2-3 hours:

```bash
# On the Pi:
git clone --depth=1 -b rpi-6.12.y https://github.com/raspberrypi/linux
cd linux
KERNEL=kernel8
make bcm2711_defconfig
# Edit .config for PREEMPT_RT
make -j4 Image modules dtbs
sudo make modules_install
sudo cp arch/arm64/boot/Image /boot/kernel8-rt.img
```

---

## 7. Implementation Roadmap

### Phase 1: Base OS Setup (this phase)
1. Flash RPiOS Lite 64-bit to SD card
2. First boot: enable SSH, set hostname, update packages
3. Disable unnecessary services (WiFi, Bluetooth, avahi, etc.)
4. Apply boot optimizations (config.txt, cmdline.txt)
5. Configure CPU isolation (isolcpus=1-3)
6. Test baseline boot time with `systemd-analyze`

### Phase 2: RT Kernel Build (next phase, P2)
1. Set up cross-compilation toolchain on x86 host
2. Clone `rpi-6.12.y` kernel
3. Apply PREEMPT_RT + tuning config
4. Build kernel, modules, DTBs
5. Install on Pi and verify with `cyclictest`
6. Target: max latency < 100µs

### Phase 3: Read-Only Rootfs (follows audio stack validation)
1. Test OverlayFS initramfs approach
2. Create /data partition for persistent state
3. Configure tmpfs for /var, /tmp, /run
4. Test update procedure (remount-rw → apt → remount-ro)

### Phase 4: Production Image (optional, post-validation)
1. Evaluate Buildroot for appliance image
2. Create squashfs rootfs with initramfs
3. Generate reproducible SD card image
4. CI pipeline for kernel + rootfs builds

---

## 8. Risk Register

| Risk | Likelihood | Impact | Mitigation |
|------|-----------|--------|------------|
| Cross-compilation toolchain mismatch (glibc version skew) | Low | Medium | Use matching toolchain; fall back to native build |
| PREEMPT_RT kernel causes USB/xHCI regressions | Medium | High | Test thoroughly; maintain stock kernel as fallback in /boot |
| SD card wear from audio logging | Medium | Low | Log to tmpfs; archive to /data periodically |
| USB SSD storage conflicts with audio isochronous transfers | Medium | High | Use SD card or NVMe HAT; avoid USB storage during audio operation |
| RPi kernel rpi-6.12.y diverges from mainline PREEMPT_RT | Low | Medium | Monitor kernel releases; test each new kernel version |
| Read-only rootfs breaks apt/package management | Low | Medium | Document remount-rw procedure; automate in update script |

---

## 9. References

1. Raspberry Pi Linux kernel: https://github.com/raspberrypi/linux (branch: rpi-6.12.y)
2. Buildroot manual: https://buildroot.org/downloads/manual/manual.html
3. Buildroot RPi4 64-bit defconfig: https://github.com/buildroot/buildroot/tree/master/configs/raspberrypi4_64_defconfig
4. Yocto meta-raspberrypi: https://github.com/agherzan/meta-raspberrypi
5. Yocto Project docs: https://docs.yoctoproject.org/
6. PREEMPT_RT wiki: https://wiki.linuxfoundation.org/realtime/start
7. Bootlin toolchains: https://toolchains.bootlin.com/
8. Zynthian OS (RPi RT audio reference): https://github.com/zynthian/zynthian-sys
9. Linux kernel RT config docs: https://www.kernel.org/doc/html/latest/locking/rt-mutex-design.html
10. RPi boot options: https://www.raspberrypi.com/documentation/computers/config_txt.html#boot-options
11. OverlayFS kernel docs: https://www.kernel.org/doc/html/latest/filesystems/overlayfs.html
12. systemd-analyze: https://www.freedesktop.org/software/systemd/man/systemd-analyze.html