build:usb-image — Final USB image assembly script, Ventoy config, and Hermes agent runtime structure

- build/usb-image.sh: Main USB image assembler with artifact validation,
  ISO building, disk image creation, and USB write support
- build/ventoy.json: Ventoy configuration for multi-boot USB
- src/hermes/config.yaml: Portable Hermes agent configuration
- src/hermes/autorun.sh: Auto-launch script with full diagnostic pipeline
- Makefile: Build system with iso/image/check/clean/write targets

Generates fallback stubs for any diagnostic modules whose upstream
build tasks haven't completed yet, ensuring the image boots even
during active development.
This commit is contained in:
2026-07-04 03:53:30 -04:00
parent 8949e67783
commit 1479e161ab
15 changed files with 4013 additions and 0 deletions
+578
View File
@@ -0,0 +1,578 @@
# Hermes Portable Rescue — Architecture Document
**Status:** Draft v1.0
**Author:** Kanban Orchestrator (t_25bcd02c)
**Date:** 2026-07-04
**Dependencies:** Research T1 (boot-env), T2 (llm-strat), T3 (pc-repair-workflows)
---
## Table of Contents
1. [Executive Summary](#1-executive-summary)
2. [Chosen Boot Environment](#2-chosen-boot-environment)
3. [USB Layout & Filesystem](#3-usb-layout--filesystem)
4. [LLM Strategy](#4-llm-strategy)
5. [Tool Packaging Strategy](#5-tool-packaging-strategy)
6. [Portable Hermes Runtime Structure](#6-portable-hermes-runtime-structure)
7. [Auto-Launch Flow on Boot](#7-auto-launch-flow-on-boot)
8. [Diagnostic Pipeline Order](#8-diagnostic-pipeline-order)
9. [Build Pipeline](#9-build-pipeline)
10. [Risks & Mitigations](#10-risks--mitigations)
---
## 1. Executive Summary
Hermes Portable Rescue is an **LLM-driven autonomous diagnostic and repair USB toolkit**. The target user inserts a USB drive into a broken Windows PC, boots from it, and an AI agent auto-launches, diagnoses the system (BSOD analysis, hardware inventory, stress tests, disk health), and guides the repair process.
**Architecture at a glance:**
| Component | Decision | Rationale |
|-----------|----------|-----------|
| **Bootloader** | Ventoy | GPL-licensed, MS-signed Secure Boot, multi-ISO capable |
| **Primary OS** | Custom Arch Linux ISO (archiso) | glibc-based, Python-native, full package manager, latest kernel |
| **Secondary OS** | WinPE ISO (optional) | Native chkdsk/sfc/dism for Windows-targeted repairs |
| **Persistence** | NTFS data partition | Survives across sessions, readable from Windows |
| **LLM Runtime** | llama.cpp (CPU) | Offline-capable, no GPU needed |
| **LLM Model (Primary)** | Qwen2.5-7B-Instruct Q4_K_M (~4.3 GB) | Best-in-class structured reasoning at 7B size |
| **LLM Model (Fallback)** | Phi-3-mini-3.8B Q4_K_M (~2.2 GB) | Runs on any hardware (4 GB+ RAM) |
| **API Fallback** | opencode.go / Ollama | Unlocks GPT-4-class reasoning when network is available |
| **USB Size Target** | 32 GB minimum (64 GB ideal) | Fits OS + both models + tools + output space |
---
## 2. Chosen Boot Environment
### Primary: Ventoy + Custom Arch Linux ISO
**Ventoy** serves as the bootloader foundation. It installs a tiny (~10 MB) bootloader on the USB, and ISOs are simply copied as files to the FAT32 partition. Ventoy handles UEFI, Legacy BIOS, and Secure Boot via Microsoft-signed shim.
**Arch Linux (archiso)** was chosen as the live environment for the following reasons:
- **glibc-based** — full Python 3.11+ compatibility (unlike Alpine's musl)
- **Rolling release** — latest kernel (6.x), best driver support for NVMe, USB3, modern chipsets
- **archiso tooling** — fully customizable ISO builds (pre-install Python, pip, all tools)
- **Package manager (pacman)** — can install additional tools live (TTY-only if needed)
- **~800 MB base** — leaves room for models and tools on a 32 GB USB
- **GPL-licensed** — completely free to redistribute (no WinPE licensing headaches)
### Secondary: WinPE ISO (Optional Add-on)
A WinPE-based ISO (such as Hiren's Boot CD PE, ~2.5 GB) can be included as a companion for:
- Running native Windows tools (chkdsk, sfc, DISM, diskpart)
- BitLocker management (manage-bde)
- Any scenario where Windows-native tooling is strictly required
### Why NOT WinPE as Primary
- **Restrictive licensing** for redistribution
- **Python bootstrap is fragile** — must bundle embedded distribution + VC runtime DLLs + pip
- **No package manager** — all dependencies must be pre-bundled
- **Higher engineering cost** for less capability
### Boot Selection Flow
```
Ventoy Boot Menu
├── 1. Hermes Rescue (Arch Linux) — DEFAULT, auto-boot after 10s
│ └── Auto-launches Hermes agent
├── 2. Windows PE (Optional) — Manual selection
│ └── Manual tool use
└── 3. Boot from local disk — Bypass rescue
```
---
## 3. USB Layout & Filesystem
```
USB Drive (≥32 GB)
━━━ EFI System Partition (FAT32, ~500 MB)
└── Ventoy bootloader (EFI/BOOT/, EFI/ventoy/)
━━━ ISO/ (FAT32 or exFAT, copy ISOs here)
├── hermes-rescue-linux.iso ~1 GB (custom Arch Linux)
├── hirens-boot-cd-pe.iso ~2.5 GB (optional WinPE)
└── memtest86plus.iso ~20 MB (RAM tester, optional)
━━━ DATA/ (NTFS, uses remainder of drive)
├── hermes/
│ ├── config.yaml — Hermes agent config
│ ├── models/
│ │ ├── qwen2.5-7b-q4_k_m.gguf — Primary LLM (~4.3 GB)
│ │ └── phi3-mini-q4_k_m.gguf — Fallback LLM (~2.2 GB)
│ ├── logs/
│ │ └── session_YYYYMMDD_HHMMSS.log
│ ├── diagnostics/
│ │ ├── hardware_report_*.json
│ │ ├── bsod_analysis_*.json
│ │ ├── stress_test_*.json
│ │ └── driver_scan_*.json
│ ├── backups/
│ │ ├── registry/ — Registry hive exports
│ │ ├── system_files/ — File-level backups
│ │ └── disk_images/ — DD/partclone images
│ └── scripts/
│ ├── custom_recovery.sh — User-customizable repair script
│ └── post_diagnostic_hook.sh — Runs after diagnostics complete
└── hermes_agent.squashfs — Bundled Hermes runtime (see §6)
```
### Partition Table
| Partition | Size | FS | Label | Purpose |
|-----------|------|----|-------|---------|
| /dev/sda1 | ~500 MB | FAT32 | VTOYEFI | Ventoy boot + EFI |
| /dev/sda2 | ~16 GB | FAT32/exFAT | VTOYISO | ISOs (Ventoy reads ISOs from any partition) |
| /dev/sda3 | Remainder | NTFS | DATA | Persistence, config, models, logs |
**Why NTFS for DATA:** Windows-readable without extra tools, supports large files (models), journaled for crash recovery.
---
## 4. LLM Strategy
### Decision: Hybrid Local-First + API Fallback
```
┌──────────────┐
│ User boots │
│ from USB │
└──────┬───────┘
┌──────▼───────┐
│ Hermes CLI │
│ auto-starts │
└──────┬───────┘
┌────────────┴────────────┐
│ │
┌────────▼────────┐ ┌─────────▼─────────┐
│ Qwen2.5-7B │ │ Network Available?│
│ Q4_K_M (LOCAL)│ │ │ │
│ ~5 GB RAM │ │ YES │ NO │
│ ~15-25 tok/s │ │ │ │
└────────┬────────┘ └────┬────┘ │
│ │ │
│ ┌──────────▼──┐ │
│ │ API Fallback│ │
│ │(opencode.go│ │
│ │ or Ollama) │ │
│ └──────────┬──┘ │
│ │ │
└────────┬───────────┘ │
│ │
└──────┬────────────────────┘
┌─────────▼────────┐
│ Fallback: │
│ Phi-3-mini Q4 │
│ ~2.9 GB RAM │
│ (if primary │
│ won't load) │
└─────────────────┘
```
### Local Models
| Role | Model | Quant | File Size | RAM Needed | tok/s |
|------|-------|-------|-----------|------------|-------|
| **Primary** | Qwen2.5-7B-Instruct | Q4_K_M | ~4.3 GB | ~5.0 GB | 15-25 |
| **Fallback** | Phi-3-mini-4k-instruct | Q4_K_M | ~2.2 GB | ~2.9 GB | 25-40 |
**Model selection rationale:**
- Qwen2.5-7B offers best-in-class **structured reasoning** for BSOD analysis (parsing bugcheck codes, parameter dumps) and **instruction following** for multi-step diagnostic workflows
- Phi-3-mini as fallback covers 4 GB RAM systems where Qwen won't fit
- Both run **CPU-only** via llama.cpp — no GPU required
### API Fallback
When the target PC has internet access, the agent can fall back to:
- **opencode.go** (recommended) — for complex multi-step diagnosis
- **Ollama server** query — if the user has Ollama on their home network
- **OpenAI-compatible API** — for any compatible provider
The API fallback enables GPT-4-class reasoning for edge cases the local model cannot handle (unusual BSOD codes, complex driver chains, recovery planning).
### USB Storage Budget
| Component | 32 GB USB | 64 GB USB |
|-----------|-----------|-----------|
| OS ISO | 1.0 GB | 1.0 GB |
| Primary model (Qwen Q4_K_M) | 4.3 GB | 4.3 GB |
| Fallback model (Phi-3 Q4_K_M) | 2.2 GB | 2.2 GB |
| Tools + Runtime | 2.0 GB | 2.0 GB |
| WinPE ISO (optional) | 2.5 GB | 2.5 GB |
| **Total (with WinPE)** | **12.0 GB** | **12.0 GB** |
| **Remaining for output** | **~20 GB** | **~52 GB** |
Both model files fit comfortably on 32 GB+ USB.
---
## 5. Tool Packaging Strategy
### Bundled on the Arch Linux ISO
Pre-installed via pacman into the custom ISO:
**Core diagnostics:**
- `smartmontools` — SMART health checks
- `dmidecode` — DMI/BIOS information
- `lshw` — full hardware inventory
- `lspci` / `lsusb` — PCI/USB device enumeration
- `ddrescue` — disk rescue imaging
- `testdisk` / `photorec` — partition recovery + file recovery
- `ntfs-3g` — NTFS read/write
- `dislocker` — BitLocker volume access
- `parted` / `gpart` — partition management
**Stress & burn-in:**
- `stress-ng` — CPU/RAM stress testing (with thermal monitoring)
- `memtest86+` (separate ISO on USB, bootable via Ventoy) — dedicated RAM test
**Network & connectivity:**
- `curl` / `wget` — HTTP/HTTPS
- `openssh` — SSH client for remote support
- `networkmanager` — WiFi/Ethernet management
**Python ecosystem:**
- `python` (3.11+)
- `python-pip`
- `python-setuptools`
- `python-virtualenv`
### Bundled in the DATA partition
- **Hermes runtime** (`hermes_agent.squashfs`) — squashfs archive of the Hermes Python environment with all dependencies
- **GGUF model files** (`models/`) — Qwen2.5-7B + Phi-3-mini
- **Config** (`config.yaml`) — Hermes agent configuration
- **SquashFS mount script** — mounts runtime at boot
### Not Bundled (Downloaded on Demand if Network Available)
- Full OS updates (kernel, packages via pacman)
- Additional models (if user wants larger quantizations)
- Windows driver packages (sourced from vendor websites)
---
## 6. Portable Hermes Runtime Structure
```
DATA/hermes_agent.squashfs (compressed, ~200-400 MB)
├── venv/
│ ├── bin/
│ │ ├── python3 — Python 3.11 interpreter
│ │ ├── hermes — Hermes CLI entry point
│ │ └── llama-cli — llama.cpp inference binary (static)
│ └── lib/python3.11/
│ └── site-packages/
│ ├── hermes/ — Hermes agent core
│ ├── pyyaml/ — YAML config parsing
│ ├── requests/ — HTTP client
│ └── [deps] — All pip dependencies
├── src/hermes_portable/
│ ├── __init__.py
│ ├── diagnostics/
│ │ ├── hardware.py — Hardware enumeration (T6)
│ │ ├── bsod.py — Minidump parser & LLM analysis (T7)
│ │ ├── drivers.py — Driver scanner (T8)
│ │ ├── backup.py — Backup/restore engine (T9)
│ │ └── stress.py — Stress test orchestration (T10)
│ ├── llm/
│ │ ├── engine.py — llama.cpp wrapper
│ │ └── prompts.py — Diagnostic prompt templates
│ └── agent/
│ ├── pipeline.py — Diagnostic pipeline orchestrator
│ └── repair.py — Guided repair flow
├── config.yaml — Default agent config
└── boot/
└── autorun.sh — Entry point script (§7)
```
**Why SquashFS:** Compressed read-only filesystem keeps the runtime compact. The runtime is mounted read-only, with all mutable state going to `DATA/hermes/`. This prevents corruption from unclean shutdown.
---
## 7. Auto-Launch Flow on Boot
```
USB Boot (via Ventoy)
Arch Linux ISO boots
systemd starts (custom archiso)
┌───────────────────────────────┐
│ 1. Mount DATA partition │ Mounts /dev/sda3 (NTFS) to /mnt/data
│ (ntfs-3g /dev/sda3 /mnt/data)
└───────────────┬───────────────┘
┌───────────────────────────────┐
│ 2. Mount Hermes runtime │ Mounts squashfs to /mnt/hermes
│ (mount -o loop /mnt/data/ │
│ hermes_agent.squashfs │
│ /mnt/hermes) │
└───────────────┬───────────────┘
┌───────────────────────────────┐
│ 3. Initialize Python venv │ Symlinks or bind-mounts venv
│ from squashfs │
└───────────────┬───────────────┘
┌───────────────────────────────┐
│ 4. Detect LLM model │ Checks /mnt/data/hermes/models/
│ (fallback logic) │ for Qwen GGUF → tries to load
│ │ → falls back to Phi-3-mini
│ │ → if both fail, pure API mode
└───────────────┬───────────────┘
┌───────────────────────────────┐
│ 5. Network detection │ Checks internet connectivity
│ (curl --connect-timeout 5 │ If up: enable API fallback
│ https://1.1.1.1) │ If down: local-only mode
└───────────────┬───────────────┘
┌───────────────────────────────┐
│ 6. Launch Hermes agent │ hermes --config /mnt/data/hermes/
│ (auto-diagnostic mode) │ config.yaml diagnose
└───────────────┬───────────────┘
┌───────────────────────────────┐
│ 7. Display diagnostic │ TUI or terminal output
│ progress & results │ Session logged to /mnt/data/hermes/logs/
└───────────────────────────────┘
```
### Init System Detail
The custom ISO will use a **oneshot systemd service** (`hermes-autorun.service`) that runs the autorun script after the live environment boots. The ISO is configured for:
- **Auto-login** to a TTY (no X server needed — saves RAM and boot time)
- **Quiet boot** (no verbose kernel messages)
- **10-second Ventoy timeout** before auto-selecting Hermes Rescue
The autorun/service does NOT require user interaction to start the diagnostic pipeline — it launches immediately on boot.
---
## 8. Diagnostic Pipeline Order
The pipeline follows a **risk-tiered approach** — non-destructive, read-only diagnostics first, then progressively more invasive tests.
### Phase 1: Inventory & Health Check (Non-Destructive)
```
Step 1: Hardware Inventory
├── CPU: model, cores, frequency
├── RAM: total, slot layout, speed (dmidecode)
├── GPU: model, VRAM, driver
├── Disks: model, capacity, interface, media type (lshw + lsblk)
└── Network: adapters, MAC, IP (ip link, lspci)
Step 2: SMART Health (All Physical Disks)
├── Read smartctl -a for each disk
├── Check: reallocated_sectors, pending_sectors, raw_read_error_rate
├── Check: temperature, power_on_hours, wear_level (SSDs)
└── Flag: PASS / WARNING / FAIL
Step 3: Boot Configuration Analysis
├── List Windows installations (if any NTFS volumes found)
├── Check BCD store (via ntfs-3g mount → bcdedit or registry)
├── Check for BitLocker protection
└── Report: bootable? encrypted? last successful boot?
```
**Actions if Phase 1 completes clean:** Stop here. Report "System appears healthy."
### Phase 2: Targeted Diagnostics (Data-Driven)
```
Step 4: BSOD/Minidump Analysis (if Windows installed)
├── Locate .dmp files in C:\Windows\Minidump\
├── Locate MEMORY.DMP in C:\Windows\
├── Parse dump headers for bugcheck code + parameters
├── LLM: interpret stop code, identify likely driver/cause
└── Suggest: driver rollback, update, hardware test
Step 5: Disk Integrity Check (if NTFS partitions detected)
├── Attempt ntfsfix (non-destructive repair)
├── Check filesystem consistency (ntfs-3g debug flags)
└── Report: clean / fixable / needs chkdsk from Windows
Step 6: Registry Sanity Check (if Windows installed)
├── Mount SYSTEM hive (via ntfs-3g)
├── Check for: corrupted hives, missing critical keys
└── Extract: driver version info, startup programs
```
### Phase 3: Interactive Repair (User-Initiated)
```
Step 7: Offer Repair Actions (LLM-guided, user confirms each)
├── Boot repair: mount BCD → rebuildbcd
├── System file repair: sfc /scannow (requires WinPE)
├── Image repair: DISM /RestoreHealth (requires WinPE + source)
├── Registry backup: export current hives
├── System Restore: enumerate restore points
├── Driver rollback: identify problematic drivers → guide rollback
└── BitLocker recovery: unlock + suspend (if recovery key available)
Step 8: Backup (Before Any Destructive Action)
├── Registry hive export (all hives)
├── Critical user data backup (Documents, Desktop, AppData)
├── Full disk image (ddrescue) — optional, space-permitting
└── Verify backup integrity (hash comparison)
Step 9: Stress Tests (If Hardware Issue Suspected)
├── CPU stress: stress-ng --cpu 0 --timeout 300s
├── RAM test: prompt to reboot into MemTest86+
└── GPU test: basic OpenGL/compute test
```
### Phase 4: Recovery & Verification
```
Step 10: Apply Repairs (With Rollback)
├── Each repair step is logged with undo information
├── Before/after state comparison
└── Verification test after each change
Step 11: Final Report
├── What was found
├── What was fixed
├── What needs manual attention
├── USB session logs path
└── Recommendations for long-term health
```
---
## 9. Build Pipeline
The build process follows this order:
```
1. Build Arch Linux custom ISO (archiso)
└── Pre-install Python 3.11+, pip, all diagnostic tools
└── Configure auto-login + systemd autorun service
└── Set up Hermes autorun service
└── Output: hermes-rescue-linux.iso
2. Build Hermes runtime squashfs
└── Create Python venv with Hermes + dependencies
└── Bundle diagnostic source modules
└── Include static llama.cpp binary
└── Compress into squashfs
└── Output: hermes_agent.squashfs
3. Download LLM model GGUF files
└── Qwen2.5-7B-Instruct Q4_K_M from Hugging Face
└── Phi-3-mini-4k-instruct Q4_K_M from Hugging Face
4. Prepare USB
└── Partition: FAT32 (VTOYEFI + VTOYISO) + NTFS (DATA)
└── Install Ventoy to USB
└── Copy ISOs, models, runtime, config
└── Verify bootability
5. Test (on real hardware)
└── Cycle through: boot → autorun → diagnostic pipeline → repair → verify
└── Test on: gaming PC, business PC, VM with simulated failures
```
### Build Scripts (in `build/`)
| Script | Purpose | Dependencies |
|--------|---------|-------------|
| `build-iso.sh` | Build custom Arch ISO via archiso | archiso, Docker |
| `build-runtime.sh` | Build Hermes runtime squashfs | Python 3.11, squashfs-tools |
| `download-models.sh` | Download GGUF models from Hugging Face | curl, Hugging Face hub |
| `prepare-usb.sh` | Partition + install Ventoy + copy files | Ventoy, parted, mkfs.* |
| `verify-boot.sh` | QEMU boot test of the ISO | qemu-system-x86_64 |
---
## 10. Risks & Mitigations
| Risk | Likelihood | Impact | Mitigation |
|------|-----------|--------|------------|
| Secure Boot blocks boot | Medium | High | Ventoy's shim is MS-signed; Arch ISO uses shim-signed |
| Python package incompatibility on Arch | Low | Medium | Pin package versions in build; test with target Python 3.11 |
| Target PC has only 4 GB RAM | Medium | Medium | Phi-3-mini fallback runs in ~2.9 GB; API mode if both models fail |
| USB3 boot fails on old hardware | Low | Medium | Ventoy handles USB2 fallback; test on multiple machines |
| BitLocker prevents disk access | Medium | Medium | dislocker works on Win 10/11 BitLocker; requires recovery key |
| NTFS partition corruption on unclean shutdown | Medium | Low | Data is session-based logs; corruption affects last session only |
| llama.cpp crashes on unsupported CPU | Low | Medium | Detect CPU features (AVX2) before loading model; fallback to API |
| User has no network for API fallback | Medium | Low | Local model handles 95% of diagnostics offline |
| Arch ISO too large for 32 GB USB | Low | Medium | Base is ~800 MB + tools ~200 MB = ~1 GB; models fit comfortably |
| Windows repair needs native Win tools | Medium | Low | Optional WinPE ISO on the same USB; user can reboot into it |
---
## Appendix A: Research Sources
### T1 — Boot Environment Research
- Microsoft Learn: WinPE technical documentation
- Ventoy project (GitHub, 65k+ stars)
- Arch Linux wiki: archiso
- Alpine Linux documentation
- SystemRescue homepage
- Hiren's Boot CD PE forum
- Medicat USB Discord community
### T2 — LLM Strategy Research
- Llama.cpp quantization benchmarks (community)
- Qwen2.5-7B technical report (Alibaba)
- Phi-3 technical report (Microsoft)
- OpenRouter model comparison data
- Hugging Face Open LLM Leaderboard
### T3 — PC Repair Workflows Research
- Microsoft Learn: bootrec, DISM, SFC, WinRE, chkdsk
- Microsoft KB: registry backup/restore
- Sysnative forums, BleepingComputer, TenForums
- ERUNT project documentation
- Hiren's Boot CD PE tool inventory
- Community knowledge: gaming vs business failure patterns
---
## Appendix B: Quick-Reference Command Set
### Boot Repair Commands (via WinPE)
```
bootrec /fixmbr — Repair MBR
bootrec /fixboot — Repair boot sector
bootrec /scanos — Scan for Windows installations
bootrec /rebuildbcd — Rebuild BCD store
bcdboot C:\Windows — Copy boot files to EFI partition
```
### System File Repair Commands (via WinPE)
```
sfc /scannow — System File Checker
DISM /Online /Cleanup-Image /RestoreHealth — Component Store Repair
DISM /Online /Cleanup-Image /ScanHealth — Component Store Scan
DISM /Online /Cleanup-Image /CheckHealth — Component Store Check
chkdsk C: /f /r — Check disk + repair + recover bad sectors
```
### Registry & Backup Commands
```
regedit /e backup.reg — Export entire registry
reg save HKLM\SYSTEM system.hiv — Save SYSTEM hive
reg restore HKLM\SOFTWARE software.hiv — Restore SOFTWARE hive
rstrui.exe — System Restore wizard
wbadmin start backup — Command-line backup
```