feat: update OCR docs for Ollama vision pipeline

2026-05-29 09:00:55 -04:00
parent 5e0313443a
commit 9d90f15ad4
2 changed files with 52 additions and 4 deletions
+23 -4
@@ -1,6 +1,6 @@
# Asset Label Photo Matching
**Added:** 2026-05-29
**Updated:** 2026-05-29
New system for matching label photos (serial numbers, barcodes, QR codes) against the assets database.
@@ -21,10 +21,17 @@ Examples:
`find_asset_by_normalized_id()` searches across `serial_number`, `connect_id`, `equipment_id`, `machine_id`, and `barcode` columns — normalizing all before comparing.
## OCR Pipeline (priority order)
1. **Ollama vision model** (qwen2.5vl:3b) — runs on Windows gaming PC (192.168.0.181) via persistent SSH tunnel (systemd `ollama-tunnel.service` on localhost:11434). Most accurate. Resizes images to 640x480 before sending.
2. **Tesseract** (pytesseract) — local fallback, only used if Ollama is unavailable.
Response includes `ocr_source` field (`"ollama"` or `"tesseract"`).
## API Endpoints
### `POST /api/ocr`
Existing photo upload endpoint enhanced. Now returns `matched_assets` in addition to the legacy `machine_id` field. Uses Tesseract OCR. Falls through gracefully when OCR produces garbage.
Existing photo upload endpoint enhanced. Now returns `matched_assets` and `ocr_source` in addition to the legacy `machine_id` field. Ollama takes priority over Tesseract.
### `POST /api/match-text`
New endpoint for client-side processing:
@@ -41,6 +48,18 @@ python3 scripts/match_label_photo.py photo.jpg
python3 scripts/match_label_photo.py --text "S/N: 2500.0100.0025534"
```
## Vision Fallback
## Windows PC Setup
The vision model at opencode.ai (`mimo-v2-omni`) is used when Tesseract fails on dark-background or complex labels. The app can accept text from client-side vision processing via `/api/match-text`.
- **Host:** gamingpc (100.84.53.121 / 192.168.0.181)
- **Service:** Ollama with qwen2.5vl:3b model
- **Connection:** SSH tunnel via systemd `ollama-tunnel.service`
- **Auth:** SSH key at `~/.ssh/id_comfyui`
- **Tunnel:** `ssh -N -L 11434:127.0.0.1:11434 gamingpc`
- **Auto-start:** `sudo systemctl enable ollama-tunnel.service`
Results with qwen2.5vl:3b on test photos:
| Photo | Raw Text | DB Match |
|-------|----------|----------|
| Keurig label | `S/N: 2500.0100.0025534` | ✅ Asset #5144 (636671 / 956 Cypress Way) |
| Coca-Cola cooler | `2010378A00039 / RCUCC095.6` | ❌ Not in DB (bottler tags) |
| Telemetry device | `ID# 44343331624226353 / Monyx ID 48602143` | ❌ Not in DB (GPS tracker) |
+29
@@ -0,0 +1,29 @@
# Ollama Vision Tunnel Service
## Install
```bash
sudo tee /etc/systemd/system/ollama-tunnel.service << 'EOF'
[Unit]
Description=SSH Tunnel to Windows PC Ollama
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=oplabs
ExecStart=/usr/bin/ssh -o ConnectTimeout=5 -o ServerAliveInterval=30 -o ServerAliveCountMax=3 -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -N -L 11434:127.0.0.1:11434 gamingpc
ExecStop=/usr/bin/kill $MAINPID
Restart=always
RestartSec=10
StartLimitIntervalSec=60
StartLimitBurst=5
[Install]
WantedBy=multi-user.target
EOF
sudo systemctl daemon-reload
sudo systemctl enable ollama-tunnel.service
sudo systemctl start ollama-tunnel.service
```