diff --git a/Asset-Label-Matching.md b/Asset-Label-Matching.md index ec1fdd0..1b2df68 100644 --- a/Asset-Label-Matching.md +++ b/Asset-Label-Matching.md @@ -1,6 +1,6 @@ # Asset Label Photo Matching -**Added:** 2026-05-29 +**Updated:** 2026-05-29 New system for matching label photos (serial numbers, barcodes, QR codes) against the assets database. @@ -21,10 +21,17 @@ Examples: `find_asset_by_normalized_id()` searches across `serial_number`, `connect_id`, `equipment_id`, `machine_id`, and `barcode` columns — normalizing all before comparing. +## OCR Pipeline (priority order) + +1. **Ollama vision model** (qwen2.5vl:3b) — runs on Windows gaming PC (192.168.0.181) via persistent SSH tunnel (systemd `ollama-tunnel.service` on localhost:11434). Most accurate. Resizes images to 640x480 before sending. +2. **Tesseract** (pytesseract) — local fallback, only used if Ollama is unavailable. + +Response includes `ocr_source` field (`"ollama"` or `"tesseract"`). + ## API Endpoints ### `POST /api/ocr` -Existing photo upload endpoint enhanced. Now returns `matched_assets` in addition to the legacy `machine_id` field. Uses Tesseract OCR. Falls through gracefully when OCR produces garbage. +Existing photo upload endpoint enhanced. Now returns `matched_assets` and `ocr_source` in addition to the legacy `machine_id` field. Ollama takes priority over Tesseract. ### `POST /api/match-text` New endpoint for client-side processing: @@ -41,6 +48,18 @@ python3 scripts/match_label_photo.py photo.jpg python3 scripts/match_label_photo.py --text "S/N: 2500.0100.0025534" ``` -## Vision Fallback +## Windows PC Setup -The vision model at opencode.ai (`mimo-v2-omni`) is used when Tesseract fails on dark-background or complex labels. The app can accept text from client-side vision processing via `/api/match-text`. +- **Host:** gamingpc (100.84.53.121 / 192.168.0.181) +- **Service:** Ollama with qwen2.5vl:3b model +- **Connection:** SSH tunnel via systemd `ollama-tunnel.service` +- **Auth:** SSH key at `~/.ssh/id_comfyui` +- **Tunnel:** `ssh -N -L 11434:127.0.0.1:11434 gamingpc` +- **Auto-start:** `sudo systemctl enable ollama-tunnel.service` + +Results with qwen2.5vl:3b on test photos: +| Photo | Raw Text | DB Match | +|-------|----------|----------| +| Keurig label | `S/N: 2500.0100.0025534` | ✅ Asset #5144 (636671 / 956 Cypress Way) | +| Coca-Cola cooler | `2010378A00039 / RCUCC095.6` | ❌ Not in DB (bottler tags) | +| Telemetry device | `ID# 44343331624226353 / Monyx ID 48602143` | ❌ Not in DB (GPS tracker) | diff --git a/Ollama-Vision-Tunnel.md b/Ollama-Vision-Tunnel.md new file mode 100644 index 0000000..93e6728 --- /dev/null +++ b/Ollama-Vision-Tunnel.md @@ -0,0 +1,29 @@ +# Ollama Vision Tunnel Service + +## Install + +```bash +sudo tee /etc/systemd/system/ollama-tunnel.service << 'EOF' +[Unit] +Description=SSH Tunnel to Windows PC Ollama +After=network-online.target +Wants=network-online.target + +[Service] +Type=simple +User=oplabs +ExecStart=/usr/bin/ssh -o ConnectTimeout=5 -o ServerAliveInterval=30 -o ServerAliveCountMax=3 -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -N -L 11434:127.0.0.1:11434 gamingpc +ExecStop=/usr/bin/kill $MAINPID +Restart=always +RestartSec=10 +StartLimitIntervalSec=60 +StartLimitBurst=5 + +[Install] +WantedBy=multi-user.target +EOF + +sudo systemctl daemon-reload +sudo systemctl enable ollama-tunnel.service +sudo systemctl start ollama-tunnel.service +```