feat: update OCR docs for Ollama vision pipeline
+23
-4
@@ -1,6 +1,6 @@
|
|||||||
# Asset Label Photo Matching
|
# Asset Label Photo Matching
|
||||||
|
|
||||||
**Added:** 2026-05-29
|
**Updated:** 2026-05-29
|
||||||
|
|
||||||
New system for matching label photos (serial numbers, barcodes, QR codes) against the assets database.
|
New system for matching label photos (serial numbers, barcodes, QR codes) against the assets database.
|
||||||
|
|
||||||
@@ -21,10 +21,17 @@ Examples:
|
|||||||
|
|
||||||
`find_asset_by_normalized_id()` searches across `serial_number`, `connect_id`, `equipment_id`, `machine_id`, and `barcode` columns — normalizing all before comparing.
|
`find_asset_by_normalized_id()` searches across `serial_number`, `connect_id`, `equipment_id`, `machine_id`, and `barcode` columns — normalizing all before comparing.
|
||||||
|
|
||||||
|
## OCR Pipeline (priority order)
|
||||||
|
|
||||||
|
1. **Ollama vision model** (qwen2.5vl:3b) — runs on Windows gaming PC (192.168.0.181) via persistent SSH tunnel (systemd `ollama-tunnel.service` on localhost:11434). Most accurate. Resizes images to 640x480 before sending.
|
||||||
|
2. **Tesseract** (pytesseract) — local fallback, only used if Ollama is unavailable.
|
||||||
|
|
||||||
|
Response includes `ocr_source` field (`"ollama"` or `"tesseract"`).
|
||||||
|
|
||||||
## API Endpoints
|
## API Endpoints
|
||||||
|
|
||||||
### `POST /api/ocr`
|
### `POST /api/ocr`
|
||||||
Existing photo upload endpoint enhanced. Now returns `matched_assets` in addition to the legacy `machine_id` field. Uses Tesseract OCR. Falls through gracefully when OCR produces garbage.
|
Existing photo upload endpoint enhanced. Now returns `matched_assets` and `ocr_source` in addition to the legacy `machine_id` field. Ollama takes priority over Tesseract.
|
||||||
|
|
||||||
### `POST /api/match-text`
|
### `POST /api/match-text`
|
||||||
New endpoint for client-side processing:
|
New endpoint for client-side processing:
|
||||||
@@ -41,6 +48,18 @@ python3 scripts/match_label_photo.py photo.jpg
|
|||||||
python3 scripts/match_label_photo.py --text "S/N: 2500.0100.0025534"
|
python3 scripts/match_label_photo.py --text "S/N: 2500.0100.0025534"
|
||||||
```
|
```
|
||||||
|
|
||||||
## Vision Fallback
|
## Windows PC Setup
|
||||||
|
|
||||||
The vision model at opencode.ai (`mimo-v2-omni`) is used when Tesseract fails on dark-background or complex labels. The app can accept text from client-side vision processing via `/api/match-text`.
|
- **Host:** gamingpc (100.84.53.121 / 192.168.0.181)
|
||||||
|
- **Service:** Ollama with qwen2.5vl:3b model
|
||||||
|
- **Connection:** SSH tunnel via systemd `ollama-tunnel.service`
|
||||||
|
- **Auth:** SSH key at `~/.ssh/id_comfyui`
|
||||||
|
- **Tunnel:** `ssh -N -L 11434:127.0.0.1:11434 gamingpc`
|
||||||
|
- **Auto-start:** `sudo systemctl enable ollama-tunnel.service`
|
||||||
|
|
||||||
|
Results with qwen2.5vl:3b on test photos:
|
||||||
|
| Photo | Raw Text | DB Match |
|
||||||
|
|-------|----------|----------|
|
||||||
|
| Keurig label | `S/N: 2500.0100.0025534` | ✅ Asset #5144 (636671 / 956 Cypress Way) |
|
||||||
|
| Coca-Cola cooler | `2010378A00039 / RCUCC095.6` | ❌ Not in DB (bottler tags) |
|
||||||
|
| Telemetry device | `ID# 44343331624226353 / Monyx ID 48602143` | ❌ Not in DB (GPS tracker) |
|
||||||
|
|||||||
@@ -0,0 +1,29 @@
|
|||||||
|
# Ollama Vision Tunnel Service
|
||||||
|
|
||||||
|
## Install
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo tee /etc/systemd/system/ollama-tunnel.service << 'EOF'
|
||||||
|
[Unit]
|
||||||
|
Description=SSH Tunnel to Windows PC Ollama
|
||||||
|
After=network-online.target
|
||||||
|
Wants=network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
User=oplabs
|
||||||
|
ExecStart=/usr/bin/ssh -o ConnectTimeout=5 -o ServerAliveInterval=30 -o ServerAliveCountMax=3 -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -N -L 11434:127.0.0.1:11434 gamingpc
|
||||||
|
ExecStop=/usr/bin/kill $MAINPID
|
||||||
|
Restart=always
|
||||||
|
RestartSec=10
|
||||||
|
StartLimitIntervalSec=60
|
||||||
|
StartLimitBurst=5
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
EOF
|
||||||
|
|
||||||
|
sudo systemctl daemon-reload
|
||||||
|
sudo systemctl enable ollama-tunnel.service
|
||||||
|
sudo systemctl start ollama-tunnel.service
|
||||||
|
```
|
||||||
Reference in New Issue
Block a user