feat: update OCR docs for Ollama vision pipeline
+23
-4
@@ -1,6 +1,6 @@
|
||||
# Asset Label Photo Matching
|
||||
|
||||
**Added:** 2026-05-29
|
||||
**Updated:** 2026-05-29
|
||||
|
||||
New system for matching label photos (serial numbers, barcodes, QR codes) against the assets database.
|
||||
|
||||
@@ -21,10 +21,17 @@ Examples:
|
||||
|
||||
`find_asset_by_normalized_id()` searches across `serial_number`, `connect_id`, `equipment_id`, `machine_id`, and `barcode` columns — normalizing all before comparing.
|
||||
|
||||
## OCR Pipeline (priority order)
|
||||
|
||||
1. **Ollama vision model** (qwen2.5vl:3b) — runs on Windows gaming PC (192.168.0.181) via persistent SSH tunnel (systemd `ollama-tunnel.service` on localhost:11434). Most accurate. Resizes images to 640x480 before sending.
|
||||
2. **Tesseract** (pytesseract) — local fallback, only used if Ollama is unavailable.
|
||||
|
||||
Response includes `ocr_source` field (`"ollama"` or `"tesseract"`).
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### `POST /api/ocr`
|
||||
Existing photo upload endpoint enhanced. Now returns `matched_assets` in addition to the legacy `machine_id` field. Uses Tesseract OCR. Falls through gracefully when OCR produces garbage.
|
||||
Existing photo upload endpoint enhanced. Now returns `matched_assets` and `ocr_source` in addition to the legacy `machine_id` field. Ollama takes priority over Tesseract.
|
||||
|
||||
### `POST /api/match-text`
|
||||
New endpoint for client-side processing:
|
||||
@@ -41,6 +48,18 @@ python3 scripts/match_label_photo.py photo.jpg
|
||||
python3 scripts/match_label_photo.py --text "S/N: 2500.0100.0025534"
|
||||
```
|
||||
|
||||
## Vision Fallback
|
||||
## Windows PC Setup
|
||||
|
||||
The vision model at opencode.ai (`mimo-v2-omni`) is used when Tesseract fails on dark-background or complex labels. The app can accept text from client-side vision processing via `/api/match-text`.
|
||||
- **Host:** gamingpc (100.84.53.121 / 192.168.0.181)
|
||||
- **Service:** Ollama with qwen2.5vl:3b model
|
||||
- **Connection:** SSH tunnel via systemd `ollama-tunnel.service`
|
||||
- **Auth:** SSH key at `~/.ssh/id_comfyui`
|
||||
- **Tunnel:** `ssh -N -L 11434:127.0.0.1:11434 gamingpc`
|
||||
- **Auto-start:** `sudo systemctl enable ollama-tunnel.service`
|
||||
|
||||
Results with qwen2.5vl:3b on test photos:
|
||||
| Photo | Raw Text | DB Match |
|
||||
|-------|----------|----------|
|
||||
| Keurig label | `S/N: 2500.0100.0025534` | ✅ Asset #5144 (636671 / 956 Cypress Way) |
|
||||
| Coca-Cola cooler | `2010378A00039 / RCUCC095.6` | ❌ Not in DB (bottler tags) |
|
||||
| Telemetry device | `ID# 44343331624226353 / Monyx ID 48602143` | ❌ Not in DB (GPS tracker) |
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
# Ollama Vision Tunnel Service
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
sudo tee /etc/systemd/system/ollama-tunnel.service << 'EOF'
|
||||
[Unit]
|
||||
Description=SSH Tunnel to Windows PC Ollama
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=oplabs
|
||||
ExecStart=/usr/bin/ssh -o ConnectTimeout=5 -o ServerAliveInterval=30 -o ServerAliveCountMax=3 -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -N -L 11434:127.0.0.1:11434 gamingpc
|
||||
ExecStop=/usr/bin/kill $MAINPID
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
StartLimitIntervalSec=60
|
||||
StartLimitBurst=5
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
EOF
|
||||
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable ollama-tunnel.service
|
||||
sudo systemctl start ollama-tunnel.service
|
||||
```
|
||||
Reference in New Issue
Block a user