canteen-asset-tracker/docs/FIELD_MAP.md

# Field Map — Canteen Asset Tracker

Every column in `assets.db`, how each pipeline touches it.

## Source Legend

| Source | Pipeline | File(s) | How to Run |
|--------|----------|---------|------------|
| **MSFS** | ADB pull → Dynamics 365 extraction | `pull_msfs_data.py` → `sync_assets_to_canteen.py` | Button on extraction viewer `:8910` **Manual** |
| **CANT** | Cantaloupe staged import | `cantaloupe_sync.py` in admin server | Dashboard upload **Manual** |
| **SEED** | Seed Excel import | `parser.py` → `db_writer.py` → `sync.py` | CLI `--mode=seed` or `--mode=update` **Manual** |
| **APP** | Main app (technician check-in, photo, GPS) | `server.py` | Used by techs in field |
| **ADMIN** | Admin dashboard edits | `admin_server.py` | Dashboard Cleanup/Mass Edit |

## Policy Legend

- **always** — pipeline overwrites this field every run
- **if_empty** — only filled when currently NULL/empty
- **never** — pipeline never touches this column
- **if_numeric** — only overwritten if value is bare number (Cantaloupe default ID)

---

## 🔵 MSFS Sync — Field Policy

| Column | Policy | Dynamics 365 Source |
|--------|--------|---------------------|
| `serial_number` | always | `hsl_serialnumber` |
| `connect_id` | always | `hsl_connectid` |
| `equipment_id` | always | `hsl_equipmentid` |
| `canteen_connect_guid` | always | `hsl_canteenconnectguid` |
| `barcode` | always | `hsl_barcode` |
| `manufacturer` | always | `hsl_manufacturertext` |
| `model` | always | `hsl_modeltext` or `hsl_modelname` |
| `route_name` | always | `hsl_routeidname` |
| `install_date` | always | `hsl_opendate` |
| `device` | always | `msdyn_deviceid` |
| `name` | if_numeric | `msdyn_name` or `hsl_name` |
| `company` | if_empty | Account `name` |
| `customer_name` | if_empty | Account `name` |
| `place` | if_empty | Account `address1_city` |
| `address` | if_empty | Account `address1_line1` + `line2` |
| `make` | **never** | _(uses `manufacturer` → separate column)_ |
| `building_name` | **never** | — |
| `floor` | **never** | — |
| `room` | **never** | — |
| `building_number` | **never** | — |
| `trailer_number` | **never** | — |
| `location_area` | **never** | — |
| `latitude` | **never** | _(mapped but only if NULL in code)_ |
| `longitude` | **never** | _(mapped but only if NULL in code)_ |
| `photo_path` | **never** | — |
| `category` | **never** | — |
| `status` | **never** | — |
| `description` | **never** | — |

> All other columns (sales data, alerts, disney flags, etc.) are **never** touched by MSFS sync.

---

## 🟡 Cantaloupe Sync — Field Policy

| Column | Policy | Notes |
|--------|--------|-------|
| `machine_id` | never (match key) | — |
| `serial_number` | always | — |
| `name` | always | — |
| `description` | always | — |
| `category` | always | Validated against categories table |
| `status` | always | Validated against statuses |
| `make` | always | — |
| `model` | always | — |
| `address` | always | — |
| `building_name` | always | — |
| `building_number` | always | — |
| `floor` | always | — |
| `room` | always | — |
| `trailer_number` | always | — |
| `walking_directions` | always | — |
| `map_link` | always | — |
| `parking_location` | always | — |
| `photo_path` | always | — |
| `latitude` | always | — |
| `longitude` | always | — |
| `geofence_radius_meters` | always | — |
| `dex_report_date` | always | — |
| `deployed` | always | — |
| `pulled_date` | always | — |
| `customer_id` | always | Via customer name lookup/creation |
| `location_id` | always | Via location name lookup/creation |
| `is_disney` | always | Derived from customer name prefix `D-` |

> **No preserve logic.** Any non-empty field in the import file overwrites the existing value.

---

## 🟢 Seed Import — Field Policy (db_writer.py)

### Seed Mode (`--mode=seed`)
**DESTROYS AND RE-INSERTS** — `DELETE FROM assets` first, then inserts all rows fresh.

### Update Mode (`--mode=update`)
Upsert by `machine_id`. **MSFS is primary data source** — seed fills gaps only.

**ALL fields** are treated as ONLY_IF_EMPTY for existing records:
| Column | Policy | Excel Source / Generation |
|--------|--------|--------------------------|
| `serial_number` | if_empty | Serial Number column + OCR corrections |
| `name` | if_empty | Generated: `{make} @ {place}` |
| `make` | if_empty | Parsed from Type column |
| `model` | if_empty | Parsed from Type column |
| `class` | if_empty | Parsed from Type column |
| `is_glass_front` | if_empty | Parsed from Type column |
| `company` | if_empty | Parsed from Customer column |
| `place` | if_empty | Parsed from Place column |
| `building_name` | if_empty | Parsed from Place column |
| `suite` | if_empty | Parsed from Place column |
| `room` | if_empty | Parsed from Place column |
| `trailer` | if_empty | Parsed from Place column |
| `floor` | if_empty | Parsed from Place column |
| `zone` | if_empty | Parsed from Place column |
| `zone_type` | if_empty | Parsed from Place column |
| `address` | if_empty | Address column |
| `location_area` | if_empty | City column |
| `latitude` | if_empty | Excel (existing GPS preserved) |
| `longitude` | if_empty | Excel (existing GPS preserved) |
| `photo_path` | if_empty | Excel (existing photo preserved) |
| `install_date` | if_empty | Added Date (existing date preserved) |
| `deployed` | if_empty | Deployed column (existing preserved) |
| `pulled_date` | if_empty | Pulled Date (existing preserved) |
| `dex_report_date` | if_empty | Last Dex Report Time |
| `disney_park` | if_empty | Parsed from Customer column |
| `disney_area` | if_empty | Derived from disney_park |
| `remote_pricing_status` | if_empty | Status column |
| `alerts` | if_empty | Alerts column + Coil/Product Alerts |
| `telemetry_provider` | if_empty | Parsed from Device column |
| `card_reader_brand` | if_empty | Parsed from Device column |
| `priority` | if_empty | Computed from yearly_sales |
| `yearly_sales` | if_empty | Yearly Sales column |
| `monthly_sales` | if_empty | Monthly Sales column |
| `weekly_sales` | if_empty | Weekly Sales column |
| `days_since_restock` | if_empty | Days Since Restock column |
| `prepick_group` | if_empty | Prepick Group column |
| `has_cashless` | if_empty | Has Cashless column |

> **New assets** (not found in DB by machine_id) are inserted with full seed data.
> **Existing assets**: only empty/null fields are filled. Any field already populated by MSFS, cleanup, or field use is preserved.

---

## 🟠 Seed Import — Sync (`sync.py` → main app DB)

After `db_writer.py`, this script pushes the seed DB into the main app's schema.
**Now follows ONLY_IF_EMPTY policy** for existing assets — same as db_writer update mode.

| Column | Policy |
|--------|--------|
| `machine_id` | never (match key) |
| `serial_number` | if_empty |
| `name` | if_empty |
| `description` | if_empty |
| `category` | if_empty |
| `status` | if_empty |
| `make` | if_empty |
| `model` | if_empty |
| `address` | if_empty |
| `building_name` | if_empty |
| `building_number` | if_empty |
| `floor` | if_empty |
| `room` | if_empty |
| `trailer_number` | if_empty |
| `walking_directions` | if_empty |
| `map_link` | if_empty |
| `parking_location` | if_empty |
| `photo_path` | if_empty |
| `latitude` | if_empty |
| `longitude` | if_empty |
| `geofence_radius_meters` | if_empty |
| `disney_park` | if_empty |
| `is_disney` | if_empty |
| `dex_report_date` | if_empty |
| `install_date` | if_empty |
| `deployed` | if_empty |
| `pulled_date` | if_empty |
| `customer_id` | if_empty (via upsert customers table) |
| `location_id` | if_empty (via upsert locations table) |

> **Also creates/updates** `customers` and `locations` tables.

---

## ⚪ Admin Tools — Policy

### Batch Update (Cleanup page)
- `make`, `model`, `name`, `company`, `place`, `building_name`, `building_number`, `floor`, `room`, `location_area`, `address`, `category`, `status`, `customer_name`
- **Always overwrites** with user-provided value.

### Clear GPS
- `latitude`, `longitude` — set to NULL

### Mass Edit (existing tool)
- Same allowed fields as Batch Update.

---

## ⚫ Main App (Tech Field) — What Touches What

| Action | Fields Affected |
|--------|----------------|
| Barcode scan check-in | `updated_at` |
| GPS check-in | `latitude`, `longitude`, `updated_at` |
| Photo upload | `photo_path`, `updated_at` |
| OCR scan (name search) | (read-only) |
| Register new asset | All columns with defaults |

---

## Pipeline Activity Status

| Pipeline | Currently Scheduled? | Frequency | Risk to Cleanup |
|----------|--------------------|-----------|-----------------|
| **MSFS Sync** | 🔴 **Manual only** | On-demand via extraction viewer | ✅ GPS never touched. Preserves name/company/place/building/floor. Only `model` overwritten. |
| **Cantaloupe Sync** | 🔴 **No cron** | Manual upload on admin dashboard | ⚠️ Preserves GPS now (fixed — none from Cantaloupe). Still overwrites other fields. |
| **Seed Import** `--mode=update` | 🔴 **Manual CLI only** | `python3 main.py -m update` | ✅ **GPS never touched** (check-in only). Fills gaps only for other fields. |
| **Seed Import** `--mode=seed` | 🔴 **Manual CLI only** | `python3 main.py -m seed` | 🔥 **NUCLEAR** — `DELETE FROM assets`. Full re-insert. GPS set to NULL. |