Diff stage: snapshot/live collection modes, to_add + to_update outputs
compute_diff is a pure function over matches + collection items: new games land in to_add.csv (carrying a confident version when matching produced one); owned version-less entries with version_auto/approved matches produce additive to_update.csv rows keyed by collid, consuming distinct collids when photos show two editions; entries that already carry a version are never touched — disagreements are reported in the summary. Unseen collection entries are listed informationally. Live API mode activates when BGG_API_TOKEN + username exist; otherwise the two hand-pulled snapshot XMLs (real 2018 collection, 79 unique copies after collid dedupe) are used, and they double as parsing fixtures in tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -77,13 +77,16 @@ Each stage reads the previous stage's artifact and writes its own. Re-running a
|
||||
- Fetch my current collection: `https://boardgamegeek.com/xmlapi2/collection?username=<me>&own=1` (handle the 202-retry queue; also pass `&subtype=boardgameexpansion` in a second call — the collection endpoint excludes expansions from the default subtype).
|
||||
- Output `to_add.csv`: approved/auto matches whose IDs are **not** already in the collection.
|
||||
- **Multiple editions of the same game**: collection items are identified by `collid` (one per copy), not just `objectid`. If matches contain two entries for the same `bgg_id` with different `version_id`s, both belong in the collection as separate entries. Diff logic: a (bgg_id, version_id) pair is "already owned" only if a collection item matches both; a bare bgg_id with `version_unknown` is "already owned" if any copy of that game exists.
|
||||
- Print a summary: N recognized, N already owned, N to add (including second editions), N rejected/unmatched.
|
||||
- **Improvement pass (to_update)**: for games already owned whose collection entry has NO version set, where photo matching produced a `version_auto`/`version_approved` — emit `to_update.csv` (`collid, bgg_id, bgg_name, version_id, version_name`). This upgrades the hand-entered 2018 entries with edition data from the shelves. Strictly additive: only fill empty version fields; if the collection entry already has a version, never touch it (even if the photo disagrees — report the disagreement in the summary instead).
|
||||
- Informational only: list collection entries not seen in any photo (possible missing/loaned/sold games) in the summary. No action taken.
|
||||
- Print a summary: N recognized, N already owned, N to add (including second editions), N version updates, N rejected/unmatched.
|
||||
|
||||
### Stage 5 — `upload`: Add games via Playwright
|
||||
|
||||
- Log in to boardgamegeek.com with credentials from env vars (`BGG_USERNAME`, `BGG_PASSWORD`). Never write credentials to disk or logs. Persist the browser session/storage state locally so repeat runs don't re-login.
|
||||
- For each row in `to_add.csv`: navigate to the game page, use the "Add to Collection" flow, set status **Owned**, and — when a `version_id` is present — set the specific version in the collection item's version picker before saving. Manually walk this flow once and document the selectors before automating; the version UI is the most fragile part.
|
||||
- Adding a second copy of an already-owned game must create a NEW collection entry, not edit the existing one.
|
||||
- **Update mode** (rows from `to_update.csv`): open the EXISTING collection entry (keyed by `collid`) rather than the add flow, set the version, save. Must never create a duplicate entry and never change any other field of the entry. Verify the already-owned dialog behavior manually first — flagged as unverified in `docs/bgg-upload-flow.md`.
|
||||
- Log every attempt to `upload_log.csv`: `bgg_id, name, status (added|already_present|failed), timestamp, error`.
|
||||
- Idempotent: skip IDs already logged as `added`; re-verify against a fresh collection fetch on `--verify`.
|
||||
- Deliberately slow: 2–4 s randomized delay between games. This is a real account on a community site — behave like a polite human.
|
||||
@@ -103,7 +106,8 @@ All artifacts are flat files in a `data/` directory — human-readable, git-frie
|
||||
- `titles.json` — extraction output (stage 1)
|
||||
- `bgg_cache/` — cached XML API responses
|
||||
- `matches.csv` — the master matching table (stages 2–3)
|
||||
- `to_add.csv` — upload queue (stage 4)
|
||||
- `to_add.csv` — upload queue, new entries (stage 4)
|
||||
- `to_update.csv` — version upgrades for existing version-less entries (stage 4)
|
||||
- `upload_log.csv` — audit trail (stage 5)
|
||||
- `games.json` — full game + version metadata (stage 6); seed data for the future frontend
|
||||
|
||||
@@ -130,6 +134,7 @@ All artifacts are flat files in a `data/` directory — human-readable, git-frie
|
||||
6. Credentials never appear in any file, log, or error message.
|
||||
7. Where photos show legible edition cues, the matched version survives to the BGG collection entry; where they don't, the entry is added version-less rather than with a guessed version.
|
||||
8. A game I own in two editions ends up as two distinct collection entries, and `games.json` contains full metadata for every game in the collection.
|
||||
9. Version upgrades land on existing collection entries (same `collid`) with no duplicate entries created and no non-version fields changed; entries that already have a version are never modified.
|
||||
|
||||
## Suggested Build Order
|
||||
|
||||
|
||||
Reference in New Issue
Block a user