eric/bggpipe
1
0
Fork 0
Code Issues Pull Requests Releases 2 Activity

RPGs pull real RPGGeek data; off-BGG games get facts and a cover photo

Browse Source 
Two gaps at the edges of the library, both closed.

RPGGeek items live in the same database but use their own link types —
rpgdesigner, rpgpublisher, rpggenre, rpgcategory, rpgmechanic — so a
board-game-only parser found none of them and both RPG entries showed
just a year and a description. parse_things_full now reads both
vocabularies (plus rpgproducer/rpgseries): .dungeon gains John Battle
and Project Nerves, Parsely gains Jared A. Sorensen and its genres.

An off-BGG game has no API to enrich it and no publisher art to fetch,
so its detail page now hosts the only source it will ever have: a form
for title, year, players, playing time, publishers, designers and
notes, plus a cover photo upload. Both persist in data/local_games.json
and data/local_art/ (committed, like every other curation store) and
enrich merges them over the photo reads, so a rebuild can't erase them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016jXZFSTZQKzAC8fqpWSz9g
This commit is contained in:
Eric Wagoner
2026-08-05 23:45:09 -04:00
co-authored by Claude Fable 5
parent f5e949bd62
commit 7e95ed607d
10 changed files with 582 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+3 -3
View File
@@ -47,11 +47,11 @@ Full design lives in `bgg-shelf-pipeline-spec.md` (read it before changing pipel
- Base game vs. expansion vs. new edition is the top failure mode — bias matching toward `ambiguous` over auto-match ("Wingspan Europe" must not match base Wingspan).
- Editions/versions matter: Eric owns multiple editions of some games — each is a separate collection entry (keyed by `collid` on BGG). Never guess a version: no legible cues → `version_unknown` and a version-less collection entry.
- Normalize titles (casefold, strip punctuation/articles, special chars like é/&/:) identically on both sides of a match; dedupe across photos but keep `source_photos` provenance.
- **Human curation is durable**: `data/title_splits.json` (photo-scoped split-into-copies decisions, honored by extract's dedupe AND resolve's dedupe), `data/title_edits.json` (corrected reads/cues, applied before dedupe on every titles.json rebuild), `data/title_removals.json` (lines removed from the catalog — filtered out of every rebuild; delete the record to undo), and `data/title_additions.json` (games added without a photo — joined into every rebuild; a later photo sighting dedupe-merges with them) persist forever. Row-level decisions persist via the `dedupe_veto` column — edits never drop veto'd rows (a rename retitles them in place); removal drops them (explicitly discarding the line).
- **RPGs are local-only citizens**: when the board-game search runs dry, resolve falls back to `type=rpgitem` (same geekdo API/token). Matched rpgitems enrich into the library but diff routes them to `local_only` — they must never reach `to_add.csv`/upload (their collection lives on RPGGeek, out of scope).
- **Human curation is durable**: `data/title_splits.json` (photo-scoped split-into-copies decisions, honored by extract's dedupe AND resolve's dedupe), `data/title_edits.json` (corrected reads/cues, applied before dedupe on every titles.json rebuild), `data/title_removals.json` (lines removed from the catalog — filtered out of every rebuild; delete the record to undo), `data/title_additions.json` (games added without a photo — joined into every rebuild; a later photo sighting dedupe-merges with them), and `data/local_games.json` + `data/local_art/` (hand-written facts and a cover photo for off-BGG games — the ONLY source for them, merged over the photo reads by enrich) persist forever. Row-level decisions persist via the `dedupe_veto` column — edits never drop veto'd rows (a rename retitles them in place); removal drops them (explicitly discarding the line).
- **RPGs are local-only citizens**: when the board-game search runs dry, resolve falls back to `type=rpgitem` (same geekdo API/token). RPGGeek items carry their OWN link types (`rpgdesigner`, `rpgpublisher`, `rpggenre`, `rpgcategory`, `rpgmechanic`) — a board-game-only parser silently returns nothing for them. Matched rpgitems enrich into the library but diff routes them to `local_only` — they must never reach `to_add.csv`/upload (their collection lives on RPGGeek, out of scope).
- Detailed BGG API behavior (202 queueing, collection-endpoint quirks, endpoints): use the `bgg-api` skill. **If the spec's BGG behavior changes, update the `bgg-api` skill to match** — they must not drift.
## Git
- Remote is self-hosted Gitea 1.26 (`git.kestrelsnest.social/eric/bggpipe`), **not GitHub** — `gh` CLI does not work here.
- Commit `data/matches.csv`, `data/to_add.csv`, `data/to_update.csv`, `data/upload_log.csv`, `data/titles.json`, `data/unidentified.json`, `data/unidentified_dismissed.json`, `data/title_splits.json`, `data/title_edits.json`, `data/title_removals.json`, `data/title_additions.json`, `data/games.json`, `data/STUB_DATA.marker` (while it applies), and the collection snapshot XMLs. Never commit `data/bgg_cache/`, `data/extract_raw/`, `photos/`, `data/.lan_key`, Playwright storage state, or `.env`.
- Commit `data/matches.csv`, `data/to_add.csv`, `data/to_update.csv`, `data/upload_log.csv`, `data/titles.json`, `data/unidentified.json`, `data/unidentified_dismissed.json`, `data/title_splits.json`, `data/title_edits.json`, `data/title_removals.json`, `data/title_additions.json`, `data/local_games.json`, `data/local_art/`, `data/games.json`, `data/STUB_DATA.marker` (while it applies), and the collection snapshot XMLs. Never commit `data/bgg_cache/`, `data/extract_raw/`, `photos/`, `data/.lan_key`, Playwright storage state, or `.env`.