Eric WagonerandClaude Fable 5
1dd72d2688
Eric's question cut to the bone: "How would a user know? It matched
wiz-war and that IS the game." The auto looked unanimous because the
matcher discarded the evidence of doubt before anyone saw it — and
worse, BGG's search hides evidence of its own: results truncate
unordered in the several-hundreds (the game named "Dungeon!" appears
in NEITHER the "Dungeon!" nor the "Dungeon" search), and punctuation
can bury matches.
Three matcher changes: every title is searched raw AND depuncted,
merged by id; a name that becomes exact once its trailing
parenthetical is stripped ("Wiz-War (Eighth Edition)") is a sibling
edition — BGG files new editions as separate games — and enters the
candidate set at exact grade, so same-named lineages land in review as
a visible choice; and a LONE candidate must now earn trust (stats
fetched, sibling-grade never autos alone, true exacts must clear the
dominance ownership floor) — closing the fast path both impostors
(.dungeon at 31 owners, then Dungeon (ICP)) walked through.
Recorded outcomes: WIZ-WAR → ambiguous with all three lineages on the
ballot; Dungeon! → ambiguous (its true match is beyond BGG's search
horizon — that's what manual id is for); every legitimate auto in the
fixture set held. And the answer to Eric's second question is now
structural: re-match never re-decides — it demotes to unmatched and
the HUMAN picks from re-search or manual id; the machine only chooses
on first resolve, and it now chooses more humbly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016jXZFSTZQKzAC8fqpWSz9g
84 lines
7.4 KiB
Markdown
84 lines
7.4 KiB
Markdown
---
|
||
name: bgg-api
|
||
description: Reference for BoardGameGeek's XML API2 and website automation — endpoints, 202 queueing, rate limits, collection quirks, upload etiquette. Use when writing or debugging any code that talks to boardgamegeek.com (resolve, diff, or upload stages).
|
||
---
|
||
|
||
# BoardGameGeek API & site automation reference
|
||
|
||
## Authentication (required since 2025)
|
||
|
||
Every XML API request must carry `Authorization: Bearer <token>` or BGG
|
||
returns **401 Unauthorized**. Tokens come from a registered application:
|
||
create one at `https://boardgamegeek.com/applications` (non-commercial
|
||
license is free; approval can take a week or more), then generate a token
|
||
under "Tokens". `bggpipe` reads it from the `BGG_API_TOKEN` env var — never
|
||
put it in config.toml, code, or logs. Requests must go to
|
||
`boardgamegeek.com` **without** a leading `www` or the token is ignored.
|
||
Exception: downloading your own collection while logged in on the website
|
||
needs no registration — relevant to the Playwright stages, not the API
|
||
client. Usage is monitored per-application at `/applications` → "Usage".
|
||
Open-source note: each user of this tool registers their OWN application
|
||
and supplies their own token — never ship or share a token in the repo.
|
||
Future frontend note: public-facing apps must display the "Powered by
|
||
BGG" logo linking back to boardgamegeek.com.
|
||
|
||
## Endpoints (XML API2 — the only sanctioned read API)
|
||
|
||
- Search: `https://boardgamegeek.com/xmlapi2/search?query=<title>&type=boardgame,boardgameexpansion`
|
||
- RPG search: same endpoint with `type=rpgitem` — the geekdo database is shared across BGG/RPGGeek, so the same API and token serve RPG products. bggpipe uses this as a fallback for titles absent from the board-game types; matched rpgitems are LOCAL-ONLY (enriched, browsable, never uploaded — RPG collections live on rpggeek.com, outside this pipeline's write scope).
|
||
- Thing (details/stats): `https://boardgamegeek.com/xmlapi2/thing?id=<id1,id2,...>&stats=1` — accepts comma-separated IDs; batch (~20) to reduce request count.
|
||
- Thing versions: `https://boardgamegeek.com/xmlapi2/thing?id=<id>&versions=1` — lists every published edition/version of a game, each with its own version id, name, publisher, year, and language. Used to match photo edition cues to a concrete version.
|
||
- Collection: `https://boardgamegeek.com/xmlapi2/collection?username=<user>&own=1` — add `&version=1` to include version info on collection items.
|
||
|
||
All responses are XML. There is no JSON API and **no write API** — writes automate the website via Playwright with a logged-in session.
|
||
|
||
## HTTP 202 queueing (collection endpoint)
|
||
|
||
The first `/collection` call typically returns **HTTP 202** with a "please retry" message: BGG queues the export and serves it on a later request. Retry schedule: **2s, 5s, 10s, 30s, give up after ~5 tries** with a clear error message. Treat 202 as normal flow, not an error.
|
||
|
||
## Collection endpoint quirks
|
||
|
||
- The default subtype **excludes expansions**. Make a second call with `&subtype=boardgameexpansion` and merge, or owned expansions will be invisible to `diff` and re-uploaded.
|
||
- `own=1` filters to owned items; other statuses (wishlist, previously owned) exist and must not be counted as owned.
|
||
- Each collection item has a **`collid`** (unique per copy) alongside `objectid` (the game). Owning two editions of one game = two items, same `objectid`, different `collid`s. Diff on (objectid, version) pairs, not bare objectid, when versions are known.
|
||
- With `&version=1`, items that have a version set include it; items without one simply don't — version-less entries are legal and common.
|
||
|
||
## Rate limiting
|
||
|
||
- **≤1 request every 2 seconds** to any BGG endpoint (API or website). Jittered exponential backoff on 429/503.
|
||
- Cache every API response on disk under `data/bgg_cache/` keyed by query/ID; check cache before hitting the network so re-runs are free.
|
||
- Upload stage is deliberately slower: **2–4 s randomized delay** between games — this is a real account on a community site.
|
||
|
||
## Search & matching heuristics
|
||
|
||
0. BGG's search is UNRELIABLE for generic and punctuated queries: results
|
||
truncate in the several-hundreds unordered (the game named "Dungeon!"
|
||
appears in neither the "Dungeon!" nor the "Dungeon" search), and
|
||
punctuation can hide matches. Every title is searched raw AND with
|
||
punctuation stripped, merged by id. A LONE surviving candidate never
|
||
auto-matches unless it's a true exact match whose owned-count clears
|
||
the dominance floor — the sole survivor may be an impostor standing
|
||
where a famous game should be, and the real match may need review's
|
||
manual-id entry.
|
||
1. Exact normalized-name match → strong candidate. A name that becomes
|
||
exact once its trailing parenthetical is stripped ("Wiz-War (Eighth
|
||
Edition)") is a SIBLING EDITION — BGG files new editions as separate
|
||
games — and counts as exact-grade so the choice between lineages
|
||
reaches review instead of hiding behind a confident auto.
|
||
2. Fuzzy match: `rapidfuzz` `token_sort_ratio ≥ 90` → good candidate.
|
||
3. Ties: fetch `/thing?stats=1` for top ~5 candidates; prefer higher owned-count / better BGG rank (well-known game beats obscure duplicate of the same name).
|
||
- Search results include `boardgameexpansion` as a distinct `type` — keep expansions but tag them so review catches base/expansion confusion.
|
||
- Classify every result: `auto` (single confident match) / `ambiguous` (store all candidates) / `unmatched`. When in doubt between editions or base-vs-expansion, choose `ambiguous`.
|
||
|
||
## Website automation (upload stage)
|
||
|
||
- Login with `BGG_USERNAME` / `BGG_PASSWORD` env vars; persist Playwright storage state locally (gitignored) so repeat runs skip login. Never write credentials to disk, logs, or error messages.
|
||
- Per game: navigate to the game page → "Add to Collection" flow → status **Owned** → if a version_id is known, set it in the collection item's version picker → save. Never guess a version — omit it when unknown. The flow has been walked and documented: see `docs/bgg-upload-flow.md` for the dialog structure, version-picker behavior (paginated, no search — match by canonical version NAME from the XML API), and observed automation gotchas (stale elements, hidden-not-removed dialogs, hydration races).
|
||
- A second copy of an owned game must be a NEW collection entry (new collid), not an edit of the existing item. Conversely, version UPGRADES from to_update.csv must edit the EXISTING item (same collid) — additive only: fill the empty version field, change nothing else, and skip any entry that already has a version.
|
||
- Expect UI fragility: wrap each game in its own try/except, log the failure to `upload_log.csv`, and continue. `--retry-failed` re-attempts failures; `--dry-run` logs without touching the site.
|
||
- Idempotency: skip IDs already logged `added`; `--verify` re-fetches the collection to confirm.
|
||
|
||
## Policy note
|
||
|
||
BGG's 2025 policy (version 2025-07-02, boardgamegeek.com/using_the_xml_api) locked the API behind registered applications and Bearer tokens, breaking older community tools. Do not use undocumented endpoints, scraped JSON blobs, or third-party mirrors — only the authenticated XML API2 and the public website. Third-party services that proxy BGG data to other applications are explicitly prohibited. The policy asks for server-side requests, aggressive caching, and minimal request counts — the cache-first client design is mandatory, not optional. Policies can change at any time; changes are announced in BGG's Geek Tools News forum.
|