Eric Wagoner
2109e3544a
bggpipe resolve works end to end: search -> exact/fuzzy candidate scoring -> auto/ambiguous/unmatched classification with owned-count tie-breaks (mixed base/expansion candidates never auto-match), version scoring from edition cues (never guessed; no cues -> version_unknown), idempotent matches.csv appends. Discovered mid-build: BGG now requires registered-application Bearer tokens on the XML API (2025 policy change) and returns 401 otherwise. Client sends Authorization from BGG_API_TOKEN and raises an actionable BGGAuthError; CLAUDE.md and the bgg-api skill are updated to match. Live fixture recording is blocked until registration is approved, so tests replay hand-crafted stub fixtures via a network-refusing transport; scripts/record_fixtures.py re-records real XML under the same cache keys once a token exists. One live read-only smoke test is skipped unless --run-live. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
66 lines
5.1 KiB
Markdown
66 lines
5.1 KiB
Markdown
---
|
||
name: bgg-api
|
||
description: Reference for BoardGameGeek's XML API2 and website automation — endpoints, 202 queueing, rate limits, collection quirks, upload etiquette. Use when writing or debugging any code that talks to boardgamegeek.com (resolve, diff, or upload stages).
|
||
---
|
||
|
||
# BoardGameGeek API & site automation reference
|
||
|
||
## Authentication (required since 2025)
|
||
|
||
Every XML API request must carry `Authorization: Bearer <token>` or BGG
|
||
returns **401 Unauthorized**. Tokens come from a registered application:
|
||
create one at `https://boardgamegeek.com/applications` (non-commercial
|
||
license is free; approval can take a week or more), then generate a token
|
||
under "Tokens". `bggpipe` reads it from the `BGG_API_TOKEN` env var — never
|
||
put it in config.toml, code, or logs. Requests must go to
|
||
`boardgamegeek.com` **without** a leading `www` or the token is ignored.
|
||
Exception: downloading your own collection while logged in on the website
|
||
needs no registration — relevant to the Playwright stages, not the API
|
||
client. Usage is monitored per-application at `/applications` → "Usage".
|
||
|
||
## Endpoints (XML API2 — the only sanctioned read API)
|
||
|
||
- Search: `https://boardgamegeek.com/xmlapi2/search?query=<title>&type=boardgame,boardgameexpansion`
|
||
- Thing (details/stats): `https://boardgamegeek.com/xmlapi2/thing?id=<id1,id2,...>&stats=1` — accepts comma-separated IDs; batch (~20) to reduce request count.
|
||
- Thing versions: `https://boardgamegeek.com/xmlapi2/thing?id=<id>&versions=1` — lists every published edition/version of a game, each with its own version id, name, publisher, year, and language. Used to match photo edition cues to a concrete version.
|
||
- Collection: `https://boardgamegeek.com/xmlapi2/collection?username=<user>&own=1` — add `&version=1` to include version info on collection items.
|
||
|
||
All responses are XML. There is no JSON API and **no write API** — writes automate the website via Playwright with a logged-in session.
|
||
|
||
## HTTP 202 queueing (collection endpoint)
|
||
|
||
The first `/collection` call typically returns **HTTP 202** with a "please retry" message: BGG queues the export and serves it on a later request. Retry schedule: **2s, 5s, 10s, 30s, give up after ~5 tries** with a clear error message. Treat 202 as normal flow, not an error.
|
||
|
||
## Collection endpoint quirks
|
||
|
||
- The default subtype **excludes expansions**. Make a second call with `&subtype=boardgameexpansion` and merge, or owned expansions will be invisible to `diff` and re-uploaded.
|
||
- `own=1` filters to owned items; other statuses (wishlist, previously owned) exist and must not be counted as owned.
|
||
- Each collection item has a **`collid`** (unique per copy) alongside `objectid` (the game). Owning two editions of one game = two items, same `objectid`, different `collid`s. Diff on (objectid, version) pairs, not bare objectid, when versions are known.
|
||
- With `&version=1`, items that have a version set include it; items without one simply don't — version-less entries are legal and common.
|
||
|
||
## Rate limiting
|
||
|
||
- **≤1 request every 2 seconds** to any BGG endpoint (API or website). Jittered exponential backoff on 429/503.
|
||
- Cache every API response on disk under `data/bgg_cache/` keyed by query/ID; check cache before hitting the network so re-runs are free.
|
||
- Upload stage is deliberately slower: **2–4 s randomized delay** between games — this is a real account on a community site.
|
||
|
||
## Search & matching heuristics
|
||
|
||
1. Exact normalized-name match → strong candidate.
|
||
2. Fuzzy match: `rapidfuzz` `token_sort_ratio ≥ 90` → good candidate.
|
||
3. Ties: fetch `/thing?stats=1` for top ~5 candidates; prefer higher owned-count / better BGG rank (well-known game beats obscure duplicate of the same name).
|
||
- Search results include `boardgameexpansion` as a distinct `type` — keep expansions but tag them so review catches base/expansion confusion.
|
||
- Classify every result: `auto` (single confident match) / `ambiguous` (store all candidates) / `unmatched`. When in doubt between editions or base-vs-expansion, choose `ambiguous`.
|
||
|
||
## Website automation (upload stage)
|
||
|
||
- Login with `BGG_USERNAME` / `BGG_PASSWORD` env vars; persist Playwright storage state locally (gitignored) so repeat runs skip login. Never write credentials to disk, logs, or error messages.
|
||
- Per game: navigate to the game page → "Add to Collection" flow → status **Owned** → if a version_id is known, set it in the collection item's version picker → save. Never guess a version — omit it when unknown. The version picker is the most fragile part of the UI: walk it manually once and document the selectors before automating.
|
||
- A second copy of an owned game must be a NEW collection entry (new collid), not an edit of the existing item.
|
||
- Expect UI fragility: wrap each game in its own try/except, log the failure to `upload_log.csv`, and continue. `--retry-failed` re-attempts failures; `--dry-run` logs without touching the site.
|
||
- Idempotency: skip IDs already logged `added`; `--verify` re-fetches the collection to confirm.
|
||
|
||
## Policy note
|
||
|
||
BGG changed API access policies in 2025 and broke older community tools. Do not use undocumented endpoints, scraped JSON blobs, or third-party mirrors — only XML API2 and the public website.
|