From 96a1e09956cd5578dc8eac66b295900e059673f6 Mon Sep 17 00:00:00 2001 From: Eric Wagoner Date: Sat, 1 Aug 2026 13:05:15 -0400 Subject: [PATCH] Docs: BGG auth policy in spec/README, upload-flow recon notes Co-Authored-By: Claude Fable 5 --- .claude/skills/bgg-api/SKILL.md | 8 +++- README.md | 6 +-- bgg-shelf-pipeline-spec.md | 1 + docs/bgg-upload-flow.md | 80 +++++++++++++++++++++++++++++++++ 4 files changed, 90 insertions(+), 5 deletions(-) create mode 100644 docs/bgg-upload-flow.md diff --git a/.claude/skills/bgg-api/SKILL.md b/.claude/skills/bgg-api/SKILL.md index 850dc31..b493739 100644 --- a/.claude/skills/bgg-api/SKILL.md +++ b/.claude/skills/bgg-api/SKILL.md @@ -17,6 +17,10 @@ put it in config.toml, code, or logs. Requests must go to Exception: downloading your own collection while logged in on the website needs no registration — relevant to the Playwright stages, not the API client. Usage is monitored per-application at `/applications` → "Usage". +Open-source note: each user of this tool registers their OWN application +and supplies their own token — never ship or share a token in the repo. +Future frontend note: public-facing apps must display the "Powered by +BGG" logo linking back to boardgamegeek.com. ## Endpoints (XML API2 — the only sanctioned read API) @@ -55,11 +59,11 @@ The first `/collection` call typically returns **HTTP 202** with a "please retry ## Website automation (upload stage) - Login with `BGG_USERNAME` / `BGG_PASSWORD` env vars; persist Playwright storage state locally (gitignored) so repeat runs skip login. Never write credentials to disk, logs, or error messages. -- Per game: navigate to the game page → "Add to Collection" flow → status **Owned** → if a version_id is known, set it in the collection item's version picker → save. Never guess a version — omit it when unknown. The version picker is the most fragile part of the UI: walk it manually once and document the selectors before automating. +- Per game: navigate to the game page → "Add to Collection" flow → status **Owned** → if a version_id is known, set it in the collection item's version picker → save. Never guess a version — omit it when unknown. The flow has been walked and documented: see `docs/bgg-upload-flow.md` for the dialog structure, version-picker behavior (paginated, no search — match by canonical version NAME from the XML API), and observed automation gotchas (stale elements, hidden-not-removed dialogs, hydration races). - A second copy of an owned game must be a NEW collection entry (new collid), not an edit of the existing item. - Expect UI fragility: wrap each game in its own try/except, log the failure to `upload_log.csv`, and continue. `--retry-failed` re-attempts failures; `--dry-run` logs without touching the site. - Idempotency: skip IDs already logged `added`; `--verify` re-fetches the collection to confirm. ## Policy note -BGG changed API access policies in 2025 and broke older community tools. Do not use undocumented endpoints, scraped JSON blobs, or third-party mirrors — only XML API2 and the public website. +BGG's 2025 policy (version 2025-07-02, boardgamegeek.com/using_the_xml_api) locked the API behind registered applications and Bearer tokens, breaking older community tools. Do not use undocumented endpoints, scraped JSON blobs, or third-party mirrors — only the authenticated XML API2 and the public website. Third-party services that proxy BGG data to other applications are explicitly prohibited. The policy asks for server-side requests, aggressive caching, and minimal request counts — the cache-first client design is mandatory, not optional. Policies can change at any time; changes are announced in BGG's Geek Tools News forum. diff --git a/README.md b/README.md index fa38e29..bae58bb 100644 --- a/README.md +++ b/README.md @@ -30,14 +30,14 @@ Every stage is idempotent and resumable: kill it mid-run, restart, lose nothing. - macOS or Linux, Python 3.12+, [uv](https://docs.astral.sh/uv/) - An [Anthropic API key](https://console.anthropic.com/) (vision extraction) -- A BoardGameGeek account +- A BoardGameGeek account **and a registered BGG application** — as of BGG's [2025 API policy](https://boardgamegeek.com/using_the_xml_api), the XML API requires a Bearer token from a registered app. Register a free non-commercial application at [boardgamegeek.com/applications](https://boardgamegeek.com/applications) (approval can take a week or more — apply early), then create a token. Each user needs their own — tokens must not be shared. - Playwright Chromium: `uv run playwright install chromium` -Secrets come from environment variables only — `ANTHROPIC_API_KEY`, `BGG_USERNAME`, `BGG_PASSWORD` — and are never written to disk or logs. +Secrets come from environment variables only — `ANTHROPIC_API_KEY`, `BGG_USERNAME`, `BGG_PASSWORD`, `BGG_API_TOKEN` — and are never written to disk or logs. ## A note on being a good BGG citizen -This tool is **not affiliated with or supported by BoardGameGeek**. It uses only the sanctioned XML API2 for reads and drives the regular website for writes, deliberately slowly (one request every couple of seconds, slower for uploads). Please keep it that way: BGG is a community resource running on community goodwill. You are responsible for your own account — review the dry-run output before a real upload. +This tool is **not affiliated with or supported by BoardGameGeek**. It uses only the sanctioned XML API2 for reads (with your own registered application token, per BGG's current policy) and drives the regular website for writes, deliberately slowly (one request every couple of seconds, slower for uploads). Please keep it that way: BGG is a community resource running on community goodwill. You are responsible for your own account — review the dry-run output before a real upload. ## License diff --git a/bgg-shelf-pipeline-spec.md b/bgg-shelf-pipeline-spec.md index 2028af4..95c36b4 100644 --- a/bgg-shelf-pipeline-spec.md +++ b/bgg-shelf-pipeline-spec.md @@ -11,6 +11,7 @@ Beyond the bare game, capture **which edition/version I own** wherever the photo ## Constraints & Context - BGG has **no write API**. Reads go through the XML API2 (`https://boardgamegeek.com/xmlapi2/`); writes must automate the website itself with a real login session. +- The XML API **requires a registered application** (boardgamegeek.com/using_the_xml_api, policy 2025-07-02): every request carries `Authorization: Bearer ` from the `BGG_API_TOKEN` env var, sent to `boardgamegeek.com` without a leading `www`. Register a free non-commercial application at boardgamegeek.com/applications — approval can take a week+, so development runs on recorded/stub XML fixtures until then. - BGG's XML API queues collection requests: a first call may return HTTP 202 ("try again"). Retry with backoff. - BGG will throttle aggressive clients. Target ≤1 request every 2 seconds to any BGG endpoint, with jittered backoff on 429/503. - BGG changed API access policies in 2025; some older community tools broke. Don't depend on undocumented endpoints beyond XML API2 and the public website. diff --git a/docs/bgg-upload-flow.md b/docs/bgg-upload-flow.md new file mode 100644 index 0000000..07ed3a4 --- /dev/null +++ b/docs/bgg-upload-flow.md @@ -0,0 +1,80 @@ +# BGG "Add to Collection" flow — recon notes for the upload stage + +Recorded 2026-08-01 by walking the flow manually on boardgamegeek.com (Wingspan, +id 266192) while logged in. Dialog opened, inspected, and **cancelled without +saving**. These notes are the selector documentation the spec requires before +automating Stage 5. + +## Entry point + +- Game page has **two** "Add To" buttons (one in the game header module, one + lower on the page) — target the first, but match by accessible name, not + position. Button's accessible name is "Add To" with adjacent text + "Collection". +- Clicking opens a `role="dialog"` containing a `
`. Dialog heading shows + a "Loading..." span before content settles — **wait for the game-name + heading** (e.g. `getByRole('heading', {name: gameName})`) before + interacting. + +## Main dialog structure + +- **Status checkboxes**, each wrapped in a `