Audit round 8: the export earns its publishing promises
Five blind reviewers over the day-old export stage; ~30 findings verified, the big ones sharing one root — a static-site generator makes promises a pipeline stage doesn't, and the first cut kept none of them. URL stability was empirically false two ways: adding an edition whose key sorted first STOLE the base slug (every colliding URL reshuffled), and removing the base holder renumbered survivors over the stale pages' corpses — wrong content at live URLs, not even 404s. And nothing ever deleted anything: removed games stayed published forever. One mechanism fixes all of it — a manifest (.bggpipe-export.json) in the output directory records which slugs the export owns and which source URL produced each cover. Slugs persist across runs (a published URL never moves and can never be stolen), stale pages are removed (only ever manifest-claimed ones — user files are not ours to touch), replaced box art re-fetches when its URL changes, and "art" is a reserved name so a game called Art can't move into the asset dir. Trust-the-network fixes: a 200 response must LOOK like an image (magic bytes + size) before it's cached, else a CDN interstitial became a permanent "cover" that re-runs skipped forever; downloads go through fsio.atomic_write_bytes instead of a hand-rolled fixed-tmp-name dance (the exact hazard fsio's own docstring warns about); a missing hand-added cover counts as a failure instead of silently shipping coverless; the badge file is sniffed too; CDN pacing raised to 1s and written into the spec as an adjudicated carve-out rather than a code comment's private opinion. Ship-shape: pages write atomically with the index LAST (a killed run can't publish links to pages that don't exist); the CLI exits nonzero on failures so `export && rsync` can't publish an incomplete site; footer/fine-print contrast now clears WCAG AA on the sky background; meta description, og:title/og:image and a favicon stop bare unfurls; the BGG link moved out of the h1; the noart tile is aria-hidden; the search box gained a no-matches message; numeric fields from enrich render instead of crashing the join; years and ids are escaped; the players/playtime formatters are aligned with their JS twins and both sides carry keep-in-sync constraint comments; export moved after enrich in the CLI listing. Twelve export tests now, including the previously-vacuous atomicity test rebuilt to actually interrupt a write. One honest loose end: one full cover re-fetch occurred during rollout that the identical naming code can't explain; the manifest's URL records make any recurrence diagnosable. 352 tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016jXZFSTZQKzAC8fqpWSz9g
This commit is contained in:
co-authored by
Claude Fable 5
parent
f291b9c190
commit
c7fdc60f87
@@ -18,6 +18,7 @@ Beyond the bare game, capture **which edition/version I own** wherever the photo
|
||||
- The XML API **requires a registered application** (boardgamegeek.com/using_the_xml_api, policy 2025-07-02): every request carries `Authorization: Bearer <token>` from the `BGG_API_TOKEN` env var, sent to `boardgamegeek.com` without a leading `www`. Register a free non-commercial application at boardgamegeek.com/applications — approval can take a week+; offline development and tests run on recorded XML fixtures.
|
||||
- BGG's XML API queues collection requests: a first call may return HTTP 202 ("try again"). Retry with backoff.
|
||||
- BGG will throttle aggressive clients. Target ≤1 request every 2 seconds to any BGG endpoint, with jittered backoff on 429/503.
|
||||
- Image-CDN carve-out (adjudicated 2026-08-06): cover downloads from BGG's image CDN (`cf.geekdo-images.com`) during `export` pace at ≥1 second apart, fetch only files not already on disk, and re-fetch a file only when its source URL changes. The CDN is not the API, but it is still BGG's infrastructure — the spacing is mandatory, not advisory.
|
||||
- BGG changed API access policies in 2025; some older community tools broke. Don't depend on undocumented endpoints beyond XML API2 and the public website.
|
||||
- Vision extraction uses the **Anthropic API** (Claude with vision). Assume `ANTHROPIC_API_KEY` in the environment.
|
||||
- Runs on macOS. Prefer **Python 3.12+** with `uv` for dependency management. Browser automation via **Playwright** (not Selenium).
|
||||
|
||||
Reference in New Issue
Block a user