Audit round 8: the export earns its publishing promises

Five blind reviewers over the day-old export stage; ~30 findings
verified, the big ones sharing one root — a static-site generator
makes promises a pipeline stage doesn't, and the first cut kept none
of them.

URL stability was empirically false two ways: adding an edition whose
key sorted first STOLE the base slug (every colliding URL reshuffled),
and removing the base holder renumbered survivors over the stale
pages' corpses — wrong content at live URLs, not even 404s. And
nothing ever deleted anything: removed games stayed published forever.
One mechanism fixes all of it — a manifest (.bggpipe-export.json) in
the output directory records which slugs the export owns and which
source URL produced each cover. Slugs persist across runs (a published
URL never moves and can never be stolen), stale pages are removed
(only ever manifest-claimed ones — user files are not ours to touch),
replaced box art re-fetches when its URL changes, and "art" is a
reserved name so a game called Art can't move into the asset dir.

Trust-the-network fixes: a 200 response must LOOK like an image (magic
bytes + size) before it's cached, else a CDN interstitial became a
permanent "cover" that re-runs skipped forever; downloads go through
fsio.atomic_write_bytes instead of a hand-rolled fixed-tmp-name dance
(the exact hazard fsio's own docstring warns about); a missing
hand-added cover counts as a failure instead of silently shipping
coverless; the badge file is sniffed too; CDN pacing raised to 1s and
written into the spec as an adjudicated carve-out rather than a code
comment's private opinion.

Ship-shape: pages write atomically with the index LAST (a killed run
can't publish links to pages that don't exist); the CLI exits nonzero
on failures so `export && rsync` can't publish an incomplete site;
footer/fine-print contrast now clears WCAG AA on the sky background;
meta description, og:title/og:image and a favicon stop bare unfurls;
the BGG link moved out of the h1; the noart tile is aria-hidden; the
search box gained a no-matches message; numeric fields from enrich
render instead of crashing the join; years and ids are escaped; the
players/playtime formatters are aligned with their JS twins and both
sides carry keep-in-sync constraint comments; export moved after
enrich in the CLI listing.

Twelve export tests now, including the previously-vacuous atomicity
test rebuilt to actually interrupt a write. One honest loose end: one
full cover re-fetch occurred during rollout that the identical naming
code can't explain; the manifest's URL records make any recurrence
diagnosable. 352 tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016jXZFSTZQKzAC8fqpWSz9g
This commit is contained in:
Eric Wagoner
2026-08-06 18:51:05 -04:00
co-authored by Claude Fable 5
parent f291b9c190
commit c7fdc60f87
5 changed files with 385 additions and 89 deletions
+1
View File
@@ -18,6 +18,7 @@ Beyond the bare game, capture **which edition/version I own** wherever the photo
- The XML API **requires a registered application** (boardgamegeek.com/using_the_xml_api, policy 2025-07-02): every request carries `Authorization: Bearer <token>` from the `BGG_API_TOKEN` env var, sent to `boardgamegeek.com` without a leading `www`. Register a free non-commercial application at boardgamegeek.com/applications — approval can take a week+; offline development and tests run on recorded XML fixtures.
- BGG's XML API queues collection requests: a first call may return HTTP 202 ("try again"). Retry with backoff.
- BGG will throttle aggressive clients. Target ≤1 request every 2 seconds to any BGG endpoint, with jittered backoff on 429/503.
- Image-CDN carve-out (adjudicated 2026-08-06): cover downloads from BGG's image CDN (`cf.geekdo-images.com`) during `export` pace at ≥1 second apart, fetch only files not already on disk, and re-fetch a file only when its source URL changes. The CDN is not the API, but it is still BGG's infrastructure — the spacing is mandatory, not advisory.
- BGG changed API access policies in 2025; some older community tools broke. Don't depend on undocumented endpoints beyond XML API2 and the public website.
- Vision extraction uses the **Anthropic API** (Claude with vision). Assume `ANTHROPIC_API_KEY` in the environment.
- Runs on macOS. Prefer **Python 3.12+** with `uv` for dependency management. Browser automation via **Playwright** (not Selenium).