Eric WagonerandClaude Fable 5
6041150c8d
Eric's read on the first-visitor experience: 190 lines of pitch, manual, gallery, and contributor doc is intimidating when the visitor only needs the first 40. Split three ways: README.md is now the front door — what it is, why it exists (told in first person now, since it IS a personal itch scratched), how the six stages work, requirements, quick start, one hero screenshot, and the development/citizenship/license notes. Sixty percent shorter. docs/tour.md carries the full gallery: all seven pages, the game detail view, and the phone set, captions intact. docs/guide.md is the complete user's guide: credentials and config, the stages and their flags, phone pairing, photo technique, curation stores, RPG handling, upload safety (including the collection-export lag), the no-token-yet path, and the keep-data-out-of-git rationale. Every relative link and README→guide anchor machine-verified to resolve. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016jXZFSTZQKzAC8fqpWSz9g
80 lines
6.7 KiB
Markdown
80 lines
6.7 KiB
Markdown
# bggpipe — Shelf-to-BoardGameGeek Collection Pipeline
|
|
|
|
<img src="assets/logo-full.jpeg" alt="the bggpipe piper — a bagpiper whose bag is a board game box" width="220" align="right">
|
|
|
|
Photograph your board game shelves. End up with your whole collection — including which *edition* of each game you own — cataloged on [BoardGameGeek](https://boardgamegeek.com).
|
|
|
|
```
|
|
photos/ → [1 extract] → titles.json → [2 resolve] → matches.csv
|
|
→ [3 review] → matches.csv (approved) → [4 diff] → to_add.csv
|
|
→ [5 upload] → upload_log.csv
|
|
→ [6 enrich] → games.json
|
|
```
|
|
|
|
> **Status: battle-tested end to end.** The full pipeline has run against a live BGG account: shelf photos → 136 identified games → 62 additions and 36 version updates on a real collection. Still sensible on a first run: `--dry-run`, then `--limit 1`.
|
|
|
|
## Why this exists
|
|
|
|
BGG has no bulk import and no write API. Cataloging a few hundred games by hand means hours of searching, clicking, and second-guessing which of five editions you own. I wanted to point a camera at my shelves instead. This pipeline replaces the typing with: take photos, run a command, settle a handful of ambiguous matches in a review step, done — and every judgment call along the way is yours, made in a review UI, never guessed by the machine.
|
|
|
|
## How it works
|
|
|
|
1. **extract** — Shelf photos go to a vision model (Claude by default; any OpenAI-compatible endpoint or a local Ollama model works), which reads game titles off spines and boxes along with edition cues: publisher, edition wording, print year, language.
|
|
2. **resolve** — Titles are matched to BGG game IDs via the [XML API2](https://boardgamegeek.com/wiki/page/BGG_XML_API2) (exact + fuzzy matching, popularity tiebreaks), then edition cues are matched against BGG's version list for each game. Anything uncertain is flagged rather than guessed.
|
|
3. **review** — A local review step for ambiguous matches: pick the right game/version, or leave the version blank. Wrong guesses never reach your collection.
|
|
4. **diff** — Your existing BGG collection is fetched and compared, per copy (owning one edition of a game doesn't hide a second edition you also own).
|
|
5. **upload** — A Playwright browser session logs into your BGG account and adds each game (with its version, when known) politely and slowly. Dry-run mode, per-game logging, and resumability included.
|
|
6. **enrich** — Full metadata for every game (designers, player counts, weight, rank, mechanics, artwork, version details) lands in `data/games.json`, feeding a browsable library of your shelves.
|
|
|
|
Everything runs locally, every stage survives being killed mid-run, and all artifacts are flat CSV/JSON files you can inspect and edit. You can drive it from the terminal or from a local web app:
|
|
|
|
|
|
|
|
**➔ [See the full tour](docs/tour.md)** — all seven pages, the game-detail view, and the phone experience.
|
|
|
|
## Requirements
|
|
|
|
- Python 3.12+ and [uv](https://docs.astral.sh/uv/), on macOS, Linux, or Windows. (Development happens on macOS; Windows is untested but nothing is platform-specific. One caveat: the owner-only file permissions bggpipe sets on `.env` and browser session state are POSIX-only — on Windows, keep those files in a directory protected by your account.)
|
|
- A vision model for extraction — an [Anthropic API key](https://console.anthropic.com/) by default, or any OpenAI-compatible endpoint including a free local [Ollama](https://ollama.com/) model.
|
|
- A BoardGameGeek account **and a registered BGG application** — as of BGG's [2025 API policy](https://boardgamegeek.com/using_the_xml_api), the XML API requires a Bearer token from a registered app. Register a free non-commercial application at [boardgamegeek.com/applications](https://boardgamegeek.com/applications) (approval can take a week or more, so **apply on day one**), then create a token. Each user needs their own; tokens must not be shared. ([You can start before it arrives.](docs/guide.md#running-before-your-bgg-token-arrives))
|
|
|
|
## Quick start
|
|
|
|
```sh
|
|
uv tool install git+https://git.kestrelsnest.social/eric/bggpipe
|
|
mkdir shelves && cd shelves # any directory of your own — NOT a clone of this repo
|
|
bggpipe init # guided setup: folders, credentials, browser download
|
|
bggpipe web # the whole app at http://127.0.0.1:8377/
|
|
```
|
|
|
|
Drop shelf photos on the Photos page (or straight [from your phone's camera](docs/guide.md#from-your-phone)) and follow the pipeline left to right. Your photos and every pipeline artifact live in the directory where you run it, and `uv tool upgrade bggpipe` picks up fixes without going anywhere near your data — which is also why running inside a clone of this repo is the one unsupported setup: this repo carries its author's live pipeline data at git-tracked paths, and [a `git pull` on top of yours could destroy it](docs/guide.md#keeping-your-data-safe-from-git). bggpipe warns if it catches you doing this.
|
|
|
|
**➔ [The user's guide](docs/guide.md)** — credentials, every stage and flag, phone pairing, RPG handling, fixing misreads, uploading safely, and running before your token arrives.
|
|
|
|
## Development
|
|
|
|
```sh
|
|
git clone https://git.kestrelsnest.social/eric/bggpipe.git
|
|
cd bggpipe
|
|
uv sync # installs Python deps
|
|
uv run pytest # offline test suite (recorded fixtures, no network)
|
|
uv run pytest --run-live # + a read-only live-API smoke test (needs token)
|
|
uv run ruff check src tests # lint
|
|
uv run bggpipe web --dev # the app with code hot-reload
|
|
```
|
|
|
|
Run your own *pipeline* from a different directory — the clone's `data/` is the author's live data. Architecture and contributor guidance: [CLAUDE.md](CLAUDE.md) and the [full spec](bgg-shelf-pipeline-spec.md); BGG automation notes: [docs/bgg-upload-flow.md](docs/bgg-upload-flow.md).
|
|
|
|
## A note on being a good BGG citizen
|
|
|
|
This tool is **not affiliated with or supported by BoardGameGeek**. It uses only the sanctioned XML API2 for reads (with your own registered application token, per BGG's current policy) and drives the regular website for writes, deliberately slowly (one request every couple of seconds, slower for uploads). Please keep it that way: BGG is a community resource running on community goodwill. You are responsible for your own account — review the dry-run output before a real upload.
|
|
|
|
## License
|
|
|
|
MIT — see [LICENSE](LICENSE).
|
|
|
|
BoardGameGeek and BGG are trademarks of BoardGameGeek, LLC. bggpipe is an
|
|
independent project, not affiliated with or endorsed by BoardGameGeek.
|
|
|
|
Mascot art by Juniper, used with pride.
|