Compare commits
108
Commits
4bf7481f9b
...
v1.0.0
@@ -25,6 +25,7 @@ BGG" logo linking back to boardgamegeek.com.
|
||||
## Endpoints (XML API2 — the only sanctioned read API)
|
||||
|
||||
- Search: `https://boardgamegeek.com/xmlapi2/search?query=<title>&type=boardgame,boardgameexpansion`
|
||||
- RPG search: same endpoint with `type=rpgitem` — the geekdo database is shared across BGG/RPGGeek, so the same API and token serve RPG products. bggpipe uses this as a fallback for titles absent from the board-game types; matched rpgitems are LOCAL-ONLY (enriched, browsable, never uploaded — RPG collections live on rpggeek.com, outside this pipeline's write scope).
|
||||
- Thing (details/stats): `https://boardgamegeek.com/xmlapi2/thing?id=<id1,id2,...>&stats=1` — accepts comma-separated IDs; batch (~20) to reduce request count.
|
||||
- Thing versions: `https://boardgamegeek.com/xmlapi2/thing?id=<id>&versions=1` — lists every published edition/version of a game, each with its own version id, name, publisher, year, and language. Used to match photo edition cues to a concrete version.
|
||||
- Collection: `https://boardgamegeek.com/xmlapi2/collection?username=<user>&own=1` — add `&version=1` to include version info on collection items.
|
||||
@@ -50,7 +51,20 @@ The first `/collection` call typically returns **HTTP 202** with a "please retry
|
||||
|
||||
## Search & matching heuristics
|
||||
|
||||
1. Exact normalized-name match → strong candidate.
|
||||
0. BGG's search is UNRELIABLE for generic and punctuated queries: results
|
||||
truncate in the several-hundreds unordered (the game named "Dungeon!"
|
||||
appears in neither the "Dungeon!" nor the "Dungeon" search), and
|
||||
punctuation can hide matches. Every title is searched raw AND with
|
||||
punctuation stripped, merged by id. A LONE surviving candidate never
|
||||
auto-matches unless it's a true exact match whose owned-count clears
|
||||
the dominance floor — the sole survivor may be an impostor standing
|
||||
where a famous game should be, and the real match may need review's
|
||||
manual-id entry.
|
||||
1. Exact normalized-name match → strong candidate. A name that becomes
|
||||
exact once its trailing parenthetical is stripped ("Wiz-War (Eighth
|
||||
Edition)") is a SIBLING EDITION — BGG files new editions as separate
|
||||
games — and counts as exact-grade so the choice between lineages
|
||||
reaches review instead of hiding behind a confident auto.
|
||||
2. Fuzzy match: `rapidfuzz` `token_sort_ratio ≥ 90` → good candidate.
|
||||
3. Ties: fetch `/thing?stats=1` for top ~5 candidates; prefer higher owned-count / better BGG rank (well-known game beats obscure duplicate of the same name).
|
||||
- Search results include `boardgameexpansion` as a distinct `type` — keep expansions but tag them so review catches base/expansion confusion.
|
||||
|
||||
@@ -8,4 +8,6 @@ BGG_USERNAME=
|
||||
BGG_PASSWORD=
|
||||
|
||||
# Anthropic API key for the vision extract stage
|
||||
# default vision provider; not needed for an openai-compatible/local
|
||||
# setup — see the [vision.*] blocks in config.toml
|
||||
ANTHROPIC_API_KEY=
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
# Secrets & credential-adjacent state
|
||||
.env
|
||||
*.storage_state.json
|
||||
data/.lan_key
|
||||
data/.own_repo
|
||||
playwright/.auth/
|
||||
storage_state.json
|
||||
|
||||
|
||||
@@ -4,31 +4,54 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
|
||||
|
||||
## Project
|
||||
|
||||
`bggpipe` — a resumable, idempotent CLI pipeline: shelf photos → Claude vision title + edition-cue extraction → BoardGameGeek ID and version matching → human review → collection diff → upload to BGG via Playwright → metadata enrichment (`games.json`). Full design: @bgg-shelf-pipeline-spec.md. **The repo is greenfield — no code exists yet.** Follow the spec's "Suggested Build Order" when scaffolding; re-run `/init` once code exists.
|
||||
`bggpipe` — a resumable, idempotent CLI pipeline that turns shelf photos into a BoardGameGeek collection, in six stages:
|
||||
|
||||
## Stack (decided, not yet scaffolded)
|
||||
| # | Command | What it does | Status |
|
||||
|---|---------|--------------|--------|
|
||||
| 1 | `bggpipe extract` | Claude vision reads titles + edition cues from `photos/` | working |
|
||||
| 2 | `bggpipe resolve` | match titles to BGG IDs and versions via XML API2 | working (real API data since 2026-08-05) |
|
||||
| 3 | `bggpipe review` | human review of ambiguous/unmatched items; `--web` serves a FastAPI UI on port 8377 | working |
|
||||
| 4 | `bggpipe diff` | diff approved matches against the existing BGG collection | working |
|
||||
| 5 | `bggpipe upload` | add games via a logged-in Playwright session | working — all browser flows verified live 2026-08-06 (62 adds + 36 version updates landed) |
|
||||
| 6 | `bggpipe enrich` | fetch full game/version metadata into `games.json` | working |
|
||||
|
||||
- Python 3.12+, deps via **uv** (`uv add`, `uv run`), CLI framework **typer**, tests **pytest**, lint/format **ruff**.
|
||||
- Browser automation: **Playwright** (not Selenium). Needs `uv run playwright install chromium` after install.
|
||||
- Vision: Anthropic API, latest Sonnet model.
|
||||
Full design lives in `docs/spec.md` (read it before changing pipeline semantics); the upload-stage walkthrough is in `docs/bgg-upload-flow.md`.
|
||||
|
||||
## Commands
|
||||
|
||||
- `uv sync` — install deps (Python 3.12+, managed by **uv**; use `uv add`, never pip). `uv run bggpipe init` handles first-run setup (folders, .env credentials, the one-time `playwright install chromium`).
|
||||
- `uv run bggpipe web` — the app: seven pages (Pipeline `/`, Photos, Titles, Review, Queue, Library, Help) in a shared sidebar shell (responsive: hamburger nav + stacked tables under 900px); stage runs execute one-at-a-time in a background job. `--lan` binds 0.0.0.0 behind a per-device access key: persisted in `data/.lan_key` (gitignored), printed as a QR at startup, cookie-paired for a year, required on EVERY network request (loopback clients and `/static/*` are exempt; the Host/Origin guard still applies). Phone camera uploads (generic `image.jpg` names) get minted `shelf-<timestamp>` names — only explicitly-named files trigger the replace-to-reshoot flow.
|
||||
- `uv run bggpipe <stage>` — run a pipeline stage. Non-secret settings come from `config.toml` (dirs, rate limit, `vision_provider` + per-provider `[vision.*]` blocks — "anthropic" or any OpenAI-compatible endpoint incl. local Ollama); `--config` overrides the path.
|
||||
- `uv run pytest` — the suite runs fully offline against fixtures. Tests marked `live` hit the real BGG API (read-only) and are skipped unless you pass `--run-live`.
|
||||
- `uv run ruff check` / `uv run ruff format` — lint (rules E, F, I, UP, B, SIM) and format.
|
||||
|
||||
## Layout
|
||||
|
||||
- `src/bggpipe/` — `cli.py` (typer app), one module per stage (`extract`, `resolve`, `review` + `webreview`, `diff`, `upload`, `enrich`); `templates/shell.html` + `templates/pages/*` + `static/app.{css,js}` are the web UI (the stylesheet is the design system — tokens derive from the mascot art), plus `bgg_client.py` (rate-limited XML API2 client that caches responses to `data/bgg_cache/`), `jobs.py` (single-slot background stage runner for the web UI), `normalize.py` (title normalization), `models.py` (dataclasses), `config.py`, `fsio.py` (atomic writes), `init_wizard.py` (first-run setup).
|
||||
- `scripts/` — `write_stub_fixtures.py` / `write_photo_fixtures.py` generate synthetic fixtures; `record_fixtures.py` re-records real API responses once a token exists.
|
||||
- `tests/fixtures/bgg_cache/` — recorded real XML fixtures the offline tests run against.
|
||||
- `data/` — pipeline state (CSV/JSON artifacts are committed; caches are not — see Git).
|
||||
|
||||
## Hard rules (from spec — never violate)
|
||||
|
||||
- **≤1 request every 2 seconds** to any BGG endpoint; jittered backoff on 429/503. Upload stage: 2–4 s randomized delay between games.
|
||||
- **Credentials never touch disk or logs.** `ANTHROPIC_API_KEY`, `BGG_USERNAME`, `BGG_PASSWORD`, `BGG_API_TOKEN` come from env vars only. Playwright storage state is credential-adjacent — keep it gitignored.
|
||||
- **The XML API requires a registered app token** (`Authorization: Bearer`, from `BGG_API_TOKEN`) — unregistered requests get 401. Until Eric's registration at boardgamegeek.com/applications is approved, tests run on the stub fixtures in `tests/fixtures/bgg_cache/`; re-record them with `scripts/record_fixtures.py` once the token exists.
|
||||
- **The XML API requires a registered app token** (`Authorization: Bearer`, from `BGG_API_TOKEN`) — unregistered requests get 401. The token is ACTIVE and `tests/fixtures/bgg_cache/` holds real recorded responses; add fixtures for new tests with `scripts/record_fixtures.py`.
|
||||
- **Every stage is idempotent and resumable** — killing mid-run and restarting must lose no work; re-runs skip already-processed items.
|
||||
- Use only the XML API2 and the public website — no undocumented BGG endpoints (BGG tightened access policies in 2025).
|
||||
- BGG has **no write API**: writes drive the real website with a logged-in Playwright session.
|
||||
- **Synthetic data must never reach upload.** The stub era ended 2026-08-05: `matches.csv` and `tests/fixtures/bgg_cache/` now hold real API data. The guard mechanism stays armed: the stub-fixture generators write `data/bgg_cache/STUB_FIXTURES.marker` (gitignored) and `data/STUB_DATA.marker` (committed), and the upload stage MUST refuse to run while either exists — regenerating stubs re-locks upload automatically.
|
||||
|
||||
## Domain gotchas
|
||||
|
||||
- Base game vs. expansion vs. new edition is the top failure mode — bias matching toward `ambiguous` over auto-match ("Wingspan Europe" must not match base Wingspan).
|
||||
- Editions/versions matter: Eric owns multiple editions of some games — each is a separate collection entry (keyed by `collid` on BGG). Never guess a version: no legible cues → `version_unknown` and a version-less collection entry.
|
||||
- Normalize titles (casefold, strip punctuation/articles, special chars like é/&/:) identically on both sides of a match; dedupe across photos but keep `source_photos` provenance.
|
||||
- **Human curation is durable**: `data/title_splits.json` (photo-scoped split-into-copies decisions, honored by extract's dedupe AND resolve's dedupe), `data/title_edits.json` (corrected reads/cues, applied before dedupe on every titles.json rebuild), `data/title_removals.json` (lines removed from the catalog — filtered out of every rebuild; delete the record to undo), `data/title_additions.json` (games added without a photo — joined into every rebuild; a later photo sighting dedupe-merges with them), and `data/local_games.json` + `data/local_art/` (hand-written facts and a cover photo for off-BGG games — the ONLY source for them, merged over the photo reads by enrich) persist forever. Row-level decisions persist via the `dedupe_veto` column — edits never drop veto'd rows (a rename retitles them in place); removal drops them (explicitly discarding the line).
|
||||
- **RPGs are local-only citizens**: when the board-game search runs dry, resolve falls back to `type=rpgitem` (same geekdo API/token). RPGGeek items carry their OWN link types (`rpgdesigner`, `rpgpublisher`, `rpggenre`, `rpgcategory`, `rpgmechanic`) — a board-game-only parser silently returns nothing for them. Matched rpgitems enrich into the library but diff routes them to `local_only` — they must never reach `to_add.csv`/upload (their collection lives on RPGGeek, out of scope).
|
||||
- Detailed BGG API behavior (202 queueing, collection-endpoint quirks, endpoints): use the `bgg-api` skill. **If the spec's BGG behavior changes, update the `bgg-api` skill to match** — they must not drift.
|
||||
|
||||
## Git
|
||||
|
||||
- Remote is self-hosted Gitea 1.26 (`git.kestrelsnest.social`), **not GitHub** — `gh` CLI does not work here.
|
||||
- Commit `data/matches.csv`, `data/to_add.csv`, `data/upload_log.csv`, `data/titles.json`, `data/unidentified.json`, `data/games.json`. Never commit `data/bgg_cache/`, `photos/`, Playwright storage state, or `.env`.
|
||||
- Remote is self-hosted Gitea 1.26 (`git.kestrelsnest.social/eric/bggpipe`), **not GitHub** — `gh` CLI does not work here.
|
||||
- Commit `data/matches.csv`, `data/to_add.csv`, `data/to_update.csv`, `data/upload_log.csv`, `data/titles.json`, `data/unidentified.json`, `data/unidentified_dismissed.json`, `data/title_splits.json`, `data/title_edits.json`, `data/title_removals.json`, `data/title_additions.json`, `data/local_games.json`, `data/local_art/`, `data/games.json`, `data/STUB_DATA.marker` (while it applies), and the collection snapshot XMLs. Never commit `data/bgg_cache/`, `data/extract_raw/`, `photos/`, `data/.lan_key`, Playwright storage state, or `.env`.
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# bggpipe — Shelf-to-BoardGameGeek Collection Pipeline
|
||||
|
||||
<img src="assets/logo-full.jpeg" alt="the bggpipe piper — a bagpiper whose bag is a board game box" width="220" align="right">
|
||||
|
||||
Photograph your board game shelves. End up with your whole collection — including which *edition* of each game you own — cataloged on [BoardGameGeek](https://boardgamegeek.com).
|
||||
|
||||
```
|
||||
@@ -9,31 +11,69 @@ photos/ → [1 extract] → titles.json → [2 resolve] → matches.csv
|
||||
→ [6 enrich] → games.json
|
||||
```
|
||||
|
||||
> **Status: pre-release.** The design is complete ([full spec](bgg-shelf-pipeline-spec.md)); the code is being built. Nothing below works yet.
|
||||
> **Status: battle-tested end to end.** The full pipeline has run against a live BGG account: shelf photos → 136 identified games → 62 additions and 36 version updates on a real collection. The result is public — [the author's collection on BGG](https://boardgamegeek.com/collection/user/ewagoner) is what this pipeline built. Still sensible on a first run: `--dry-run`, then `--limit 1`.
|
||||
|
||||
## Why this exists
|
||||
|
||||
BGG has no bulk import and no write API. Cataloging a few hundred games by hand means hours of searching, clicking, and second-guessing which of five editions you own. This pipeline replaces that with: take photos, run a command, resolve a handful of ambiguous matches in a review step, done.
|
||||
BGG has no bulk import and no write API. Cataloging a few hundred games by hand means hours of searching, clicking, and second-guessing which of five editions you own. I wanted to point a camera at my shelves instead. This pipeline replaces the typing with: take photos, run a command, settle a handful of ambiguous matches in a review step, done — and every judgment call along the way is yours, made in a review UI, never guessed by the machine.
|
||||
|
||||
## How it works
|
||||
|
||||
1. **extract** — Shelf photos go to the Anthropic API (Claude vision), which reads game titles off spines and boxes along with edition cues: publisher, edition wording, print year, language.
|
||||
1. **extract** — Shelf photos go to a vision model (Claude by default; any OpenAI-compatible endpoint or a local Ollama model works), which reads game titles off spines and boxes along with edition cues: publisher, edition wording, print year, language.
|
||||
2. **resolve** — Titles are matched to BGG game IDs via the [XML API2](https://boardgamegeek.com/wiki/page/BGG_XML_API2) (exact + fuzzy matching, popularity tiebreaks), then edition cues are matched against BGG's version list for each game. Anything uncertain is flagged rather than guessed.
|
||||
3. **review** — A local review step for ambiguous matches: pick the right game/version, or leave the version blank. Wrong guesses never reach your collection.
|
||||
4. **diff** — Your existing BGG collection is fetched and compared, per copy (owning one edition of a game doesn't hide a second edition you also own).
|
||||
5. **upload** — A Playwright browser session logs into your BGG account and adds each game — with its version, when known — politely and slowly, with dry-run mode, per-game logging, and resumability.
|
||||
6. **enrich** — Full metadata for every game (designers, player counts, weight, rank, mechanics, artwork URLs, version details) lands in `data/games.json`, ready to power whatever you build next.
|
||||
5. **upload** — A Playwright browser session logs into your BGG account and adds each game (with its version, when known) politely and slowly. Dry-run mode, per-game logging, and resumability included.
|
||||
6. **enrich** — Full metadata for every game (designers, player counts, weight, rank, mechanics, artwork, version details) lands in `data/games.json`, feeding a browsable library of your shelves.
|
||||
|
||||
Every stage is idempotent and resumable: kill it mid-run, restart, lose nothing. All artifacts are flat CSV/JSON files you can inspect and edit.
|
||||
Everything runs locally, every stage survives being killed mid-run, and all artifacts are flat CSV/JSON files you can inspect and edit. You can drive it from the terminal or from a local web app:
|
||||
|
||||
|
||||
|
||||
**➔ [See the full tour](docs/tour.md)** — all seven pages, the game-detail view, and the phone experience.
|
||||
|
||||
## Requirements
|
||||
|
||||
- macOS or Linux, Python 3.12+, [uv](https://docs.astral.sh/uv/)
|
||||
- An [Anthropic API key](https://console.anthropic.com/) (vision extraction)
|
||||
- A BoardGameGeek account **and a registered BGG application** — as of BGG's [2025 API policy](https://boardgamegeek.com/using_the_xml_api), the XML API requires a Bearer token from a registered app. Register a free non-commercial application at [boardgamegeek.com/applications](https://boardgamegeek.com/applications) (approval can take a week or more — apply early), then create a token. Each user needs their own — tokens must not be shared.
|
||||
- Playwright Chromium: `uv run playwright install chromium`
|
||||
- Python 3.12+ and [uv](https://docs.astral.sh/uv/), on macOS, Linux, or Windows. (Development happens on macOS; Windows is untested but nothing is platform-specific. One caveat: the owner-only file permissions bggpipe sets on `.env` and browser session state are POSIX-only — on Windows, keep those files in a directory protected by your account.)
|
||||
- A vision model for extraction — an [Anthropic API key](https://console.anthropic.com/) by default, or any OpenAI-compatible endpoint including a free local [Ollama](https://ollama.com/) model. **Cost expectation:** extracting the author's whole collection — 65 shelf photos, 136 games — cost under a dollar with the default model (Claude Sonnet), one-time. Re-runs are free: every photo's read is cached, and only new or replaced photos go back to the model.
|
||||
- A BoardGameGeek account **and a registered BGG application** — as of BGG's [2025 API policy](https://boardgamegeek.com/using_the_xml_api), the XML API requires a Bearer token from a registered app. Register a free non-commercial application at [boardgamegeek.com/applications](https://boardgamegeek.com/applications) (approval can take a week or more, so **apply on day one**), then create a token. Each user needs their own; tokens must not be shared. ([You can start before it arrives.](docs/guide.md#running-before-your-bgg-token-arrives))
|
||||
|
||||
Secrets come from environment variables only — `ANTHROPIC_API_KEY`, `BGG_USERNAME`, `BGG_PASSWORD`, `BGG_API_TOKEN` — and are never written to disk or logs.
|
||||
### Why it asks for your BGG password — and where your credentials go
|
||||
|
||||
BGG has no write API: the **only** way to add games to a collection is the website itself. So the upload stage signs into boardgamegeek.com in a real browser window, on your machine, and clicks the same buttons you would — you can literally watch it work (the browser is visible by default). That's the whole reason the password is needed, and it's used for exactly that login and nothing else.
|
||||
|
||||
Everything stays on your computer. There is no bggpipe server, no telemetry, no analytics, and no account with me — I never see your credentials, your collection, or anything else, and the code is right here to check. Credentials live in a local `.env` file (owner-only permissions, never written to logs) and are sent only to boardgamegeek.com itself. The only other service the pipeline ever contacts is the vision provider *you* configure, which receives your shelf photos and nothing more — and with a local Ollama model, even those never leave the house.
|
||||
|
||||
## Quick start
|
||||
|
||||
```sh
|
||||
uv tool install git+https://git.kestrelsnest.social/eric/bggpipe
|
||||
mkdir shelves && cd shelves # any directory of your own — NOT a clone of this repo
|
||||
bggpipe init # guided setup: folders, credentials, browser download
|
||||
bggpipe web # the whole app at http://127.0.0.1:8377/
|
||||
```
|
||||
|
||||
Drop shelf photos on the Photos page (or straight [from your phone's camera](docs/guide.md#from-your-phone)) and follow the pipeline left to right. Your photos and every pipeline artifact live in the directory where you run it, and `uv tool upgrade bggpipe` picks up fixes without going anywhere near your data — which is also why running inside a clone of this repo is the one unsupported setup: this repo carries its author's live pipeline data at git-tracked paths, and [a `git pull` on top of yours could destroy it](docs/guide.md#keeping-your-data-safe-from-git). bggpipe warns if it catches you doing this.
|
||||
|
||||
**➔ [The user's guide](docs/guide.md)** — credentials, every stage and flag, phone pairing, RPG handling, fixing misreads, uploading safely, and running before your token arrives.
|
||||
|
||||
## Development
|
||||
|
||||
```sh
|
||||
git clone https://git.kestrelsnest.social/eric/bggpipe.git
|
||||
cd bggpipe
|
||||
uv sync # installs Python deps
|
||||
uv run pytest # offline test suite (recorded fixtures, no network)
|
||||
uv run pytest --run-live # + a read-only live-API smoke test (needs token)
|
||||
uv run ruff check src tests # lint
|
||||
uv run bggpipe web --dev # the app with code hot-reload
|
||||
```
|
||||
|
||||
Run your own *pipeline* from a different directory — the clone's `data/` is the author's live data. Architecture and contributor guidance: [CLAUDE.md](CLAUDE.md) and the [design contract](docs/spec.md); BGG automation notes: [docs/bgg-upload-flow.md](docs/bgg-upload-flow.md).
|
||||
|
||||
## Questions, bugs, ideas
|
||||
|
||||
Email [eric@ericwagoner.com](mailto:eric@ericwagoner.com), or find me on Mastodon at [@eric@toots.kestrelsnest.social](https://toots.kestrelsnest.social/@eric) or Bluesky at [@kestrelsnest.social](https://bsky.app/profile/kestrelsnest.social) — bug reports, confusions, and "it worked, here's my collection" notes all welcome. (This Gitea instance doesn't take public registrations, so there's no issue tracker to file into yet; if enough people show up, one will materialize.)
|
||||
|
||||
## A note on being a good BGG citizen
|
||||
|
||||
@@ -42,3 +82,8 @@ This tool is **not affiliated with or supported by BoardGameGeek**. It uses only
|
||||
## License
|
||||
|
||||
MIT — see [LICENSE](LICENSE).
|
||||
|
||||
BoardGameGeek and BGG are trademarks of BoardGameGeek, LLC. bggpipe is an
|
||||
independent project, not affiliated with or endorsed by BoardGameGeek.
|
||||
|
||||
Mascot art by Juniper, used with pride.
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 507 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 483 KiB |
+17
-7
@@ -1,11 +1,21 @@
|
||||
# Non-secret configuration for bggpipe. Secrets (ANTHROPIC_API_KEY,
|
||||
# BGG_PASSWORD) come from env vars only and must never appear here.
|
||||
|
||||
# TODO(eric): set your BGG username — the live smoke test and diff/upload
|
||||
# stages need it. Leave empty to skip the live smoke test.
|
||||
bgg_username = ""
|
||||
# Non-secret knobs for bggpipe. Everything account-related — including
|
||||
# your BGG username — lives in .env (see .env.example), not here.
|
||||
|
||||
photos_dir = "photos"
|
||||
data_dir = "data"
|
||||
model = "claude-sonnet-5"
|
||||
rate_limit_seconds = 2.0
|
||||
|
||||
# Which vision backend reads your shelf photos. Both recipes below stay
|
||||
# on file; this line picks one.
|
||||
vision_provider = "anthropic"
|
||||
|
||||
[vision.anthropic]
|
||||
# reads ANTHROPIC_API_KEY from the environment
|
||||
model = "claude-sonnet-5"
|
||||
|
||||
[vision."openai-compatible"]
|
||||
# OpenAI, OpenRouter, or a local runtime (Ollama, LM Studio, vLLM).
|
||||
# key_env names the env var holding the key; "" = endpoint needs none.
|
||||
base_url = "http://localhost:11434/v1"
|
||||
model = "qwen2.5vl:7b"
|
||||
key_env = ""
|
||||
|
||||
+7618
File diff suppressed because it is too large
Load Diff
+137
-28
File diff suppressed because one or more lines are too long
@@ -0,0 +1,32 @@
|
||||
[
|
||||
{
|
||||
"title_raw": "Civilization: West Extension Map",
|
||||
"confidence": "high",
|
||||
"publisher_hint": "",
|
||||
"edition_hint": "",
|
||||
"year_hint": null,
|
||||
"language_hint": "",
|
||||
"art_notes": "",
|
||||
"source_photos": []
|
||||
},
|
||||
{
|
||||
"title_raw": "Artistry: Delightful Doorways Mini Expansion",
|
||||
"confidence": "high",
|
||||
"publisher_hint": "",
|
||||
"edition_hint": "",
|
||||
"year_hint": null,
|
||||
"language_hint": "",
|
||||
"art_notes": "",
|
||||
"source_photos": []
|
||||
},
|
||||
{
|
||||
"title_raw": "castle panic wizard's tower",
|
||||
"confidence": "high",
|
||||
"publisher_hint": "",
|
||||
"edition_hint": "",
|
||||
"year_hint": null,
|
||||
"language_hint": "",
|
||||
"art_notes": "",
|
||||
"source_photos": []
|
||||
}
|
||||
]
|
||||
@@ -0,0 +1,99 @@
|
||||
[
|
||||
{
|
||||
"match": "Hangermuger 4 800",
|
||||
"title_raw": "Huggermugger"
|
||||
},
|
||||
{
|
||||
"match": "DOCTOR WHO THE GAME OF TIME",
|
||||
"title_raw": "DOCTOR WHO THE card GAME"
|
||||
},
|
||||
{
|
||||
"match": "CAPTAIN MARVEL SECRET SKRULLS",
|
||||
"confidence": "high"
|
||||
},
|
||||
{
|
||||
"match": "CHECKERS",
|
||||
"title_raw": "Super Mario CHECKERS",
|
||||
"confidence": "high"
|
||||
},
|
||||
{
|
||||
"match": "DOCTOR WHO THE card GAME",
|
||||
"confidence": "high"
|
||||
},
|
||||
{
|
||||
"match": "DUNGEONS & DRAGONS",
|
||||
"confidence": "high"
|
||||
},
|
||||
{
|
||||
"match": "Huggermugger",
|
||||
"confidence": "high"
|
||||
},
|
||||
{
|
||||
"match": "Patch Work",
|
||||
"confidence": "high"
|
||||
},
|
||||
{
|
||||
"match": "The Ain't It Cool Trivia Game",
|
||||
"title_raw": "The Rocky Horror Trivia Game",
|
||||
"confidence": "high"
|
||||
},
|
||||
{
|
||||
"match": "Verdant",
|
||||
"confidence": "high"
|
||||
},
|
||||
{
|
||||
"match": "Wiz-War",
|
||||
"photos": [
|
||||
"IMG_4502.jpeg"
|
||||
],
|
||||
"confidence": "high"
|
||||
},
|
||||
{
|
||||
"match": "ACTION CASTLE",
|
||||
"title_raw": "Parsely",
|
||||
"confidence": "high"
|
||||
},
|
||||
{
|
||||
"match": "Patch Work",
|
||||
"title_raw": "Patchwork",
|
||||
"confidence": "high"
|
||||
},
|
||||
{
|
||||
"match": "SMALLWORLD",
|
||||
"title_raw": "SMALL WORLD",
|
||||
"confidence": "high"
|
||||
},
|
||||
{
|
||||
"match": "ART NOUVEAU ARTISTRY",
|
||||
"title_raw": "ARTISTRY",
|
||||
"confidence": "high"
|
||||
},
|
||||
{
|
||||
"match": "MAGE WARS ACADEMY CORE SET",
|
||||
"title_raw": "MAGE WARS ACADEMY",
|
||||
"confidence": "high"
|
||||
},
|
||||
{
|
||||
"match": "MANTIS FALLS (a game of trust)",
|
||||
"title_raw": "MANTIS FALLS",
|
||||
"confidence": "high"
|
||||
},
|
||||
{
|
||||
"match": "BOTANY EXPANSION TANTALIZING TREES",
|
||||
"title_raw": "BOTANY: TANTALIZING TREES",
|
||||
"confidence": "high"
|
||||
},
|
||||
{
|
||||
"match": "BOTANY EXPANSION PERILOUS PERFUMES POISONOUS, CARNIVOROUS, PARASITIC AND BIZARRE PLANTS",
|
||||
"title_raw": "BOTANY: PERILOUS PERFUMES",
|
||||
"confidence": "high"
|
||||
},
|
||||
{
|
||||
"match": "WIZ-WAR",
|
||||
"photos": [
|
||||
"IMG_4504.jpeg"
|
||||
],
|
||||
"year_hint": 2012,
|
||||
"confidence": "high"
|
||||
}
|
||||
]
|
||||
@@ -0,0 +1,86 @@
|
||||
[
|
||||
{
|
||||
"title": "THE CATAN CARD GAME",
|
||||
"photos": [
|
||||
"IMG_4523.jpeg"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "SLUGFEST GAMES",
|
||||
"photos": [
|
||||
"IMG_4528.jpeg"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "SCRABBLE",
|
||||
"photos": [
|
||||
"IMG_4508.jpeg"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "SCERU MERRY MEN",
|
||||
"photos": [
|
||||
"IMG_4518.jpeg"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "Patchwork",
|
||||
"photos": [
|
||||
"IMG_4556.jpeg"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "Jokin Ha...",
|
||||
"photos": [
|
||||
"IMG_4506.jpeg"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "CHEAPASS GAMES",
|
||||
"photos": [
|
||||
"IMG_4519.jpeg"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "CATAN",
|
||||
"photos": [
|
||||
"IMG_4523.jpeg"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "CAR WARS",
|
||||
"photos": [
|
||||
"IMG_4532.jpeg"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "Bird on Your Bread?!",
|
||||
"photos": [
|
||||
"IMG_4517.jpeg"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "Super Mario CHECKERS",
|
||||
"photos": [
|
||||
"IMG_4507.jpeg"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "ACTION CASTLE I",
|
||||
"photos": [
|
||||
"IMG_4573.jpeg"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "SIX-GUN SHOWDOWN",
|
||||
"photos": [
|
||||
"IMG_4573.jpeg"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "Blackboa[rd]",
|
||||
"photos": [
|
||||
"IMG_4573.jpeg"
|
||||
]
|
||||
}
|
||||
]
|
||||
@@ -0,0 +1,3 @@
|
||||
[
|
||||
"Wiz-War"
|
||||
]
|
||||
+1510
-82
File diff suppressed because it is too large
Load Diff
+1
-9
@@ -1,9 +1 @@
|
||||
bgg_id,bgg_name,year,type,version_id,version_name,title_raw,source_photos
|
||||
235096,Cat Crimes,2017,boardgame,360982,English edition,CAT CRIMES,IMG_4499.jpeg
|
||||
1078,Skip-Bo,1967,boardgame,,,SKIP-BO,IMG_4501.jpeg
|
||||
268163,A Gentle Rain,2019,boardgame,268164,Bloom Edition,a Gentle Rain,IMG_4501.jpeg
|
||||
163412,Patchwork,2014,boardgame,,,PATCH WORK,IMG_4501.jpeg
|
||||
181254,Utter Nonsense,2015,boardgame,,,Utter Nonsense!,IMG_4501.jpeg
|
||||
50381,Cards Against Humanity,2009,boardgame,,,Cards Against Humanity,IMG_4501.jpeg
|
||||
195314,Herbaceous,2017,boardgame,,,Hebarceos,IMG_4502.jpeg
|
||||
218866,Scrawl,2017,boardgame,,,SCRAWL,IMG_4502.jpeg
|
||||
bgg_id,bgg_name,year,type,version_id,version_name,title_raw,source_photos,second_copy
|
||||
|
||||
|
@@ -1,10 +1 @@
|
||||
collid,bgg_id,bgg_name,version_id,version_name
|
||||
53429559,240,Britannia,25668,Avalon Hill English edition
|
||||
53429530,71,Civilization,71001,Avalon Hill English edition
|
||||
53429542,177,Advanced Civilization,177001,Avalon Hill English edition
|
||||
53429504,3090,Doctor Who: The Game of Time & Space,309001,Games Workshop English edition
|
||||
53429482,3585,Sorcerer,358501,SPI Designer's Edition
|
||||
53429367,2524,StarForce 'Alpha Centauri': Interstellar Conflict in the 25th Century,252401,SPI Designer's Edition
|
||||
53430387,193621,Joking Hazard,19362101,First edition
|
||||
53429831,26,Age of Renaissance,2601,Avalon Hill English edition
|
||||
53429816,224,History of the World,22401,Avalon Hill English edition
|
||||
|
||||
|
+376
-1
@@ -1 +1,376 @@
|
||||
{}
|
||||
{
|
||||
"IMG_4501.jpeg": [
|
||||
{
|
||||
"location": "Top shelf, far left edge, cut off by frame; near 'a Gentle Rain' box",
|
||||
"partial_text": "",
|
||||
"art_notes": "Dark/black box, only a thin sliver visible"
|
||||
},
|
||||
{
|
||||
"location": "Right side, middle shelf, black spine sticking out between Santorini and purple box below",
|
||||
"partial_text": "T",
|
||||
"art_notes": "Black spine, single letter visible, rest obscured"
|
||||
},
|
||||
{
|
||||
"location": "Middle shelf, purple box beneath Joking Hazard, right side of stack",
|
||||
"partial_text": "A Boardgame for 1 to 6 players, ages 8 and up.",
|
||||
"art_notes": "Purple/violet box, text describing player count, title not legible from this angle"
|
||||
},
|
||||
{
|
||||
"location": "Bottom shelf, left portion, above 'Exploding Kittens' box",
|
||||
"partial_text": "EXPANSION",
|
||||
"art_notes": "Small box with partial text visible, colorful design, title cut off by frame edge"
|
||||
},
|
||||
{
|
||||
"location": "Bottom shelf, right side, near Exploding Kittens box",
|
||||
"partial_text": "COSMIC",
|
||||
"art_notes": "Colorful box spine, blue/green tones, title partially visible but not fully confirmable"
|
||||
}
|
||||
],
|
||||
"IMG_4502.jpeg": [
|
||||
{
|
||||
"location": "top-left corner of shelf, above Santorini and to the left of Wiz-War",
|
||||
"partial_text": "possibly 'Herbaceous' or similar, text partially cut off",
|
||||
"art_notes": "cream/white box with floral yellow illustration, thin spine"
|
||||
},
|
||||
{
|
||||
"location": "far left side of shelf, stacked horizontally beneath the yellow-floral box, left of Santorini",
|
||||
"partial_text": "",
|
||||
"art_notes": "small colorful box spines, red and white coloring, too angled/blurry to read title"
|
||||
},
|
||||
{
|
||||
"location": "left side of shelf, below the horizontal stack, left of Santorini and above Joking Hazard",
|
||||
"partial_text": "",
|
||||
"art_notes": "blue box with white illustration, appears to be a game box but title not legible"
|
||||
},
|
||||
{
|
||||
"location": "bottom-left corner of shelf, beneath the History of the World area, left of Dungeon!",
|
||||
"partial_text": "A Boardgame for 1 to 6 players, ages 8 and up",
|
||||
"art_notes": "purple/tan box with descriptive text visible but no title readable"
|
||||
}
|
||||
],
|
||||
"IMG_4505.jpeg": [
|
||||
{
|
||||
"location": "Top shelf, far left edge behind Cat Crimes, cut off by frame edge",
|
||||
"partial_text": "",
|
||||
"art_notes": "Dark brown/maroon spine with faint gold decorative border, partially visible vertical box"
|
||||
}
|
||||
],
|
||||
"IMG_4506.jpeg": [
|
||||
{
|
||||
"location": "Top shelf, spine visible above Santorini box, partially cut off at top of frame",
|
||||
"partial_text": "",
|
||||
"art_notes": "Multicolored patchwork/tile pattern spine in yellow, tan, and blue tones with geometric designs; appears to be a board game box edge but no title text is legible"
|
||||
},
|
||||
{
|
||||
"location": "Upper right corner of image, above and right of Santorini spine, next to unidentified patterned box",
|
||||
"partial_text": "",
|
||||
"art_notes": "Small red and blue box corner visible, mostly cut off by frame edge, no discernible text"
|
||||
},
|
||||
{
|
||||
"location": "Right side of image, behind Santorini box, dark box with small visible logo",
|
||||
"partial_text": "",
|
||||
"art_notes": "Dark/black box with a small circular or square colored logo element, mostly obscured by foreground Santorini box"
|
||||
}
|
||||
],
|
||||
"IMG_4507.jpeg": [
|
||||
{
|
||||
"location": "Top of stack, right of center, between 'a Gentle Rain' and 'SANToRINI'",
|
||||
"partial_text": "PATCH / WORK (possibly)",
|
||||
"art_notes": "Colorful patchwork quilt-style pattern on spine, small box"
|
||||
},
|
||||
{
|
||||
"location": "Below 'LABYRINTH', left side of shelf, partially cut off at frame edge",
|
||||
"partial_text": "DRAGO...",
|
||||
"art_notes": "Green/dark box, only top edge visible, text cut off"
|
||||
}
|
||||
],
|
||||
"IMG_4508.jpeg": [
|
||||
{
|
||||
"location": "Top shelf, far left, partially cut off by frame edge, next to a green-framed picture and above the Super Mario Checkers box",
|
||||
"partial_text": "",
|
||||
"art_notes": "Teal/green colored box edge, mostly obscured, shape suggests a game box but no readable text"
|
||||
},
|
||||
{
|
||||
"location": "Top shelf, center, small yellow tin sitting next to a black box near the stack of papers/magazines, above the Labyrinth box",
|
||||
"partial_text": "",
|
||||
"art_notes": "Small yellow tin container, no visible text, could be a card game tin"
|
||||
}
|
||||
],
|
||||
"IMG_4511.jpeg": [
|
||||
{
|
||||
"location": "Middle shelf, behind Superfight boxes, green/black box with skeletal figure artwork - between Superfight stack and the Rook box",
|
||||
"partial_text": "...CTHULHU",
|
||||
"art_notes": "Dark green box, illustrated skeletal/robed figure, likely a Cthulhu-themed game (e.g., Cthulhu Fluxx or Cthulhu Gloom) but exact title obscured"
|
||||
},
|
||||
{
|
||||
"location": "Middle shelf, black spine box stacked above Rook, to the left of the dice tray",
|
||||
"partial_text": "...Yourself",
|
||||
"art_notes": "Black box with orange/white text, only partial word 'Yourself' legible, title obscured by angle"
|
||||
},
|
||||
{
|
||||
"location": "Right side of shelf, blue box partially cut off by frame edge, next to Throw Throw Avocado box above",
|
||||
"partial_text": "HEX",
|
||||
"art_notes": "Blue box, small visible portion, rest of title cut off by image edge"
|
||||
},
|
||||
{
|
||||
"location": "Top right corner of shelf, box mostly out of frame, above the Hex box",
|
||||
"partial_text": "Drag...",
|
||||
"art_notes": "Orange and blue box, only a fragment of text visible, likely a dragon-themed game title"
|
||||
}
|
||||
],
|
||||
"IMG_4517.jpeg": [
|
||||
{
|
||||
"location": "Middle shelf area, second box from left, between 'Pie Face!' and 'Bird on Your Bread?!'",
|
||||
"partial_text": "...renade... Nintendo?",
|
||||
"art_notes": "Narrow tall box, red and tan coloring, partially obscured by shadow and angle, appears to have small figure illustration"
|
||||
}
|
||||
],
|
||||
"IMG_4519.jpeg": [
|
||||
{
|
||||
"location": "Middle shelf area, two matching red boxes with orange circular 'G' logo, positioned between the white D&D boxes on the left and the white spined books on the right",
|
||||
"partial_text": "G (stylized orange circular logo)",
|
||||
"art_notes": "Two identical red boxes, narrow spine width, orange/yellow circular logo with letter G, taped/worn edges"
|
||||
},
|
||||
{
|
||||
"location": "Left shelf, red spine with black text between the cream D&D box and the red-orange boxes",
|
||||
"partial_text": "THE BEST OF RA... (possibly 'RAMPHA' or similar)",
|
||||
"art_notes": "Dark red spine with illustrated figure artwork, book-like thickness"
|
||||
},
|
||||
{
|
||||
"location": "Right side of shelf, tall white spine positioned next to 'Before I Kill You, Mister Bond' box",
|
||||
"partial_text": "KILL DR ... LUCKY",
|
||||
"art_notes": "White spine, black vertical text, appears to reference a 'kill' themed card game"
|
||||
}
|
||||
],
|
||||
"IMG_4520.jpeg": [
|
||||
{
|
||||
"location": "Middle shelf, fourth position from left, between Carcassonne and Evolution: The Beginning",
|
||||
"partial_text": "",
|
||||
"art_notes": "Plain wooden-colored box, appears unlabeled or title not visible, vertical wood-grain texture"
|
||||
}
|
||||
],
|
||||
"IMG_4521.jpeg": [
|
||||
{
|
||||
"location": "Top of shelf, above the Sleeping Gods box, partially cut off by frame edge",
|
||||
"partial_text": "",
|
||||
"art_notes": "Dark spines of several boxes, indistinct due to shadow and angle, colors appear dark blue/black and tan"
|
||||
}
|
||||
],
|
||||
"IMG_4527.jpeg": [
|
||||
{
|
||||
"location": "Right edge of shelf, behind/beside the Red Dragon Inn box, partially cut off by frame",
|
||||
"partial_text": "4, 3",
|
||||
"art_notes": "Dark colored spine/box edge, numbers visible but title obscured"
|
||||
}
|
||||
],
|
||||
"IMG_4528.jpeg": [
|
||||
{
|
||||
"location": "Top shelf, above Wiz-War box, partially obscured by rolled poster/tube on left side",
|
||||
"partial_text": "SLUGFEST GAMES",
|
||||
"art_notes": "Red and orange colored box with illustrated fantasy/comic style artwork, only top edge visible"
|
||||
}
|
||||
],
|
||||
"IMG_4529.jpeg": [
|
||||
{
|
||||
"location": "Top of stack, above the 1000-piece puzzle box, partially cut off at top of frame",
|
||||
"partial_text": "V, 3",
|
||||
"art_notes": "Dark spines with minimal visible text, one shows a circular 'V' logo, another shows number '3' on purple/dark background; too obscured to identify title"
|
||||
},
|
||||
{
|
||||
"location": "Middle of stack, between the puzzle box and the Risk Lord of the Rings box, showing an armored warrior figure",
|
||||
"partial_text": "The Middle-earth, NEW LINE CINEMA",
|
||||
"art_notes": "Dark box spine/edge showing armored fantasy warriors, likely related to a Lord of the Rings themed game; small circular logo visible with Asian-style characters, could be the same Risk LOTR box viewed from spine angle or a separate title"
|
||||
}
|
||||
],
|
||||
"IMG_4532.jpeg": [
|
||||
{
|
||||
"location": "Top right corner of frame, above Gloomhaven box, partially cut off",
|
||||
"partial_text": "GLO...",
|
||||
"art_notes": "Fiery orange/red background with dark textured lettering, appears to be another large box possibly related to Gloomhaven expansion"
|
||||
}
|
||||
],
|
||||
"IMG_4535.jpeg": [
|
||||
{
|
||||
"location": "Top shelf, far right edge of frame, next to the last visible Etherfields big box",
|
||||
"partial_text": "",
|
||||
"art_notes": "Black box spine, partially cut off by frame edge, appears similar in size to adjacent Etherfields big boxes"
|
||||
}
|
||||
],
|
||||
"IMG_4542.jpeg": [
|
||||
{
|
||||
"location": "Top left corner of shelf, partially cut off by frame edge, above the Skip-Bo box",
|
||||
"partial_text": "...O / From the Makers of...",
|
||||
"art_notes": "Red box edge with white lettering, likely a card game box, mostly obscured"
|
||||
},
|
||||
{
|
||||
"location": "Top area, small red square box with white 'M' logo, near Dragon Land",
|
||||
"partial_text": "M",
|
||||
"art_notes": "Small red square box with large white letter M, possibly a classic board game logo"
|
||||
},
|
||||
{
|
||||
"location": "Right side shelf, below CATAN, dark spine with red/white text",
|
||||
"partial_text": "T...T / Ride (possible)",
|
||||
"art_notes": "Dark red/maroon spine with train-like graphic element, narrow box spine"
|
||||
},
|
||||
{
|
||||
"location": "Bottom right shelf area, dark spine next to unidentified train-themed box",
|
||||
"partial_text": "",
|
||||
"art_notes": "Dark spine, illegible text, appears to be a board game box based on shape"
|
||||
}
|
||||
],
|
||||
"IMG_4543.jpeg": [
|
||||
{
|
||||
"location": "Top right corner of frame, behind and above the Scrabble Slam box, partially cut off by frame edge",
|
||||
"partial_text": "Dogs Kitchen",
|
||||
"art_notes": "Small box or book with illustrated cartoon-style cover, colorful, appears to feature a dog character; too small/blurry to confirm if it's a game"
|
||||
},
|
||||
{
|
||||
"location": "Right edge of frame, middle row, next to the Scrabble Slam box",
|
||||
"partial_text": "CAT...",
|
||||
"art_notes": "Dark colored box spine, partially cut off at right edge of photo, red/dark background with light text"
|
||||
},
|
||||
{
|
||||
"location": "Background, lower right area behind the hand holding Scrabble Slam, blurred and out of focus",
|
||||
"partial_text": "",
|
||||
"art_notes": "Indistinct colorful box shapes, cannot make out title or artwork details due to heavy blur"
|
||||
}
|
||||
],
|
||||
"IMG_4544.jpeg": [
|
||||
{
|
||||
"location": "Background shelf behind the Cat Stax box, top of frame, multiple stacked boxes with no clearly identifiable neighbors",
|
||||
"partial_text": "",
|
||||
"art_notes": "Blurry stack of colorful boxes (orange, white, dark tones) in background, out of focus, too indistinct to read any text"
|
||||
}
|
||||
],
|
||||
"IMG_4545.jpeg": [
|
||||
{
|
||||
"location": "Top shelf, right side, above the 'Ravensburger' labeled box, partial title visible at top edge of frame",
|
||||
"partial_text": "DRAGON ...and",
|
||||
"art_notes": "Orange/red box with dragon and fantasy artwork, cut off at top of image"
|
||||
},
|
||||
{
|
||||
"location": "Top right shelf, box with 'Ravensburger' text visible, title obscured",
|
||||
"partial_text": "Ravensburger",
|
||||
"art_notes": "Colorful box, brand name visible but game title not legible"
|
||||
},
|
||||
{
|
||||
"location": "Lower left shelf area, dark colored box partially visible behind hand holding Pocket Farkel",
|
||||
"partial_text": "",
|
||||
"art_notes": "Dark/black box edge visible, no legible text"
|
||||
},
|
||||
{
|
||||
"location": "Bottom right corner of image, yellowish box edge visible",
|
||||
"partial_text": "",
|
||||
"art_notes": "Yellow/tan colored box spine, text not legible due to angle and cropping"
|
||||
}
|
||||
],
|
||||
"IMG_4547.jpeg": [
|
||||
{
|
||||
"location": "Top left corner of shelf, above the main held box, leftmost visible spine",
|
||||
"partial_text": "...AZE",
|
||||
"art_notes": "Blue and green box, partial text visible, cut off at frame edge"
|
||||
},
|
||||
{
|
||||
"location": "Top left area, next to the '...AZE' box, second spine from left",
|
||||
"partial_text": "UTTLES or similar",
|
||||
"art_notes": "Bright blue box with light green accents, small toy-like image visible on top"
|
||||
},
|
||||
{
|
||||
"location": "Top right corner of shelf, above and to the right of the main held box",
|
||||
"partial_text": "Dark Terr... or Dark Territory",
|
||||
"art_notes": "Dark navy/black box with red and white text, partially obscured by other boxes"
|
||||
},
|
||||
{
|
||||
"location": "Top right area, red-orange box between the dark box and top edge",
|
||||
"partial_text": "Roll Pla...",
|
||||
"art_notes": "Red/orange box with illustrated artwork, text partially cut off"
|
||||
},
|
||||
{
|
||||
"location": "Top center-right, small red/orange box near top edge",
|
||||
"partial_text": "",
|
||||
"art_notes": "Small reddish box with indistinct artwork, mostly cut off at top frame"
|
||||
}
|
||||
],
|
||||
"IMG_4552.jpeg": [
|
||||
{
|
||||
"location": "Upper right edge of frame, partially cut off, behind the Dark Cults game",
|
||||
"partial_text": "...ard Game, -6 Players, ...uring the, o your evil lair, llet in his head, ...ng the temptation, our prize, to tell, plans, to let him, eath and blow up, ...ir in the process, agine winning, Yeah. Right.",
|
||||
"art_notes": "White/light colored box or insert with black text, appears to be another horror/villain themed party or card game, partially obscured by hand and folded materials"
|
||||
},
|
||||
{
|
||||
"location": "Left edge of frame, vertical spine, sharply cropped",
|
||||
"partial_text": "Giv... / Ci...",
|
||||
"art_notes": "Dark spine with white lettering, partially visible artwork, cropped by frame edge"
|
||||
}
|
||||
],
|
||||
"IMG_4553.jpeg": [
|
||||
{
|
||||
"location": "Top right corner of image, partially cut off by frame edge, to the right of Give Me The Brain",
|
||||
"partial_text": "Cards, ...6 Players",
|
||||
"art_notes": "Small dark card game box, black background with white text, silhouette figure visible on partial box edge"
|
||||
}
|
||||
],
|
||||
"IMG_4555.jpeg": [
|
||||
{
|
||||
"location": "Background, top left corner of image, partially visible behind main box",
|
||||
"partial_text": "",
|
||||
"art_notes": "Colorful box edge, mostly obscured by the Kill Doctor Lucky box in foreground"
|
||||
},
|
||||
{
|
||||
"location": "Background, top area, right of the colorful box in top left",
|
||||
"partial_text": "",
|
||||
"art_notes": "Yellow/tan colored box top, mostly obscured"
|
||||
},
|
||||
{
|
||||
"location": "Right side of image, vertical spine",
|
||||
"partial_text": "Z-WAR or similar fragment",
|
||||
"art_notes": "Dark spine with partial text visible, appears to be a game box on shelf"
|
||||
}
|
||||
],
|
||||
"IMG_4556.jpeg": [
|
||||
{
|
||||
"location": "Top left corner, above the green dinosaur-patterned spine and left of 'Ravensbu...' spine",
|
||||
"partial_text": "",
|
||||
"art_notes": "Dark spine, mostly obscured, edge of frame"
|
||||
},
|
||||
{
|
||||
"location": "Top row, second shelf, spine reading 'Ravensbu...' between an unidentified dark box and green dinosaur-patterned spine",
|
||||
"partial_text": "Ravensbu...",
|
||||
"art_notes": "White spine with blue text, likely Ravensburger logo/publisher rather than title, top cut off"
|
||||
},
|
||||
{
|
||||
"location": "Top row, green textured spine with cartoon dinosaur, left of 'Ravensbu...' spine",
|
||||
"partial_text": "",
|
||||
"art_notes": "Green speckled/scaly texture spine with small cartoon dinosaur illustration"
|
||||
},
|
||||
{
|
||||
"location": "Top right corner, quilted patchwork-pattern box, right of the box with 'TCH ORK' text",
|
||||
"partial_text": "",
|
||||
"art_notes": "Multicolored quilt/patchwork square pattern box, title not legible, partially cut at frame edge"
|
||||
},
|
||||
{
|
||||
"location": "Bottom right, below 'SANTO...' red box",
|
||||
"partial_text": "",
|
||||
"art_notes": "Small box with light blue/white pattern, mostly obscured by Utter Nonsense box"
|
||||
}
|
||||
],
|
||||
"IMG_4566.jpeg": [
|
||||
{
|
||||
"location": "Top shelf, background, partially obscured behind and above the two Alice Is Missing boxes",
|
||||
"partial_text": "TOWN...FUKU (possibly Japanese text)",
|
||||
"art_notes": "Colorful box with cartoon character illustrations, appears to be a small/medium sized game box, mostly obscured by foreground items"
|
||||
}
|
||||
],
|
||||
"IMG_4573.jpeg": [
|
||||
{
|
||||
"location": "Upper right area of the book cover art, stacked above 'ACTION CASTLE I' box",
|
||||
"partial_text": "AC... (partially obscured by hand/fingers)",
|
||||
"art_notes": "Red/orange box spine, appears to be part of a series with other 'Action Castle' related titles"
|
||||
},
|
||||
{
|
||||
"location": "Bottom right of cover art, near 'Blackboard' box",
|
||||
"partial_text": "",
|
||||
"art_notes": "Small dark box with logo icon, title illegible due to size and angle"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -0,0 +1,114 @@
|
||||
[
|
||||
"IMG_4501.jpeg|Bottom shelf, left portion, above 'Exploding Kittens' box|EXPANSION|Small box with partial text visible, colorful design, title cut off by frame edge",
|
||||
"IMG_4501.jpeg|Bottom shelf, right side, near Exploding Kittens box|COSMIC|Colorful box spine, blue/green tones, title partially visible but not fully confirmable",
|
||||
"IMG_4501.jpeg|Middle shelf, purple box beneath Joking Hazard, right side of stack|A Boardgame for 1 to 6 players, ages 8 and up.|Purple/violet box, text describing player count, title not legible from this angle",
|
||||
"IMG_4501.jpeg|Right side, middle shelf, black spine sticking out between Santorini and purple box below|T|Black spine, single letter visible, rest obscured",
|
||||
"IMG_4501.jpeg|Top shelf, far left edge, cut off by frame; near 'a Gentle Rain' box||Dark/black box, only a thin sliver visible",
|
||||
"IMG_4502.jpeg|bottom-left corner of shelf, beneath the History of the World area, left of Dungeon!|A Boardgame for 1 to 6 players, ages 8 and up|purple/tan box with descriptive text visible but no title readable",
|
||||
"IMG_4502.jpeg|far left side of shelf, stacked horizontally beneath the yellow-floral box, left of Santorini||small colorful box spines, red and white coloring, too angled/blurry to read title",
|
||||
"IMG_4502.jpeg|left side of shelf, below the horizontal stack, left of Santorini and above Joking Hazard||blue box with white illustration, appears to be a game box but title not legible",
|
||||
"IMG_4502.jpeg|top-left corner of shelf, above Santorini and to the left of Wiz-War|possibly 'Herbaceous' or similar, text partially cut off|cream/white box with floral yellow illustration, thin spine",
|
||||
"IMG_4505.jpeg|Bottom area, left side near Mantis Falls, small figure/box partially visible||Small gold/tan colored object, possibly a game piece or small box, mostly obscured",
|
||||
"IMG_4505.jpeg|Top shelf, far left edge behind Cat Crimes, cut off by frame edge||Dark brown/maroon spine with faint gold decorative border, partially visible vertical box",
|
||||
"IMG_4505.jpeg|Top shelf, far left edge, partially cut off by frame, to the left of Cat Crimes||Dark brown/maroon box spine with gold decorative swirl design, portion visible at left edge of image",
|
||||
"IMG_4506.jpeg|Middle area, below and left of Santorini, dark red/maroon box spine with small square icons||Dark reddish-brown spine with small colored square icons in a row, text not legible",
|
||||
"IMG_4506.jpeg|Right side of image, behind Santorini box, dark box with small visible logo||Dark/black box with a small circular or square colored logo element, mostly obscured by foreground Santorini box",
|
||||
"IMG_4506.jpeg|Right side, background behind Santorini box, partially visible dark box with small game piece icon||Dark background box with a small blue/white icon, mostly obscured by Santorini box in foreground",
|
||||
"IMG_4506.jpeg|Top shelf, above Santorini spine, partially visible box with patterned tiles/squares in yellow, blue, and cream tones||Patchwork/quilt-like pattern of colored squares (yellow, tan, blue) along the top edge of the box, style suggests a tile-based board game",
|
||||
"IMG_4506.jpeg|Top shelf, right side near blue corner box, small red 'X' symbols visible on light blue background|X X|Light blue box edge with red X pattern, possibly a tally or scoring game box",
|
||||
"IMG_4506.jpeg|Top shelf, spine visible above Santorini box, partially cut off at top of frame||Multicolored patchwork/tile pattern spine in yellow, tan, and blue tones with geometric designs; appears to be a board game box edge but no title text is legible",
|
||||
"IMG_4506.jpeg|Upper right corner of image, above and right of Santorini spine, next to unidentified patterned box||Small red and blue box corner visible, mostly cut off by frame edge, no discernible text",
|
||||
"IMG_4507.jpeg|Below 'LABYRINTH', left side of shelf, partially cut off at frame edge|DRAGO...|Green/dark box, only top edge visible, text cut off",
|
||||
"IMG_4507.jpeg|Middle shelf area, partially hidden behind Labyrinth box and below Utter Nonsense, left side of the stack|Dragon...|Green/dark colored box spine, text cut off by adjacent boxes, only 'Dragon' prefix visible",
|
||||
"IMG_4507.jpeg|Top of stack, right of center, between 'a Gentle Rain' and 'SANToRINI'|PATCH / WORK (possibly)|Colorful patchwork quilt-style pattern on spine, small box",
|
||||
"IMG_4507.jpeg|Upper right area near Santorini and a Gentle Rain, appears to be a narrow spine box|PATCH WORK or similar|Colorful patchwork/quilt-like pattern visible on spine, tan and multicolor squares design",
|
||||
"IMG_4508.jpeg|Top shelf, center, small yellow tin sitting next to a black box near the stack of papers/magazines, above the Labyrinth box||Small yellow tin container, no visible text, could be a card game tin",
|
||||
"IMG_4508.jpeg|Top shelf, center, yellow tin box sitting next to the Super Mario Checkers box||Small yellow tin, no visible title text, could be a card game tin",
|
||||
"IMG_4508.jpeg|Top shelf, far left, behind the stack of papers/notebooks, next to the framed picture||Small black box with unreadable logo, mostly obscured by papers",
|
||||
"IMG_4508.jpeg|Top shelf, far left, partially cut off by frame edge, next to a green-framed picture and above the Super Mario Checkers box||Teal/green colored box edge, mostly obscured, shape suggests a game box but no readable text",
|
||||
"IMG_4508.jpeg|Top shelf, right of center, wooden box with metal clasp between the checkers box and the papers stack||Wooden box with brass hardware, no visible title, appears decorative or a game storage box",
|
||||
"IMG_4508.jpeg|Top shelf, right side, partially visible box behind the wooden box, between the stack of papers and the edge of the frame|SCRABBLE (partial letters visible)|Red and teal colored box edge, appears to be a Scrabble-branded box but mostly obscured",
|
||||
"IMG_4511.jpeg|Middle shelf, behind Superfight boxes, green/black box with skeletal figure artwork - between Superfight stack and the Rook box|...CTHULHU|Dark green box, illustrated skeletal/robed figure, likely a Cthulhu-themed game (e.g., Cthulhu Fluxx or Cthulhu Gloom) but exact title obscured",
|
||||
"IMG_4511.jpeg|Middle shelf, black box stacked below SUPERFIGHT, above ROOK; left side of shelf|...Go... Yourself|Black spine box, small white/colored text, similar size to Superfight box",
|
||||
"IMG_4511.jpeg|Middle shelf, black spine box stacked above Rook, to the left of the dice tray|...Yourself|Black box with orange/white text, only partial word 'Yourself' legible, title obscured by angle",
|
||||
"IMG_4511.jpeg|Right edge of shelf, partially cut off by frame, next to Cthulhu box|HEX|Blue box with yellow/tan geometric pattern, only edge visible",
|
||||
"IMG_4511.jpeg|Top right corner of shelf, box mostly out of frame, above the Hex box|Drag...|Orange and blue box, only a fragment of text visible, likely a dragon-themed game title",
|
||||
"IMG_4517.jpeg|Middle of shelf, between Pie Face! (left) and the blue box with bird artwork (right)|...Grenade...|Tall narrow box, dark red/brown color scheme, appears to have a hand grenade or bomb-like illustration",
|
||||
"IMG_4517.jpeg|Middle of shelf, right of the grenade-themed box and left of Walk the Plank!|Bird ... Bird ...|Blue box with tall bird-like cartoon character illustration, small/narrow box shape",
|
||||
"IMG_4517.jpeg|Middle shelf area, second box from left, between 'Pie Face!' and 'Bird on Your Bread?!'|...renade... Nintendo?|Narrow tall box, red and tan coloring, partially obscured by shadow and angle, appears to have small figure illustration",
|
||||
"IMG_4518.jpeg|Between Mage Wars Academy Core Set and Saberu Merry Men||Thin black spine, largely obscured, possibly a second small game box",
|
||||
"IMG_4519.jpeg|Far right of shelf, white spine box next to the two red 'G' logo boxes|Before I Kill You, Mister Bond... / Kill Dr. No Lucky|White worn spine with black text, possibly James Bond themed game, 'CHEAPEST GAMES' text also visible on spine which may be a store sticker rather than title",
|
||||
"IMG_4519.jpeg|Left shelf, red spine with black text between the cream D&D box and the red-orange boxes|THE BEST OF RA... (possibly 'RAMPHA' or similar)|Dark red spine with illustrated figure artwork, book-like thickness",
|
||||
"IMG_4519.jpeg|Middle shelf area, two matching red boxes with orange circular 'G' logo, positioned between the white D&D boxes on the left and the white spined books on the right|G (stylized orange circular logo)|Two identical red boxes, narrow spine width, orange/yellow circular logo with letter G, taped/worn edges",
|
||||
"IMG_4519.jpeg|Right-center of shelf, two matching red boxes standing between the D&D group on the left and the white 'Cheapest Games' spine on the right|Orange 'G' logo visible on both spines|Two identical red boxes with worn/taped edges, orange circular 'G' emblem, appears to be same game duplicated",
|
||||
"IMG_4520.jpeg|Middle shelf, between Carcassonne and Evolution: The Beginning||Plain wooden-colored vertical box, no visible text, appears to be a wood grain finish game box or insert",
|
||||
"IMG_4521.jpeg|Top edge of frame, above the Sleeping Gods box, partially cut off||Dark spines/boxes barely visible at top, too obscured by shadow and cropping to identify",
|
||||
"IMG_4521.jpeg|Top of shelf, above the Sleeping Gods box, partially cut off by frame edge||Dark spines of several boxes, indistinct due to shadow and angle, colors appear dark blue/black and tan",
|
||||
"IMG_4527.jpeg|Left edge of shelf, partially visible spines to the left of the main game box, on the top shelf|01, T...|Appears to be book-like spines, red and dark covers, stacked vertically, likely not games but partially obscured",
|
||||
"IMG_4527.jpeg|Right edge of shelf, behind/beside the Red Dragon Inn box, partially cut off by frame|4, 3|Dark colored spine/box edge, numbers visible but title obscured",
|
||||
"IMG_4528.jpeg|Top shelf, above Wiz-War box, partially obscured by rolled poster/tube on left side|SLUGFEST GAMES|Red and orange colored box with illustrated fantasy/comic style artwork, only top edge visible",
|
||||
"IMG_4528.jpeg|Top shelf, back corner above the Wiz-War box; only the publisher logo area is visible, partially obscured by other items|SLUGFEST GAMES|Colorful red/orange box art, only top publisher banner visible, rest of box obscured",
|
||||
"IMG_4529.jpeg|Below the 1000-piece puzzle box, spine showing armored warrior artwork, above the face-out Risk Lord of the Rings box|The Middle-earth|Dark battle scene with armored figures, likely same LOTR-themed game box viewed from spine angle",
|
||||
"IMG_4529.jpeg|Middle of stack between the puzzle box and Risk Lord of the Rings, tan/cream colored spine with Asian characters and blue castle icon|unreadable Asian characters, blue icon|Tan spine with small blue building/castle graphic and Asian script text",
|
||||
"IMG_4529.jpeg|Top of stack, above the 1000-piece puzzle box, partially cut off at top of frame|V, 3|Dark spines with minimal visible text, one shows a circular 'V' logo, another shows number '3' on purple/dark background; too obscured to identify title",
|
||||
"IMG_4529.jpeg|Top of stack, second partial box edge, above the puzzle box|0|White/light colored box edge with a circular icon, only corner visible",
|
||||
"IMG_4529.jpeg|Very top of stack, box behind/above the numbered fragments, purple/dark spine||Dark purple or navy spine, mostly obscured, edge of frame",
|
||||
"IMG_4529.jpeg|Very top of stack, partial box edge visible above the puzzle box|3|Dark colored spine/edge, only a sliver visible, number '3' printed in white",
|
||||
"IMG_4532.jpeg|Top right corner of frame, above Gloomhaven box, partially cut off|GLO...|Fiery orange/red background with dark textured lettering, appears to be another large box possibly related to Gloomhaven expansion",
|
||||
"IMG_4532.jpeg|Top right corner of image, partially cut off, above and behind the Gloomhaven box|GLO... (possibly part of another title starting with GLO)|Dark box with fiery orange/red background and large white curved text, only edge visible",
|
||||
"IMG_4535.jpeg|Top shelf, far right edge of frame, next to the last visible Etherfields big box||Black box spine, partially cut off by frame edge, appears similar in size to adjacent Etherfields big boxes",
|
||||
"IMG_4542.jpeg|Bottom right shelf area, dark spine next to unidentified train-themed box||Dark spine, illegible text, appears to be a board game box based on shape",
|
||||
"IMG_4542.jpeg|Right side shelf, below CATAN, dark spine with red/white text|T...T / Ride (possible)|Dark red/maroon spine with train-like graphic element, narrow box spine",
|
||||
"IMG_4542.jpeg|Right side shelf, below Catan box, next to unidentified black-spined game|T...C...|Teal/orange box with train-like graphic, partially obscured, could be a train-themed game",
|
||||
"IMG_4542.jpeg|Right side shelf, lower row near 'Catan' box||Dark spine with red accent, text too small/blurry to read",
|
||||
"IMG_4542.jpeg|Top area, small red square box with white 'M' logo, near Dragon Land|M|Small red square box with large white letter M, possibly a classic board game logo",
|
||||
"IMG_4542.jpeg|Top left corner of shelf, partially cut off by frame edge, above the Skip-Bo box|...O / From the Makers of...|Red box edge with white lettering, likely a card game box, mostly obscured",
|
||||
"IMG_4542.jpeg|Top right shelf, partially hidden behind Dragon Land box|From the Makers of, M|Red box with a large stylized 'M' logo and small icons, likely a Mattel-published card game (possibly UNO), edge cut off by frame",
|
||||
"IMG_4543.jpeg|Background, lower right area behind the hand holding Scrabble Slam, blurred and out of focus||Indistinct colorful box shapes, cannot make out title or artwork details due to heavy blur",
|
||||
"IMG_4543.jpeg|Right edge of frame, middle row, next to the Scrabble Slam box|CAT...|Dark colored box spine, partially cut off at right edge of photo, red/dark background with light text",
|
||||
"IMG_4543.jpeg|Top right corner of frame, behind and above the Scrabble Slam box, partially cut off by frame edge|Dogs Kitchen|Small box or book with illustrated cartoon-style cover, colorful, appears to feature a dog character; too small/blurry to confirm if it's a game",
|
||||
"IMG_4543.jpeg|right edge of image, behind Scrabble Slam box|CAT|Yellow/orange box edge visible, partial text 'CAT' in bold letters, rest obscured by foreground box",
|
||||
"IMG_4543.jpeg|top right corner of image, above and behind the main Scrabble Slam box|Dogs Kitchen|Small blue/purple box partially cut off at top edge, illustrated cartoon-style artwork",
|
||||
"IMG_4544.jpeg|Background shelf behind the Cat Stax box, blurred and out of focus, appears to be a row of game/book boxes||Multiple colorful spines/boxes including orange, white, and dark tones, too blurry to make out any text",
|
||||
"IMG_4544.jpeg|Background shelf behind the Cat Stax box, top of frame, multiple stacked boxes with no clearly identifiable neighbors||Blurry stack of colorful boxes (orange, white, dark tones) in background, out of focus, too indistinct to read any text",
|
||||
"IMG_4545.jpeg|Bottom right corner of image, yellowish box edge visible||Yellow/tan colored box spine, text not legible due to angle and cropping",
|
||||
"IMG_4545.jpeg|Lower left shelf area, dark colored box partially visible behind hand holding Pocket Farkel||Dark/black box edge visible, no legible text",
|
||||
"IMG_4545.jpeg|Top right shelf, box with 'Ravensburger' text visible, title obscured|Ravensburger|Colorful box, brand name visible but game title not legible",
|
||||
"IMG_4545.jpeg|Top shelf, far right edge, partially cut off by frame border|Ravens...|Yellow/orange box with small logo resembling a bird, likely Ravensburger branding but game title not legible",
|
||||
"IMG_4545.jpeg|Top shelf, right side, above the 'Ravensburger' labeled box, partial title visible at top edge of frame|DRAGON ...and|Orange/red box with dragon and fantasy artwork, cut off at top of image",
|
||||
"IMG_4545.jpeg|Top shelf, upper right corner, above and behind the Pocket Farkel tin, to the right of 'Bugs in the Kitchen'|DRAGON, land|Colorful fantasy-style box art, orange/red hues, dragon imagery visible, title cut off at top edge of frame",
|
||||
"IMG_4547.jpeg|Top area, right of the maze-like box, small orange/red box||Small red/orange box, text illegible, largely obscured",
|
||||
"IMG_4547.jpeg|Top center-right, small red/orange box near top edge||Small reddish box with indistinct artwork, mostly cut off at top frame",
|
||||
"IMG_4547.jpeg|Top left area, next to the '...AZE' box, second spine from left|UTTLES or similar|Bright blue box with light green accents, small toy-like image visible on top",
|
||||
"IMG_4547.jpeg|Top left corner of image, partially visible behind main box|AZE|Blue and green box, appears to be a puzzle or maze-themed game, mostly cut off by frame edge",
|
||||
"IMG_4547.jpeg|Top left corner of shelf, above the main held box, leftmost visible spine|...AZE|Blue and green box, partial text visible, cut off at frame edge",
|
||||
"IMG_4547.jpeg|Top right area, red-orange box between the dark box and top edge|Roll Pla...|Red/orange box with illustrated artwork, text partially cut off",
|
||||
"IMG_4547.jpeg|Top right corner of image|Dark T...|Dark blue/black box with white text, appears to be 'Dark T' something, possibly 'Dark Tower' or similar, cut off at frame edge",
|
||||
"IMG_4547.jpeg|Top right corner of shelf, above and to the right of the main held box|Dark Terr... or Dark Territory|Dark navy/black box with red and white text, partially obscured by other boxes",
|
||||
"IMG_4547.jpeg|Top right, near 'Dark T' box||Colorful box with red/orange tones, illustration unclear, partially obscured by other boxes",
|
||||
"IMG_4552.jpeg|Left edge of frame, vertical spine, sharply cropped|Giv... / Ci...|Dark spine with white lettering, partially visible artwork, cropped by frame edge",
|
||||
"IMG_4552.jpeg|Top left corner of frame, behind and above the Dark Cults package, no clearly identified games adjacent due to cropping|'Giv...'|Dark colored box with white/red text fragment, likely a card or board game box mostly out of frame",
|
||||
"IMG_4552.jpeg|Top right edge of frame, partially obscured behind the plastic-wrapped Dark Cults package; no identifiable neighboring games visible due to cropping|'...ard Game', '-6 Players', 'Yeah, Right.'|White box with black text, appears to be a separate board game box edge cut off by frame, cannot determine title",
|
||||
"IMG_4552.jpeg|Upper right edge of frame, partially cut off, behind the Dark Cults game|...ard Game, -6 Players, ...uring the, o your evil lair, llet in his head, ...ng the temptation, our prize, to tell, plans, to let him, eath and blow up, ...ir in the process, agine winning, Yeah. Right.|White/light colored box or insert with black text, appears to be another horror/villain themed party or card game, partially obscured by hand and folded materials",
|
||||
"IMG_4553.jpeg|Top right corner of image, partially cut off by frame edge, to the right of Give Me The Brain|Cards, ...6 Players|Small dark card game box, black background with white text, silhouette figure visible on partial box edge",
|
||||
"IMG_4553.jpeg|Top right corner of image, partially visible behind/beside the main game, edge of frame||White box with a black silhouette figure, appears to be a card or board game box edge, mostly cut off by frame",
|
||||
"IMG_4555.jpeg|Background, top area, right of the colorful box in top left||Yellow/tan colored box top, mostly obscured",
|
||||
"IMG_4555.jpeg|Background, top left corner of image, partially visible behind main box||Colorful box edge, mostly obscured by the Kill Doctor Lucky box in foreground",
|
||||
"IMG_4555.jpeg|Right side of image, vertical spine|Z-WAR or similar fragment|Dark spine with partial text visible, appears to be a game box on shelf",
|
||||
"IMG_4555.jpeg|top left corner of image, above the Kill Doctor Lucky box||Colorful box top edge visible, mostly cropped out of frame",
|
||||
"IMG_4555.jpeg|top right corner of image, partially visible behind and above the main game box|Z-WAR (or similar, partially cut off)|Dark spine, appears to be a book or game box, mostly obscured by hand holding main item",
|
||||
"IMG_4556.jpeg|Bottom of frame, left of 'Utter nonsense' box, partially stacked papers/boxes||White/light colored edge visible, mostly obscured by foreground objects",
|
||||
"IMG_4556.jpeg|Bottom right corner, right of the 'Cards Against' box||Purple/violet box, partially visible, text not legible",
|
||||
"IMG_4556.jpeg|Bottom right, below 'SANTO...' red box||Small box with light blue/white pattern, mostly obscured by Utter Nonsense box",
|
||||
"IMG_4556.jpeg|Far right edge of shelf, beside the 'SANTO' box||Colorful patterned box (patchwork-like quilt design), largely cut off at right frame edge",
|
||||
"IMG_4556.jpeg|Left side, lower area, below the Ravensburger-labeled box, left of Utter Nonsense box||Green box with dinosaur/reptile illustration, mostly cut off at left frame edge",
|
||||
"IMG_4556.jpeg|Left side, second row, to the left of the green dinosaur-themed box and below the dark unmarked box|Ravensbu...|Blue box with white text, appears to be a puzzle or game brand box, partially cut off at left edge",
|
||||
"IMG_4556.jpeg|Top left corner, above the green dinosaur-patterned spine and left of 'Ravensbu...' spine||Dark spine, mostly obscured, edge of frame",
|
||||
"IMG_4556.jpeg|Top left of shelf, above the 'Ravensburger'-labeled box, left of the dark unmarked box||Dark navy/black box, no legible text visible, top left corner of frame",
|
||||
"IMG_4556.jpeg|Top right corner, quilted patchwork-pattern box, right of the box with 'TCH ORK' text||Multicolored quilt/patchwork square pattern box, title not legible, partially cut at frame edge",
|
||||
"IMG_4556.jpeg|Top row, green textured spine with cartoon dinosaur, left of 'Ravensbu...' spine||Green speckled/scaly texture spine with small cartoon dinosaur illustration",
|
||||
"IMG_4556.jpeg|Top row, second shelf, spine reading 'Ravensbu...' between an unidentified dark box and green dinosaur-patterned spine|Ravensbu...|White spine with blue text, likely Ravensburger logo/publisher rather than title, top cut off",
|
||||
"IMG_4566.jpeg|Top shelf, background, partially obscured behind and above the two Alice Is Missing boxes|TOWN...FUKU (possibly Japanese text)|Colorful box with cartoon character illustrations, appears to be a small/medium sized game box, mostly obscured by foreground items",
|
||||
"IMG_4573.jpeg|Bottom right of cover art, near 'Blackboard' box||Small dark box with logo icon, title illegible due to size and angle",
|
||||
"IMG_4573.jpeg|Upper right area of the book cover art, stacked above 'ACTION CASTLE I' box|AC... (partially obscured by hand/fingers)|Red/orange box spine, appears to be part of a series with other 'Action Castle' related titles"
|
||||
]
|
||||
@@ -0,0 +1,117 @@
|
||||
action,bgg_id,collid,name,version_id,second_copy,status,timestamp,error
|
||||
add,334011,,A Gentle Rain,701315,,failed,2026-08-06T02:15:34+00:00,"TimeoutError: Locator.wait_for: Timeout 15000ms exceeded. Call log: - waiting for get_by_role(""dialog"").get_by_role(""heading"", name=re.compile(r""A\ Gentle\ Rain"", re.IGNORECASE)) to be visible"
|
||||
add,589,,Wiz-War,27352,,failed,2026-08-06T02:20:33+00:00,"Error: Locator.check: Error: strict mode violation: get_by_role(""dialog"").get_by_label(""Own"") resolved to 2 elements: 1) <input type=""checkbox"" ng-model=""item.status.own"" class=""ng-pristine ng-untouched ng-valid ng-empty""/> aka get_by_role(""checkbox"", name=""Own"", exact=True) 2) <input type=""checkbox"" ng-model=""item.status.prevowned"" class=""ng-pristine ng-untouched ng-valid ng-empty""/> aka get_by_role(""checkbox"", name=""Prev. Owned"") Call log: - waiting for get_by_role(""dialog"").get_by_label(""Own"")"
|
||||
add,334011,,A Gentle Rain,701315,,added,2026-08-06T02:21:32+00:00,
|
||||
add,125921,,Catan: Junior,476263,,added,2026-08-06T02:24:22+00:00,
|
||||
add,31260,,Agricola,297589,,added,2026-08-06T02:24:28+00:00,
|
||||
add,181304,,Mysterium,536289,,added_no_version,2026-08-06T02:24:34+00:00,version 'English edition 2018-2' not in picker; added without version
|
||||
add,312786,,Poetry for Neanderthals,514050,,added_no_version,2026-08-06T02:24:40+00:00,version 'English edition 2020' not in picker; added without version
|
||||
add,273240,,The Red Dragon Inn Smorgasbox,446069,,added,2026-08-06T02:24:46+00:00,
|
||||
add,419687,,Munchkin Big Box,711400,,failed,2026-08-06T02:38:12+00:00,"TimeoutError: Locator.click: Timeout 30000ms exceeded. Call log: - waiting for get_by_role(""dialog"").locator(""ul.pagination a[title=\""First Page\""]"").first - locator resolved to <a href="""" role=""menuitem"" title=""First Page"" class=""pagination-pager"" ng-click=""selectPage(1)""> ⇆⇆⇆⇆First ⇆⇆⇆</a> - attempting click action 2 × waiting for element to be visible, enabled and stable - element is not visible - retrying click action - waiting 20ms 2 × waiting for element to be visible, enabled and stable - element is not visible - retrying click action - waiting 100ms 55 × waiting for element to be visible, enabled and stable - element is not visible - retrying click action - waiting 500ms"
|
||||
add,252153,,Tang Garden,404702,,failed,2026-08-06T02:38:48+00:00,"TimeoutError: Locator.click: Timeout 30000ms exceeded. Call log: - waiting for get_by_role(""dialog"").locator(""ul.pagination a[title=\""First Page\""]"").first - locator resolved to <a href="""" role=""menuitem"" title=""First Page"" class=""pagination-pager"" ng-click=""selectPage(1)""> ⇆⇆⇆⇆First ⇆⇆⇆</a> - attempting click action 2 × waiting for element to be visible, enabled and stable - element is not visible - retrying click action - waiting 20ms 2 × waiting for element to be visible, enabled and stable - element is not visible - retrying click action - waiting 100ms 56 × waiting for element to be visible, enabled and stable - element is not visible - retrying click action - waiting 500ms"
|
||||
add,255984,,Sleeping Gods,701034,,failed,2026-08-06T02:39:23+00:00,"TimeoutError: Locator.click: Timeout 30000ms exceeded. Call log: - waiting for get_by_role(""dialog"").locator(""ul.pagination a[title=\""First Page\""]"").first - locator resolved to <a href="""" role=""menuitem"" title=""First Page"" class=""pagination-pager"" ng-click=""selectPage(1)""> ⇆⇆⇆⇆First ⇆⇆⇆</a> - attempting click action 2 × waiting for element to be visible, enabled and stable - element is not visible - retrying click action - waiting 20ms 2 × waiting for element to be visible, enabled and stable - element is not visible - retrying click action - waiting 100ms 56 × waiting for element to be visible, enabled and stable - element is not visible - retrying click action - waiting 500ms"
|
||||
add,589,,Wiz-War,27352,,added,2026-08-06T02:45:10+00:00,
|
||||
add,419687,,Munchkin Big Box,711400,,added,2026-08-06T02:45:16+00:00,
|
||||
add,252153,,Tang Garden,404702,,added,2026-08-06T02:45:23+00:00,
|
||||
add,255984,,Sleeping Gods,701034,,failed,2026-08-06T02:45:58+00:00,"TimeoutError: Locator.click: Timeout 30000ms exceeded. Call log: - waiting for get_by_role(""dialog"").locator(""li:has(.summary-item-thumbnail)"").filter(has_text=re.compile(r""Sleeping\ Gods\ \(English\ Gamefound\ edition\)\ \(2023\)"", re.IGNORECASE)).first"
|
||||
add,283766,,Sleeping Gods: Tides of Ruin,463140,,added,2026-08-06T02:46:07+00:00,
|
||||
add,255984,,Sleeping Gods,701034,,failed,2026-08-06T02:49:44+00:00,RuntimeError: 'Sleeping Gods (English Gamefound edition) (2023)' was listed a moment ago but vanished on the second pass — retryable
|
||||
add,280794,,Etherfields,458300,,added,2026-08-06T02:49:53+00:00,
|
||||
add,380837,,Botany,650545,,added,2026-08-06T02:50:00+00:00,
|
||||
add,393672,,Gloomhaven: Buttons & Bugs,669782,,failed,2026-08-06T02:50:07+00:00,RuntimeError: 'Gloomhaven: Buttons & Bugs (English edition) (2024)' was listed a moment ago but vanished on the second pass — retryable
|
||||
add,14535,,SPANC: Space Pirate Amazon Ninja Catgirls,29075,,added,2026-08-06T02:50:15+00:00,
|
||||
add,255984,,Sleeping Gods,701034,,added,2026-08-06T02:54:05+00:00,
|
||||
add,393672,,Gloomhaven: Buttons & Bugs,669782,,added,2026-08-06T02:54:12+00:00,
|
||||
add,9666,,Greed Quest,215137,,added,2026-08-06T02:54:19+00:00,
|
||||
add,40508,,Scrabble Slam!,644326,,added,2026-08-06T02:54:27+00:00,
|
||||
add,3181,,Farkle,87931,,added,2026-08-06T02:54:36+00:00,
|
||||
add,193485,,Dastardly Dirigibles,300446,,added,2026-08-06T02:54:54+00:00,
|
||||
add,226065,,Sheriff of Nottingham: Merry Men,353389,,added,2026-08-06T02:55:01+00:00,
|
||||
add,143884,,Machi Koro,453051,,added,2026-08-06T02:55:09+00:00,
|
||||
add,358880,,Etherfields: Kittenburg Expansion,603694,,added,2026-08-06T02:55:15+00:00,
|
||||
add,432858,,Artistry,734890,,added,2026-08-06T02:55:21+00:00,
|
||||
add,417060,,Botany: Tantalizing Trees,706506,,added,2026-08-06T02:55:27+00:00,
|
||||
add,417064,,Botany: Perilous Perfumes,706510,,added,2026-08-06T02:55:33+00:00,
|
||||
add,161928,,Utter Nonsense,,,added,2026-08-06T02:55:40+00:00,
|
||||
add,50381,,Cards Against Humanity,,,added,2026-08-06T02:55:44+00:00,
|
||||
add,1269,,Skip-Bo,,,added,2026-08-06T02:55:50+00:00,
|
||||
add,202982,,Scrawl,,,added,2026-08-06T02:55:56+00:00,
|
||||
add,195314,,Herbaceous,,,added,2026-08-06T02:56:02+00:00,
|
||||
add,320202,,Dragon Land,,,added,2026-08-06T02:56:07+00:00,
|
||||
add,2452,,Jenga,,,added,2026-08-06T02:56:12+00:00,
|
||||
add,42063,,Code Master,,,added,2026-08-06T02:56:18+00:00,
|
||||
add,340420,,Throw Throw Avocado,,,added,2026-08-06T02:56:23+00:00,
|
||||
add,1260,,Rook,,,added,2026-08-06T02:56:28+00:00,
|
||||
add,116,,Guillotine,,,added,2026-08-06T02:56:34+00:00,
|
||||
add,169124,,Flick 'em Up!,,,added,2026-08-06T02:56:40+00:00,
|
||||
add,20866,,The Rocky Horror Trivia Game,,,added,2026-08-06T02:56:45+00:00,
|
||||
add,269564,,Captain Marvel: Secret Skrulls,,,added,2026-08-06T02:57:37+00:00,
|
||||
add,18755,,Pie Face!,,,added,2026-08-06T02:57:41+00:00,
|
||||
add,140509,,Dungeons & Dragons,,,failed,2026-08-06T02:58:01+00:00,"TimeoutError: Locator.wait_for: Timeout 15000ms exceeded. Call log: - waiting for get_by_role(""dialog"").get_by_role(""heading"", name=re.compile(r""Dungeons\ \&\ Dragons"", re.IGNORECASE)) to be visible"
|
||||
add,201248,,Evolution: The Beginning,,,added,2026-08-06T02:58:05+00:00,
|
||||
add,362205,,Sleeping Gods: Primeval Peril,,,added,2026-08-06T02:58:11+00:00,
|
||||
add,453845,,Murder at the Manor,,,added,2026-08-06T02:58:16+00:00,
|
||||
add,329230,,Bluffaneer,,,added,2026-08-06T02:58:20+00:00,
|
||||
add,358320,,Sleeping Gods: Distant Skies,,,added,2026-08-06T02:58:26+00:00,
|
||||
add,389113,,Rivals,,,added,2026-08-06T02:58:31+00:00,
|
||||
add,324937,,Wiz-War (9th Edition),,,added,2026-08-06T02:58:35+00:00,
|
||||
add,295770,,Frosthaven,,,added,2026-08-06T02:58:40+00:00,
|
||||
add,174430,,Gloomhaven,,,added,2026-08-06T02:58:46+00:00,
|
||||
add,442312,,Deadly Doodles,,,added,2026-08-06T02:58:51+00:00,
|
||||
add,332398,,Everdell: The Complete Collection,,,added,2026-08-06T02:58:56+00:00,
|
||||
add,334065,,Verdant,,,added,2026-08-06T02:59:01+00:00,
|
||||
add,283155,,Calico,,,added,2026-08-06T02:59:06+00:00,
|
||||
add,291457,,Gloomhaven: Jaws of the Lion,,,added,2026-08-06T02:59:12+00:00,
|
||||
add,150925,,Hold Your Breath!,,,added,2026-08-06T02:59:18+00:00,
|
||||
add,2980,,The Simpsons Trivia Game,,,added,2026-08-06T02:59:23+00:00,
|
||||
add,1197,,Blink,,,added,2026-08-06T02:59:29+00:00,
|
||||
add,140509,,Dungeons & Dragons,,,failed,2026-08-06T03:05:04+00:00,"TimeoutError: Locator.wait_for: Timeout 15000ms exceeded. Call log: - waiting for get_by_role(""dialog"").get_by_role(""heading"", name=re.compile(r""Dungeons\ \&\ Dragons"", re.IGNORECASE)) to be visible"
|
||||
add,352819,,The Office Trivia Game,,,added,2026-08-06T03:05:09+00:00,
|
||||
add,163412,,Patchwork,,,added,2026-08-06T03:05:14+00:00,
|
||||
add,291847,,Mantis Falls,,,added,2026-08-06T03:05:19+00:00,
|
||||
add,2058,,Civilization: West Extension Map,,,added,2026-08-06T03:05:24+00:00,
|
||||
add,452833,,Artistry: Delightful Doorways Mini Expansion,,,added,2026-08-06T03:05:31+00:00,
|
||||
update,240,53429559,Britannia,24621,,failed,2026-08-06T03:06:04+00:00,"TimeoutError: Locator.click: Timeout 30000ms exceeded. Call log: - waiting for get_by_role(""row"").filter(has_text=re.compile(r""Britannia"", re.IGNORECASE)).first.get_by_role(""link"", name=re.compile(r""own"", re.IGNORECASE)).first"
|
||||
update,71,53429530,Civilization,24006,,failed,2026-08-06T03:06:39+00:00,"TimeoutError: Locator.click: Timeout 30000ms exceeded. Call log: - waiting for get_by_role(""row"").filter(has_text=re.compile(r""Civilization"", re.IGNORECASE)).first.get_by_role(""link"", name=re.compile(r""own"", re.IGNORECASE)).first"
|
||||
update,177,53429542,Advanced Civilization,24972,,failed,2026-08-06T03:07:12+00:00,"TimeoutError: Locator.click: Timeout 30000ms exceeded. Call log: - waiting for get_by_role(""row"").filter(has_text=re.compile(r""Advanced\ Civilization"", re.IGNORECASE)).first.get_by_role(""link"", name=re.compile(r""own"", re.IGNORECASE)).first"
|
||||
update,240,53429559,Britannia,24621,,failed,2026-08-06T03:08:42+00:00,"TimeoutError: Locator.click: Timeout 30000ms exceeded. Call log: - waiting for get_by_role(""row"").filter(has_text=re.compile(r""Britannia"", re.IGNORECASE)).first.get_by_role(""link"", name=re.compile(r""own"", re.IGNORECASE)).first"
|
||||
update,71,53429530,Civilization,24006,,failed,2026-08-06T03:09:17+00:00,"TimeoutError: Locator.click: Timeout 30000ms exceeded. Call log: - waiting for get_by_role(""row"").filter(has_text=re.compile(r""Civilization"", re.IGNORECASE)).first.get_by_role(""link"", name=re.compile(r""own"", re.IGNORECASE)).first"
|
||||
update,177,53429542,Advanced Civilization,24972,,failed,2026-08-06T03:09:50+00:00,"TimeoutError: Locator.click: Timeout 30000ms exceeded. Call log: - waiting for get_by_role(""row"").filter(has_text=re.compile(r""Advanced\ Civilization"", re.IGNORECASE)).first.get_by_role(""link"", name=re.compile(r""own"", re.IGNORECASE)).first"
|
||||
update,240,53429559,Britannia,24621,,updated,2026-08-06T03:17:05+00:00,
|
||||
update,71,53429530,Civilization,24006,,failed,2026-08-06T03:17:11+00:00,"Error: Page.goto: net::ERR_NETWORK_CHANGED at https://boardgamegeek.com/collection/user/***?objectid=71&own=1 Call log: - navigating to ""https://boardgamegeek.com/collection/user/***?objectid=71&own=1"", waiting until ""domcontentloaded"""
|
||||
update,177,53429542,Advanced Civilization,24972,,updated,2026-08-06T03:17:17+00:00,
|
||||
update,71,53429530,Civilization,24006,,updated,2026-08-06T03:17:36+00:00,
|
||||
update,3090,53429504,Doctor Who: The Game of Time & Space,25782,,updated,2026-08-06T03:17:55+00:00,
|
||||
update,3585,53429482,Sorcerer: The Game of Magical Conflict,211154,,updated,2026-08-06T03:18:02+00:00,
|
||||
update,2524,53429367,StarForce 'Alpha Centauri': Interstellar Conflict in the 25th Century,219137,,updated,2026-08-06T03:18:06+00:00,
|
||||
update,224,53429816,History of the World,20727,,updated,2026-08-06T03:18:11+00:00,
|
||||
update,9209,53429636,Ticket to Ride,294188,,updated,2026-08-06T03:18:17+00:00,
|
||||
update,13,53430595,Catan,347121,,updated,2026-08-06T03:18:24+00:00,
|
||||
update,1219,53430485,Labyrinth,488621,,updated,2026-08-06T03:18:32+00:00,
|
||||
update,137909,53429649,Bugs in the Kitchen,216474,,updated,2026-08-06T03:18:38+00:00,
|
||||
update,27710,53430142,Catan Dice Game,606657,,updated,2026-08-06T03:18:44+00:00,
|
||||
update,169784,53429953,Castle Panic: The Dark Titan,260454,,updated,2026-08-06T03:18:49+00:00,
|
||||
update,147370,53429666,Robot Turtles,231664,,updated,2026-08-06T03:18:55+00:00,
|
||||
update,43443,53430559,Castle Panic,103068,,updated,2026-08-06T03:19:02+00:00,
|
||||
update,24310,53429632,The Red Dragon Inn,374258,,updated,2026-08-06T03:19:06+00:00,
|
||||
update,39856,53429748,Dixit,296529,,updated,2026-08-06T03:19:11+00:00,
|
||||
update,35505,53430257,Walk the Plank!,781669,,updated,2026-08-06T03:19:15+00:00,
|
||||
update,164,53430020,"Before I Kill You, Mister Bond",28465,,updated,2026-08-06T03:19:20+00:00,
|
||||
update,181304,148198097,Mysterium,536289,,updated,2026-08-06T03:19:27+00:00,
|
||||
update,65244,53429887,Forbidden Island,31208,,updated,2026-08-06T03:19:33+00:00,
|
||||
update,95386,53429908,Tempurra,290572,,updated,2026-08-06T03:19:41+00:00,
|
||||
update,258,53430200,Fluxx,20840,,updated,2026-08-06T03:19:47+00:00,
|
||||
update,312786,148198100,Poetry for Neanderthals,514050,,updated,2026-08-06T03:19:52+00:00,
|
||||
update,188614,53430332,Simon's Cat Card Game,779792,,updated,2026-08-06T03:19:57+00:00,
|
||||
update,4324,53429850,Risk: The Lord of the Rings,26859,,updated,2026-08-06T03:20:01+00:00,
|
||||
update,1927,53429933,Munchkin,28144,,updated,2026-08-06T03:20:06+00:00,
|
||||
update,1784,53430007,Dark Cults,110926,,updated,2026-08-06T03:20:11+00:00,
|
||||
update,176,53429986,Give Me the Brain!,27673,,updated,2026-08-06T03:20:16+00:00,
|
||||
update,257,53430040,Kill Doctor Lucky,520364,,updated,2026-08-06T03:20:21+00:00,
|
||||
update,153999,53612880,"...and then, we held hands.",280845,,updated,2026-08-06T03:20:26+00:00,
|
||||
update,166976,53612866,Consentacle,410382,,updated,2026-08-06T03:20:32+00:00,
|
||||
update,231302,53430652,The Cat Game,361880,,updated,2026-08-06T03:20:36+00:00,
|
||||
update,40692,53430517,Small World,294178,,updated,2026-08-06T03:20:45+00:00,
|
||||
update,172503,53429718,Mage Wars Academy,265107,,updated,2026-08-06T03:20:49+00:00,
|
||||
update,104710,53429642,Wiz-War (Eighth Edition),117685,,updated,2026-08-06T03:20:53+00:00,
|
||||
update,1339,53430097,Dungeon!,34390,,updated,2026-08-06T04:01:15+00:00,
|
||||
|
@@ -78,3 +78,111 @@ runs): pagination controls in the version sub-view, exact post-save behavior
|
||||
(toast? dialog close? redirect?), and how the dialog differs when the game is
|
||||
ALREADY in the collection (second-copy flow must create a new entry, not edit
|
||||
the existing one).
|
||||
|
||||
## Login page (recon 2026-08-01, anonymous probe via Playwright)
|
||||
|
||||
- **Cloudflare Turnstile blocks headless browsers outright**: the headless
|
||||
shell never gets past "Just a moment..." (`cf-turnstile-response` hidden
|
||||
input, no form). A normal **headed** Chromium passed the check without
|
||||
interaction. Hence `bggpipe upload` runs headed by default; `--headless`
|
||||
exists but expect login to fail there. A first login in headed mode may
|
||||
still need one human click on the challenge widget; the session then
|
||||
persists via `storage_state.json` (gitignored).
|
||||
- **BGG pages never reach Playwright's `networkidle`** — ad/analytics
|
||||
requests poll forever. Navigate with `wait_until="domcontentloaded"` and
|
||||
rely on element-level auto-waiting.
|
||||
- Verified form selectors at `/login`: `#inputUsername` (name=`username`,
|
||||
formcontrolname=`username`), `#inputPassword`, and a button with
|
||||
accessible name **"Sign In"** (`type="button"` — Angular handles submit,
|
||||
so click the button rather than pressing Enter and hoping for a form
|
||||
submit). Labels "Username"/"Password" point at those ids. Cookie-consent
|
||||
checkboxes (Essential, Performance Analytics, ...) render on the same
|
||||
page but did not overlay the form in the probe.
|
||||
- Logged-in detection heuristic (unverified): the header shows a "Sign In"
|
||||
link only when logged out.
|
||||
|
||||
|
||||
## Verified against the live site (2026-08-06, first real uploads)
|
||||
|
||||
The add flow works end to end; every failure on the way was in code the
|
||||
earlier walkthrough had marked *verified*, and the parts marked
|
||||
*unverified* were mostly right. Corrections:
|
||||
|
||||
- **Sign In is an `<a class="btn">` with no `href`.** It therefore has no
|
||||
implicit `link` role: `get_by_role("link", name="Sign In")` matches
|
||||
nothing in any state. The header also hydrates after
|
||||
`domcontentloaded`, so for a moment neither Sign In nor Sign Out
|
||||
exists — a check resting on one absence silently concludes "signed in"
|
||||
and browses anonymously. Poll until one control or the other proves the
|
||||
state; treat "neither, after 30s" as an error.
|
||||
- **`get_by_label("Own")` also matches "Prev. Owned."** Use
|
||||
`get_by_role("checkbox", name="Own", exact=True)`.
|
||||
- **Version rows** are the `<li>`s carrying a thumbnail:
|
||||
`li:has(.summary-item-thumbnail)`. Plain `listitem` also catches the
|
||||
paging `<li>`s ("First", "Prev", "1", "…").
|
||||
- **Paging is an AngularJS `<ul class="pagination">` of anchors**, not
|
||||
buttons: `a[title="Next Page"]`, with the parent `<li>` gaining
|
||||
`disabled` at the end. Paging is client-side over an already-loaded
|
||||
list — no request per page.
|
||||
- **Every paging control renders TWICE**: a desktop set and a mobile set
|
||||
inside `<li class="visible-xs-*">`. A selector matches both, and
|
||||
`.first` may be the hidden one — Playwright then waits for it to become
|
||||
visible until it times out. Always click the first *visible* match.
|
||||
First/Prev may be unclickable on a desktop viewport (mobile-only
|
||||
variant). To return to page 1, click the visible numbered **"1"**
|
||||
anchor: the sub-view's paging state SURVIVES closing and reopening it
|
||||
(Angular keeps the scope), so a reopen lands wherever it was left, not
|
||||
on page 1.
|
||||
- **Match row text in Python, not with `has_text`.** Playwright's
|
||||
`has_text` regex tests raw `textContent`, which carries the markup's
|
||||
tabs and newlines; a whitespace-normalized capture will never equal it.
|
||||
Normalize both sides yourself and click the row by index.
|
||||
- **Row text is `<game name> (<version name>) (<year>)`**, and the game
|
||||
name is localized (a Czech edition's row starts "Spící bohové"). Match
|
||||
the version name inside its parentheses.
|
||||
- **The API's version name is not always the picker's string.** BGG's
|
||||
XML gives e.g. `English edition 2018-2` where the picker shows
|
||||
`(English edition) (2018)`. Match the full name first, then retry with
|
||||
a trailing year/printing qualifier stripped — and if that relaxed match
|
||||
hits more than one row, refuse and add version-less (never guess).
|
||||
- **An owned game's page has no "Add To" button**; it reads
|
||||
"In Collections (Own…)". That is the update flow's entry point.
|
||||
|
||||
### The update flow (verified 2026-08-06)
|
||||
|
||||
Do **not** go through the game page's dialog for an existing entry — it
|
||||
cannot say which copy it edits. The collection table can:
|
||||
|
||||
1. `/collection/user/<user>?objectid=<bgg_id>&own=1`.
|
||||
2. The version cell carries its own collid:
|
||||
`td.collection_version[onclick*="<collid>"]`. Clicking it opens an
|
||||
inline editor (the cell shows *Editing* meanwhile).
|
||||
3. The editor is a radio list whose **values are version ids**:
|
||||
`form[id^="form_version"] input[type="radio"][value="<version_id>"]`.
|
||||
Both the copy and the edition are therefore addressed exactly — no
|
||||
name matching, no pagination.
|
||||
4. Clicking the radio fires `CE_SaveData(cellid, collid, 'version')`
|
||||
itself. **There is no Save button.** The save has landed when the
|
||||
cell's text stops reading *Editing*.
|
||||
|
||||
This is strictly additive: it sets one field on one collid and cannot
|
||||
create a duplicate entry.
|
||||
|
||||
### BGG's collection export lags the site
|
||||
|
||||
After a successful update the website shows the new version immediately
|
||||
(the row's version cell reads e.g. "English first edition Year: 2012",
|
||||
and reopening the editor shows that radio checked), but the XML API's
|
||||
`/collection` export can still report the entry with no version — even
|
||||
with a cache-busting re-request. `diff` reads the API, so it will
|
||||
re-queue work that has already landed.
|
||||
|
||||
Consequences to keep in mind rather than "fix":
|
||||
|
||||
- The upload log is the authority on what this tool did; the API is the
|
||||
authority on what BGG has published. They disagree for a while.
|
||||
- `build_queue` skipping a job whose log says done is CORRECT here — the
|
||||
work is applied, and re-running it would be a no-op at best.
|
||||
- Anything user-facing should count PENDING jobs (queue rows minus what
|
||||
the log completed), never raw queue rows, or settled work reads as
|
||||
outstanding until the next diff.
|
||||
|
||||
+118
@@ -0,0 +1,118 @@
|
||||
# The bggpipe user's guide
|
||||
|
||||
Everything past the [README](../README.md)'s quick start: credentials, every stage and its flags, the web app, phones, RPGs, fixing the model's mistakes, and running without a BGG token. Screenshots of everything described here: the [tour](tour.md).
|
||||
|
||||
## Contents
|
||||
|
||||
- [Credentials and configuration](#credentials-and-configuration)
|
||||
- [The web app](#the-web-app)
|
||||
- [From your phone](#from-your-phone)
|
||||
- [The stages, from the terminal](#the-stages-from-the-terminal)
|
||||
- [Taking good shelf photos](#taking-good-shelf-photos)
|
||||
- [Fixing what the model gets wrong](#fixing-what-the-model-gets-wrong)
|
||||
- [RPGs on your shelves](#rpgs-on-your-shelves)
|
||||
- [Uploading safely](#uploading-safely)
|
||||
- [Running before your BGG token arrives](#running-before-your-bgg-token-arrives)
|
||||
- [Keeping your data safe from git](#keeping-your-data-safe-from-git)
|
||||
|
||||
## Credentials and configuration
|
||||
|
||||
The `init` wizard is idempotent — re-run it anytime to check status or add keys you skipped. It prompts for the credentials below (hidden input, saved to a `.env` it creates with owner-only permissions) and offers the one-time Playwright Chromium download. Prefer doing it by hand? Copy [.env.example](../.env.example) beside your data, fill it in, and run `playwright install chromium` yourself.
|
||||
|
||||
Secrets live in environment variables only, never in config files, code, or logs, and `.env` is gitignored. Every `bggpipe` command loads `.env` from the working directory by itself — real environment variables always win over the file, so [direnv](https://direnv.net/) users and CI overrides keep working unchanged.
|
||||
|
||||
Your credentials never leave your machine except to sign in to boardgamegeek.com itself — [the README spells out the full privacy picture](../README.md#why-it-asks-for-your-bgg-password--and-where-your-credentials-go). The password exists solely because BGG has no write API: adding games means driving the real website, in a visible browser window, on your computer. The saved browser session (`storage_state.json`) is credential-adjacent — it stays local and gitignored too.
|
||||
|
||||
| Variable | Used by | What it is |
|
||||
|---|---|---|
|
||||
| `ANTHROPIC_API_KEY` | extract | Anthropic API key |
|
||||
| `BGG_API_TOKEN` | resolve, diff, enrich | Bearer token from your registered BGG application |
|
||||
| `BGG_USERNAME` | diff, upload, enrich | Your BGG username (public, but kept in `.env` so it lives in one place) |
|
||||
| `BGG_PASSWORD` | upload (website login) | Your BGG password |
|
||||
|
||||
Non-secret knobs live in `config.toml`: `photos_dir`, `data_dir`, the BGG rate limit, and the vision setup — a `[vision.<provider>]` block per provider ("anthropic" or any OpenAI-compatible endpoint, including a local [Ollama](https://ollama.com/)), with `vision_provider` picking one. Local models read spines noticeably worse than frontier ones — expect a longer proofread pass on the Titles page, not a broken pipeline.
|
||||
|
||||
**What extraction costs:** the vision call is the pipeline's only paid step, and it's small — the author's full collection (65 shelf photos, 136 games) came to under a dollar on the default model (Claude Sonnet). It's also one-time: raw reads are cached per photo, so re-running extract is free, and only new or replaced photos are ever sent again. Everything BGG-side is free (the API token costs nothing).
|
||||
|
||||
## The web app
|
||||
|
||||
```sh
|
||||
bggpipe web # opens http://127.0.0.1:8377/ — the whole app in the browser
|
||||
```
|
||||
|
||||
Seven pages — Pipeline, Photos, Titles, Review, Queue, Library, and Help — all [pictured in the tour](tour.md). Stage runs execute one at a time in the background with live output:
|
||||
|
||||
|
||||
|
||||
Stage runs execute one at a time; every decision saves immediately; the pages live-follow the data files, so a stage run in another terminal shows up without a refresh. The real upload sits behind a confirmation (and behind a stub-data lock if synthetic test fixtures ever regenerate). The in-app **Help** page documents every status chip and keyboard shortcut.
|
||||
|
||||
## From your phone
|
||||
|
||||
The app is localhost-only by default. To use it from a phone or tablet on your network — proofreading from the couch, or shooting shelf photos straight into the pipeline — serve it to the LAN instead:
|
||||
|
||||
```sh
|
||||
bggpipe web --lan # localhost + your network, behind an access key
|
||||
```
|
||||
|
||||
Startup prints a pairing link (`?k=...`) and a QR code: point the phone's camera at the terminal and tap. Pairing is one-time per device — the key persists across restarts (`data/.lan_key`; delete it to revoke every paired device) and the cookie lasts a year. Save the page to the phone's home screen for the full-screen treatment, piper icon included.
|
||||
|
||||
To photograph shelves from the phone: on the Photos page, tap the drop zone and choose "Take Photo." The upload narrates its progress, and camera captures get unique `shelf-<timestamp>` names so rapid-fire shots never overwrite each other. The key is the only lock — there is no login behind it — so still prefer networks you trust (or use a device VPN like Tailscale against the localhost default instead).
|
||||
|
||||
## The stages, from the terminal
|
||||
|
||||
Every stage is also a command, and the two interfaces share all state:
|
||||
|
||||
```sh
|
||||
bggpipe extract # photos → titles.json (+ retake prompts)
|
||||
bggpipe resolve # titles → BGG ids/versions in matches.csv
|
||||
bggpipe review --web # review UI only
|
||||
bggpipe diff # compare against your BGG collection
|
||||
bggpipe upload --dry-run # ALWAYS inspect this first
|
||||
bggpipe upload --limit 1 # then one game, then small batches
|
||||
bggpipe enrich # full metadata → data/games.json
|
||||
```
|
||||
|
||||
Each stage skips work it has already done; `--force`/`--refresh` flags redo it. Every stage is idempotent and resumable: kill it mid-run, restart, lose nothing. All artifacts are flat CSV/JSON files you can inspect and edit. `review` without `--web` runs in the terminal instead.
|
||||
|
||||
## Taking good shelf photos
|
||||
|
||||
Straight-on, one shelf (or part of one) per shot, close enough that spine text is legible to a human. If you can't read it, the model can't either. Overlap between shots is fine: duplicate reads are deduped automatically, with the merge shown (and veto-able) in review. Boxes the model spots but can't identify become retake prompts in `unidentified.json` and the Photos page's "reshoot" tickets: photograph those boxes up close, drop the new photo in, and run `extract` again.
|
||||
|
||||
## Fixing what the model gets wrong
|
||||
|
||||
Vision reads aren't perfect, and you know things the photos don't show. The Titles page lets you **edit** a title (fix a misspelling, add publisher/edition/year/language cues you know offhand), **split** a line into per-photo copies when one title is actually several boxes, **remove** lines that aren't games at all, and **add** a game no photo caught. Every one of these is durable: the decision lands in a small committed store (`data/title_edits.json`, `data/title_splits.json`, `data/title_removals.json`, `data/title_additions.json`) that is replayed on every rebuild — re-running extract or resolve can never undo your curation. Undo any decision by deleting its record from the store.
|
||||
|
||||
Games BGG doesn't have at all can be kept as **local** library citizens: their detail page in the Library takes hand-written facts (players, playtime, publisher, notes) and a cover photo of your own, stored in `data/local_games.json` and `data/local_art/` — the only source such a game will ever have.
|
||||
|
||||
## RPGs on your shelves
|
||||
|
||||
Tabletop RPGs aren't in BGG's board-game database — they live on RPGGeek, which shares the same underlying API. When a title isn't found as a board game, bggpipe retries as an RPG: matches are identified, enriched (designers, publishers, genres from RPGGeek), and browsable in the library (filter: RPGs), but they stay **local only** — they're never uploaded, since your BGG collection can't hold them. When the automatic search can't reach the right database (BGG has board games named "Dungeons & Dragons" too), every Review card has explicit **search BGG** / **search RPGGeek** buttons.
|
||||
|
||||
## Uploading safely
|
||||
|
||||
`upload` drives a real logged-in browser session against your real account, so it is deliberately careful:
|
||||
|
||||
- `--dry-run` logs what would happen without touching the site — always read it first, then `--limit 1`, then small batches:
|
||||
|
||||
|
||||
- The browser runs **headed** by default — BGG's Cloudflare check blocks headless ones, and a first login may need one human click before the session is saved locally and reused.
|
||||
- Requests are slow on purpose (seconds between actions, per BGG's API policy); the Queue page shows exactly what will run before it runs, and `upload_log.csv` keeps a permanent record of every attempt.
|
||||
- `--retry-failed` re-attempts failures; `--verify` re-fetches your collection and cross-checks the log. Note that BGG's collection export can lag the website by hours — freshly-landed work may look missing to `diff`/`--verify` until it catches up.
|
||||
- Review decisions outrank the queue: re-deciding a match after `diff` retires its queued job automatically.
|
||||
|
||||
## Running before your BGG token arrives
|
||||
|
||||
BGG application approval can take a week or more. Until then: `extract` works immediately (it only needs the vision key), and `resolve` does what it can, parking the rest as "waiting on BGG API token" — it picks them up automatically once the token exists. Everything is saved as you go.
|
||||
|
||||
`diff` normally fetches your collection live, but there's a logged-in-browser exemption that needs no token: while signed in to BGG, save these two URLs as `data/collection_snapshot_base.xml` and `data/collection_snapshot_expansions.xml` (if you get a "queued" message, refresh after a few seconds):
|
||||
|
||||
- `https://boardgamegeek.com/xmlapi2/collection?username=YOU&own=1&version=1`
|
||||
- `https://boardgamegeek.com/xmlapi2/collection?username=YOU&own=1&version=1&subtype=boardgameexpansion`
|
||||
|
||||
## Keeping your data safe from git
|
||||
|
||||
The [README's quick start](../README.md#quick-start) — installed as a tool, run in a directory of your own — is the only supported way to use bggpipe on your collection. Everything the pipeline produces lives where you run it, and `uv tool upgrade bggpipe` picks up fixes without going anywhere near your data.
|
||||
|
||||
**Why not clone and run?** The source repo doubles as its author's live pipeline: `data/` ships with their real artifacts, committed and updated often. Run inside a clone and *your* data lands at git-tracked paths — the next `git pull` will refuse to merge, and the usual remedies (`git reset --hard`, `git checkout .`, `git stash`, `git clean -fdx`) would destroy your review decisions, hand-written games, upload log, and photos. bggpipe detects this arrangement and warns at `init` and on the web dashboard; don't ignore it.
|
||||
|
||||
One committed file of the author's data deserves a word: `data/STUB_DATA.marker` is normally absent. It appears only if the synthetic stub fixtures (from `scripts/write_stub_fixtures.py`) regenerate the CSVs, and the upload stage refuses to run while it exists — so placeholder data can never reach a real BGG account.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 224 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 748 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.1 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 588 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 654 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 332 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 445 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 731 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 841 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 695 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 798 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 390 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 360 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 378 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 468 KiB |
@@ -1,4 +1,8 @@
|
||||
# Shelf-to-BGG Collection Pipeline — Build Spec
|
||||
# Shelf-to-BGG Collection Pipeline — Design Contract
|
||||
|
||||
> This is the pipeline's contract — what must stay true, and why — kept
|
||||
> current as the design evolves. Read it before changing pipeline
|
||||
> semantics. How to *use* the pipeline is the [user's guide](guide.md).
|
||||
|
||||
## Goal
|
||||
|
||||
@@ -11,7 +15,7 @@ Beyond the bare game, capture **which edition/version I own** wherever the photo
|
||||
## Constraints & Context
|
||||
|
||||
- BGG has **no write API**. Reads go through the XML API2 (`https://boardgamegeek.com/xmlapi2/`); writes must automate the website itself with a real login session.
|
||||
- The XML API **requires a registered application** (boardgamegeek.com/using_the_xml_api, policy 2025-07-02): every request carries `Authorization: Bearer <token>` from the `BGG_API_TOKEN` env var, sent to `boardgamegeek.com` without a leading `www`. Register a free non-commercial application at boardgamegeek.com/applications — approval can take a week+, so development runs on recorded/stub XML fixtures until then.
|
||||
- The XML API **requires a registered application** (boardgamegeek.com/using_the_xml_api, policy 2025-07-02): every request carries `Authorization: Bearer <token>` from the `BGG_API_TOKEN` env var, sent to `boardgamegeek.com` without a leading `www`. Register a free non-commercial application at boardgamegeek.com/applications — approval can take a week+; offline development and tests run on recorded XML fixtures.
|
||||
- BGG's XML API queues collection requests: a first call may return HTTP 202 ("try again"). Retry with backoff.
|
||||
- BGG will throttle aggressive clients. Target ≤1 request every 2 seconds to any BGG endpoint, with jittered backoff on 429/503.
|
||||
- BGG changed API access policies in 2025; some older community tools broke. Don't depend on undocumented endpoints beyond XML API2 and the public website.
|
||||
@@ -60,12 +64,13 @@ Each stage reads the previous stage's artifact and writes its own. Re-running a
|
||||
- `unmatched` — nothing plausible found.
|
||||
- Expansions: BGG returns `boardgameexpansion` as a distinct type. Keep them — I own expansions and want them in the collection — but tag them so review can catch base-game/expansion confusion (a spine reading "Wingspan Europe" must not match base Wingspan).
|
||||
- Cache all BGG responses on disk (keyed by query/ID) so re-runs don't re-hit the API.
|
||||
- **Post-resolve dedupe**: rows resolving to the same (bgg_id, version_id — or both version-unknown) are the same physical game read twice (typo, partial spine) unless their extraction cues conflict (two editions). Losers get `match_status=merged` + a `merged_into` column pointing at the survivor — rows never silently disappear, the survivor keeps the combined photo provenance downstream, and review surfaces every merge with a veto that restores the row as a distinct approved match.
|
||||
- **Version resolution**: once a game ID is settled (auto or approved), fetch `/thing?id=<id>&versions=1` and score the version list against the extraction's edition cues (publisher, year, language, edition wording). Same three-way classification: a single clear winner is `version_auto`; multiple plausible → `version_ambiguous` (goes to review); no cues at all → `version_unknown` (acceptable — BGG allows collection entries with no version set, and guessing wrong is worse than leaving it blank).
|
||||
- Output: `matches.csv` with columns: `title_raw, bgg_id, bgg_name, year, type, match_status (auto|ambiguous|unmatched|approved|rejected), version_id, version_name, version_status (version_auto|version_ambiguous|version_unknown|version_approved), candidates_json, version_candidates_json, source_photos`.
|
||||
|
||||
### Stage 3 — `review`: Human review of ambiguous/unmatched
|
||||
|
||||
- A minimal local review flow. A TUI is fine (e.g., `rich`/`textual`), or a tiny localhost web page — builder's choice, but keep it dependency-light.
|
||||
- A minimal local review flow. A TUI is fine (e.g., `rich`/`textual`), or a tiny localhost web page — builder's choice, but keep it dependency-light. *(As built: both — a rich TUI and a six-page FastAPI app sharing one decision engine.)*
|
||||
- For each `ambiguous` item: show the raw title, source photo filename, and candidate list (name, year, type, BGG rank, owned count) → pick one, skip, or reject.
|
||||
- For each `unmatched` item: allow manual BGG ID entry or a free-text re-search.
|
||||
- For each `version_ambiguous` item: show the version candidates (version name, publisher, year, language) next to the edition cues from the photo → pick one, or mark `version_unknown`. Keep this pass optional/skippable — version review shouldn't block getting games uploaded.
|
||||
@@ -87,7 +92,7 @@ Each stage reads the previous stage's artifact and writes its own. Re-running a
|
||||
- Log in to boardgamegeek.com with credentials from env vars (`BGG_USERNAME`, `BGG_PASSWORD`). Never write credentials to disk or logs. Persist the browser session/storage state locally so repeat runs don't re-login.
|
||||
- For each row in `to_add.csv`: navigate to the game page, use the "Add to Collection" flow, set status **Owned**, and — when a `version_id` is present — set the specific version in the collection item's version picker before saving. Manually walk this flow once and document the selectors before automating; the version UI is the most fragile part.
|
||||
- Adding a second copy of an already-owned game must create a NEW collection entry, not edit the existing one.
|
||||
- **Update mode** (rows from `to_update.csv`): open the EXISTING collection entry (keyed by `collid`) rather than the add flow, set the version, save. Must never create a duplicate entry and never change any other field of the entry. Verify the already-owned dialog behavior manually first — flagged as unverified in `docs/bgg-upload-flow.md`.
|
||||
- **Update mode** (rows from `to_update.csv`): open the EXISTING collection entry (keyed by `collid`) rather than the add flow, set the version, save. Must never create a duplicate entry and never change any other field of the entry. The verified route (2026-08-06, documented in `docs/bgg-upload-flow.md`): the collection table's version cell (`td.collection_version[onclick*="<collid>"]`) opens an inline editor whose radio values ARE version ids; clicking a radio saves immediately.
|
||||
- Log every attempt to `upload_log.csv`: `bgg_id, name, status (added|already_present|failed), timestamp, error`.
|
||||
- Idempotent: skip IDs already logged as `added`; re-verify against a fresh collection fetch on `--verify`.
|
||||
- Deliberately slow: 2–4 s randomized delay between games. This is a real account on a community site — behave like a polite human.
|
||||
@@ -106,6 +111,8 @@ All artifacts are flat files in a `data/` directory — human-readable, git-frie
|
||||
|
||||
- `titles.json` — extraction output (stage 1)
|
||||
- `unidentified.json` — game boxes seen but not identified (stage 1); retake prompts
|
||||
- `title_edits.json`, `title_splits.json`, `title_removals.json` — durable human curation (corrected reads/cues, split-into-copies decisions, removed lines); replayed on every titles.json rebuild so re-extraction never undoes them
|
||||
- `unidentified_dismissed.json` — dismissed retake prompts (kept apart so rebuilds can't resurrect them)
|
||||
- `bgg_cache/` — cached XML API responses
|
||||
- `matches.csv` — the master matching table (stages 2–3)
|
||||
- `to_add.csv` — upload queue, new entries (stage 4)
|
||||
@@ -121,7 +128,7 @@ All artifacts are flat files in a `data/` directory — human-readable, git-frie
|
||||
|
||||
- **202 queue** on `/collection`: retry with backoff (2s, 5s, 10s, 30s; give up after ~5 tries with a clear message).
|
||||
- **HEIC photos** from iPhone: convert transparently.
|
||||
- **Duplicate copies**: a title appearing in multiple photos with consistent edition cues is one game (dedupe). But genuinely distinct editions of the same game ARE in scope — they stay separate entries end-to-end and become separate BGG collection entries. Identical duplicate copies of the same edition are out of scope (assume dedupe).
|
||||
- **Duplicate copies**: a title appearing in multiple photos with consistent edition cues is one game (dedupe). But genuinely distinct editions of the same game ARE in scope — they stay separate entries end-to-end and become separate BGG collection entries. *(Amended as built:)* identical duplicate copies of the same edition are supported too, as an explicit human decision — a photo-scoped "split into copies" recorded in `data/title_splits.json`, honored by every dedupe pass thereafter.
|
||||
- **Non-game items** on shelves (books, card sleeves, storage boxes): the vision prompt should be instructed to include only board/card games; anything that slips through will fail resolution and land in review.
|
||||
- **Special characters** in titles (é, colons, ampersands): normalize consistently on both sides of the match.
|
||||
- **Base game vs. expansion vs. new edition**: the most common failure mode. Bias toward surfacing these as `ambiguous` rather than auto-matching.
|
||||
@@ -0,0 +1,48 @@
|
||||
# A tour of bggpipe
|
||||
|
||||
Seven pages in one local app — Pipeline, Photos, Titles, Review, Queue, Library, and Help. Every screenshot below is the real app on the author's real shelves, and the end product is public: [the author's BGG collection](https://boardgamegeek.com/collection/user/ewagoner) is what this pipeline built. (Back to the [README](../README.md) · how to use it all: the [user's guide](guide.md).)
|
||||
|
||||
**The Pipeline page** — every stage is a card with live counts and a Run button; blockers (missing token, stub-data lock) surface as banners, not surprises. Here: the settled state after a full run — 136 titles read, 115 matched, 62 added.
|
||||
|
||||
|
||||
|
||||
**Photos** — drag shelf photos in; boxes the vision model saw but couldn't read become illustrated reshoot tickets with shelf directions a human can follow.
|
||||
|
||||
|
||||
|
||||
**Titles** — every read off the shelves, alphabetized with status and source photos. This is the proofreading checkpoint: edit misreads, split multi-copy lines, remove non-games, add a game no photo caught.
|
||||
|
||||
|
||||
|
||||
**Review** — keyboard-first decisions on ambiguous matches. Each card shows the shelf photo beside the candidates (with rank, owner counts, and a view-on-BGG link per candidate), plus hand-steered re-searches of BGG or RPGGeek when the automatic search can't reach the right database. Here: is that box base Agricola or the Revised Edition?
|
||||
|
||||
|
||||
|
||||
**The editions pass** — after matches are settled, an optional pass picks which *printing* each copy is, scored against the cues read off the box. Skippable per game, and never blocks uploads. Here: five English Catan editions to choose between.
|
||||
|
||||
|
||||
|
||||
**Queue** — exactly what upload will do before it does it, and a permanent log of every attempt ever made. Here: six adds and one version update, pending.
|
||||
|
||||
|
||||
|
||||
**Library** — the enriched collection: searchable across titles, designers and mechanics, filterable by player count, sortable by rank/weight/year/time. Every card opens a detail page joining BGG's data with your own shelf photos; off-BGG games take hand-written facts and a cover photo there.
|
||||
|
||||
|
||||
|
||||
**A game's detail page** — BGG's stats, chips, and description joined with what only the pipeline knows: *your* edition, and the shelf photo it was read from.
|
||||
|
||||
|
||||
|
||||
**Help** — the whole flow, every page, every status, and every keyboard shortcut, documented in-app.
|
||||
|
||||
|
||||
|
||||
**And on a phone** (`--lan`) — the same app, paired once by QR code: the hamburger menu, shooting shelf photos straight into the pipeline from the camera, proofreading titles from the couch, and the piper on the Help page.
|
||||
|
||||
<p>
|
||||
<img src="screenshots/08-phone-menu.jpeg" alt="Phone view: the hamburger menu open over the Photos page, showing all seven pages with attention badges" width="24%">
|
||||
<img src="screenshots/09-phone-camera.jpeg" alt="Phone view: tapping the photo drop zone offers iOS's Photo Library / Take Photo / Choose Files sheet" width="24%">
|
||||
<img src="screenshots/10-phone-titles.jpeg" alt="Phone view: the Titles page as stacked cards with status chips, edit and split buttons, and the shaky-reads filter" width="24%">
|
||||
<img src="screenshots/11-phone-piper.jpeg" alt="Phone view: the Help page's credits card with Juniper's full piper artwork and the BGG trademark attribution" width="24%">
|
||||
</p>
|
||||
+35
-1
@@ -1,8 +1,26 @@
|
||||
[project]
|
||||
name = "bggpipe"
|
||||
version = "0.1.0"
|
||||
dynamic = ["version"]
|
||||
description = "Shelf-to-BGG collection pipeline: photos in, BoardGameGeek collection out"
|
||||
readme = "README.md"
|
||||
license = "MIT"
|
||||
license-files = ["LICENSE"]
|
||||
requires-python = ">=3.12"
|
||||
authors = [{ name = "Eric Wagoner", email = "eric@ericwagoner.com" }]
|
||||
keywords = ["boardgamegeek", "bgg", "board-games", "collection", "vision", "cataloging"]
|
||||
classifiers = [
|
||||
# Beta is deliberate while the user base is one person on one platform,
|
||||
# however battle-tested that person's collection is
|
||||
"Development Status :: 4 - Beta",
|
||||
"Environment :: Console",
|
||||
"Environment :: Web Environment",
|
||||
"Intended Audience :: End Users/Desktop",
|
||||
"Operating System :: MacOS",
|
||||
"Operating System :: POSIX :: Linux",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
"Topic :: Games/Entertainment :: Board Games",
|
||||
]
|
||||
|
||||
dependencies = [
|
||||
"typer>=0.12",
|
||||
"httpx>=0.27",
|
||||
@@ -12,8 +30,21 @@ dependencies = [
|
||||
"pillow>=12.3.0",
|
||||
"pillow-heif>=1.5.0",
|
||||
"rich>=15.0.0",
|
||||
"fastapi>=0.141.1",
|
||||
"uvicorn>=0.52.1",
|
||||
"playwright>=1.62.0",
|
||||
"pydantic>=2.13.4",
|
||||
"python-multipart>=0.0.32",
|
||||
"qrcode>=8.2",
|
||||
]
|
||||
|
||||
[project.urls]
|
||||
Homepage = "https://kestrelsnest.social"
|
||||
Repository = "https://git.kestrelsnest.social/eric/bggpipe"
|
||||
Documentation = "https://git.kestrelsnest.social/eric/bggpipe/src/branch/main/docs/guide.md"
|
||||
Mastodon = "https://toots.kestrelsnest.social/@eric"
|
||||
Bluesky = "https://bsky.app/profile/kestrelsnest.social"
|
||||
|
||||
[project.scripts]
|
||||
bggpipe = "bggpipe.cli:app"
|
||||
|
||||
@@ -23,6 +54,9 @@ dev = [
|
||||
"ruff>=0.5",
|
||||
]
|
||||
|
||||
[tool.hatch.version]
|
||||
path = "src/bggpipe/__init__.py"
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
"""Shared plumbing for the two stub-fixture generators.
|
||||
|
||||
Both write into the same cache dirs, so they must agree on XML escaping
|
||||
(a title containing & or " must not produce malformed XML) and on the
|
||||
provenance-marker text the upload guard depends on.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from bggpipe.config import STUB_CACHE_MARKER_NAME, STUB_DATA_MARKER_NAME
|
||||
|
||||
FIXTURE_CACHE = Path("tests/fixtures/bgg_cache")
|
||||
|
||||
CACHE_MARKER_TEXT = (
|
||||
"This cache contains hand-written stub XML, not real BGG "
|
||||
"responses. Data resolved from it must not be uploaded.\n"
|
||||
)
|
||||
DATA_MARKER_TEXT = (
|
||||
"The CSVs in this directory were resolved from hand-written stub "
|
||||
"fixtures, not real BGG data — version_ids are SYNTHETIC. The "
|
||||
"upload stage refuses to run while this file exists. Delete it "
|
||||
"only after re-resolving against real recorded fixtures "
|
||||
"(BGG_API_TOKEN + scripts/record_fixtures.py + resolve --force).\n"
|
||||
)
|
||||
|
||||
|
||||
def esc(text: str) -> str:
|
||||
"""Minimal XML attribute/text escaping for hand-built fixture strings."""
|
||||
return str(text).replace("&", "&").replace("<", "<").replace('"', """)
|
||||
|
||||
|
||||
def write_cache_marker(target: Path) -> None:
|
||||
(target / STUB_CACHE_MARKER_NAME).write_text(CACHE_MARKER_TEXT)
|
||||
|
||||
|
||||
def write_data_marker(data_dir: Path = Path("data")) -> None:
|
||||
data_dir.mkdir(parents=True, exist_ok=True)
|
||||
(data_dir / STUB_DATA_MARKER_NAME).write_text(DATA_MARKER_TEXT)
|
||||
@@ -13,11 +13,11 @@ from __future__ import annotations
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from fixture_common import FIXTURE_CACHE
|
||||
|
||||
from bggpipe.bgg_client import BGGClient
|
||||
from bggpipe.resolve import load_titles, resolve_entry
|
||||
|
||||
FIXTURE_CACHE = Path("tests/fixtures/bgg_cache")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
if not os.environ.get("BGG_API_TOKEN"):
|
||||
|
||||
@@ -15,12 +15,13 @@ Usage: uv run python scripts/write_photo_fixtures.py
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from fixture_common import FIXTURE_CACHE, esc, write_cache_marker, write_data_marker
|
||||
|
||||
from bggpipe.bgg_client import cache_key
|
||||
from bggpipe.bgg_client import SEARCH_TYPES, cache_key
|
||||
from bggpipe.config import Config
|
||||
|
||||
TARGETS = (FIXTURE_CACHE, Config().cache_dir)
|
||||
|
||||
TARGETS = (Path("tests/fixtures/bgg_cache"), Path("data/bgg_cache"))
|
||||
SEARCH_TYPES = "boardgame,boardgameexpansion"
|
||||
|
||||
BG, EXP = "boardgame", "boardgameexpansion"
|
||||
|
||||
@@ -98,6 +99,28 @@ SEARCHES: dict[str, list[tuple]] = {
|
||||
"History of the World": [(224, "History of the World", 1991, BG, "primary")],
|
||||
"Dungeon!": [(1339, "Dungeon!", 1975, BG, "primary")],
|
||||
"DUNGEON!": [(1339, "Dungeon!", 1975, BG, "primary")], # spelling drift
|
||||
# Alice Is Missing (IMG_4566) is an RPG — a geekdo rpgitem, so the
|
||||
# boardgame,boardgameexpansion search realistically returns nothing for
|
||||
# the full titles or any truncation head. It belongs on RPGGeek, not in
|
||||
# the BGG boardgame collection; expect these to stay unmatched.
|
||||
"ALICE IS MISSING: A SILENT ROLE PLAYING GAME": [],
|
||||
"ALICE IS MISSING: A SILENT ROLE PLAYING": [],
|
||||
"ALICE IS MISSING: SILENT FALLS EXPANSION": [],
|
||||
"ALICE IS MISSING": [],
|
||||
"ALICE IS": [],
|
||||
}
|
||||
|
||||
# rpgitem-search results for titles that live on RPGGeek, not BGG. Ids are
|
||||
# SYNTHETIC like everything else here; the marker guards them. Queries not
|
||||
# listed get an empty rpgitem stub automatically.
|
||||
RPG = "rpgitem"
|
||||
RPG_SEARCHES: dict[str, list[tuple]] = {
|
||||
"ALICE IS MISSING: A SILENT ROLE PLAYING GAME": [
|
||||
(400001, "Alice Is Missing: A Silent Role Playing Game", 2020, RPG, "primary")
|
||||
],
|
||||
"ALICE IS MISSING: SILENT FALLS EXPANSION": [
|
||||
(400002, "Alice Is Missing: Silent Falls Expansion", 2023, RPG, "primary")
|
||||
],
|
||||
}
|
||||
|
||||
# ids-param -> [(bgg_id, name, year, type, owned, rank, [publishers])]
|
||||
@@ -134,6 +157,8 @@ THING_STATS: dict[str, list[tuple]] = {
|
||||
|
||||
# bgg_id -> [(version_id, name, year, [publishers], [languages])]
|
||||
VERSIONS: dict[int, list[tuple]] = {
|
||||
400001: [], # Alice Is Missing (rpgitem): no cataloged versions in stubs
|
||||
400002: [],
|
||||
235096: [(360982, "English edition", 2017, ["ThinkFun"], ["English"])],
|
||||
240: [
|
||||
(
|
||||
@@ -281,7 +306,7 @@ VERSIONS: dict[int, list[tuple]] = {
|
||||
def search_xml(results: list[tuple]) -> str:
|
||||
items = "".join(
|
||||
f'<item type="{type_}" id="{bgg_id}">'
|
||||
f'<name type="{name_type}" value="{_esc(name)}"/>'
|
||||
f'<name type="{name_type}" value="{esc(name)}"/>'
|
||||
f'<yearpublished value="{year}"/></item>'
|
||||
for bgg_id, name, year, type_, name_type in results
|
||||
)
|
||||
@@ -292,12 +317,12 @@ def stats_xml(things: list[tuple]) -> str:
|
||||
items = ""
|
||||
for bgg_id, name, year, type_, owned, rank, publishers in things:
|
||||
links = "".join(
|
||||
f'<link type="boardgamepublisher" id="1" value="{_esc(p)}"/>'
|
||||
f'<link type="boardgamepublisher" id="1" value="{esc(p)}"/>'
|
||||
for p in publishers
|
||||
)
|
||||
items += (
|
||||
f'<item type="{type_}" id="{bgg_id}">'
|
||||
f'<name type="primary" value="{_esc(name)}"/>'
|
||||
f'<name type="primary" value="{esc(name)}"/>'
|
||||
f'<yearpublished value="{year}"/>{links}'
|
||||
f"<statistics><ratings>"
|
||||
f'<owned value="{owned}"/>'
|
||||
@@ -312,14 +337,14 @@ def versions_xml(bgg_id: int, versions: list[tuple]) -> str:
|
||||
version_items = ""
|
||||
for vid, name, year, publishers, languages in versions:
|
||||
links = "".join(
|
||||
f'<link type="boardgamepublisher" id="1" value="{_esc(p)}"/>'
|
||||
f'<link type="boardgamepublisher" id="1" value="{esc(p)}"/>'
|
||||
for p in publishers
|
||||
) + "".join(
|
||||
f'<link type="language" id="1" value="{_esc(lang)}"/>' for lang in languages
|
||||
f'<link type="language" id="1" value="{esc(lang)}"/>' for lang in languages
|
||||
)
|
||||
version_items += (
|
||||
f'<item type="boardgameversion" id="{vid}">'
|
||||
f'<name type="primary" value="{_esc(name)}"/>'
|
||||
f'<name type="primary" value="{esc(name)}"/>'
|
||||
f'<yearpublished value="{year}"/>{links}</item>'
|
||||
)
|
||||
return (
|
||||
@@ -329,15 +354,13 @@ def versions_xml(bgg_id: int, versions: list[tuple]) -> str:
|
||||
)
|
||||
|
||||
|
||||
def _esc(text: str) -> str:
|
||||
return text.replace("&", "&").replace("<", "<").replace('"', """)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
files: dict[str, str] = {}
|
||||
for query, results in SEARCHES.items():
|
||||
key = cache_key("search", {"query": query, "type": SEARCH_TYPES})
|
||||
files[key] = search_xml(results)
|
||||
rpg_key = cache_key("search", {"query": query, "type": "rpgitem"})
|
||||
files[rpg_key] = search_xml(RPG_SEARCHES.get(query, []))
|
||||
for ids, things in THING_STATS.items():
|
||||
files[cache_key("thing", {"id": ids, "stats": "1"})] = stats_xml(things)
|
||||
for bgg_id, versions in VERSIONS.items():
|
||||
@@ -348,7 +371,12 @@ def main() -> None:
|
||||
target.mkdir(parents=True, exist_ok=True)
|
||||
for name, xml in files.items():
|
||||
(target / name).write_text(xml)
|
||||
# provenance marker: anything resolved from this cache is stub-derived
|
||||
# and NOT upload-ready; re-recording real fixtures removes the marker
|
||||
write_cache_marker(target)
|
||||
write_data_marker()
|
||||
print(f"Wrote {len(files)} fixture file(s) to {' and '.join(map(str, TARGETS))}")
|
||||
print("Wrote data/STUB_DATA.marker (committed; upload refuses while it exists)")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -12,19 +12,16 @@ Usage: uv run python scripts/write_stub_fixtures.py
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from fixture_common import FIXTURE_CACHE, esc, write_cache_marker
|
||||
|
||||
from bggpipe.bgg_client import cache_key
|
||||
|
||||
FIXTURE_CACHE = Path("tests/fixtures/bgg_cache")
|
||||
SEARCH_TYPES = "boardgame,boardgameexpansion"
|
||||
from bggpipe.bgg_client import SEARCH_TYPES, cache_key
|
||||
|
||||
|
||||
def search_item(bgg_id: int, name: str, year: int | None, type_: str) -> str:
|
||||
year_xml = f'<yearpublished value="{year}"/>' if year else ""
|
||||
return (
|
||||
f'<item type="{type_}" id="{bgg_id}">'
|
||||
f'<name type="primary" value="{name}"/>{year_xml}</item>'
|
||||
f'<name type="primary" value="{esc(name)}"/>{year_xml}</item>'
|
||||
)
|
||||
|
||||
|
||||
@@ -104,10 +101,15 @@ THINGS = {
|
||||
|
||||
def main() -> None:
|
||||
FIXTURE_CACHE.mkdir(parents=True, exist_ok=True)
|
||||
write_cache_marker(FIXTURE_CACHE)
|
||||
for query, items in SEARCHES.items():
|
||||
key = cache_key("search", {"query": query, "type": SEARCH_TYPES})
|
||||
total = items.count("<item ")
|
||||
(FIXTURE_CACHE / key).write_text(f'<items total="{total}">{items}</items>')
|
||||
# resolve falls back to an rpgitem search whenever the board-game
|
||||
# search runs dry: every known query needs an (empty) answer there
|
||||
rpg_key = cache_key("search", {"query": query, "type": "rpgitem"})
|
||||
(FIXTURE_CACHE / rpg_key).write_text('<items total="0"></items>')
|
||||
for (ids, flavor), xml in THINGS.items():
|
||||
key = cache_key("thing", {"id": ids, flavor: "1"})
|
||||
(FIXTURE_CACHE / key).write_text(xml)
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
"""Shelf-to-BGG collection pipeline."""
|
||||
|
||||
__version__ = "0.1.0"
|
||||
__version__ = "1.0.0"
|
||||
|
||||
+55
-14
@@ -18,16 +18,25 @@ from urllib.parse import urlencode
|
||||
|
||||
import httpx
|
||||
|
||||
from bggpipe import __version__
|
||||
from bggpipe.config import Config
|
||||
from bggpipe.fsio import atomic_write_text
|
||||
from bggpipe.models import (
|
||||
BGGResponseError,
|
||||
CollectionItem,
|
||||
SearchResult,
|
||||
ThingDetails,
|
||||
parse_collection,
|
||||
parse_search,
|
||||
parse_things,
|
||||
parse_things_full,
|
||||
validate_response,
|
||||
)
|
||||
|
||||
BASE_URL = "https://boardgamegeek.com/xmlapi2"
|
||||
# One home for the search-type filter: the fixture generators must build
|
||||
# cache keys with the byte-identical string or every lookup silently misses.
|
||||
SEARCH_TYPES = "boardgame,boardgameexpansion"
|
||||
QUEUE_BACKOFF = (2.0, 5.0, 10.0, 30.0) # sleeps between the 5 attempts (spec)
|
||||
MAX_ATTEMPTS = 5
|
||||
_UNSAFE = re.compile(r"[^A-Za-z0-9._=,-]+")
|
||||
@@ -64,7 +73,7 @@ class BGGClient:
|
||||
self._monotonic = monotonic
|
||||
self._rng = rng or random.Random()
|
||||
self._last_request: float | None = None
|
||||
headers = {"User-Agent": "bggpipe/0.1 (shelf-collection pipeline)"}
|
||||
headers = {"User-Agent": f"bggpipe/{__version__} (shelf-collection pipeline)"}
|
||||
# BGG requires registered applications since 2025: the token from
|
||||
# https://boardgamegeek.com/applications must accompany every request.
|
||||
# Env var only — never config, disk, or logs.
|
||||
@@ -83,10 +92,14 @@ class BGGClient:
|
||||
if wait > 0:
|
||||
self._sleep(wait)
|
||||
|
||||
def get_xml(self, endpoint: str, params: dict[str, str]) -> str:
|
||||
"""Fetch one endpoint, serving from and filling the disk cache."""
|
||||
def get_xml(
|
||||
self, endpoint: str, params: dict[str, str], *, refresh: bool = False
|
||||
) -> str:
|
||||
"""Fetch one endpoint, serving from and filling the disk cache.
|
||||
refresh=True skips the cache read (still writes) — for data that
|
||||
drifts over time, like ranks and ratings."""
|
||||
cache_path = self.cache_dir / cache_key(endpoint, params)
|
||||
if cache_path.exists():
|
||||
if cache_path.exists() and not refresh:
|
||||
return cache_path.read_text()
|
||||
|
||||
for attempt in range(MAX_ATTEMPTS):
|
||||
@@ -112,8 +125,13 @@ class BGGClient:
|
||||
continue
|
||||
|
||||
response.raise_for_status()
|
||||
self.cache_dir.mkdir(parents=True, exist_ok=True)
|
||||
cache_path.write_text(response.text)
|
||||
try:
|
||||
# error documents AND malformed/truncated bodies must never
|
||||
# reach the cache — they would poison every future run
|
||||
validate_response(response.text)
|
||||
except BGGResponseError as err:
|
||||
raise BGGResponseError(f"/{endpoint}: {err}") from err
|
||||
atomic_write_text(cache_path, response.text)
|
||||
return response.text
|
||||
|
||||
raise BGGQueueTimeout(
|
||||
@@ -124,14 +142,13 @@ class BGGClient:
|
||||
|
||||
# -- typed endpoint wrappers ------------------------------------------
|
||||
|
||||
def search(
|
||||
self, query: str, types: str = "boardgame,boardgameexpansion"
|
||||
) -> list[SearchResult]:
|
||||
def search(self, query: str, types: str = SEARCH_TYPES) -> list[SearchResult]:
|
||||
return parse_search(self.get_xml("search", {"query": query, "type": types}))
|
||||
|
||||
def things(
|
||||
self,
|
||||
ids: Iterable[int],
|
||||
*,
|
||||
stats: bool = False,
|
||||
versions: bool = False,
|
||||
) -> list[ThingDetails]:
|
||||
@@ -142,22 +159,46 @@ class BGGClient:
|
||||
params["versions"] = "1"
|
||||
return parse_things(self.get_xml("thing", params))
|
||||
|
||||
def things_full(self, ids: Iterable[int], *, refresh: bool = False) -> list[dict]:
|
||||
"""Full metadata dicts for the enrich stage."""
|
||||
params = {"id": ",".join(str(i) for i in ids), "stats": "1"}
|
||||
return parse_things_full(self.get_xml("thing", params, refresh=refresh))
|
||||
|
||||
def collection(
|
||||
self,
|
||||
username: str,
|
||||
*,
|
||||
subtype: str | None = None,
|
||||
version: bool = True,
|
||||
refresh: bool = False,
|
||||
) -> list[CollectionItem]:
|
||||
params = {"username": username, "own": "1"}
|
||||
if subtype:
|
||||
params["subtype"] = subtype
|
||||
if version:
|
||||
params["version"] = "1"
|
||||
return parse_collection(self.get_xml("collection", params))
|
||||
return parse_collection(self.get_xml("collection", params, refresh=refresh))
|
||||
|
||||
def collection_full(self, username: str) -> list[CollectionItem]:
|
||||
"""Owned items incl. expansions (excluded from the default subtype)."""
|
||||
base = self.collection(username)
|
||||
expansions = self.collection(username, subtype="boardgameexpansion")
|
||||
def collection_full(
|
||||
self, username: str, *, refresh: bool = False
|
||||
) -> list[CollectionItem]:
|
||||
"""Owned items incl. expansions (excluded from the default subtype).
|
||||
refresh=True bypasses the cache — upload --verify must see the live
|
||||
collection, not the snapshot resolve ran against."""
|
||||
base = self.collection(username, refresh=refresh)
|
||||
expansions = self.collection(
|
||||
username, subtype="boardgameexpansion", refresh=refresh
|
||||
)
|
||||
seen = {item.coll_id for item in base}
|
||||
return base + [e for e in expansions if e.coll_id not in seen]
|
||||
|
||||
|
||||
def client_for(cfg: Config) -> BGGClient:
|
||||
"""The standard injection fallback: every stage's `client or client_for(cfg)`."""
|
||||
return BGGClient(cfg.cache_dir, cfg.rate_limit_seconds)
|
||||
|
||||
|
||||
def cached_paths(cache_dir: Path, endpoint: str) -> list[Path]:
|
||||
"""Cache files for one endpoint. Glob the cache only through this
|
||||
helper so the filename layout stays private to cache_key."""
|
||||
return sorted(cache_dir.glob(f"{endpoint}_*.xml"))
|
||||
|
||||
+111
-14
@@ -7,24 +7,37 @@ from typing import Annotated
|
||||
|
||||
import typer
|
||||
|
||||
from bggpipe.config import load_config
|
||||
from bggpipe.config import DEFAULT_REVIEW_PORT, load_config
|
||||
|
||||
app = typer.Typer(
|
||||
help="Shelf-to-BGG collection pipeline.",
|
||||
no_args_is_help=True,
|
||||
)
|
||||
|
||||
|
||||
@app.callback()
|
||||
def _load_dotenv() -> None:
|
||||
# credentials live in ./.env (written by init); load them so a fresh
|
||||
# directory works without direnv or manual sourcing. Real environment
|
||||
# variables always take precedence over the file.
|
||||
from bggpipe.init_wizard import load_env_file
|
||||
|
||||
load_env_file(Path(".env"))
|
||||
|
||||
|
||||
ConfigOpt = Annotated[
|
||||
Path | None,
|
||||
typer.Option("--config", help="Path to config.toml (default: ./config.toml)"),
|
||||
]
|
||||
|
||||
|
||||
def _not_implemented(stage: str, build_order: int) -> None:
|
||||
typer.echo(
|
||||
f"bggpipe {stage}: not implemented yet (build-order step {build_order})."
|
||||
)
|
||||
raise typer.Exit(code=1)
|
||||
@app.command()
|
||||
def init(config: ConfigOpt = None) -> None:
|
||||
"""Guided first-run setup: folders, config, credentials, browser."""
|
||||
from bggpipe.init_wizard import run_init
|
||||
|
||||
cfg = load_config(config)
|
||||
run_init(cfg)
|
||||
|
||||
|
||||
@app.command()
|
||||
@@ -59,14 +72,66 @@ def resolve(
|
||||
|
||||
|
||||
@app.command()
|
||||
def review(config: ConfigOpt = None) -> None:
|
||||
def review(
|
||||
web: Annotated[
|
||||
bool, typer.Option("--web", help="Serve the review UI on localhost")
|
||||
] = False,
|
||||
port: Annotated[
|
||||
int, typer.Option("--port", help="Port for --web")
|
||||
] = DEFAULT_REVIEW_PORT,
|
||||
dev: Annotated[
|
||||
bool, typer.Option("--dev", help="With --web: restart on source changes")
|
||||
] = False,
|
||||
config: ConfigOpt = None,
|
||||
) -> None:
|
||||
"""Stage 3: human review of ambiguous/unmatched items."""
|
||||
cfg = load_config(config)
|
||||
if web:
|
||||
from bggpipe.webreview import run_web_review
|
||||
|
||||
run_web_review(cfg, port=port, dev=dev, config_path=config)
|
||||
else:
|
||||
from bggpipe.review import run_review
|
||||
|
||||
cfg = load_config(config)
|
||||
run_review(cfg)
|
||||
|
||||
|
||||
@app.command()
|
||||
def web(
|
||||
port: Annotated[
|
||||
int, typer.Option("--port", help="Port to serve on")
|
||||
] = DEFAULT_REVIEW_PORT,
|
||||
dev: Annotated[
|
||||
bool, typer.Option("--dev", help="Restart on source changes")
|
||||
] = False,
|
||||
no_browser: Annotated[
|
||||
bool, typer.Option("--no-browser", help="Don't open a browser tab")
|
||||
] = False,
|
||||
lan: Annotated[
|
||||
bool,
|
||||
typer.Option(
|
||||
"--lan",
|
||||
help="Also serve to your local network behind a per-run access "
|
||||
"key (trusted networks only)",
|
||||
),
|
||||
] = False,
|
||||
config: ConfigOpt = None,
|
||||
) -> None:
|
||||
"""The whole pipeline in a local web UI: photos, stages, review."""
|
||||
from bggpipe.webreview import run_web_review
|
||||
|
||||
cfg = load_config(config)
|
||||
run_web_review(
|
||||
cfg,
|
||||
port=port,
|
||||
dev=dev,
|
||||
lan=lan,
|
||||
config_path=config,
|
||||
landing="/",
|
||||
open_browser=not no_browser,
|
||||
)
|
||||
|
||||
|
||||
@app.command()
|
||||
def diff(config: ConfigOpt = None) -> None:
|
||||
"""Stage 4: diff approved matches against the existing BGG collection."""
|
||||
@@ -78,19 +143,51 @@ def diff(config: ConfigOpt = None) -> None:
|
||||
|
||||
@app.command()
|
||||
def upload(
|
||||
dry_run: Annotated[bool, typer.Option("--dry-run")] = False,
|
||||
verify: Annotated[bool, typer.Option("--verify")] = False,
|
||||
retry_failed: Annotated[bool, typer.Option("--retry-failed")] = False,
|
||||
dry_run: Annotated[
|
||||
bool, typer.Option("--dry-run", help="Show the queue without a browser")
|
||||
] = False,
|
||||
verify: Annotated[
|
||||
bool, typer.Option("--verify", help="Re-fetch the collection and cross-check")
|
||||
] = False,
|
||||
retry_failed: Annotated[
|
||||
bool, typer.Option("--retry-failed", help="Re-attempt previously failed games")
|
||||
] = False,
|
||||
limit: Annotated[
|
||||
int | None, typer.Option("--limit", help="Upload at most N games this run")
|
||||
] = None,
|
||||
headless: Annotated[
|
||||
bool,
|
||||
typer.Option(
|
||||
"--headless",
|
||||
help="Run the browser headless (Cloudflare may block login)",
|
||||
),
|
||||
] = False,
|
||||
config: ConfigOpt = None,
|
||||
) -> None:
|
||||
"""Stage 5: add games to the BGG collection via Playwright."""
|
||||
_not_implemented("upload", 5)
|
||||
from bggpipe.upload import run_upload
|
||||
|
||||
cfg = load_config(config)
|
||||
run_upload(
|
||||
cfg,
|
||||
dry_run=dry_run,
|
||||
verify=verify,
|
||||
retry_failed=retry_failed,
|
||||
limit=limit,
|
||||
headless=headless,
|
||||
)
|
||||
|
||||
|
||||
@app.command()
|
||||
def enrich(
|
||||
refresh: Annotated[bool, typer.Option("--refresh")] = False,
|
||||
refresh: Annotated[
|
||||
bool,
|
||||
typer.Option("--refresh", help="Re-fetch metadata (ranks/ratings drift)"),
|
||||
] = False,
|
||||
config: ConfigOpt = None,
|
||||
) -> None:
|
||||
"""Stage 6: fetch full game + version metadata into games.json."""
|
||||
_not_implemented("enrich", 6)
|
||||
from bggpipe.enrich import run_enrich
|
||||
|
||||
cfg = load_config(config)
|
||||
run_enrich(cfg, refresh=refresh)
|
||||
|
||||
+181
-5
@@ -1,18 +1,29 @@
|
||||
"""Configuration: config.toml at the project root, env vars override.
|
||||
"""Configuration: non-secret knobs from config.toml at the project root.
|
||||
|
||||
Secrets (ANTHROPIC_API_KEY, BGG_PASSWORD) are never stored here — they are
|
||||
read from the environment at the point of use and must never be written to
|
||||
disk or logs.
|
||||
Everything account-related lives in the environment (.env via direnv):
|
||||
secrets (ANTHROPIC_API_KEY, BGG_PASSWORD, BGG_API_TOKEN) because they must
|
||||
never touch disk or logs, and BGG_USERNAME — public, but kept there so the
|
||||
account has exactly one home.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import tomllib
|
||||
import warnings
|
||||
from dataclasses import dataclass, replace
|
||||
from pathlib import Path
|
||||
|
||||
DEFAULT_CONFIG_PATH = Path("config.toml")
|
||||
DEFAULT_REVIEW_PORT = 8377
|
||||
# Provenance marker filenames — the upload guard and both fixture
|
||||
# generators must agree on these exactly.
|
||||
STUB_CACHE_MARKER_NAME = "STUB_FIXTURES.marker"
|
||||
STUB_DATA_MARKER_NAME = "STUB_DATA.marker"
|
||||
|
||||
|
||||
VISION_PROVIDERS = ("anthropic", "openai-compatible")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -22,6 +33,11 @@ class Config:
|
||||
data_dir: Path = Path("data")
|
||||
model: str = "claude-sonnet-5"
|
||||
rate_limit_seconds: float = 2.0
|
||||
# "anthropic" (default) or "openai-compatible" — the latter covers
|
||||
# OpenAI, OpenRouter, and local runtimes (Ollama, LM Studio, vLLM)
|
||||
vision_provider: str = "anthropic"
|
||||
vision_base_url: str = "" # e.g. http://localhost:11434/v1 for Ollama
|
||||
vision_key_env: str = "OPENAI_API_KEY" # "" = endpoint needs no key
|
||||
|
||||
@property
|
||||
def cache_dir(self) -> Path:
|
||||
@@ -39,21 +55,181 @@ class Config:
|
||||
def matches_path(self) -> Path:
|
||||
return self.data_dir / "matches.csv"
|
||||
|
||||
@property
|
||||
def extract_raw_dir(self) -> Path:
|
||||
return self.data_dir / "extract_raw"
|
||||
|
||||
@property
|
||||
def to_add_path(self) -> Path:
|
||||
return self.data_dir / "to_add.csv"
|
||||
|
||||
@property
|
||||
def to_update_path(self) -> Path:
|
||||
return self.data_dir / "to_update.csv"
|
||||
|
||||
@property
|
||||
def upload_log_path(self) -> Path:
|
||||
return self.data_dir / "upload_log.csv"
|
||||
|
||||
@property
|
||||
def games_path(self) -> Path:
|
||||
return self.data_dir / "games.json"
|
||||
|
||||
def tracked_data_warning(self) -> str | None:
|
||||
"""The clone-and-run trap: pipeline data written at git-TRACKED
|
||||
paths is one `git pull` + one panicked `git reset --hard` away
|
||||
from destruction. Warn whenever artifacts under data_dir are
|
||||
tracked — unless the owner has marked this checkout as their own
|
||||
repo (the marker is gitignored, so a fresh clone never inherits
|
||||
the exemption)."""
|
||||
if (self.data_dir / ".own_repo").exists():
|
||||
return None
|
||||
try:
|
||||
tracked = subprocess.run( # noqa: S603
|
||||
["git", "ls-files", "--", str(self.data_dir)], # noqa: S607
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=10,
|
||||
cwd=self.data_dir.resolve().parent,
|
||||
)
|
||||
except (OSError, subprocess.TimeoutExpired):
|
||||
return None # no git, no repo, no trap
|
||||
if tracked.returncode != 0 or not tracked.stdout.strip():
|
||||
return None
|
||||
return (
|
||||
f"your pipeline data ({self.data_dir}) sits at git-TRACKED "
|
||||
"paths — a `git pull` here can refuse to merge, and the usual "
|
||||
"remedies (reset --hard, checkout ., stash) would destroy your "
|
||||
"review decisions, hand-written games, and upload log. Run "
|
||||
"bggpipe from a directory outside this repository (see README: "
|
||||
"Bring your own shelves). If this repo is genuinely where you "
|
||||
f"version your own data, `touch {self.data_dir}/.own_repo` to "
|
||||
"accept the arrangement and silence this warning."
|
||||
)
|
||||
|
||||
@property
|
||||
def local_games_path(self) -> Path:
|
||||
# hand-written metadata for games BGG doesn't have — the only
|
||||
# source of truth for them, so it is committed like the other stores
|
||||
return self.data_dir / "local_games.json"
|
||||
|
||||
@property
|
||||
def local_art_dir(self) -> Path:
|
||||
# cover photos for off-BGG games (committed: nothing else has them)
|
||||
return self.data_dir / "local_art"
|
||||
|
||||
@property
|
||||
def title_additions_path(self) -> Path:
|
||||
# games the human added without a photo (expansions stored inside
|
||||
# base boxes, unshelved games) — joined into every rebuild
|
||||
return self.data_dir / "title_additions.json"
|
||||
|
||||
@property
|
||||
def title_removals_path(self) -> Path:
|
||||
# titles the human removed from the catalog (not a game, misread) —
|
||||
# filtered out of every titles.json rebuild
|
||||
return self.data_dir / "title_removals.json"
|
||||
|
||||
@property
|
||||
def title_edits_path(self) -> Path:
|
||||
# human corrections to extracted reads (misspellings, known cues) —
|
||||
# replayed on every titles.json rebuild
|
||||
return self.data_dir / "title_edits.json"
|
||||
|
||||
@property
|
||||
def title_splits_path(self) -> Path:
|
||||
# titles the human declared to be MULTIPLE physical copies: extract
|
||||
# and resolve dedupe must never cross-photo-merge them again
|
||||
return self.data_dir / "title_splits.json"
|
||||
|
||||
@property
|
||||
def dismissed_path(self) -> Path:
|
||||
return self.data_dir / "unidentified_dismissed.json"
|
||||
|
||||
@property
|
||||
def snapshot_paths(self) -> tuple[Path, Path]:
|
||||
return (
|
||||
self.data_dir / "collection_snapshot_base.xml",
|
||||
self.data_dir / "collection_snapshot_expansions.xml",
|
||||
)
|
||||
|
||||
@property
|
||||
def lan_key_path(self) -> Path:
|
||||
# the --lan access key, persisted so restarts don't strand phones'
|
||||
# cookies; credential-adjacent like Playwright state — gitignored
|
||||
return self.data_dir / ".lan_key"
|
||||
|
||||
@property
|
||||
def storage_state_path(self) -> Path:
|
||||
# cwd-relative on purpose (credential-adjacent, gitignored) but
|
||||
# centralized here with every other artifact path
|
||||
return Path("storage_state.json")
|
||||
|
||||
@property
|
||||
def stub_marker_paths(self) -> tuple[Path, Path]:
|
||||
# gitignored (travels with the stub XML) + committed (guards clones)
|
||||
return (
|
||||
self.cache_dir / STUB_CACHE_MARKER_NAME,
|
||||
self.data_dir / STUB_DATA_MARKER_NAME,
|
||||
)
|
||||
|
||||
|
||||
def load_config(path: Path | None = None) -> Config:
|
||||
cfg = Config()
|
||||
p = path or DEFAULT_CONFIG_PATH
|
||||
if path is not None and not path.exists():
|
||||
# an EXPLICIT --config that doesn't exist must not silently fall
|
||||
# back to defaults — that reads photos/ while the user believes
|
||||
# their prod config is active
|
||||
raise FileNotFoundError(f"--config {path} does not exist")
|
||||
if p.exists():
|
||||
raw = tomllib.loads(p.read_text())
|
||||
known = {
|
||||
"bgg_username": str,
|
||||
"photos_dir": Path,
|
||||
"data_dir": Path,
|
||||
"model": str,
|
||||
"rate_limit_seconds": float,
|
||||
"vision_provider": str,
|
||||
}
|
||||
updates = {key: caster(raw[key]) for key, caster in known.items() if key in raw}
|
||||
if unknown := sorted(raw.keys() - known.keys() - {"vision"}):
|
||||
# a typo'd key silently falling back to defaults is a debugging
|
||||
# trap ("No photos found in photos/") — say so up front
|
||||
warnings.warn(
|
||||
f"{p}: ignoring unknown key(s): {', '.join(unknown)}",
|
||||
stacklevel=2,
|
||||
)
|
||||
# [vision.<provider>] blocks: every provider's recipe can live in
|
||||
# the committed file; only the ACTIVE provider's block applies
|
||||
vision_blocks = raw.get("vision") or {}
|
||||
if stray := sorted(vision_blocks.keys() - set(VISION_PROVIDERS)):
|
||||
warnings.warn(
|
||||
f"{p}: ignoring [vision.*] block(s) for unknown provider(s): "
|
||||
f"{', '.join(stray)}",
|
||||
stacklevel=2,
|
||||
)
|
||||
provider = updates.get("vision_provider", cfg.vision_provider)
|
||||
block = vision_blocks.get(provider) or {}
|
||||
block_known = {
|
||||
"model": "model",
|
||||
"base_url": "vision_base_url",
|
||||
"key_env": "vision_key_env",
|
||||
}
|
||||
if odd := sorted(block.keys() - block_known.keys()):
|
||||
warnings.warn(
|
||||
f"{p}: [vision.{provider}]: ignoring unknown key(s): {', '.join(odd)}",
|
||||
stacklevel=2,
|
||||
)
|
||||
for key, field in block_known.items():
|
||||
if key in block:
|
||||
updates[field] = str(block[key])
|
||||
cfg = replace(cfg, **updates)
|
||||
if cfg.vision_provider not in VISION_PROVIDERS:
|
||||
# a typo here would surface as a confusing extract failure later
|
||||
raise ValueError(
|
||||
f"{p}: vision_provider must be one of {', '.join(VISION_PROVIDERS)}"
|
||||
f" (got {cfg.vision_provider!r})"
|
||||
)
|
||||
if username := os.environ.get("BGG_USERNAME"):
|
||||
cfg = replace(cfg, bgg_username=username)
|
||||
return cfg
|
||||
|
||||
+185
-59
@@ -8,25 +8,37 @@ Two collection sources:
|
||||
logged-in-user exemption.
|
||||
|
||||
Outputs both artifacts:
|
||||
- to_add.csv — recognized games not in the collection (new entries);
|
||||
- to_add.csv — recognized games not in the collection, plus additional
|
||||
copies once every owned copy is claimed by another match row;
|
||||
- to_update.csv — owned, VERSION-LESS entries where matching produced a
|
||||
confident version (version_auto/version_approved). Strictly additive:
|
||||
entries that already carry a version are never touched — a photo/version
|
||||
disagreement is reported in the summary instead.
|
||||
confident version. Strictly additive: entries that already carry a
|
||||
version are never touched — a version mismatch against an unclaimed
|
||||
copy is reported as a disagreement, nothing more.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import csv
|
||||
import os
|
||||
from collections import Counter
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
import typer
|
||||
|
||||
from bggpipe.bgg_client import BGGClient
|
||||
from bggpipe.bgg_client import (
|
||||
BGGAuthError,
|
||||
BGGClient,
|
||||
BGGQueueTimeout,
|
||||
client_for,
|
||||
)
|
||||
from bggpipe.config import Config
|
||||
from bggpipe.models import CollectionItem, parse_collection
|
||||
from bggpipe.fsio import atomic_write_csv
|
||||
from bggpipe.models import (
|
||||
RECOGNIZED_MATCH_STATUSES,
|
||||
CollectionItem,
|
||||
is_confident_version,
|
||||
parse_collection,
|
||||
)
|
||||
from bggpipe.resolve import read_matches
|
||||
|
||||
TO_ADD_COLUMNS = [
|
||||
@@ -38,11 +50,11 @@ TO_ADD_COLUMNS = [
|
||||
"version_name",
|
||||
"title_raw",
|
||||
"source_photos",
|
||||
"second_copy", # "1": game already had copies — verify can't confirm it
|
||||
]
|
||||
TO_UPDATE_COLUMNS = ["collid", "bgg_id", "bgg_name", "version_id", "version_name"]
|
||||
|
||||
SNAPSHOT_FILES = ("collection_snapshot_base.xml", "collection_snapshot_expansions.xml")
|
||||
_CONFIDENT_VERSION = ("version_auto", "version_approved")
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -50,10 +62,13 @@ class DiffResult:
|
||||
to_add: list[dict] = field(default_factory=list)
|
||||
to_update: list[dict] = field(default_factory=list)
|
||||
already_owned: list[str] = field(default_factory=list) # title_raw
|
||||
disagreements: list[str] = field(default_factory=list)
|
||||
local_only: list[str] = field(default_factory=list) # RPGs etc: never uploaded
|
||||
second_copies: list[str] = field(default_factory=list) # notes for adds
|
||||
disagreements: list[str] = field(default_factory=list) # report-only
|
||||
unseen: list[CollectionItem] = field(default_factory=list)
|
||||
pending: list[str] = field(default_factory=list) # ambiguous/unmatched titles
|
||||
rejected: int = 0
|
||||
merged: int = 0
|
||||
recognized: int = 0
|
||||
|
||||
|
||||
@@ -62,8 +77,7 @@ def load_snapshot_collection(data_dir: Path) -> list[CollectionItem]:
|
||||
same physical copy can appear in both responses)."""
|
||||
items: list[CollectionItem] = []
|
||||
seen: set[int] = set()
|
||||
for name in SNAPSHOT_FILES:
|
||||
path = data_dir / name
|
||||
for path in Config(data_dir=data_dir).snapshot_paths:
|
||||
if not path.exists():
|
||||
raise FileNotFoundError(
|
||||
f"{path} not found — pull your collection while logged in "
|
||||
@@ -86,27 +100,21 @@ def compute_diff(rows: list[dict], collection: list[CollectionItem]) -> DiffResu
|
||||
seen_object_ids: set[int] = set()
|
||||
consumed_collids: set[int] = set()
|
||||
|
||||
# photos of merged-away duplicate reads belong to their survivor
|
||||
merged_photos: dict[str, set[str]] = {}
|
||||
for row in rows:
|
||||
status = row["match_status"]
|
||||
if status == "rejected":
|
||||
result.rejected += 1
|
||||
continue
|
||||
if status not in ("auto", "approved") or not row["bgg_id"]:
|
||||
result.pending.append(row["title_raw"])
|
||||
continue
|
||||
if row["match_status"] == "merged" and row.get("merged_into"):
|
||||
merged_photos.setdefault(row["merged_into"], set()).update(
|
||||
p for p in row["source_photos"].split(";") if p
|
||||
)
|
||||
|
||||
result.recognized += 1
|
||||
bgg_id = int(row["bgg_id"])
|
||||
copies = by_object.get(bgg_id, [])
|
||||
if copies:
|
||||
seen_object_ids.add(bgg_id)
|
||||
queued_adds: Counter[int] = Counter() # per game, across all passes
|
||||
|
||||
confident = row["version_status"] in _CONFIDENT_VERSION and row["version_id"]
|
||||
version_id = int(row["version_id"]) if confident else None
|
||||
|
||||
if not copies:
|
||||
result.to_add.append(
|
||||
{
|
||||
def add_row(row: dict, confident: bool, second_copy: bool = False) -> dict:
|
||||
queued_adds[int(row["bgg_id"])] += 1
|
||||
photos = {p for p in row["source_photos"].split(";") if p}
|
||||
photos |= merged_photos.get(row["title_raw"], set())
|
||||
return {
|
||||
"bgg_id": row["bgg_id"],
|
||||
"bgg_name": row["bgg_name"],
|
||||
"year": row["year"],
|
||||
@@ -114,25 +122,74 @@ def compute_diff(rows: list[dict], collection: list[CollectionItem]) -> DiffResu
|
||||
"version_id": row["version_id"] if confident else "",
|
||||
"version_name": row["version_name"] if confident else "",
|
||||
"title_raw": row["title_raw"],
|
||||
"source_photos": row["source_photos"],
|
||||
"source_photos": ";".join(sorted(photos)),
|
||||
"second_copy": "1" if second_copy else "",
|
||||
}
|
||||
)
|
||||
continue
|
||||
|
||||
if not confident:
|
||||
# bare id with unknown version: owned if any copy exists
|
||||
result.already_owned.append(row["title_raw"])
|
||||
recognized: list[dict] = []
|
||||
for row in rows:
|
||||
status = row["match_status"]
|
||||
if status == "rejected":
|
||||
result.rejected += 1
|
||||
continue
|
||||
|
||||
if any(c.version_id == version_id for c in copies):
|
||||
result.already_owned.append(row["title_raw"])
|
||||
if status == "merged":
|
||||
result.merged += 1 # represented by its survivor row
|
||||
continue
|
||||
if status == "local":
|
||||
# the human's call: a real game BGG doesn't have — a library
|
||||
# citizen only, never queued
|
||||
result.local_only.append(row["title_raw"])
|
||||
continue
|
||||
if status not in RECOGNIZED_MATCH_STATUSES or not row["bgg_id"]:
|
||||
result.pending.append(row["title_raw"])
|
||||
continue
|
||||
if row["type"] == "rpgitem":
|
||||
# a real, identified game — but it lives on RPGGeek, not in the
|
||||
# BGG collection: a library citizen only, never queued
|
||||
result.local_only.append(row["title_raw"])
|
||||
continue
|
||||
result.recognized += 1
|
||||
recognized.append(row)
|
||||
if by_object.get(int(row["bgg_id"])):
|
||||
seen_object_ids.add(int(row["bgg_id"]))
|
||||
|
||||
versionless = [
|
||||
c
|
||||
for c in copies
|
||||
if c.version_id is None and c.coll_id not in consumed_collids
|
||||
def unconsumed(bgg_id: int) -> list[CollectionItem]:
|
||||
return [
|
||||
c for c in by_object.get(bgg_id, []) if c.coll_id not in consumed_collids
|
||||
]
|
||||
|
||||
# Ordered sub-passes over the confident rows: greedy per-row handling
|
||||
# would let an earlier row's disagreement consume the exact-version copy
|
||||
# a later row matches, manufacturing a duplicate upload. Claims settle
|
||||
# strongest-first across all rows — exact version matches, then
|
||||
# versionless upgrades, then disagreement/second-copy handling.
|
||||
confident_rows = [r for r in recognized if is_confident_version(r)]
|
||||
leftover: list[dict] = []
|
||||
|
||||
# 1a — exact (bgg_id, version) matches consume first
|
||||
for row in confident_rows:
|
||||
bgg_id = int(row["bgg_id"])
|
||||
if not by_object.get(bgg_id):
|
||||
result.to_add.append(add_row(row, True))
|
||||
continue
|
||||
matching = [
|
||||
c for c in unconsumed(bgg_id) if c.version_id == int(row["version_id"])
|
||||
]
|
||||
if matching:
|
||||
# consume, so a SECOND row with the same version (a vetoed
|
||||
# duplicate = a real second copy) falls through to 1b/1c
|
||||
consumed_collids.add(matching[0].coll_id)
|
||||
result.already_owned.append(row["title_raw"])
|
||||
else:
|
||||
leftover.append(row)
|
||||
|
||||
# 1b — versionless copies get upgraded. The update flow addresses the
|
||||
# copy by collid and the edition by radio value, so a versioned
|
||||
# sibling copy is never at risk.
|
||||
still_left: list[dict] = []
|
||||
for row in leftover:
|
||||
bgg_id = int(row["bgg_id"])
|
||||
versionless = [c for c in unconsumed(bgg_id) if c.version_id is None]
|
||||
if versionless:
|
||||
target = versionless[0]
|
||||
consumed_collids.add(target.coll_id)
|
||||
@@ -147,13 +204,64 @@ def compute_diff(rows: list[dict], collection: list[CollectionItem]) -> DiffResu
|
||||
}
|
||||
)
|
||||
else:
|
||||
# every copy already carries a (different) version — never touch it
|
||||
still_left.append(row)
|
||||
|
||||
# 1c — what remains disagrees with an unclaimed copy (report-only) or
|
||||
# is an additional physical copy (every copy claimed by another row)
|
||||
for row in still_left:
|
||||
remaining = unconsumed(int(row["bgg_id"]))
|
||||
if remaining:
|
||||
# most likely the same physical box mis-scored — report, never
|
||||
# touch, never duplicate (spec: report the disagreement)
|
||||
consumed_collids.add(remaining[0].coll_id)
|
||||
result.already_owned.append(row["title_raw"])
|
||||
result.disagreements.append(
|
||||
f"{row['title_raw']}: photo suggests version "
|
||||
f"{row['version_name']!r} ({row['version_id']}) but the "
|
||||
"collection entry already has a version set — not changing it"
|
||||
"remaining collection entry carries a different version — "
|
||||
"left untouched"
|
||||
)
|
||||
else:
|
||||
result.to_add.append(add_row(row, True, second_copy=True))
|
||||
result.second_copies.append(
|
||||
f"{row['title_raw']}: adding as a NEW copy with version "
|
||||
f"{row['version_name']!r} ({row['version_id']}) — every "
|
||||
"existing entry of this game keeps its current version"
|
||||
)
|
||||
|
||||
# Pass 2 — bare (version-unknown) rows. Spec: a bare id is owned if ANY
|
||||
# copy exists — only a human veto (dedupe_veto) makes an extra bare row
|
||||
# a genuine additional copy.
|
||||
for row in (r for r in recognized if not is_confident_version(r)):
|
||||
bgg_id = int(row["bgg_id"])
|
||||
copies = by_object.get(bgg_id, [])
|
||||
if not copies:
|
||||
if queued_adds[bgg_id] and not row.get("dedupe_veto"):
|
||||
# an unvetoed bare row is a typo-read sibling: a pass-1 add
|
||||
# for the same absent game already covers the physical box —
|
||||
# queueing it again would upload a duplicate entry
|
||||
result.already_owned.append(row["title_raw"])
|
||||
else:
|
||||
result.to_add.append(
|
||||
add_row(row, False, second_copy=bool(queued_adds[bgg_id]))
|
||||
)
|
||||
continue
|
||||
remaining = unconsumed(bgg_id)
|
||||
if remaining:
|
||||
consumed_collids.add(remaining[0].coll_id)
|
||||
result.already_owned.append(row["title_raw"])
|
||||
elif row.get("dedupe_veto"):
|
||||
result.to_add.append(add_row(row, False, second_copy=True))
|
||||
result.second_copies.append(
|
||||
f"{row['title_raw']}: adding as a NEW version-less copy — "
|
||||
"human-vetoed duplicate, every existing entry claimed by "
|
||||
"another match row"
|
||||
)
|
||||
else:
|
||||
# unvetoed bare row, all copies claimed: per spec still owned
|
||||
# (a typo-read sibling of a confident row must not become a
|
||||
# spurious upload)
|
||||
result.already_owned.append(row["title_raw"])
|
||||
|
||||
result.unseen = [
|
||||
item for item in collection if item.object_id not in seen_object_ids
|
||||
@@ -161,24 +269,31 @@ def compute_diff(rows: list[dict], collection: list[CollectionItem]) -> DiffResu
|
||||
return result
|
||||
|
||||
|
||||
def _write_csv(path: Path, columns: list[str], rows: list[dict]) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open("w", newline="") as f:
|
||||
writer = csv.DictWriter(f, fieldnames=columns, extrasaction="ignore")
|
||||
writer.writeheader()
|
||||
writer.writerows(rows)
|
||||
|
||||
|
||||
def run_diff(cfg: Config, *, client: BGGClient | None = None) -> DiffResult:
|
||||
rows = read_matches(cfg.matches_path)
|
||||
if not rows:
|
||||
typer.echo(f"{cfg.matches_path} is empty — run `bggpipe resolve` first.")
|
||||
raise typer.Exit(code=1)
|
||||
|
||||
if os.environ.get("BGG_API_TOKEN") and cfg.bgg_username:
|
||||
has_token = bool(os.environ.get("BGG_API_TOKEN"))
|
||||
if has_token and cfg.bgg_username:
|
||||
typer.echo("Fetching live collection from BGG…")
|
||||
client = client or BGGClient(cfg.cache_dir, cfg.rate_limit_seconds)
|
||||
collection = client.collection_full(cfg.bgg_username)
|
||||
client = client or client_for(cfg)
|
||||
try:
|
||||
collection = client.collection_full(cfg.bgg_username, refresh=True)
|
||||
except (BGGAuthError, BGGQueueTimeout) as err:
|
||||
# a present-but-invalid token must not traceback when the
|
||||
# snapshot fallback is sitting right there
|
||||
typer.echo(f"Live fetch failed ({err}) — using snapshot files.")
|
||||
collection = load_snapshot_collection(cfg.data_dir)
|
||||
else:
|
||||
if has_token:
|
||||
# saying "No BGG_API_TOKEN" here would be false and misdirect
|
||||
# the user's debugging — the missing half is the username
|
||||
typer.echo(
|
||||
"BGG_API_TOKEN is set but BGG_USERNAME is not (is .env "
|
||||
"loaded?) — using snapshot files instead of live mode."
|
||||
)
|
||||
else:
|
||||
typer.echo(
|
||||
"No BGG_API_TOKEN — using collection snapshot files in "
|
||||
@@ -188,17 +303,28 @@ def run_diff(cfg: Config, *, client: BGGClient | None = None) -> DiffResult:
|
||||
|
||||
result = compute_diff(rows, collection)
|
||||
|
||||
_write_csv(cfg.data_dir / "to_add.csv", TO_ADD_COLUMNS, result.to_add)
|
||||
_write_csv(cfg.data_dir / "to_update.csv", TO_UPDATE_COLUMNS, result.to_update)
|
||||
# atomic: a killed diff never leaves a torn upload queue
|
||||
atomic_write_csv(cfg.to_add_path, TO_ADD_COLUMNS, result.to_add)
|
||||
atomic_write_csv(cfg.to_update_path, TO_UPDATE_COLUMNS, result.to_update)
|
||||
|
||||
merged_note = f" · {result.merged} merged duplicate(s)" if result.merged else ""
|
||||
local_note = (
|
||||
f" · {len(result.local_only)} local-only (RPGs, off-BGG games)"
|
||||
if result.local_only
|
||||
else ""
|
||||
)
|
||||
typer.echo(
|
||||
f"\n{result.recognized} recognized · {len(result.already_owned)} already "
|
||||
f"owned · {len(result.to_add)} to add · {len(result.to_update)} version "
|
||||
f"update(s) · {len(result.pending)} pending review · "
|
||||
f"{result.rejected} rejected"
|
||||
f"{result.rejected} rejected{merged_note}{local_note}"
|
||||
)
|
||||
if result.second_copies:
|
||||
typer.echo("\nSecond copies (verify these on the dry run before upload):")
|
||||
for line in result.second_copies:
|
||||
typer.echo(f" - {line}")
|
||||
if result.disagreements:
|
||||
typer.echo("\nVersion disagreements (left untouched):")
|
||||
typer.echo("\nVersion disagreements (reported only — nothing changed):")
|
||||
for line in result.disagreements:
|
||||
typer.echo(f" - {line}")
|
||||
if result.pending:
|
||||
|
||||
@@ -0,0 +1,212 @@
|
||||
"""Stage 6 — enrich: full game + version metadata into games.json.
|
||||
|
||||
The seed data for the future web frontend, keyed by bgg_id (or
|
||||
"bgg_id:version_id" when a version is settled). Cheap by design:
|
||||
- ids are batched (~20 per /thing call) and sorted so batch cache keys
|
||||
stay stable across runs;
|
||||
- already-enriched keys are skipped entirely (no request, no cache read)
|
||||
unless --refresh, which bypasses the cache read because ranks and
|
||||
ratings drift over time;
|
||||
- the chosen version's details come from matches.csv's stored version
|
||||
candidates — no extra API calls.
|
||||
Degrades gracefully without BGG_API_TOKEN: whatever is cached enriches,
|
||||
the rest waits for the token.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
import typer
|
||||
|
||||
from bggpipe.bgg_client import (
|
||||
BGGAuthError,
|
||||
BGGClient,
|
||||
BGGQueueTimeout,
|
||||
client_for,
|
||||
)
|
||||
from bggpipe.config import Config
|
||||
from bggpipe.fsio import atomic_write_text
|
||||
from bggpipe.models import is_confident_version, is_recognized
|
||||
from bggpipe.normalize import normalize_title
|
||||
from bggpipe.resolve import load_titles, read_matches
|
||||
|
||||
BATCH_SIZE = 20
|
||||
|
||||
|
||||
def _version_info(row: dict) -> dict | None:
|
||||
if not is_confident_version(row):
|
||||
return None
|
||||
version_id = int(row["version_id"])
|
||||
for cand in json.loads(row["version_candidates_json"] or "[]"):
|
||||
if cand.get("version_id") == version_id:
|
||||
return {
|
||||
"version_id": version_id,
|
||||
"name": cand.get("name", ""),
|
||||
"year": cand.get("year"),
|
||||
"publishers": cand.get("publishers") or [],
|
||||
"languages": cand.get("languages") or [],
|
||||
}
|
||||
# candidates were pruned (e.g. manual review) — keep what the row knows
|
||||
return {
|
||||
"version_id": version_id,
|
||||
"name": row["version_name"],
|
||||
"year": None,
|
||||
"publishers": [],
|
||||
"languages": [],
|
||||
}
|
||||
|
||||
|
||||
def run_enrich(
|
||||
cfg: Config, *, refresh: bool = False, client: BGGClient | None = None
|
||||
) -> dict:
|
||||
rows = read_matches(cfg.matches_path)
|
||||
if not rows:
|
||||
typer.echo(f"{cfg.matches_path} is empty — run `bggpipe resolve` first.")
|
||||
raise typer.Exit(code=1)
|
||||
|
||||
targets: list[tuple[str, int, dict | None]] = []
|
||||
for row in rows:
|
||||
if not is_recognized(row):
|
||||
continue
|
||||
version = _version_info(row)
|
||||
key = f"{row['bgg_id']}:{version['version_id']}" if version else row["bgg_id"]
|
||||
targets.append((key, int(row["bgg_id"]), version))
|
||||
|
||||
games_path = cfg.games_path
|
||||
games: dict = json.loads(games_path.read_text()) if games_path.exists() else {}
|
||||
|
||||
need = sorted({bgg_id for key, bgg_id, _ in targets if refresh or key not in games})
|
||||
client = client or client_for(cfg)
|
||||
|
||||
fetched: dict[int, dict] = {}
|
||||
blocked = False
|
||||
for start in range(0, len(need), BATCH_SIZE):
|
||||
batch = need[start : start + BATCH_SIZE]
|
||||
try:
|
||||
for thing in client.things_full(batch, refresh=refresh):
|
||||
fetched[thing["bgg_id"]] = thing
|
||||
except (BGGAuthError, BGGQueueTimeout):
|
||||
blocked = True
|
||||
break
|
||||
|
||||
updated = 0
|
||||
for key, bgg_id, version in targets:
|
||||
if bgg_id in fetched:
|
||||
games[key] = {**fetched[bgg_id], "version": version}
|
||||
updated += 1
|
||||
|
||||
# games the human ruled off-BGG: library entries built from their own
|
||||
# photo reads — no API involved, so a blocked run still lands them
|
||||
local_keys: set[str] = set()
|
||||
local_rows = [r for r in rows if r["match_status"] == "local"]
|
||||
# hand-written metadata wins over the photo reads: for an off-BGG game
|
||||
# it is the only real source there is
|
||||
hand: dict = {}
|
||||
if cfg.local_games_path.exists():
|
||||
try:
|
||||
hand = json.loads(cfg.local_games_path.read_text())
|
||||
except json.JSONDecodeError as err:
|
||||
raise ValueError(
|
||||
f"{cfg.local_games_path} is corrupt ({err}) — it holds "
|
||||
"hand-written game data, so check git history before deleting"
|
||||
) from err
|
||||
if local_rows:
|
||||
try:
|
||||
cues = {
|
||||
(e.title_raw, ";".join(e.source_photos)): e
|
||||
for e in load_titles(cfg.titles_path)
|
||||
}
|
||||
except FileNotFoundError:
|
||||
cues = {}
|
||||
for row in local_rows:
|
||||
key = f"local:{normalize_title(row['title_raw'])}:{row['source_photos']}"
|
||||
local_keys.add(key)
|
||||
entry = cues.get((row["title_raw"], row["source_photos"]))
|
||||
games[key] = {
|
||||
"bgg_id": None,
|
||||
"name": row["title_raw"],
|
||||
"year": entry.year_hint if entry else None,
|
||||
"type": "localgame",
|
||||
"publishers": [entry.publisher_hint]
|
||||
if entry and entry.publisher_hint
|
||||
else [],
|
||||
"source_photos": [p for p in row["source_photos"].split(";") if p],
|
||||
}
|
||||
games[key].update(
|
||||
{k: v for k, v in (hand.get(key) or {}).items() if v not in (None, "")}
|
||||
)
|
||||
# the key embeds the photo list, so a new sighting or a title edit
|
||||
# strands hand data under a key no row produces anymore. Same
|
||||
# normalized title + exactly one candidate = unambiguous: migrate.
|
||||
# Anything else is reported, never silently dropped.
|
||||
orphans = [k for k in hand if k.startswith("local:") and k not in local_keys]
|
||||
migrated = {}
|
||||
for old_key in orphans:
|
||||
title_part = old_key.split(":", 2)[1]
|
||||
candidates = [k for k in local_keys if k.split(":", 2)[1] == title_part]
|
||||
if len(candidates) == 1 and candidates[0] not in hand:
|
||||
new_key = candidates[0]
|
||||
hand[new_key] = hand.pop(old_key)
|
||||
migrated[old_key] = new_key
|
||||
games[new_key].update(
|
||||
{k: v for k, v in hand[new_key].items() if v not in (None, "")}
|
||||
)
|
||||
else:
|
||||
typer.echo(
|
||||
f" warning: hand-written data for {old_key!r} matches "
|
||||
"no current catalog line — the facts are safe in "
|
||||
f"{cfg.local_games_path.name} but will not show in the "
|
||||
"library until the key matches again"
|
||||
)
|
||||
if migrated:
|
||||
atomic_write_text(
|
||||
cfg.local_games_path,
|
||||
json.dumps(hand, indent=2, ensure_ascii=False, sort_keys=True) + "\n",
|
||||
)
|
||||
for old_key, new_key in migrated.items():
|
||||
typer.echo(
|
||||
f" migrated hand-written data {old_key!r} -> {new_key!r} "
|
||||
"(photo set changed; same title)"
|
||||
)
|
||||
|
||||
# prune keys no current target claims: a row whose version was approved
|
||||
# after a bare-key run (or was later rejected) must not leave an orphan
|
||||
# entry in the frontend seed data. Only safe when nothing was blocked —
|
||||
# a token-less run knows too little to declare anything stale.
|
||||
if not blocked:
|
||||
current = {key for key, _, _ in targets} | local_keys
|
||||
stale = [k for k in games if k not in current]
|
||||
for k in stale:
|
||||
del games[k]
|
||||
if stale:
|
||||
typer.echo(
|
||||
f" pruned {len(stale)} stale entr{'y' if len(stale) == 1 else 'ies'}"
|
||||
)
|
||||
|
||||
games_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
atomic_write_text(
|
||||
games_path, json.dumps(games, indent=2, ensure_ascii=False) + "\n"
|
||||
)
|
||||
|
||||
# count the two populations separately: local entries are built from
|
||||
# photo reads and have no fetch target, so folding them into the API
|
||||
# tally would drive "already present or waiting" negative
|
||||
waiting = len(targets) - updated
|
||||
parts = [f"{updated} fetched from BGG"]
|
||||
if local_keys:
|
||||
parts.append(f"{len(local_keys)} local-only")
|
||||
if waiting:
|
||||
parts.append(f"{waiting} already present or waiting")
|
||||
typer.echo(
|
||||
f"games.json: {len(games)} entr{'y' if len(games) == 1 else 'ies'} "
|
||||
f"({'; '.join(parts)})."
|
||||
)
|
||||
if blocked:
|
||||
remaining = [i for i in need if i not in fetched]
|
||||
typer.echo(
|
||||
f"\n{len(remaining)} game(s) are waiting on the BGG API "
|
||||
"(set BGG_API_TOKEN and re-run enrich — everything fetched "
|
||||
"so far is saved)."
|
||||
)
|
||||
return games
|
||||
+432
-27
@@ -1,7 +1,8 @@
|
||||
"""Stage 1 — extract game titles + edition cues from shelf photos.
|
||||
|
||||
Each photo is sent to Claude vision once and the raw result is cached under
|
||||
data/extract_raw/<photo>.json (making re-runs free and --only targeted).
|
||||
Each photo is sent to the configured vision model once and the raw result
|
||||
is cached under data/extract_raw/<photo>.json (making re-runs free and
|
||||
--only targeted).
|
||||
titles.json is rebuilt from all raw files on every run, deduping identical
|
||||
titles across photos unless their edition cues conflict — conflicting cues
|
||||
mean two different editions on the shelf, which stay separate entries.
|
||||
@@ -12,6 +13,7 @@ from __future__ import annotations
|
||||
import base64
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
from collections.abc import Callable
|
||||
from pathlib import Path
|
||||
@@ -19,6 +21,7 @@ from pathlib import Path
|
||||
import typer
|
||||
|
||||
from bggpipe.config import Config
|
||||
from bggpipe.fsio import atomic_write_text
|
||||
from bggpipe.normalize import normalize_title
|
||||
|
||||
IMAGE_EXTENSIONS = {".jpg", ".jpeg", ".png", ".heic"}
|
||||
@@ -103,12 +106,18 @@ def prepare_image(path: Path) -> tuple[str, str]:
|
||||
return base64.standard_b64encode(buffer.getvalue()).decode(), "image/jpeg"
|
||||
|
||||
|
||||
def parse_vision_response(text: str) -> tuple[list[dict], list[dict]]:
|
||||
def parse_vision_response(text: str) -> tuple[list[dict], list[dict], int]:
|
||||
"""Parse the model's JSON defensively: strip code fences, locate the
|
||||
payload amid any prose. Returns (title entries, unidentified sightings).
|
||||
A bare JSON array (the pre-unidentified response shape) still parses —
|
||||
it's all titles."""
|
||||
payload amid any prose. Returns (title entries, unidentified sightings,
|
||||
dropped-malformed-entry count).
|
||||
Accepts either response shape: a bare JSON array (all titles) or an
|
||||
object with titles/unidentified keys."""
|
||||
cleaned = _CODE_FENCE.sub("", text).strip()
|
||||
if cleaned[:1] in ("[", "{"):
|
||||
# trim trailing prose after a leading JSON payload ("{...}\nNote:")
|
||||
end = cleaned.rfind("]" if cleaned[0] == "[" else "}")
|
||||
if end != -1:
|
||||
cleaned = cleaned[: end + 1]
|
||||
if cleaned[:1] not in ("[", "{"):
|
||||
starts = [i for i in (cleaned.find("["), cleaned.find("{")) if i != -1]
|
||||
if not starts:
|
||||
@@ -118,7 +127,12 @@ def parse_vision_response(text: str) -> tuple[list[dict], list[dict]]:
|
||||
if end == -1:
|
||||
raise ValueError(f"unterminated JSON in vision response: {text[:200]!r}")
|
||||
cleaned = cleaned[start : end + 1]
|
||||
try:
|
||||
data = json.loads(cleaned)
|
||||
except json.JSONDecodeError:
|
||||
# local models emit almost-JSON (trailing commas, mostly): one
|
||||
# cheap repair pass before declaring the response unusable
|
||||
data = json.loads(re.sub(r",\s*([}\]])", r"\1", cleaned))
|
||||
if isinstance(data, list):
|
||||
titles_raw, unidentified_raw = data, []
|
||||
elif isinstance(data, dict):
|
||||
@@ -133,7 +147,8 @@ def parse_vision_response(text: str) -> tuple[list[dict], list[dict]]:
|
||||
if isinstance(u, dict)
|
||||
and (u.get("location") or u.get("partial_text") or u.get("art_notes"))
|
||||
]
|
||||
return titles, unidentified
|
||||
dropped = len(titles_raw) - len(titles)
|
||||
return titles, unidentified, dropped
|
||||
|
||||
|
||||
def default_vision(model: str) -> VisionFn:
|
||||
@@ -168,11 +183,68 @@ def default_vision(model: str) -> VisionFn:
|
||||
return vision
|
||||
|
||||
|
||||
def openai_compatible_vision(model: str, base_url: str, key_env: str) -> VisionFn:
|
||||
"""Any endpoint speaking the OpenAI chat-completions format: OpenAI
|
||||
itself, OpenRouter, or a local runtime (Ollama, LM Studio, llama.cpp,
|
||||
vLLM — pass its /v1 base URL). A key is optional: local runtimes run
|
||||
without one, so an empty key_env just sends no Authorization header."""
|
||||
import httpx
|
||||
|
||||
headers = {}
|
||||
if key_env and (key := os.environ.get(key_env)):
|
||||
headers["Authorization"] = f"Bearer {key}"
|
||||
|
||||
def vision(image_b64: str, media_type: str) -> str:
|
||||
response = httpx.post(
|
||||
base_url.rstrip("/") + "/chat/completions",
|
||||
headers=headers,
|
||||
json={
|
||||
"model": model,
|
||||
"max_tokens": 4000,
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {
|
||||
"url": f"data:{media_type};base64,{image_b64}"
|
||||
},
|
||||
},
|
||||
{"type": "text", "text": VISION_PROMPT},
|
||||
],
|
||||
}
|
||||
],
|
||||
},
|
||||
timeout=300.0, # local models on modest hardware are slow
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()["choices"][0]["message"]["content"] or ""
|
||||
|
||||
return vision
|
||||
|
||||
|
||||
def vision_for(cfg: Config) -> VisionFn:
|
||||
"""The configured vision backend (config.toml: vision_provider)."""
|
||||
if cfg.vision_provider == "openai-compatible":
|
||||
if not cfg.vision_base_url:
|
||||
raise ValueError(
|
||||
"vision_provider 'openai-compatible' needs vision_base_url "
|
||||
"in config.toml (e.g. http://localhost:11434/v1 for Ollama)"
|
||||
)
|
||||
return openai_compatible_vision(
|
||||
cfg.model, cfg.vision_base_url, cfg.vision_key_env
|
||||
)
|
||||
return default_vision(cfg.model)
|
||||
|
||||
|
||||
def extract_photo(photo: Path, vision: VisionFn) -> dict:
|
||||
"""One photo -> {"titles": [...], "unidentified": [...]} (the raw-cache
|
||||
file format)."""
|
||||
image_b64, media_type = prepare_image(photo)
|
||||
raw_entries, raw_unidentified = parse_vision_response(vision(image_b64, media_type))
|
||||
raw_entries, raw_unidentified, dropped = parse_vision_response(
|
||||
vision(image_b64, media_type)
|
||||
)
|
||||
unidentified = [
|
||||
{
|
||||
"location": str(u.get("location") or "").strip(),
|
||||
@@ -198,10 +270,10 @@ def extract_photo(photo: Path, vision: VisionFn) -> dict:
|
||||
"source_photos": [photo.name],
|
||||
}
|
||||
)
|
||||
return {"titles": entries, "unidentified": unidentified}
|
||||
return {"titles": entries, "unidentified": unidentified, "dropped": dropped}
|
||||
|
||||
|
||||
def _cues_conflict(a: dict, b: dict) -> bool:
|
||||
def cues_conflict(a: dict, b: dict) -> bool:
|
||||
"""Two sightings conflict if any edition cue is set on both and differs —
|
||||
that means visibly different boxes, i.e. separate editions (spec)."""
|
||||
for key in ("publisher_hint", "edition_hint", "language_hint"):
|
||||
@@ -230,15 +302,213 @@ def _merge(a: dict, b: dict) -> dict:
|
||||
return merged
|
||||
|
||||
|
||||
def dedupe_entries(entries: list[dict]) -> list[dict]:
|
||||
"""Collapse same-normalized-title sightings unless their cues conflict."""
|
||||
# the fields a human correction may override on an extracted entry
|
||||
EDIT_FIELDS = (
|
||||
"title_raw",
|
||||
"confidence",
|
||||
"publisher_hint",
|
||||
"edition_hint",
|
||||
"year_hint",
|
||||
"language_hint",
|
||||
"art_notes",
|
||||
)
|
||||
|
||||
|
||||
def _load_store(path: Path) -> list:
|
||||
"""Curation stores hold irreplaceable human decisions and are committed
|
||||
(merge conflicts are a realistic corruption vector) — so a broken file
|
||||
must stop the pipeline loudly, never quietly reset it."""
|
||||
if not path.exists():
|
||||
return []
|
||||
try:
|
||||
return json.loads(path.read_text())
|
||||
except json.JSONDecodeError as err:
|
||||
raise ValueError(
|
||||
f"{path} is corrupt ({err}) — fix or delete it; it holds human "
|
||||
"review decisions, so check git history before deleting"
|
||||
) from err
|
||||
|
||||
|
||||
def load_title_edits(path: Path) -> list[dict]:
|
||||
"""Human corrections to raw reads (fixed misspellings, cues the owner
|
||||
knows offhand). Each record: {"match": <title as displayed when the fix
|
||||
was made>, "photos": [...] to target one copy (optional), <EDIT_FIELDS
|
||||
to override>}. Applied on every rebuild, before dedupe."""
|
||||
return _load_store(path)
|
||||
|
||||
|
||||
def record_title_edit(path: Path, record: dict) -> None:
|
||||
existing = load_title_edits(path)
|
||||
if existing and existing[-1] == record:
|
||||
return # a retried request must not double-record
|
||||
atomic_write_text(
|
||||
path, json.dumps([*existing, record], indent=2, ensure_ascii=False) + "\n"
|
||||
)
|
||||
|
||||
|
||||
def apply_title_edits(entries: list[dict], edits: list[dict]) -> list[dict]:
|
||||
"""Apply stored corrections in order. Matching is by the title an entry
|
||||
CURRENTLY carries, so a later edit made against an earlier edit's result
|
||||
chains naturally; each record applies at most once per entry."""
|
||||
if not edits:
|
||||
return entries
|
||||
for entry in entries:
|
||||
applied: set[int] = set()
|
||||
while True:
|
||||
norm = normalize_title(entry["title_raw"])
|
||||
photos = set(entry.get("source_photos") or [])
|
||||
hit = next(
|
||||
(
|
||||
i
|
||||
for i, e in enumerate(edits)
|
||||
if i not in applied
|
||||
and normalize_title(e["match"]) == norm
|
||||
and (not e.get("photos") or set(e["photos"]) & photos)
|
||||
),
|
||||
None,
|
||||
)
|
||||
if hit is None:
|
||||
break
|
||||
applied.add(hit)
|
||||
for key in EDIT_FIELDS:
|
||||
if key in edits[hit]:
|
||||
entry[key] = edits[hit][key]
|
||||
return entries
|
||||
|
||||
|
||||
def _scoped_records(path: Path) -> list[dict]:
|
||||
"""Parse a store of photo-scoped title decisions. Each stored record is
|
||||
{"title": ..., "photos": [...]} scoping the decision to the sightings
|
||||
that were on the line (a bare string is legacy: unscoped). Returns
|
||||
[{"norm": <normalized title>, "photos": set | None}]."""
|
||||
records = []
|
||||
for item in _load_store(path):
|
||||
if isinstance(item, str):
|
||||
records.append({"norm": normalize_title(item), "photos": None})
|
||||
else:
|
||||
photos = item.get("photos")
|
||||
records.append(
|
||||
{
|
||||
"norm": normalize_title(item["title"]),
|
||||
"photos": set(photos) if photos else None,
|
||||
}
|
||||
)
|
||||
return records
|
||||
|
||||
|
||||
def load_title_additions(path: Path) -> list[dict]:
|
||||
"""Entries the human added without a photo — an expansion stored inside
|
||||
a base box, a game away from the shelves. Shaped like raw reads and fed
|
||||
into every rebuild BEFORE edits and dedupe, so corrections apply and a
|
||||
later photo sighting of the same game merges instead of duplicating."""
|
||||
return _load_store(path)
|
||||
|
||||
|
||||
def record_title_addition(path: Path, entry: dict) -> bool:
|
||||
"""Record a hand-added game. False = an addition with this normalized
|
||||
title is already on file (nothing written — edit that line instead);
|
||||
a silent True here would let the caller report success for a no-op."""
|
||||
existing = load_title_additions(path)
|
||||
norm = normalize_title(entry["title_raw"])
|
||||
if any(normalize_title(e["title_raw"]) == norm for e in existing):
|
||||
return False
|
||||
atomic_write_text(
|
||||
path, json.dumps([*existing, entry], indent=2, ensure_ascii=False) + "\n"
|
||||
)
|
||||
return True
|
||||
|
||||
|
||||
def rescind_title_removal(path: Path, title: str) -> bool:
|
||||
"""Drop any UNSCOPED removal of this normalized title (a hand-added
|
||||
line removed earlier). Re-adding a title those records would filter
|
||||
from every rebuild is an explicit undo, not a conflict. Photo-scoped
|
||||
removals stay: they veto specific sightings, not the game."""
|
||||
norm = normalize_title(title)
|
||||
stored = _load_store(path)
|
||||
kept = [
|
||||
item
|
||||
for item in stored
|
||||
if not (
|
||||
(isinstance(item, str) and normalize_title(item) == norm)
|
||||
or (
|
||||
isinstance(item, dict)
|
||||
and not item.get("photos")
|
||||
and normalize_title(item["title"]) == norm
|
||||
)
|
||||
)
|
||||
]
|
||||
if len(kept) == len(stored):
|
||||
return False
|
||||
atomic_write_text(path, json.dumps(kept, indent=2, ensure_ascii=False) + "\n")
|
||||
return True
|
||||
|
||||
|
||||
def load_title_splits(path: Path) -> list[dict]:
|
||||
"""The human's split-into-copies decisions — durable: they must survive
|
||||
extract rebuilds and resolve --force."""
|
||||
return _scoped_records(path)
|
||||
|
||||
|
||||
def load_title_removals(path: Path) -> list[dict]:
|
||||
"""Titles the human removed from the catalog (not a game, a misread of
|
||||
box art, out of scope) — their sightings are filtered out of every
|
||||
rebuild, so re-extraction cannot resurrect them. Undo by deleting the
|
||||
record from the store file."""
|
||||
return _scoped_records(path)
|
||||
|
||||
|
||||
def is_split(norm: str, photos, splits: list[dict]) -> bool:
|
||||
"""Does a split decision cover this (normalized title, photo set)?
|
||||
Photo-scoped records only bind sightings from the photos that were on
|
||||
the split line — a same-named different edition keeps deduping."""
|
||||
photo_set = set(photos or [])
|
||||
return any(
|
||||
r["norm"] == norm and (r["photos"] is None or r["photos"] & photo_set)
|
||||
for r in splits
|
||||
)
|
||||
|
||||
|
||||
def _record_scoped(path: Path, title: str, photos: list[str] | None) -> None:
|
||||
if is_split(normalize_title(title), photos or [], _scoped_records(path)):
|
||||
return # an existing record already covers this line
|
||||
existing = _load_store(path)
|
||||
record = {"title": title, "photos": sorted(photos) if photos else None}
|
||||
atomic_write_text(
|
||||
path, json.dumps([*existing, record], indent=2, ensure_ascii=False) + "\n"
|
||||
)
|
||||
|
||||
|
||||
def record_title_split(path: Path, title: str, photos: list[str] | None = None) -> None:
|
||||
_record_scoped(path, title, photos)
|
||||
|
||||
|
||||
def record_title_removal(
|
||||
path: Path, title: str, photos: list[str] | None = None
|
||||
) -> None:
|
||||
_record_scoped(path, title, photos)
|
||||
|
||||
|
||||
def dedupe_entries(entries: list[dict], splits: list[dict] | None = None) -> list[dict]:
|
||||
"""Collapse same-normalized-title sightings unless their cues conflict —
|
||||
or unless the human declared them split (several physical copies):
|
||||
split sightings stay one entry per photo and never absorb new ones."""
|
||||
splits = splits or []
|
||||
|
||||
def covered(e: dict) -> bool:
|
||||
return is_split(e["title_normalized"], e.get("source_photos"), splits)
|
||||
|
||||
result: list[dict] = []
|
||||
for entry in entries:
|
||||
entry = {**entry, "title_normalized": normalize_title(entry["title_raw"])}
|
||||
if covered(entry):
|
||||
result.append(entry)
|
||||
continue
|
||||
for existing in result:
|
||||
if existing["title_normalized"] == entry[
|
||||
"title_normalized"
|
||||
] and not _cues_conflict(existing, entry):
|
||||
if (
|
||||
existing["title_normalized"] == entry["title_normalized"]
|
||||
and not covered(existing)
|
||||
and not cues_conflict(existing, entry)
|
||||
):
|
||||
existing.update(_merge(existing, entry))
|
||||
break
|
||||
else:
|
||||
@@ -246,32 +516,117 @@ def dedupe_entries(entries: list[dict]) -> list[dict]:
|
||||
return result
|
||||
|
||||
|
||||
def apply_title_removals(entries: list[dict], removals: list[dict]) -> list[dict]:
|
||||
"""Filter out sightings the human removed. Runs AFTER edits (records
|
||||
key on the title as displayed when removal was clicked) and before
|
||||
dedupe (so a removed sighting can't be absorbed into a survivor)."""
|
||||
if not removals:
|
||||
return entries
|
||||
return [
|
||||
e
|
||||
for e in entries
|
||||
if not is_split(
|
||||
normalize_title(e["title_raw"]), e.get("source_photos"), removals
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
def rebuild_artifacts(
|
||||
raw_dir: Path, titles_path: Path, unidentified_path: Path
|
||||
raw_dir: Path,
|
||||
titles_path: Path,
|
||||
unidentified_path: Path,
|
||||
splits: list[dict] | None = None,
|
||||
edits: list[dict] | None = None,
|
||||
removals: list[dict] | None = None,
|
||||
additions: list[dict] | None = None,
|
||||
) -> tuple[list[dict], dict[str, list[dict]]]:
|
||||
"""Regenerate titles.json and unidentified.json from the per-photo raw
|
||||
cache. Raw files written before the unidentified feature are bare
|
||||
arrays — still readable."""
|
||||
cache. A raw file is either an object with titles/unidentified or a bare
|
||||
array — still readable."""
|
||||
entries: list[dict] = []
|
||||
unidentified: dict[str, list[dict]] = {}
|
||||
for raw_file in sorted(raw_dir.glob("*.json")):
|
||||
try:
|
||||
data = json.loads(raw_file.read_text())
|
||||
if isinstance(data, list): # legacy format
|
||||
except json.JSONDecodeError as err:
|
||||
raise ValueError(
|
||||
f"{raw_file} is corrupt ({err}) — delete it (or re-upload "
|
||||
"the photo) and run extract again"
|
||||
) from err
|
||||
if isinstance(data, list): # bare-array shape
|
||||
entries.extend(data)
|
||||
continue
|
||||
entries.extend(data.get("titles") or [])
|
||||
photo = raw_file.name.removesuffix(".json")
|
||||
if data.get("unidentified"):
|
||||
unidentified[photo] = data["unidentified"]
|
||||
deduped = dedupe_entries(entries)
|
||||
entries = entries + [dict(e) for e in additions or []]
|
||||
entries = apply_title_removals(
|
||||
apply_title_edits(entries, edits or []), removals or []
|
||||
)
|
||||
deduped = dedupe_entries(entries, splits)
|
||||
titles_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
titles_path.write_text(json.dumps(deduped, indent=2, ensure_ascii=False) + "\n")
|
||||
unidentified_path.write_text(
|
||||
json.dumps(unidentified, indent=2, ensure_ascii=False) + "\n"
|
||||
atomic_write_text(
|
||||
titles_path, json.dumps(deduped, indent=2, ensure_ascii=False) + "\n"
|
||||
)
|
||||
atomic_write_text(
|
||||
unidentified_path, json.dumps(unidentified, indent=2, ensure_ascii=False) + "\n"
|
||||
)
|
||||
return deduped, unidentified
|
||||
|
||||
|
||||
def replay_titles(cfg: Config) -> None:
|
||||
"""Re-derive titles.json after a stored split or edit changed the rules.
|
||||
Prefers the raw caches (per-photo cue fidelity); without them (trimmed
|
||||
or hand-built data) it replays the current titles.json entries through
|
||||
the same edit + dedupe path, exploding multi-photo entries first so
|
||||
photo-level splits and targeted edits can take hold."""
|
||||
splits = load_title_splits(cfg.title_splits_path)
|
||||
edits = load_title_edits(cfg.title_edits_path)
|
||||
removals = load_title_removals(cfg.title_removals_path)
|
||||
additions = load_title_additions(cfg.title_additions_path)
|
||||
raw_dir = cfg.extract_raw_dir
|
||||
raw_photos = (
|
||||
{f.name.removesuffix(".json") for f in raw_dir.glob("*.json")}
|
||||
if raw_dir.is_dir()
|
||||
else set()
|
||||
)
|
||||
known_photos: set[str] = set()
|
||||
if cfg.titles_path.exists():
|
||||
for entry in json.loads(cfg.titles_path.read_text()):
|
||||
known_photos.update(entry.get("source_photos") or [])
|
||||
# raw caches are gitignored: a fresh clone (or a killed/partial extract)
|
||||
# can have FEWER raw files than titles.json has photos — rebuilding from
|
||||
# them would silently truncate the committed catalog
|
||||
if raw_photos and raw_photos >= known_photos:
|
||||
rebuild_artifacts(
|
||||
raw_dir,
|
||||
cfg.titles_path,
|
||||
cfg.unidentified_path,
|
||||
splits,
|
||||
edits,
|
||||
removals,
|
||||
additions,
|
||||
)
|
||||
return
|
||||
if not cfg.titles_path.exists():
|
||||
return
|
||||
exploded: list[dict] = []
|
||||
for entry in json.loads(cfg.titles_path.read_text()):
|
||||
photos = entry.get("source_photos") or []
|
||||
if len(photos) > 1:
|
||||
exploded.extend({**entry, "source_photos": [p]} for p in photos)
|
||||
else:
|
||||
exploded.append(dict(entry))
|
||||
exploded.extend(dict(e) for e in additions)
|
||||
deduped = dedupe_entries(
|
||||
apply_title_removals(apply_title_edits(exploded, edits), removals), splits
|
||||
)
|
||||
atomic_write_text(
|
||||
cfg.titles_path, json.dumps(deduped, indent=2, ensure_ascii=False) + "\n"
|
||||
)
|
||||
|
||||
|
||||
def run_extract(
|
||||
cfg: Config,
|
||||
*,
|
||||
@@ -298,28 +653,78 @@ def run_extract(
|
||||
)
|
||||
raise typer.Exit(code=1)
|
||||
|
||||
raw_dir = cfg.data_dir / "extract_raw"
|
||||
raw_dir = cfg.extract_raw_dir
|
||||
raw_dir.mkdir(parents=True, exist_ok=True)
|
||||
vision = vision or default_vision(cfg.model)
|
||||
vision = vision or vision_for(cfg)
|
||||
|
||||
failed: list[str] = []
|
||||
consecutive: tuple[str, int] = ("", 0)
|
||||
attempted = 0
|
||||
for photo in photos:
|
||||
raw_path = raw_dir / f"{photo.name}.json"
|
||||
if raw_path.exists() and not only and not force:
|
||||
try:
|
||||
json.loads(raw_path.read_text())
|
||||
typer.echo(f" {photo.name}: already extracted, skipping")
|
||||
continue
|
||||
except json.JSONDecodeError:
|
||||
typer.echo(f" {photo.name}: cache corrupt — re-extracting")
|
||||
attempted += 1
|
||||
try:
|
||||
result = extract_photo(photo, vision)
|
||||
raw_path.write_text(json.dumps(result, indent=2, ensure_ascii=False) + "\n")
|
||||
except Exception as err: # one bad photo must not block the rest
|
||||
failed.append(photo.name)
|
||||
typer.echo(f" {photo.name}: FAILED ({err}) — continuing")
|
||||
kind = type(err).__name__
|
||||
consecutive = (
|
||||
kind,
|
||||
consecutive[1] + 1 if kind == consecutive[0] else 1,
|
||||
)
|
||||
if consecutive[1] >= 3:
|
||||
typer.echo(
|
||||
" aborting — 3 identical consecutive failures look "
|
||||
"systemic (bad API key? wrong model in config.toml?), "
|
||||
"not per-photo; nothing more will be attempted"
|
||||
)
|
||||
break
|
||||
continue
|
||||
consecutive = ("", 0)
|
||||
atomic_write_text(
|
||||
raw_path, json.dumps(result, indent=2, ensure_ascii=False) + "\n"
|
||||
)
|
||||
note = (
|
||||
f" ({len(result['unidentified'])} unidentified)"
|
||||
if result["unidentified"]
|
||||
else ""
|
||||
)
|
||||
if result.get("dropped"):
|
||||
# the prompt forbids silently omitting a box; so do we
|
||||
note += (
|
||||
f" — DROPPED {result['dropped']} malformed entr"
|
||||
f"{'y' if result['dropped'] == 1 else 'ies'}; inspect "
|
||||
f"{raw_path}"
|
||||
)
|
||||
typer.echo(f" {photo.name}: {len(result['titles'])} title(s){note}")
|
||||
|
||||
deduped, unidentified = rebuild_artifacts(
|
||||
raw_dir, cfg.titles_path, cfg.unidentified_path
|
||||
raw_dir,
|
||||
cfg.titles_path,
|
||||
cfg.unidentified_path,
|
||||
load_title_splits(cfg.title_splits_path),
|
||||
load_title_edits(cfg.title_edits_path),
|
||||
load_title_removals(cfg.title_removals_path),
|
||||
load_title_additions(cfg.title_additions_path),
|
||||
)
|
||||
typer.echo(f"Wrote {len(deduped)} unique title(s) to {cfg.titles_path}.")
|
||||
if failed:
|
||||
typer.echo(
|
||||
f"\n{len(failed)} photo(s) failed extraction (re-run to retry): "
|
||||
+ ", ".join(failed)
|
||||
)
|
||||
if len(failed) == attempted:
|
||||
# every attempted photo failed: this run accomplished nothing
|
||||
# and a wrapper (or the web job runner) must not report success
|
||||
raise typer.Exit(code=1)
|
||||
|
||||
if unidentified:
|
||||
typer.echo(
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
"""Atomic file writes for every accumulated artifact.
|
||||
|
||||
A kill mid-write must never leave a torn file that poisons future runs —
|
||||
the same tmp + os.replace guarantee write_matches gives matches.csv,
|
||||
available to JSON artifacts and the XML response cache.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import csv
|
||||
import os
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def _tmp_for(path: Path) -> Path:
|
||||
# unique per writer: a FIXED tmp name lets two concurrent processes
|
||||
# interleave writes into one inode and replace garbage into place
|
||||
return path.with_name(f"{path.name}.{uuid.uuid4().hex[:8]}.tmp")
|
||||
|
||||
|
||||
def atomic_write_bytes(path: Path, data: bytes) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = _tmp_for(path)
|
||||
tmp.write_bytes(data)
|
||||
os.replace(tmp, path)
|
||||
|
||||
|
||||
def atomic_write_text(path: Path, text: str) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = _tmp_for(path)
|
||||
tmp.write_text(text)
|
||||
os.replace(tmp, path)
|
||||
|
||||
|
||||
def atomic_write_csv(path: Path, columns: list[str], rows: list[dict]) -> int:
|
||||
"""Atomic CSV rewrite (tmp + os.replace). Returns the written file's
|
||||
mtime_ns so callers tracking their own writes avoid a re-stat race."""
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = _tmp_for(path)
|
||||
with tmp.open("w", newline="") as f:
|
||||
writer = csv.DictWriter(
|
||||
f, fieldnames=columns, extrasaction="ignore", restval=""
|
||||
)
|
||||
writer.writeheader()
|
||||
writer.writerows(rows)
|
||||
# stat the TMP inode before the replace: statting the destination after
|
||||
# could hand back a FOREIGN writer's mtime landing in the gap, and the
|
||||
# caller would record someone else's write as its own
|
||||
mtime = tmp.stat().st_mtime_ns
|
||||
os.replace(tmp, path)
|
||||
return mtime
|
||||
@@ -0,0 +1,287 @@
|
||||
"""`bggpipe init` — guided first-run setup, idempotent like every stage.
|
||||
|
||||
Inspects the working directory and only fills gaps: creates photos/ and
|
||||
data/, writes a default config.toml when none exists, prompts for the
|
||||
credentials missing from the environment and .env (hidden input, appended
|
||||
to .env with 0600 permissions — values never echo and never reach logs),
|
||||
and offers the one-time Playwright browser download. Re-running reports
|
||||
status and prompts only for what is still missing; without a TTY it
|
||||
prints the status report and exits instead of hanging on a prompt.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
import typer
|
||||
|
||||
from bggpipe.config import Config
|
||||
|
||||
CONFIG_TEMPLATE = """\
|
||||
# Non-secret knobs for bggpipe. Everything account-related — including
|
||||
# your BGG username — lives in .env (see .env.example), not here.
|
||||
|
||||
photos_dir = "photos"
|
||||
data_dir = "data"
|
||||
rate_limit_seconds = 2.0
|
||||
|
||||
# Which vision backend reads your shelf photos. Both recipes below stay
|
||||
# on file; this line picks one.
|
||||
vision_provider = "anthropic"
|
||||
|
||||
[vision.anthropic]
|
||||
# reads ANTHROPIC_API_KEY from the environment
|
||||
model = "claude-sonnet-5"
|
||||
|
||||
[vision."openai-compatible"]
|
||||
# OpenAI, OpenRouter, or a local runtime (Ollama, LM Studio, vLLM).
|
||||
# key_env names the env var holding the key; "" = endpoint needs none.
|
||||
base_url = "http://localhost:11434/v1"
|
||||
model = "qwen2.5vl:7b"
|
||||
key_env = ""
|
||||
"""
|
||||
|
||||
ENV_HEADER = """\
|
||||
# bggpipe credentials — this file must stay out of version control.
|
||||
"""
|
||||
|
||||
# (key, secret?, why it's needed, where to get it)
|
||||
ENV_KEYS = (
|
||||
(
|
||||
"ANTHROPIC_API_KEY",
|
||||
True,
|
||||
"vision extraction (stage 1)",
|
||||
"https://console.anthropic.com/",
|
||||
),
|
||||
(
|
||||
"BGG_USERNAME",
|
||||
False,
|
||||
"diff/upload/enrich",
|
||||
"your boardgamegeek.com account name",
|
||||
),
|
||||
(
|
||||
"BGG_PASSWORD",
|
||||
True,
|
||||
"upload's website login (stage 5)",
|
||||
"your boardgamegeek.com password",
|
||||
),
|
||||
(
|
||||
"BGG_API_TOKEN",
|
||||
True,
|
||||
"resolve/diff/enrich (stages 2, 4, 6)",
|
||||
"https://boardgamegeek.com/applications — approval takes a week+, "
|
||||
"apply early; everything except upload works while you wait",
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class InitReport:
|
||||
created_dirs: list[str] = field(default_factory=list)
|
||||
wrote_config: bool = False
|
||||
keys_ready: list[str] = field(default_factory=list)
|
||||
keys_written: list[str] = field(default_factory=list)
|
||||
keys_missing: list[str] = field(default_factory=list)
|
||||
browser_installed: bool = False
|
||||
warnings: list[str] = field(default_factory=list)
|
||||
|
||||
|
||||
def load_env_file(env_path: Path) -> list[str]:
|
||||
"""Export .env values into this process's environment — only keys the
|
||||
environment doesn't already set (a real env var always wins). Returns
|
||||
the keys loaded. Same parsing rules as _env_file_keys; values go
|
||||
straight into os.environ and are never printed or logged."""
|
||||
if not env_path.exists():
|
||||
return []
|
||||
loaded = []
|
||||
for line in env_path.read_text().splitlines():
|
||||
line = line.strip()
|
||||
if "=" not in line or line.startswith("#"):
|
||||
continue
|
||||
key, _, value = line.partition("=")
|
||||
key = key.strip()
|
||||
if key.startswith("export "):
|
||||
key = key.removeprefix("export ").strip()
|
||||
value = value.strip()
|
||||
if len(value) >= 2 and value[0] == value[-1] and value[0] in "'\"":
|
||||
value = value[1:-1]
|
||||
if key and value and not os.environ.get(key):
|
||||
os.environ[key] = value
|
||||
loaded.append(key)
|
||||
return loaded
|
||||
|
||||
|
||||
def _env_file_keys(env_path: Path) -> set[str]:
|
||||
"""Key names with non-empty values in .env. Handles `export KEY=v` and
|
||||
quoted values; a quoted-empty value ("" / '') counts as NOT set. Values
|
||||
never outlive this parse and are never printed."""
|
||||
if not env_path.exists():
|
||||
return set()
|
||||
present = set()
|
||||
for line in env_path.read_text().splitlines():
|
||||
line = line.strip()
|
||||
if "=" not in line or line.startswith("#"):
|
||||
continue
|
||||
key, _, value = line.partition("=")
|
||||
key = key.strip()
|
||||
if key.startswith("export "):
|
||||
key = key.removeprefix("export ").strip()
|
||||
value = value.strip()
|
||||
if len(value) >= 2 and value[0] == value[-1] and value[0] in "'\"":
|
||||
value = value[1:-1]
|
||||
if value:
|
||||
present.add(key)
|
||||
return present
|
||||
|
||||
|
||||
def _quote_env_value(value: str) -> str:
|
||||
"""Single-quote for `source`/direnv safety: spaces, $, backslashes and
|
||||
quotes must survive the shell verbatim."""
|
||||
return "'" + value.replace("'", "'\\''") + "'"
|
||||
|
||||
|
||||
def _default_secret_prompt(label: str) -> str:
|
||||
return typer.prompt(label, default="", show_default=False, hide_input=True)
|
||||
|
||||
|
||||
def _default_plain_prompt(label: str) -> str:
|
||||
return typer.prompt(label, default="", show_default=False)
|
||||
|
||||
|
||||
def _default_install_browser() -> bool:
|
||||
result = subprocess.run( # noqa: S603 — fixed argv, no shell
|
||||
[sys.executable, "-m", "playwright", "install", "chromium"],
|
||||
check=False,
|
||||
)
|
||||
return result.returncode == 0
|
||||
|
||||
|
||||
def run_init(
|
||||
cfg: Config,
|
||||
*,
|
||||
project_dir: Path | None = None,
|
||||
interactive: bool | None = None,
|
||||
plain_prompt: Callable[[str], str] = _default_plain_prompt,
|
||||
secret_prompt: Callable[[str], str] = _default_secret_prompt,
|
||||
confirm: Callable[[str], bool] | None = None,
|
||||
install_browser: Callable[[], bool] = _default_install_browser,
|
||||
) -> InitReport:
|
||||
project_dir = project_dir or Path.cwd()
|
||||
if interactive is None:
|
||||
interactive = sys.stdin.isatty()
|
||||
if confirm is None:
|
||||
confirm = lambda label: typer.confirm(label, default=True) # noqa: E731
|
||||
|
||||
report = InitReport()
|
||||
typer.echo("bggpipe setup — checks what exists, fills only the gaps.\n")
|
||||
if trap := cfg.tracked_data_warning():
|
||||
typer.echo(f"WARNING: {trap}\n", err=True)
|
||||
report.warnings.append(trap)
|
||||
|
||||
# -- directories ----------------------------------------------------
|
||||
for path in (project_dir / cfg.photos_dir, project_dir / cfg.data_dir):
|
||||
if not path.is_dir():
|
||||
path.mkdir(parents=True)
|
||||
report.created_dirs.append(str(path))
|
||||
typer.echo(f" created {path}/")
|
||||
else:
|
||||
typer.echo(f" found {path}/")
|
||||
|
||||
# -- config.toml ----------------------------------------------------
|
||||
config_path = project_dir / "config.toml"
|
||||
if not config_path.exists():
|
||||
config_path.write_text(CONFIG_TEMPLATE)
|
||||
report.wrote_config = True
|
||||
typer.echo(f" wrote {config_path} (defaults — edit if you like)")
|
||||
else:
|
||||
typer.echo(f" found {config_path}")
|
||||
|
||||
# -- credentials ----------------------------------------------------
|
||||
# the vision key follows the configured provider: an openai-compatible
|
||||
# setup prompts for ITS key env (or none, for a local endpoint)
|
||||
from bggpipe.config import load_config
|
||||
|
||||
active = load_config(config_path if config_path.exists() else None)
|
||||
env_keys = list(ENV_KEYS)
|
||||
if active.vision_provider != "anthropic":
|
||||
env_keys = [k for k in env_keys if k[0] != "ANTHROPIC_API_KEY"]
|
||||
if active.vision_key_env:
|
||||
env_keys.insert(
|
||||
0,
|
||||
(
|
||||
active.vision_key_env,
|
||||
True,
|
||||
"vision extraction via "
|
||||
+ (active.vision_base_url or "the configured endpoint"),
|
||||
"your vision provider's console",
|
||||
),
|
||||
)
|
||||
else:
|
||||
typer.echo(
|
||||
" vision: openai-compatible endpoint with no key configured "
|
||||
"(local runtime) — nothing to prompt for"
|
||||
)
|
||||
env_path = project_dir / ".env"
|
||||
in_file = _env_file_keys(env_path)
|
||||
for key, secret, why, where in env_keys:
|
||||
if os.environ.get(key) or key in in_file:
|
||||
report.keys_ready.append(key)
|
||||
typer.echo(f" {key}: set")
|
||||
continue
|
||||
if not interactive:
|
||||
report.keys_missing.append(key)
|
||||
continue
|
||||
typer.echo(f"\n {key} — needed for {why}\n ({where})")
|
||||
prompt = secret_prompt if secret else plain_prompt
|
||||
value = prompt(f" {key} (enter to skip)").strip()
|
||||
if not value:
|
||||
report.keys_missing.append(key)
|
||||
continue
|
||||
is_new_file = not env_path.exists()
|
||||
# owner-only from the FIRST byte — chmod-after-write leaves a
|
||||
# world-readable window holding a credential
|
||||
fd = os.open(env_path, os.O_CREAT | os.O_WRONLY | os.O_APPEND, 0o600)
|
||||
with os.fdopen(fd, "a") as f:
|
||||
if is_new_file:
|
||||
f.write(ENV_HEADER)
|
||||
f.write(f"{key}={_quote_env_value(value)}\n")
|
||||
env_path.chmod(0o600) # older files created by hand tighten up too
|
||||
report.keys_written.append(key)
|
||||
typer.echo(f" {key}: saved to {env_path}")
|
||||
|
||||
# -- browser --------------------------------------------------------
|
||||
if interactive and confirm(
|
||||
"\nDownload/verify the Chromium browser for the upload stage? "
|
||||
"(one-time, ~100MB; skippable)"
|
||||
):
|
||||
report.browser_installed = install_browser()
|
||||
if not report.browser_installed:
|
||||
typer.echo(" browser install failed — re-run init to retry")
|
||||
|
||||
# -- summary --------------------------------------------------------
|
||||
typer.echo("\nStatus:")
|
||||
for key in report.keys_ready + report.keys_written:
|
||||
typer.echo(f" ready {key}")
|
||||
for key in report.keys_missing:
|
||||
typer.echo(f" missing {key}")
|
||||
if report.keys_missing:
|
||||
typer.echo(
|
||||
"\nMissing keys are fine to start: extract only needs "
|
||||
"ANTHROPIC_API_KEY, and re-running `bggpipe init` prompts for "
|
||||
"the rest whenever you have them."
|
||||
)
|
||||
typer.echo(
|
||||
"\nNext: drop shelf photos into "
|
||||
f"{project_dir / cfg.photos_dir}/ and run `bggpipe extract`."
|
||||
)
|
||||
if env_path.exists() and not os.environ.get("BGG_USERNAME"):
|
||||
typer.echo(
|
||||
"Load .env into your shell first: `direnv allow` (if you use "
|
||||
"direnv) or `set -a; source .env; set +a`."
|
||||
)
|
||||
return report
|
||||
@@ -0,0 +1,113 @@
|
||||
"""One-at-a-time background execution of pipeline stages for the web UI.
|
||||
|
||||
The stages share the data/ artifacts, so running two concurrently is
|
||||
unsupported everywhere in the pipeline — the runner enforces it with a
|
||||
single slot. Stage output (typer.echo goes to stdout) is captured by
|
||||
swapping sys.stdout for the job's duration; that swap is process-wide,
|
||||
which is safe here only because the runner holds the single slot and the
|
||||
web server itself never writes to stdout (uvicorn logs on stderr).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import threading
|
||||
import time
|
||||
import traceback
|
||||
from collections.abc import Callable
|
||||
from contextlib import redirect_stdout
|
||||
|
||||
import typer
|
||||
|
||||
MAX_LOG_LINES = 1000 # buffer cap; snapshots serve the last 200
|
||||
|
||||
|
||||
class _LineBuffer(io.TextIOBase):
|
||||
def __init__(self) -> None:
|
||||
self._lock = threading.Lock()
|
||||
self._lines: list[str] = []
|
||||
self._partial = ""
|
||||
|
||||
def write(self, text: str) -> int:
|
||||
with self._lock:
|
||||
self._partial += text
|
||||
*complete, self._partial = self._partial.split("\n")
|
||||
self._lines.extend(complete)
|
||||
if len(self._lines) > MAX_LOG_LINES: # bound memory on long runs
|
||||
del self._lines[: len(self._lines) - MAX_LOG_LINES]
|
||||
return len(text)
|
||||
|
||||
def lines(self) -> list[str]:
|
||||
with self._lock:
|
||||
return self._lines + ([self._partial] if self._partial else [])
|
||||
|
||||
|
||||
class JobRunner:
|
||||
"""At most one running job; finished state persists until the next start."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._lock = threading.Lock()
|
||||
self._thread: threading.Thread | None = None
|
||||
self._stage = ""
|
||||
self._status = "idle" # idle | running | done | failed
|
||||
self._buffer = _LineBuffer()
|
||||
self._error = ""
|
||||
self._started = 0.0
|
||||
self._finished = 0.0
|
||||
|
||||
def start(self, stage: str, fn: Callable[[], object]) -> bool:
|
||||
"""Begin a job; False when one is already running."""
|
||||
with self._lock:
|
||||
if self._status == "running":
|
||||
return False
|
||||
self._stage = stage
|
||||
self._status = "running"
|
||||
self._buffer = _LineBuffer()
|
||||
self._error = ""
|
||||
self._started = time.time()
|
||||
self._finished = 0.0
|
||||
self._thread = threading.Thread(target=self._run, args=(fn,), daemon=True)
|
||||
self._thread.start()
|
||||
return True
|
||||
|
||||
def _run(self, fn: Callable[[], object]) -> None:
|
||||
status, error = "done", ""
|
||||
try:
|
||||
try:
|
||||
with redirect_stdout(self._buffer):
|
||||
fn()
|
||||
except typer.Exit as exc:
|
||||
if exc.exit_code:
|
||||
status, error = "failed", f"exited with code {exc.exit_code}"
|
||||
except SystemExit as exc:
|
||||
if exc.code:
|
||||
status, error = "failed", f"exited with code {exc.code}"
|
||||
except BaseException as exc: # incl. GreenletExit et al: the
|
||||
# runner must NEVER stay "running" — that wedges every
|
||||
# future stage behind a 409 until a server restart
|
||||
status, error = "failed", f"{type(exc).__name__}: {exc}"
|
||||
# the short error names the exception; the log carries the
|
||||
# traceback, or a failure is undiagnosable from the UI
|
||||
self._buffer.write("\n" + traceback.format_exc())
|
||||
finally:
|
||||
with self._lock:
|
||||
self._status = status
|
||||
self._error = error
|
||||
self._finished = time.time()
|
||||
|
||||
def snapshot(self) -> dict:
|
||||
with self._lock:
|
||||
return {
|
||||
"stage": self._stage,
|
||||
"status": self._status,
|
||||
"log": self._buffer.lines()[-200:],
|
||||
"error": self._error,
|
||||
"started": self._started,
|
||||
"finished": self._finished,
|
||||
}
|
||||
|
||||
def wait(self, timeout: float = 10.0) -> None:
|
||||
"""Test hook: block until the current job's thread finishes."""
|
||||
thread = self._thread
|
||||
if thread is not None:
|
||||
thread.join(timeout)
|
||||
+166
-7
@@ -2,7 +2,8 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import xml.etree.ElementTree as ET # element types only; parsing goes via defusedxml
|
||||
import warnings
|
||||
import xml.etree.ElementTree as ET # parsing itself goes via defusedxml
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from defusedxml.ElementTree import fromstring as _safe_fromstring
|
||||
@@ -12,6 +13,27 @@ class BGGResponseError(Exception):
|
||||
"""The API returned a well-formed error document (e.g. bad username)."""
|
||||
|
||||
|
||||
# The two status predicates the whole pipeline shares: a version is trusted
|
||||
# for diff/upload/enrich only when matching produced it confidently or a
|
||||
# human approved it, and a row reaches diff/enrich only when its match did.
|
||||
CONFIDENT_VERSION_STATUSES = ("version_auto", "version_approved")
|
||||
RECOGNIZED_MATCH_STATUSES = ("auto", "approved")
|
||||
PENDING_MATCH_STATUSES = ("ambiguous", "unmatched")
|
||||
UNDECIDED_MATCH_STATUSES = ("ambiguous", "unmatched", "merged")
|
||||
|
||||
|
||||
def is_recognized(row: dict) -> bool:
|
||||
"""A row that reaches diff/enrich: matched and carrying a real id."""
|
||||
return row["match_status"] in RECOGNIZED_MATCH_STATUSES and bool(row["bgg_id"])
|
||||
|
||||
|
||||
def is_confident_version(row: dict) -> bool:
|
||||
"""A version trusted for upload: auto-scored or human-approved, with id."""
|
||||
return row["version_status"] in CONFIDENT_VERSION_STATUSES and bool(
|
||||
row["version_id"]
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SearchResult:
|
||||
bgg_id: int
|
||||
@@ -72,18 +94,43 @@ def _attr_int(elem: ET.Element | None, attr: str = "value") -> int | None:
|
||||
|
||||
def parse_search(xml_text: str) -> list[SearchResult]:
|
||||
results = []
|
||||
by_id: dict[int, int] = {} # bgg_id -> index in results
|
||||
skipped = 0
|
||||
for item in _root(xml_text).findall("item"):
|
||||
name_elem = item.find("name")
|
||||
if name_elem is None or item.get("id") is None:
|
||||
skipped += 1 # tolerate stragglers; wholesale drift raises below
|
||||
continue
|
||||
results.append(
|
||||
SearchResult(
|
||||
result = SearchResult(
|
||||
bgg_id=int(item.get("id")),
|
||||
name=name_elem.get("value", ""),
|
||||
name_type=name_elem.get("type", "primary"),
|
||||
year=_attr_int(item.find("yearpublished")),
|
||||
type=item.get("type", "boardgame"),
|
||||
)
|
||||
# a multi-type search lists an expansion TWICE — once per matched
|
||||
# type; collapse only SAME-NAME duplicates, preferring the specific
|
||||
# type so expansion tagging (the base-vs-expansion review guard)
|
||||
# survives. A duplicate under a DIFFERENT name stays: it may be the
|
||||
# alternate name that exact-matches the query, and dropping it
|
||||
# would silently downgrade the match to fuzzy.
|
||||
dedupe_key = (result.bgg_id, result.name.casefold())
|
||||
if dedupe_key in by_id:
|
||||
seen = results[by_id[dedupe_key]]
|
||||
if seen.type == "boardgame" and result.type != "boardgame":
|
||||
results[by_id[dedupe_key]] = result
|
||||
continue
|
||||
by_id[dedupe_key] = len(results)
|
||||
results.append(result)
|
||||
if skipped and results:
|
||||
warnings.warn(
|
||||
f"search: {skipped} unparseable item(s) tolerated — schema drift?",
|
||||
stacklevel=2,
|
||||
)
|
||||
if skipped and not results:
|
||||
raise BGGResponseError(
|
||||
f"search response had {skipped} item(s), none parseable — "
|
||||
"schema drift? Bad data must not look like no results."
|
||||
)
|
||||
return results
|
||||
|
||||
@@ -112,6 +159,8 @@ def parse_things(xml_text: str) -> list[ThingDetails]:
|
||||
for item in _root(xml_text).findall("item"):
|
||||
name = item.find("name[@type='primary']")
|
||||
rank_elem = item.find(".//ranks/rank[@name='boardgame']")
|
||||
if rank_elem is None: # RPGGeek items rank in their own family
|
||||
rank_elem = item.find(".//ranks/rank[@name='rpgitem']")
|
||||
versions = [
|
||||
v
|
||||
for v_item in item.findall("versions/item")
|
||||
@@ -119,7 +168,7 @@ def parse_things(xml_text: str) -> list[ThingDetails]:
|
||||
]
|
||||
things.append(
|
||||
ThingDetails(
|
||||
bgg_id=int(item.get("id", 0)),
|
||||
bgg_id=int(_required_attr(item, "id")),
|
||||
name=name.get("value", "") if name is not None else "",
|
||||
year=_attr_int(item.find("yearpublished")),
|
||||
type=item.get("type", "boardgame"),
|
||||
@@ -135,6 +184,110 @@ def parse_things(xml_text: str) -> list[ThingDetails]:
|
||||
return things
|
||||
|
||||
|
||||
def _attr_float(elem: ET.Element | None, attr: str = "value") -> float | None:
|
||||
if elem is None:
|
||||
return None
|
||||
try:
|
||||
return float(elem.get(attr))
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def parse_things_full(xml_text: str) -> list[dict]:
|
||||
"""Full game metadata for the enrich stage (games.json). Returns plain
|
||||
dicts — this is artifact data, not pipeline logic."""
|
||||
games = []
|
||||
for item in _root(xml_text).findall("item"):
|
||||
|
||||
def links(link_type: str, item: ET.Element = item) -> list[str]:
|
||||
return [
|
||||
link.get("value", "")
|
||||
for link in item.findall(f"link[@type='{link_type}']")
|
||||
]
|
||||
|
||||
name = item.find("name[@type='primary']")
|
||||
ratings = item.find("statistics/ratings")
|
||||
best_player_counts = []
|
||||
poll = item.find("poll[@name='suggested_numplayers']")
|
||||
if poll is not None:
|
||||
for results in poll.findall("results"):
|
||||
votes = {
|
||||
r.get("value"): int(r.get("numvotes") or 0)
|
||||
for r in results.findall("result")
|
||||
}
|
||||
best = votes.get("Best", 0)
|
||||
if (
|
||||
best
|
||||
and best >= votes.get("Recommended", 0)
|
||||
and best > votes.get("Not Recommended", 0)
|
||||
):
|
||||
best_player_counts.append(results.get("numplayers"))
|
||||
games.append(
|
||||
{
|
||||
"bgg_id": int(_required_attr(item, "id")),
|
||||
"type": item.get("type", "boardgame"),
|
||||
"name": name.get("value", "") if name is not None else "",
|
||||
"year": _attr_int(item.find("yearpublished")),
|
||||
"description": (item.findtext("description") or "").strip(),
|
||||
"image": (item.findtext("image") or "").strip(),
|
||||
"thumbnail": (item.findtext("thumbnail") or "").strip(),
|
||||
"min_players": _attr_int(item.find("minplayers")),
|
||||
"max_players": _attr_int(item.find("maxplayers")),
|
||||
"best_player_counts": best_player_counts,
|
||||
"playtime": _attr_int(item.find("playingtime")),
|
||||
"min_playtime": _attr_int(item.find("minplaytime")),
|
||||
"max_playtime": _attr_int(item.find("maxplaytime")),
|
||||
"min_age": _attr_int(item.find("minage")),
|
||||
# RPGGeek items live in the same database but use their own
|
||||
# link types, so a board-game-only reader finds none of them
|
||||
"designers": links("boardgamedesigner") + links("rpgdesigner"),
|
||||
"artists": links("boardgameartist") + links("rpgartist"),
|
||||
"publishers": links("boardgamepublisher") + links("rpgpublisher"),
|
||||
"categories": links("boardgamecategory")
|
||||
+ links("rpggenre")
|
||||
+ links("rpgcategory"),
|
||||
"mechanics": links("boardgamemechanic") + links("rpgmechanic"),
|
||||
"producers": links("rpgproducer"),
|
||||
"series": links("rpgseries"),
|
||||
"rating": _attr_float(ratings.find("average"))
|
||||
if ratings is not None
|
||||
else None,
|
||||
"weight": _attr_float(ratings.find("averageweight"))
|
||||
if ratings is not None
|
||||
else None,
|
||||
"rank": _attr_int(
|
||||
item.find(".//ranks/rank[@name='boardgame']")
|
||||
if item.find(".//ranks/rank[@name='boardgame']") is not None
|
||||
else item.find(".//ranks/rank[@name='rpgitem']")
|
||||
),
|
||||
"users_owned": _attr_int(ratings.find("owned"))
|
||||
if ratings is not None
|
||||
else None,
|
||||
}
|
||||
)
|
||||
return games
|
||||
|
||||
|
||||
def _required_attr(item: ET.Element, name: str) -> str:
|
||||
value = item.get(name)
|
||||
if not value:
|
||||
raise BGGResponseError(
|
||||
f"response item missing {name!r} — truncated or unexpected "
|
||||
"response; refusing to coerce a missing id"
|
||||
)
|
||||
return value
|
||||
|
||||
|
||||
def validate_response(xml_text: str) -> None:
|
||||
"""Raise BGGResponseError for error documents AND malformed XML — the
|
||||
client calls this before caching, so a torn or truncated 200 body can
|
||||
never poison the cache."""
|
||||
try:
|
||||
_root(xml_text)
|
||||
except ET.ParseError as err:
|
||||
raise BGGResponseError(f"malformed XML: {err}") from err
|
||||
|
||||
|
||||
def parse_collection(xml_text: str) -> list[CollectionItem]:
|
||||
items = []
|
||||
for item in _root(xml_text).findall("item"):
|
||||
@@ -143,12 +296,18 @@ def parse_collection(xml_text: str) -> list[CollectionItem]:
|
||||
version_item = item.find("version/item")
|
||||
items.append(
|
||||
CollectionItem(
|
||||
object_id=int(item.get("objectid", 0)),
|
||||
coll_id=int(item.get("collid", 0)),
|
||||
# strict: a missing id coerced to 0 would collide in the
|
||||
# collid dedupe and silently drop owned games from the diff
|
||||
object_id=int(_required_attr(item, "objectid")),
|
||||
coll_id=int(_required_attr(item, "collid")),
|
||||
name=item.findtext("name", default=""),
|
||||
subtype=item.get("subtype", "boardgame"),
|
||||
own=status is not None and status.get("own") == "1",
|
||||
year=int(year_text) if year_text and year_text.isdigit() else None,
|
||||
year=(
|
||||
int(year_text)
|
||||
if year_text and year_text.lstrip("-").isdigit()
|
||||
else None
|
||||
),
|
||||
version_id=(
|
||||
int(version_item.get("id"))
|
||||
if version_item is not None and version_item.get("id")
|
||||
|
||||
+370
-66
@@ -2,25 +2,33 @@
|
||||
|
||||
Reads data/titles.json, queries BGG search (+ thing stats for tie-breaks,
|
||||
+ versions once a game is settled), classifies each title auto/ambiguous/
|
||||
unmatched, and appends rows to data/matches.csv. Re-runs skip titles
|
||||
already present in matches.csv unless --force.
|
||||
unmatched, then post-dedupes rows resolving to the same physical game
|
||||
(losers become match_status="merged"; review can veto). The whole file is
|
||||
rewritten atomically each run; re-runs pair entries to their existing rows
|
||||
(photo overlap, then position) unless --force starts over.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import csv
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
from collections import Counter
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
import typer
|
||||
from rapidfuzz import fuzz
|
||||
|
||||
from bggpipe.bgg_client import BGGClient
|
||||
from bggpipe.bgg_client import BGGAuthError, BGGClient, BGGQueueTimeout, client_for
|
||||
from bggpipe.config import Config
|
||||
from bggpipe.models import GameVersion
|
||||
from bggpipe.extract import cues_conflict, is_split, load_title_splits
|
||||
from bggpipe.fsio import atomic_write_csv
|
||||
from bggpipe.models import (
|
||||
RECOGNIZED_MATCH_STATUSES,
|
||||
GameVersion,
|
||||
is_confident_version,
|
||||
)
|
||||
from bggpipe.normalize import normalize_title
|
||||
|
||||
FUZZY_THRESHOLD = 90
|
||||
@@ -29,6 +37,10 @@ FUZZY_THRESHOLD = 90
|
||||
DOMINANCE_MIN_OWNED = 100
|
||||
DOMINANCE_FACTOR = 10
|
||||
VERSION_PLAUSIBLE_SCORE = 2
|
||||
# "Is this the publisher on the box?" — one answer, asked in two places
|
||||
# (candidate tie-break and version scoring): the sites must move together.
|
||||
PUBLISHER_MATCH_THRESHOLD = 85
|
||||
EDITION_NAME_THRESHOLD = 80
|
||||
|
||||
MATCH_COLUMNS = [
|
||||
"title_raw",
|
||||
@@ -43,6 +55,8 @@ MATCH_COLUMNS = [
|
||||
"candidates_json",
|
||||
"version_candidates_json",
|
||||
"source_photos",
|
||||
"merged_into",
|
||||
"dedupe_veto",
|
||||
]
|
||||
|
||||
|
||||
@@ -75,6 +89,7 @@ class Candidate:
|
||||
type: str
|
||||
exact: bool
|
||||
fuzzy: float
|
||||
sibling: bool = False # exactness earned via edition-suffix stripping
|
||||
owned: int | None = None
|
||||
rank: int | None = None
|
||||
publishers: list[str] = field(default_factory=list)
|
||||
@@ -125,6 +140,8 @@ class MatchRow:
|
||||
self.version_candidates, ensure_ascii=False
|
||||
),
|
||||
"source_photos": ";".join(self.source_photos),
|
||||
"merged_into": "",
|
||||
"dedupe_veto": "",
|
||||
}
|
||||
|
||||
|
||||
@@ -139,8 +156,10 @@ def load_titles(path: Path) -> list[TitleEntry]:
|
||||
entries.append(
|
||||
TitleEntry(
|
||||
title_raw=title_raw,
|
||||
title_normalized=raw.get("title_normalized")
|
||||
or normalize_title(title_raw),
|
||||
# always recompute: a hand-written stored value would
|
||||
# silently break exact matching (both sides must normalize
|
||||
# by the current rules)
|
||||
title_normalized=normalize_title(title_raw),
|
||||
confidence=raw.get("confidence", "high"),
|
||||
publisher_hint=raw.get("publisher_hint") or "",
|
||||
edition_hint=raw.get("edition_hint") or "",
|
||||
@@ -163,19 +182,22 @@ def _truncation_heads(title_raw: str) -> list[str]:
|
||||
before the first subtitle separator, before a "(The) Game ..."
|
||||
descriptor, then the first two words as a last resort."""
|
||||
heads: list[str] = []
|
||||
sep_head = next(
|
||||
(title_raw.split(sep)[0] for sep in _SEPARATORS if sep in title_raw), None
|
||||
)
|
||||
present = [(title_raw.find(sep), sep) for sep in _SEPARATORS if sep in title_raw]
|
||||
# earliest separator wins — priority order would let " - " late in the
|
||||
# title beat an early ": ", yielding heads like "Blorvath: Quest"
|
||||
sep_head = title_raw.split(min(present)[1])[0] if present else None
|
||||
if sep_head:
|
||||
heads.append(sep_head)
|
||||
match = _GAME_WORD.search(title_raw)
|
||||
if match and match.start() > 0:
|
||||
heads.append(title_raw[: match.start()])
|
||||
# last resort: first two words — of the pre-subtitle part, so a title
|
||||
# like "Blorvath: Quest of the Zzyzx" never yields "Blorvath: Quest"
|
||||
# last resort: shrink from the right, longest first — a printed title
|
||||
# can bury the real name in the middle ("ALICE IS MISSING A SILENT ROLE
|
||||
# PLAYING GAME" is "Alice is Missing"). Only exact normalized matches
|
||||
# count for heads, so a short head cannot match loosely.
|
||||
words = (sep_head or title_raw).split()
|
||||
if len(words) > 2:
|
||||
heads.append(" ".join(words[:2]))
|
||||
for size in range(len(words) - 1, 1, -1):
|
||||
heads.append(" ".join(words[:size]))
|
||||
|
||||
seen: set[str] = {normalize_title(title_raw)}
|
||||
unique: list[str] = []
|
||||
@@ -184,19 +206,41 @@ def _truncation_heads(title_raw: str) -> list[str]:
|
||||
if norm and norm not in seen:
|
||||
seen.add(norm)
|
||||
unique.append(head)
|
||||
return unique[:3]
|
||||
return unique[:6]
|
||||
|
||||
|
||||
def _plausible_candidates(
|
||||
client: BGGClient, entry: TitleEntry, query: str, head_normalized: str = ""
|
||||
client: BGGClient,
|
||||
entry: TitleEntry,
|
||||
query: str,
|
||||
head_normalized: str = "",
|
||||
types: str | None = None,
|
||||
) -> list[Candidate]:
|
||||
"""Search BGG and keep plausible candidates, one per id: exact-normalized
|
||||
or fuzzy>=90 against the FULL title, or — on truncated retries — exact
|
||||
(only exact: truncation must stay conservative) against the head."""
|
||||
# a fully non-Latin title normalizes to "" — empty-vs-empty is not a
|
||||
# match (token_sort_ratio("", "") is 100), and searching would only
|
||||
# spend rate-limited requests to prove nothing
|
||||
if not entry.title_normalized:
|
||||
return []
|
||||
by_id: dict[int, Candidate] = {}
|
||||
for result in client.search(query):
|
||||
results = client.search(query, types) if types else client.search(query)
|
||||
for result in results:
|
||||
norm = normalize_title(result.name)
|
||||
exact = norm == entry.title_normalized
|
||||
# BGG files new editions as SEPARATE games named "X (Nth Edition)":
|
||||
# a name that equals the title once its trailing parenthetical is
|
||||
# stripped is a sibling edition — exact-grade, or the match looks
|
||||
# unanimously confident while hiding the real choice (Wiz-War has
|
||||
# three same-named lineages; the spec's top failure mode)
|
||||
sibling = False
|
||||
if not exact and result.type != "boardgameexpansion":
|
||||
sibling = (
|
||||
normalize_title(_EDITION_SUFFIX.sub("", result.name))
|
||||
== entry.title_normalized
|
||||
)
|
||||
exact = sibling
|
||||
fuzzy = fuzz.token_sort_ratio(norm, entry.title_normalized)
|
||||
if not exact and fuzzy < FUZZY_THRESHOLD:
|
||||
if not (head_normalized and norm == head_normalized):
|
||||
@@ -208,6 +252,7 @@ def _plausible_candidates(
|
||||
year=result.year,
|
||||
type=result.type,
|
||||
exact=exact,
|
||||
sibling=sibling,
|
||||
fuzzy=fuzzy,
|
||||
)
|
||||
prev = by_id.get(result.bgg_id)
|
||||
@@ -226,7 +271,23 @@ def _classify(client: BGGClient, entry: TitleEntry, cands: list[Candidate]) -> M
|
||||
return row
|
||||
|
||||
if len(cands) == 1:
|
||||
chosen = cands[0]
|
||||
# a lone candidate must still earn trust: BGG's search visibly
|
||||
# truncates generic queries (the game named "Dungeon!" appears in
|
||||
# NEITHER of its own searches), so the sole survivor may be an
|
||||
# impostor standing where the famous game should be. Sibling-grade
|
||||
# exactness never autos alone, and a true exact must clear the
|
||||
# same ownership floor the dominance rule enforces.
|
||||
only = cands[0]
|
||||
stats = {t.bgg_id: t for t in client.things([only.bgg_id], stats=True)}
|
||||
if only.bgg_id in stats:
|
||||
only.owned = stats[only.bgg_id].owned
|
||||
only.rank = stats[only.bgg_id].rank
|
||||
only.publishers = list(stats[only.bgg_id].publishers)
|
||||
row.candidates = [only]
|
||||
if only.sibling or not only.exact or (only.owned or 0) < DOMINANCE_MIN_OWNED:
|
||||
row.match_status = "ambiguous"
|
||||
return row
|
||||
chosen = only
|
||||
else:
|
||||
top = cands[:5]
|
||||
stats = {
|
||||
@@ -257,13 +318,19 @@ def _publisher_pick(entry: TitleEntry, top: list[Candidate]) -> Candidate | None
|
||||
candidate is from that publisher, that's the game."""
|
||||
if not entry.publisher_hint:
|
||||
return None
|
||||
if len({c.type for c in top}) > 1:
|
||||
# mixed base-game/expansion candidates are never auto-resolved —
|
||||
# the same veto _dominant applies (spec's top failure mode); an
|
||||
# alternate name can make an expansion "exact" too
|
||||
return None
|
||||
hint = normalize_title(entry.publisher_hint)
|
||||
matches = [
|
||||
c
|
||||
for c in top
|
||||
if c.exact
|
||||
and any(
|
||||
fuzz.partial_ratio(hint, normalize_title(p)) >= 85 for p in c.publishers
|
||||
fuzz.partial_ratio(hint, normalize_title(p)) >= PUBLISHER_MATCH_THRESHOLD
|
||||
for p in c.publishers
|
||||
)
|
||||
]
|
||||
return matches[0] if len(matches) == 1 else None
|
||||
@@ -294,7 +361,7 @@ def _score_version(entry: TitleEntry, version: GameVersion) -> int:
|
||||
if entry.publisher_hint:
|
||||
hint = normalize_title(entry.publisher_hint)
|
||||
if any(
|
||||
fuzz.partial_ratio(hint, normalize_title(p)) >= 85
|
||||
fuzz.partial_ratio(hint, normalize_title(p)) >= PUBLISHER_MATCH_THRESHOLD
|
||||
for p in version.publishers
|
||||
):
|
||||
score += 2
|
||||
@@ -310,18 +377,24 @@ def _score_version(entry: TitleEntry, version: GameVersion) -> int:
|
||||
and fuzz.token_set_ratio(
|
||||
normalize_title(entry.edition_hint), normalize_title(version.name)
|
||||
)
|
||||
>= 80
|
||||
>= EDITION_NAME_THRESHOLD
|
||||
):
|
||||
score += 2
|
||||
return score
|
||||
|
||||
|
||||
def _resolve_version(client: BGGClient, entry: TitleEntry, row: MatchRow) -> None:
|
||||
"""Fill version_* fields on an auto/approved row. Never guess (spec)."""
|
||||
def resolve_version(client: BGGClient, entry: TitleEntry, row: MatchRow) -> None:
|
||||
"""Fill version_* fields on an auto/approved row. Never guess (spec).
|
||||
Public: review's manual-id path calls this too."""
|
||||
if not entry.has_version_cues:
|
||||
row.version_status = "version_unknown"
|
||||
return
|
||||
(thing,) = client.things([row.bgg_id], versions=True)
|
||||
things = client.things([row.bgg_id], versions=True)
|
||||
if not things:
|
||||
# a manually-typed id BGG doesn't know: nothing to offer
|
||||
row.version_status = "version_unknown"
|
||||
return
|
||||
thing = things[0]
|
||||
scored = sorted(
|
||||
((v, _score_version(entry, v)) for v in thing.versions),
|
||||
key=lambda pair: -pair[1],
|
||||
@@ -339,7 +412,7 @@ def _resolve_version(client: BGGClient, entry: TitleEntry, row: MatchRow) -> Non
|
||||
"languages": list(v.languages),
|
||||
"score": s,
|
||||
}
|
||||
for v, s in plausible[:8]
|
||||
for v, s in plausible # every plausible version: no silent cap
|
||||
]
|
||||
if len(plausible) == 1 or plausible[0][1] > plausible[1][1]:
|
||||
winner = plausible[0][0]
|
||||
@@ -350,8 +423,40 @@ def _resolve_version(client: BGGClient, entry: TitleEntry, row: MatchRow) -> Non
|
||||
row.version_status = "version_ambiguous"
|
||||
|
||||
|
||||
def resolve_entry(client: BGGClient, entry: TitleEntry) -> MatchRow:
|
||||
cands = _plausible_candidates(client, entry, entry.title_raw)
|
||||
_EDITION_SUFFIX = re.compile(r"\s*\([^)]*\)\s*$")
|
||||
_PUNCT = re.compile(r"[^\w\s]", re.UNICODE)
|
||||
|
||||
|
||||
def _depunct(title: str) -> str:
|
||||
"""BGG's search engine can choke on punctuation — the game literally
|
||||
named "Dungeon!" is missing from its own 824-result search. A plain
|
||||
query recovers it."""
|
||||
return " ".join(_PUNCT.sub(" ", title).split())
|
||||
|
||||
|
||||
def _merged_candidates(
|
||||
client: BGGClient, entry: TitleEntry, query: str, types: str | None = None
|
||||
) -> list[Candidate]:
|
||||
"""Search the raw title AND its punctuation-free form, merged by id
|
||||
(strongest evidence wins)."""
|
||||
cands = _plausible_candidates(client, entry, query, types=types)
|
||||
plain = _depunct(query)
|
||||
if plain.casefold() != query.casefold():
|
||||
by_id = {c.bgg_id: c for c in cands}
|
||||
for c in _plausible_candidates(client, entry, plain, types=types):
|
||||
prev = by_id.get(c.bgg_id)
|
||||
if prev is None or (c.exact, c.fuzzy) > (prev.exact, prev.fuzzy):
|
||||
by_id[c.bgg_id] = c
|
||||
cands = sorted(by_id.values(), key=lambda c: (not c.exact, -c.fuzzy))
|
||||
return cands
|
||||
|
||||
|
||||
def find_candidates(client: BGGClient, entry: TitleEntry) -> list[Candidate]:
|
||||
"""Every search this pipeline knows, in order of confidence. Public
|
||||
because review's reopen path must search exactly as resolve did — a
|
||||
partial re-implementation there silently misses whatever the later
|
||||
steps would have found."""
|
||||
cands = _merged_candidates(client, entry, entry.title_raw)
|
||||
if not cands:
|
||||
# Long transcribed box titles ("CIVILIZATION Game of the Heroic
|
||||
# Age - ...") defeat search: retry with progressively shorter heads
|
||||
@@ -360,50 +465,149 @@ def resolve_entry(client: BGGClient, entry: TitleEntry) -> MatchRow:
|
||||
cands = _plausible_candidates(client, entry, head, normalize_title(head))
|
||||
if cands:
|
||||
break
|
||||
if not cands:
|
||||
# Not a board game? RPGs live in the same geekdo database under
|
||||
# type=rpgitem (same API, same token). A hit becomes a LOCAL
|
||||
# library citizen: identified and enriched, never uploaded (diff
|
||||
# routes rpgitem rows to local_only).
|
||||
cands = _merged_candidates(client, entry, entry.title_raw, types="rpgitem")
|
||||
if not cands:
|
||||
for head in _truncation_heads(entry.title_raw):
|
||||
cands = _plausible_candidates(
|
||||
client, entry, head, normalize_title(head), types="rpgitem"
|
||||
)
|
||||
if cands:
|
||||
break
|
||||
return cands
|
||||
|
||||
|
||||
def resolve_entry(client: BGGClient, entry: TitleEntry) -> MatchRow:
|
||||
cands = find_candidates(client, entry)
|
||||
row = _classify(client, entry, cands)
|
||||
if row.match_status == "auto":
|
||||
_resolve_version(client, entry, row)
|
||||
resolve_version(client, entry, row)
|
||||
return row
|
||||
|
||||
|
||||
def _row_key(title_raw: str, source_photos: str) -> tuple[str, str]:
|
||||
return (title_raw, source_photos)
|
||||
@dataclass(frozen=True)
|
||||
class MergeEvent:
|
||||
loser_title: str
|
||||
survivor_title: str
|
||||
bgg_name: str
|
||||
bgg_id: str
|
||||
|
||||
|
||||
def dedupe_matches(
|
||||
rows: list[dict],
|
||||
titles: list[TitleEntry],
|
||||
splits: list[dict] | None = None,
|
||||
) -> list[MergeEvent]:
|
||||
"""Post-resolve dedupe: rows resolving to the same (bgg_id, version_id —
|
||||
or both version-unknown) are the same physical game seen twice (a typo
|
||||
read, a partial spine) UNLESS their extraction cues conflict, which
|
||||
means two editions. Losers are marked match_status="merged" pointing at
|
||||
the survivor via merged_into — no row is ever deleted, and the review
|
||||
UI can veto the merge."""
|
||||
entry_by_key = {(e.title_raw, ";".join(e.source_photos)): e for e in titles}
|
||||
entry_by_title: dict[str, TitleEntry] = {}
|
||||
for e in titles:
|
||||
entry_by_title.setdefault(e.title_raw, e)
|
||||
|
||||
def cues(row: dict) -> dict:
|
||||
entry = entry_by_key.get(
|
||||
(row["title_raw"], row["source_photos"])
|
||||
) or entry_by_title.get(row["title_raw"])
|
||||
if entry is None:
|
||||
return {}
|
||||
return {
|
||||
"publisher_hint": entry.publisher_hint,
|
||||
"edition_hint": entry.edition_hint,
|
||||
"language_hint": entry.language_hint,
|
||||
"year_hint": entry.year_hint,
|
||||
}
|
||||
|
||||
groups: dict[tuple[str, str], list[dict]] = {}
|
||||
for row in rows:
|
||||
if row["match_status"] not in RECOGNIZED_MATCH_STATUSES or not row["bgg_id"]:
|
||||
continue
|
||||
if row.get("dedupe_veto"):
|
||||
# a human already ruled "this is a genuinely separate copy" —
|
||||
# re-running resolve must never overturn that (spec: re-runs
|
||||
# lose no work, least of all review decisions)
|
||||
continue
|
||||
if is_split(
|
||||
normalize_title(row["title_raw"]),
|
||||
row["source_photos"].split(";"),
|
||||
splits or [],
|
||||
):
|
||||
continue # human-split copies: per-photo rows stay separate
|
||||
key = (
|
||||
row["bgg_id"],
|
||||
row["version_id"] if is_confident_version(row) else "",
|
||||
)
|
||||
groups.setdefault(key, []).append(row)
|
||||
|
||||
events: list[MergeEvent] = []
|
||||
for (bgg_id, _version), group in groups.items():
|
||||
if len(group) < 2:
|
||||
continue
|
||||
# survivor: the row whose transcription best matches the BGG name
|
||||
group = sorted(
|
||||
group,
|
||||
key=lambda r: (
|
||||
normalize_title(r["title_raw"]) != normalize_title(r["bgg_name"] or "")
|
||||
),
|
||||
)
|
||||
survivor = group[0]
|
||||
for loser in group[1:]:
|
||||
if cues_conflict(cues(survivor), cues(loser)):
|
||||
continue # conflicting edition cues: genuinely two copies
|
||||
loser["match_status"] = "merged"
|
||||
loser["merged_into"] = survivor["title_raw"]
|
||||
events.append(
|
||||
MergeEvent(
|
||||
loser_title=loser["title_raw"],
|
||||
survivor_title=survivor["title_raw"],
|
||||
bgg_name=survivor["bgg_name"],
|
||||
bgg_id=bgg_id,
|
||||
)
|
||||
)
|
||||
# A survivor can itself lose in a later run; rows pointing at it would
|
||||
# form a chain diff's one-level photo hop can't follow. Rewrite every
|
||||
# merged row to its terminal survivor.
|
||||
status_by_title = {r["title_raw"]: r for r in rows}
|
||||
for row in rows:
|
||||
if row["match_status"] != "merged":
|
||||
continue
|
||||
target, hops = row["merged_into"], 0
|
||||
while (
|
||||
hops < 10
|
||||
and (nxt := status_by_title.get(target)) is not None
|
||||
and nxt["match_status"] == "merged"
|
||||
and nxt["merged_into"]
|
||||
):
|
||||
target = nxt["merged_into"]
|
||||
hops += 1
|
||||
row["merged_into"] = target
|
||||
return events
|
||||
|
||||
|
||||
def read_matches(path: Path) -> list[dict[str, str]]:
|
||||
if not path.exists():
|
||||
return []
|
||||
with path.open(newline="") as f:
|
||||
return list(csv.DictReader(f))
|
||||
rows = list(csv.DictReader(f))
|
||||
for row in rows: # optional columns: tolerate rows without them
|
||||
row.setdefault("merged_into", "")
|
||||
row.setdefault("dedupe_veto", "")
|
||||
return rows
|
||||
|
||||
|
||||
def write_matches(path: Path, rows: list[dict[str, str]]) -> None:
|
||||
"""Atomic full rewrite — review updates rows in place decision by decision."""
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = path.with_name(path.name + ".tmp")
|
||||
with tmp.open("w", newline="") as f:
|
||||
writer = csv.DictWriter(f, fieldnames=MATCH_COLUMNS, extrasaction="ignore")
|
||||
writer.writeheader()
|
||||
writer.writerows(rows)
|
||||
os.replace(tmp, path)
|
||||
|
||||
|
||||
def read_existing_keys(path: Path) -> set[tuple[str, str]]:
|
||||
if not path.exists():
|
||||
return set()
|
||||
with path.open(newline="") as f:
|
||||
return {_row_key(r["title_raw"], r["source_photos"]) for r in csv.DictReader(f)}
|
||||
|
||||
|
||||
def append_rows(path: Path, rows: list[MatchRow]) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
is_new = not path.exists()
|
||||
with path.open("a", newline="") as f:
|
||||
writer = csv.DictWriter(f, fieldnames=MATCH_COLUMNS)
|
||||
if is_new:
|
||||
writer.writeheader()
|
||||
for row in rows:
|
||||
writer.writerow(row.to_csv())
|
||||
def write_matches(path: Path, rows: list[dict[str, str]]) -> int:
|
||||
"""Atomic full rewrite — review updates rows in place decision by
|
||||
decision. Returns the written file's mtime_ns so the caller can record
|
||||
its own write without a re-stat race."""
|
||||
return atomic_write_csv(path, MATCH_COLUMNS, rows)
|
||||
|
||||
|
||||
def run_resolve(
|
||||
@@ -412,30 +616,130 @@ def run_resolve(
|
||||
entries = load_titles(cfg.titles_path)
|
||||
if force and cfg.matches_path.exists():
|
||||
cfg.matches_path.unlink()
|
||||
existing = read_existing_keys(cfg.matches_path)
|
||||
client = client or BGGClient(cfg.cache_dir, cfg.rate_limit_seconds)
|
||||
existing_rows = read_matches(cfg.matches_path)
|
||||
client = client or client_for(cfg)
|
||||
|
||||
# Pair entries with existing rows BY TITLE: photo-overlap first, then
|
||||
# position. Pure position breaks when titles.json order churns (a
|
||||
# reshoot photo sorting earlier reorders same-title entries); overlap
|
||||
# keeps each edition glued to its own row, and position only settles
|
||||
# entries with no photo history.
|
||||
rows_by_title: dict[str, list[dict]] = {}
|
||||
for row in existing_rows:
|
||||
rows_by_title.setdefault(row["title_raw"], []).append(row)
|
||||
|
||||
claimed_rows: set[int] = set()
|
||||
|
||||
def pair_row(entry: TitleEntry) -> dict | None:
|
||||
candidates = [
|
||||
r
|
||||
for r in rows_by_title.get(entry.title_raw, [])
|
||||
if id(r) not in claimed_rows
|
||||
]
|
||||
if not candidates:
|
||||
return None
|
||||
photos = set(entry.source_photos)
|
||||
for r in candidates:
|
||||
if photos & set(r["source_photos"].split(";")):
|
||||
claimed_rows.add(id(r))
|
||||
return r
|
||||
return None
|
||||
|
||||
def pair_row_positional(entry: TitleEntry) -> dict | None:
|
||||
candidates = [
|
||||
r
|
||||
for r in rows_by_title.get(entry.title_raw, [])
|
||||
if id(r) not in claimed_rows
|
||||
]
|
||||
if candidates:
|
||||
claimed_rows.add(id(candidates[0]))
|
||||
return candidates[0]
|
||||
return None
|
||||
|
||||
new_rows: list[MatchRow] = []
|
||||
skipped = 0
|
||||
photos_updated = False
|
||||
blocked: list[str] = []
|
||||
blocked_titles: set[str] = set()
|
||||
# overlap pass first so a reordered titles.json can't mispair editions
|
||||
paired_by_id: dict[int, dict] = {}
|
||||
for entry in entries:
|
||||
key = _row_key(entry.title_raw, ";".join(entry.source_photos))
|
||||
if key in existing:
|
||||
row_dict = pair_row(entry)
|
||||
if row_dict is not None:
|
||||
paired_by_id[id(entry)] = row_dict
|
||||
for entry in entries:
|
||||
if id(entry) not in paired_by_id:
|
||||
row_dict = pair_row_positional(entry)
|
||||
if row_dict is not None:
|
||||
paired_by_id[id(entry)] = row_dict
|
||||
|
||||
for entry in entries:
|
||||
row_dict = paired_by_id.get(id(entry))
|
||||
if row_dict is not None:
|
||||
photos = ";".join(entry.source_photos)
|
||||
if row_dict["source_photos"] != photos and not row_dict.get("dedupe_veto"):
|
||||
# provenance follows the entry — except on split/vetoed rows,
|
||||
# whose per-copy photo sets are human-authored
|
||||
row_dict["source_photos"] = photos
|
||||
photos_updated = True
|
||||
skipped += 1
|
||||
continue
|
||||
if entry.title_raw in blocked_titles:
|
||||
# an earlier same-title entry is waiting on the token: resolving
|
||||
# this one now would claim the wrong pairing slot on the next
|
||||
# run — defer the whole group
|
||||
blocked.append(entry.title_raw)
|
||||
continue
|
||||
try:
|
||||
row = resolve_entry(client, entry)
|
||||
except (BGGAuthError, BGGQueueTimeout) as err:
|
||||
# No token / BGG still queueing: cached titles still resolve;
|
||||
# the rest wait. No row is written, so a future run picks them
|
||||
# up untouched.
|
||||
blocked.append(entry.title_raw)
|
||||
blocked_titles.add(entry.title_raw)
|
||||
reason = (
|
||||
"waiting on BGG API token"
|
||||
if isinstance(err, BGGAuthError)
|
||||
else "BGG still queueing — re-run in a minute"
|
||||
)
|
||||
typer.echo(f" {entry.title_raw!r} -> {reason}")
|
||||
continue
|
||||
new_rows.append(row)
|
||||
detail = f"{row.bgg_name} ({row.bgg_id})" if row.bgg_id else "-"
|
||||
version = f" [{row.version_status}]" if row.version_status else ""
|
||||
typer.echo(f" {entry.title_raw!r} -> {row.match_status}: {detail}{version}")
|
||||
|
||||
append_rows(cfg.matches_path, new_rows)
|
||||
all_rows = existing_rows + [row.to_csv() for row in new_rows]
|
||||
merges = dedupe_matches(all_rows, entries, load_title_splits(cfg.title_splits_path))
|
||||
if new_rows or photos_updated or merges:
|
||||
write_matches(cfg.matches_path, all_rows) # atomic full rewrite
|
||||
if merges:
|
||||
typer.echo("")
|
||||
for m in merges:
|
||||
typer.echo(
|
||||
f" merged {m.loser_title!r} into {m.survivor_title!r} — same "
|
||||
f"game ({m.bgg_name}, {m.bgg_id}); veto in review if wrong"
|
||||
)
|
||||
|
||||
counts: dict[str, int] = {}
|
||||
for row in new_rows:
|
||||
counts[row.match_status] = counts.get(row.match_status, 0) + 1
|
||||
counts = Counter(row.match_status for row in new_rows)
|
||||
summary = ", ".join(f"{n} {status}" for status, n in sorted(counts.items()))
|
||||
typer.echo(
|
||||
f"Resolved {len(new_rows)} title(s) ({summary or 'nothing new'}); "
|
||||
f"skipped {skipped} already in {cfg.matches_path}."
|
||||
)
|
||||
waiting = sum(1 for r in all_rows if r.get("match_status") == "unmatched")
|
||||
if waiting:
|
||||
typer.echo(
|
||||
f" {waiting} unmatched row(s) wait for YOUR call in review "
|
||||
"(re-search or manual id) — resolve never overrides a human "
|
||||
"decision, including 'this match is wrong'"
|
||||
)
|
||||
if blocked:
|
||||
typer.echo(
|
||||
f"\n{len(blocked)} title(s) are waiting on the BGG API "
|
||||
"(register at https://boardgamegeek.com/applications, then set "
|
||||
"BGG_API_TOKEN and re-run resolve — everything above is saved): "
|
||||
+ ", ".join(blocked)
|
||||
)
|
||||
return new_rows
|
||||
|
||||
+479
-51
@@ -1,9 +1,11 @@
|
||||
"""Stage 3 — human review of ambiguous/unmatched matches, then versions.
|
||||
"""Stage 3 — the review decision engine, worn by two faces.
|
||||
|
||||
A plain rich prompt loop (deliberately dependency-light, per spec). Every
|
||||
decision rewrites matches.csv atomically, so quitting mid-review — q,
|
||||
Ctrl-C, or end of input — loses nothing; the next run resumes with
|
||||
whatever is still ambiguous/unmatched. The version pass is optional and
|
||||
ReviewSession owns all decision logic and every matches.csv write: the
|
||||
decision API (decide_*/veto_merge/pending_rows/version_rows) serves both
|
||||
this module's rich TUI prompt loop and webreview's FastAPI endpoints, and
|
||||
reload_if_changed() lets either face follow external rewrites (a resolve
|
||||
run in another terminal). Every decision rewrites matches.csv atomically,
|
||||
so quitting mid-review loses nothing. The version pass is optional and
|
||||
skippable: version review must never block getting games uploaded.
|
||||
"""
|
||||
|
||||
@@ -11,20 +13,31 @@ from __future__ import annotations
|
||||
|
||||
import json
|
||||
from collections.abc import Callable
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
import typer
|
||||
from rich.console import Console
|
||||
from rich.panel import Panel
|
||||
from rich.table import Table
|
||||
|
||||
from bggpipe.bgg_client import BGGAuthError, BGGClient, BGGQueueTimeout
|
||||
from bggpipe.bgg_client import BGGAuthError, BGGClient, BGGQueueTimeout, client_for
|
||||
from bggpipe.config import Config
|
||||
from bggpipe.models import BGGResponseError
|
||||
from bggpipe.models import (
|
||||
PENDING_MATCH_STATUSES,
|
||||
UNDECIDED_MATCH_STATUSES,
|
||||
BGGResponseError,
|
||||
)
|
||||
from bggpipe.normalize import normalize_title
|
||||
from bggpipe.resolve import (
|
||||
Candidate,
|
||||
MatchRow,
|
||||
_resolve_version,
|
||||
TitleEntry,
|
||||
_score_version,
|
||||
find_candidates,
|
||||
load_titles,
|
||||
read_matches,
|
||||
resolve_version,
|
||||
write_matches,
|
||||
)
|
||||
|
||||
@@ -56,16 +69,40 @@ class ReviewSession:
|
||||
self.cfg = cfg
|
||||
self.console = console or Console()
|
||||
self.input_fn = input_fn or (lambda prompt: self.console.input(prompt))
|
||||
self.client = client or BGGClient(cfg.cache_dir, cfg.rate_limit_seconds)
|
||||
self.rows = read_matches(cfg.matches_path)
|
||||
self.client = client or client_for(cfg)
|
||||
self.decisions = 0
|
||||
try:
|
||||
self._titles = {e.title_raw: e for e in load_titles(cfg.titles_path)}
|
||||
except FileNotFoundError:
|
||||
self._titles = {}
|
||||
self.warnings: list[str] = []
|
||||
self._load()
|
||||
|
||||
# -- plumbing -------------------------------------------------------
|
||||
|
||||
def _data_mtimes(self) -> tuple[int | None, int | None]:
|
||||
def mtime(path: Path) -> int | None:
|
||||
try:
|
||||
return path.stat().st_mtime_ns
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
|
||||
return (mtime(self.cfg.matches_path), mtime(self.cfg.titles_path))
|
||||
|
||||
def _load(self) -> None:
|
||||
self.rows = read_matches(self.cfg.matches_path)
|
||||
try:
|
||||
self.titles = load_titles(self.cfg.titles_path)
|
||||
except FileNotFoundError:
|
||||
self.titles = []
|
||||
self._titles = {e.title_raw: e for e in self.titles}
|
||||
self._loaded_mtimes = self._data_mtimes()
|
||||
|
||||
def reload_if_changed(self) -> bool:
|
||||
"""Re-read matches.csv/titles.json when another process (extract,
|
||||
resolve, a git pull) rewrote them, so the web UI never serves — or
|
||||
saves decisions over — stale rows. Returns True when it reloaded."""
|
||||
if self._data_mtimes() == self._loaded_mtimes:
|
||||
return False
|
||||
self._load()
|
||||
return True
|
||||
|
||||
def _ask(self, prompt: str) -> str:
|
||||
try:
|
||||
answer = self.input_fn(prompt).strip()
|
||||
@@ -75,8 +112,70 @@ class ReviewSession:
|
||||
raise _Quit
|
||||
return answer
|
||||
|
||||
def _save(self) -> None:
|
||||
write_matches(self.cfg.matches_path, self.rows)
|
||||
def _warn(self, message: str) -> None:
|
||||
"""Degradations must be visible in BOTH faces: the console for the
|
||||
TUI, and self.warnings for the web UI (whose console is a StringIO)."""
|
||||
self.warnings.append(message)
|
||||
self.console.print(f"[yellow]{message}[/yellow]")
|
||||
|
||||
def _adopt(self, row: dict) -> bool:
|
||||
"""Swap `row` (possibly an orphaned reference from before a reload)
|
||||
back into self.rows by identity or key. Key collisions (duplicate
|
||||
two-edition rows) prefer a still-undecided slot. False = row is gone
|
||||
from the file entirely."""
|
||||
if any(r is row for r in self.rows):
|
||||
return True
|
||||
key = (row["title_raw"], row["source_photos"])
|
||||
candidates = [
|
||||
i
|
||||
for i, r in enumerate(self.rows)
|
||||
if (r["title_raw"], r["source_photos"]) == key
|
||||
]
|
||||
if not candidates:
|
||||
return False
|
||||
undecided = [
|
||||
i
|
||||
for i in candidates
|
||||
if self.rows[i]["match_status"] in UNDECIDED_MATCH_STATUSES
|
||||
]
|
||||
if not undecided:
|
||||
# a VERSION decision targets an approved row: prefer the slot
|
||||
# still awaiting its edition, not the sibling already versioned
|
||||
undecided = [
|
||||
i
|
||||
for i in candidates
|
||||
if self.rows[i]["version_status"] == "version_ambiguous"
|
||||
]
|
||||
self.rows[(undecided or candidates)[0]] = row
|
||||
return True
|
||||
|
||||
def _save(self, row: dict | None = None) -> None:
|
||||
"""Atomic write of the in-memory rows. A TUI loop iterates row
|
||||
references snapshotted before any reload, and reload_if_changed can
|
||||
swap self.rows at every save — so the decided row must always be
|
||||
re-adopted into the CURRENT list, or the decision would be counted
|
||||
but never written."""
|
||||
self.reload_if_changed()
|
||||
if row is not None and not self._adopt(row):
|
||||
self._warn(
|
||||
f"{row['title_raw']!r} disappeared from matches.csv while "
|
||||
"you decided — decision NOT saved"
|
||||
)
|
||||
return
|
||||
self._write_rows()
|
||||
|
||||
def _write_rows(self) -> None:
|
||||
"""Atomic write of self.rows exactly as they stand — no reload, so
|
||||
a caller that just derived self.rows from a fresh reload can't have
|
||||
its result silently replaced before the write."""
|
||||
try:
|
||||
own_mtime = write_matches(self.cfg.matches_path, self.rows)
|
||||
except OSError:
|
||||
self._load() # memory must never claim what disk doesn't hold
|
||||
raise
|
||||
# record the mtime write_matches itself observed: re-statting later
|
||||
# could adopt a foreign rewrite landing in the gap as "own write"
|
||||
self._loaded_mtimes = (own_mtime, self._data_mtimes()[1])
|
||||
self.decisions += 1
|
||||
|
||||
def _apply_choice(self, row: dict, candidate: dict) -> None:
|
||||
@@ -86,21 +185,29 @@ class ReviewSession:
|
||||
row["year"] = str(candidate.get("year") or "")
|
||||
row["type"] = candidate.get("type") or "boardgame"
|
||||
self._fill_version(row)
|
||||
self._save()
|
||||
self._save(row)
|
||||
|
||||
def _fill_version(self, row: dict) -> None:
|
||||
"""Try version resolution for a just-approved row. Degrades gracefully:
|
||||
no cues, no token, or API trouble all leave version_unknown."""
|
||||
entry = self._titles.get(row["title_raw"])
|
||||
# photo-aware lookup: two same-title entries are two EDITIONS with
|
||||
# different cues — the title-only dict would hand every row the last
|
||||
# edition's cues and score the wrong version to version_auto
|
||||
entry = self.cues_for(row["title_raw"], row["source_photos"])
|
||||
if entry is None or not row["bgg_id"]:
|
||||
row["version_status"] = row["version_status"] or "version_unknown"
|
||||
return
|
||||
shim = MatchRow(title_raw=row["title_raw"], bgg_id=int(row["bgg_id"]))
|
||||
try:
|
||||
_resolve_version(self.client, entry, shim)
|
||||
resolve_version(self.client, entry, shim)
|
||||
except _BGG_ERRORS as err:
|
||||
self.console.print(f"[yellow]version lookup unavailable: {err}[/yellow]")
|
||||
row["version_status"] = "version_unknown"
|
||||
self._warn(
|
||||
f"version lookup unavailable ({err}) — re-approve this row "
|
||||
"to retry the edition lookup"
|
||||
)
|
||||
# distinct from version_unknown ("no cues"): a transient failure
|
||||
# stays visibly retryable instead of terminal
|
||||
row["version_status"] = "version_error"
|
||||
return
|
||||
row["version_status"] = shim.version_status
|
||||
row["version_id"] = str(shim.version_id or "")
|
||||
@@ -109,6 +216,327 @@ class ReviewSession:
|
||||
shim.version_candidates, ensure_ascii=False
|
||||
)
|
||||
|
||||
# -- decision API (shared by the TUI and the web UI) ----------------
|
||||
|
||||
def pending_rows(self) -> list[dict]:
|
||||
return [r for r in self.rows if r["match_status"] in ("ambiguous", "unmatched")]
|
||||
|
||||
def version_rows(self) -> list[dict]:
|
||||
return [
|
||||
r
|
||||
for r in self.rows
|
||||
if r["version_status"] == "version_ambiguous"
|
||||
and r["match_status"] != "merged"
|
||||
]
|
||||
|
||||
def merged_rows(self) -> list[dict]:
|
||||
return [r for r in self.rows if r["match_status"] == "merged"]
|
||||
|
||||
def split_row(self, row: dict) -> list[dict]:
|
||||
"""The human says one multi-photo row is actually N physical copies
|
||||
(one per photo). Replace it with per-photo rows, each veto-flagged
|
||||
so no future dedupe re-merges them, each with its own edition slot
|
||||
(different copies are usually different editions)."""
|
||||
photos = [p for p in row["source_photos"].split(";") if p]
|
||||
if len(photos) < 2:
|
||||
raise ValueError("only a multi-photo row can be split into copies")
|
||||
# same concurrent-rewrite discipline as every decision: re-adopt the
|
||||
# (possibly orphaned) row into the current list before mutating
|
||||
self.reload_if_changed()
|
||||
if not self._adopt(row):
|
||||
self._warn(
|
||||
f"{row['title_raw']!r} disappeared from matches.csv while "
|
||||
"you split — nothing changed"
|
||||
)
|
||||
return []
|
||||
has_candidates = (row.get("version_candidates_json") or "[]") != "[]"
|
||||
ix = next(i for i, r in enumerate(self.rows) if r is row)
|
||||
copies = []
|
||||
for photo in photos:
|
||||
copy = dict(row)
|
||||
copy["source_photos"] = photo
|
||||
copy["dedupe_veto"] = "1"
|
||||
copy["merged_into"] = ""
|
||||
# each physical copy picks its OWN edition in the version pass
|
||||
copy["version_id"] = ""
|
||||
copy["version_name"] = ""
|
||||
copy["version_status"] = (
|
||||
"version_ambiguous" if has_candidates else "version_unknown"
|
||||
)
|
||||
copies.append(copy)
|
||||
self.rows[ix : ix + 1] = copies
|
||||
self._save(copies[0])
|
||||
return copies
|
||||
|
||||
def drop_rows(
|
||||
self,
|
||||
title_raw: str,
|
||||
photos: list[str] | None = None,
|
||||
new_title: str | None = None,
|
||||
even_vetoed: bool = False,
|
||||
) -> int:
|
||||
"""An edit invalidated these rows — the BGG match was made against
|
||||
the uncorrected read. Remove them so resolve re-queries with the
|
||||
fix; `photos` narrows the cull to one copy of a split title.
|
||||
Rows carrying a human veto (dedupe_veto) are never dropped — the
|
||||
match itself was human-vetted, and dropping would erase the veto;
|
||||
a rename updates their title in place so they follow the entry.
|
||||
`even_vetoed` is for title REMOVAL, where the human is explicitly
|
||||
discarding the line, vetted or not."""
|
||||
self.reload_if_changed()
|
||||
norm = normalize_title(title_raw)
|
||||
keep: list[dict] = []
|
||||
dropped = renamed = 0
|
||||
for row in self.rows:
|
||||
targeted = normalize_title(row["title_raw"]) == norm and (
|
||||
photos is None
|
||||
or bool({p for p in row["source_photos"].split(";") if p} & set(photos))
|
||||
)
|
||||
if not targeted:
|
||||
keep.append(row)
|
||||
elif row.get("dedupe_veto") and not even_vetoed:
|
||||
if new_title and row["title_raw"] != new_title:
|
||||
row["title_raw"] = new_title
|
||||
renamed += 1
|
||||
keep.append(row)
|
||||
else:
|
||||
dropped += 1
|
||||
if dropped or renamed:
|
||||
self.rows = keep
|
||||
self._write_rows()
|
||||
return dropped
|
||||
|
||||
def veto_merge(self, row: dict) -> None:
|
||||
"""The human says these are NOT the same physical game: restore the
|
||||
row as a distinct, human-confirmed match."""
|
||||
row["match_status"] = "approved"
|
||||
row["merged_into"] = ""
|
||||
row["dedupe_veto"] = "1" # persists: resolve re-runs must not re-merge
|
||||
self._save(row)
|
||||
|
||||
def find_row(
|
||||
self, title_raw: str, source_photos: str, row_ix: int | None = None
|
||||
) -> dict | None:
|
||||
"""Locate a row by ordinal (duplicate two-edition rows) or by key,
|
||||
preferring a still-undecided slot on key collisions. None = gone."""
|
||||
if row_ix is not None and 0 <= row_ix < len(self.rows):
|
||||
row = self.rows[row_ix]
|
||||
if row["title_raw"] == title_raw and row["source_photos"] == source_photos:
|
||||
return row
|
||||
matches = [
|
||||
row
|
||||
for row in self.rows
|
||||
if row["title_raw"] == title_raw and row["source_photos"] == source_photos
|
||||
]
|
||||
undecided = [
|
||||
row for row in matches if row["match_status"] in UNDECIDED_MATCH_STATUSES
|
||||
]
|
||||
if undecided:
|
||||
return undecided[0]
|
||||
# both siblings decided (two-edition duplicates): prefer the one
|
||||
# whose VERSION is still open, as _adopt does — otherwise a stale
|
||||
# row_ix would land a version pick on the already-decided sibling
|
||||
version_open = [
|
||||
row for row in matches if row["version_status"] == "version_ambiguous"
|
||||
]
|
||||
if version_open:
|
||||
return version_open[0]
|
||||
return matches[0] if matches else None
|
||||
|
||||
def cues_for(
|
||||
self, title_raw: str, source_photos: str | None = None
|
||||
) -> TitleEntry | None:
|
||||
"""The extraction entry behind a row. Same-title entries (two
|
||||
editions of one game) are told apart by their photo set when given."""
|
||||
if source_photos is not None:
|
||||
for entry in self.titles:
|
||||
if (
|
||||
entry.title_raw == title_raw
|
||||
and ";".join(entry.source_photos) == source_photos
|
||||
):
|
||||
return entry
|
||||
return self._titles.get(title_raw)
|
||||
|
||||
def decide_pick(self, row: dict, candidate: dict) -> None:
|
||||
self._apply_choice(row, candidate)
|
||||
|
||||
def decide_manual(self, row: dict, bgg_id: int) -> None:
|
||||
self._manual_id(row, bgg_id)
|
||||
|
||||
def decide_reject(self, row: dict) -> None:
|
||||
row["match_status"] = "rejected"
|
||||
self._save(row)
|
||||
|
||||
def decide_local(self, row: dict) -> None:
|
||||
"""A real game BGG simply doesn't have: a local library citizen —
|
||||
listed and enriched from its own photo reads, never uploaded."""
|
||||
row["match_status"] = "local"
|
||||
row["bgg_id"] = ""
|
||||
row["bgg_name"] = ""
|
||||
row["version_status"] = ""
|
||||
row["version_id"] = ""
|
||||
row["version_name"] = ""
|
||||
self._save(row)
|
||||
|
||||
def research(self, row: dict, query: str, types: str | None = None) -> int:
|
||||
"""Re-search on the human's terms and put the results on the row's
|
||||
ballot. `types` targets a specific database — "rpgitem" for
|
||||
RPGGeek, whose entries the automatic cascade never reaches when BGG
|
||||
has a same-named BOARD game (every D&D box hits this)."""
|
||||
query = query.strip()
|
||||
if not query:
|
||||
raise ValueError("a search needs some text")
|
||||
results = (
|
||||
self.client.search(query, types) if types else self.client.search(query)
|
||||
)
|
||||
merged: dict[int, Candidate] = {}
|
||||
for r in results:
|
||||
cand = Candidate(
|
||||
bgg_id=r.bgg_id,
|
||||
name=r.name,
|
||||
year=r.year,
|
||||
type=r.type,
|
||||
exact=normalize_title(r.name) == normalize_title(query),
|
||||
fuzzy=0.0,
|
||||
)
|
||||
prior = merged.get(r.bgg_id)
|
||||
# one ballot line per game; an alternate-name row that exact-
|
||||
# matches the query outranks the primary-name row for it
|
||||
if prior is None or (cand.exact and not prior.exact):
|
||||
merged[r.bgg_id] = cand
|
||||
cands = list(merged.values())[:12]
|
||||
if cands:
|
||||
try:
|
||||
stats = {
|
||||
t.bgg_id: t
|
||||
for t in self.client.things([c.bgg_id for c in cands], stats=True)
|
||||
}
|
||||
except _BGG_ERRORS as err:
|
||||
# owned counts and ranks only decorate the ballot — losing
|
||||
# them must not lose the search the human just asked for
|
||||
self._warn(f"couldn't fetch stats for these results ({err})")
|
||||
stats = {}
|
||||
for c in cands:
|
||||
if c.bgg_id in stats:
|
||||
c.owned = stats[c.bgg_id].owned
|
||||
c.rank = stats[c.bgg_id].rank
|
||||
c.publishers = list(stats[c.bgg_id].publishers)
|
||||
row["match_status"] = "ambiguous" if cands else "unmatched"
|
||||
row["candidates_json"] = json.dumps(
|
||||
[c.as_json() for c in cands], ensure_ascii=False
|
||||
)
|
||||
# a fresh ballot supersedes any earlier verdict on this row —
|
||||
# including the VERSION verdicts, which belong to the old game: a
|
||||
# surviving version_id would ride into the next pick and put the
|
||||
# wrong game's edition on the collection entry
|
||||
row["bgg_id"] = ""
|
||||
row["bgg_name"] = ""
|
||||
row["version_status"] = ""
|
||||
row["version_id"] = ""
|
||||
row["version_name"] = ""
|
||||
row["version_candidates_json"] = "[]"
|
||||
self._save(row)
|
||||
return len(cands)
|
||||
|
||||
def reopen_match(self, row: dict) -> None:
|
||||
"""The human says a matched row is the WRONG game: clear the match
|
||||
and immediately re-search so the card comes back as a ballot — the
|
||||
machine re-OFFERS, only the human re-decides (run_resolve never
|
||||
touches unmatched rows, so without this the row would just sit).
|
||||
If BGG is unreachable the row still reopens, bare: re-search and
|
||||
manual id remain available on the card."""
|
||||
row["match_status"] = "unmatched"
|
||||
row["bgg_id"] = ""
|
||||
row["bgg_name"] = ""
|
||||
row["year"] = ""
|
||||
row["type"] = ""
|
||||
row["candidates_json"] = "[]"
|
||||
row["version_status"] = ""
|
||||
row["version_id"] = ""
|
||||
row["version_name"] = ""
|
||||
row["version_candidates_json"] = "[]"
|
||||
entry = self.cues_for(row["title_raw"], row["source_photos"]) or TitleEntry(
|
||||
title_raw=row["title_raw"],
|
||||
title_normalized=normalize_title(row["title_raw"]),
|
||||
)
|
||||
try:
|
||||
# the FULL cascade resolve uses — board games, truncation heads,
|
||||
# then RPGGeek — so a row parked as "local" gets the same look
|
||||
cands = find_candidates(self.client, entry)[:8]
|
||||
if cands:
|
||||
stats = {
|
||||
t.bgg_id: t
|
||||
for t in self.client.things([c.bgg_id for c in cands], stats=True)
|
||||
}
|
||||
for c in cands:
|
||||
if c.bgg_id in stats:
|
||||
c.owned = stats[c.bgg_id].owned
|
||||
c.rank = stats[c.bgg_id].rank
|
||||
c.publishers = list(stats[c.bgg_id].publishers)
|
||||
row["match_status"] = "ambiguous"
|
||||
row["candidates_json"] = json.dumps(
|
||||
[c.as_json() for c in cands], ensure_ascii=False
|
||||
)
|
||||
except _BGG_ERRORS as err:
|
||||
self._warn(
|
||||
f"re-search unavailable ({err}) — the row is reopened; use "
|
||||
"(f) re-search or (m) manual id on its card"
|
||||
)
|
||||
self._save(row)
|
||||
|
||||
def open_version_ballot(self, row: dict) -> int:
|
||||
"""The human knows which printing a box is even when the photo
|
||||
showed no cues: put EVERY published version on the row's ballot
|
||||
(cue-scored when cues exist) and mark it version_ambiguous so the
|
||||
normal edition pass presents it. Returns the ballot size."""
|
||||
if not row["bgg_id"]:
|
||||
raise ValueError("no BGG match on this row yet")
|
||||
entry = self.cues_for(row["title_raw"], row["source_photos"])
|
||||
things = self.client.things([int(row["bgg_id"])], versions=True)
|
||||
if not things or not things[0].versions:
|
||||
raise ValueError("BGG lists no versions for this game")
|
||||
scored = sorted(
|
||||
((v, _score_version(entry, v) if entry else 0) for v in things[0].versions),
|
||||
key=lambda pair: (-pair[1], str(pair[0].year or "")),
|
||||
)
|
||||
row["version_candidates_json"] = json.dumps(
|
||||
[
|
||||
{
|
||||
"version_id": v.version_id,
|
||||
"name": v.name,
|
||||
"year": v.year,
|
||||
"publishers": list(v.publishers),
|
||||
"languages": list(v.languages),
|
||||
"score": s,
|
||||
}
|
||||
for v, s in scored
|
||||
],
|
||||
ensure_ascii=False,
|
||||
)
|
||||
row["version_status"] = "version_ambiguous"
|
||||
row["version_id"] = ""
|
||||
row["version_name"] = ""
|
||||
self._save(row)
|
||||
return len(scored)
|
||||
|
||||
def decide_version(self, row: dict, version_id: int | None) -> None:
|
||||
"""Pick a version from the row's stored candidates, or None -> unknown."""
|
||||
if version_id is None:
|
||||
row["version_status"] = "version_unknown"
|
||||
row["version_id"] = ""
|
||||
row["version_name"] = ""
|
||||
else:
|
||||
candidates = json.loads(row["version_candidates_json"] or "[]")
|
||||
chosen = next(
|
||||
(v for v in candidates if v.get("version_id") == version_id), None
|
||||
)
|
||||
if chosen is None:
|
||||
raise ValueError(f"version {version_id} is not a stored candidate")
|
||||
row["version_status"] = "version_approved"
|
||||
row["version_id"] = str(version_id)
|
||||
row["version_name"] = chosen.get("name") or ""
|
||||
self._save(row)
|
||||
|
||||
# -- displays -------------------------------------------------------
|
||||
|
||||
def _show_item(self, row: dict, candidates: list[dict]) -> None:
|
||||
@@ -142,24 +570,26 @@ class ReviewSession:
|
||||
while True:
|
||||
self._show_item(row, candidates)
|
||||
prompt = (
|
||||
"[1-N] pick (s)kip (r)eject (m <id>) manual BGG id "
|
||||
"(f <text>) re-search (q)uit > "
|
||||
"[1-N] pick (s)kip (r)eject (l)ocal — not on BGG "
|
||||
"(m <id>) manual BGG id (f <text>) re-search (q)uit > "
|
||||
if row["match_status"] == "unmatched"
|
||||
else "[1-N] pick (s)kip (r)eject (q)uit > "
|
||||
else "[1-N] pick (s)kip (r)eject (l)ocal — not on BGG (q)uit > "
|
||||
)
|
||||
answer = self._ask(prompt)
|
||||
lowered = answer.lower()
|
||||
if lowered == "s":
|
||||
return
|
||||
if lowered == "r":
|
||||
row["match_status"] = "rejected"
|
||||
self._save()
|
||||
self.decide_reject(row)
|
||||
return
|
||||
if lowered == "l":
|
||||
self.decide_local(row)
|
||||
return
|
||||
if answer.isdigit() and 1 <= int(answer) <= len(candidates):
|
||||
self._apply_choice(row, candidates[int(answer) - 1])
|
||||
self.decide_pick(row, candidates[int(answer) - 1])
|
||||
return
|
||||
if lowered.startswith("m ") and answer[2:].strip().isdigit():
|
||||
self._manual_id(row, int(answer[2:].strip()))
|
||||
self.decide_manual(row, int(answer[2:].strip()))
|
||||
return
|
||||
if lowered.startswith("f ") and answer[2:].strip():
|
||||
candidates = self._research(answer[2:].strip()) or candidates
|
||||
@@ -169,18 +599,28 @@ class ReviewSession:
|
||||
def _manual_id(self, row: dict, bgg_id: int) -> None:
|
||||
candidate: dict = {"bgg_id": bgg_id}
|
||||
try:
|
||||
(thing,) = self.client.things([bgg_id])
|
||||
candidate.update(name=thing.name, year=thing.year, type=thing.type)
|
||||
things = self.client.things([bgg_id])
|
||||
except _BGG_ERRORS as err:
|
||||
self.console.print(
|
||||
f"[yellow]couldn't look up id {bgg_id} ({err}) — "
|
||||
"recording the id with no name[/yellow]"
|
||||
self._warn(
|
||||
f"couldn't look up id {bgg_id} ({err}) — recording the id with no name"
|
||||
)
|
||||
else:
|
||||
if things:
|
||||
thing = things[0]
|
||||
candidate.update(name=thing.name, year=thing.year, type=thing.type)
|
||||
else:
|
||||
self._warn(
|
||||
f"BGG knows no game with id {bgg_id} — recording it "
|
||||
"with no name (typo?)"
|
||||
)
|
||||
self._apply_choice(row, candidate)
|
||||
|
||||
def _research(self, query: str) -> list[dict]:
|
||||
try:
|
||||
results = self.client.search(query)
|
||||
if not results:
|
||||
# board games first, then RPGGeek — same database, same token
|
||||
results = self.client.search(query, "rpgitem")
|
||||
except _BGG_ERRORS as err:
|
||||
self.console.print(f"[yellow]search unavailable: {err}[/yellow]")
|
||||
return []
|
||||
@@ -214,32 +654,20 @@ class ReviewSession:
|
||||
if lowered == "s":
|
||||
return
|
||||
if lowered == "u":
|
||||
row["version_status"] = "version_unknown"
|
||||
row["version_id"] = ""
|
||||
row["version_name"] = ""
|
||||
self._save()
|
||||
self.decide_version(row, None)
|
||||
return
|
||||
if answer.isdigit() and 1 <= int(answer) <= len(candidates):
|
||||
chosen = candidates[int(answer) - 1]
|
||||
row["version_status"] = "version_approved"
|
||||
row["version_id"] = str(chosen.get("version_id") or "")
|
||||
row["version_name"] = chosen.get("name") or ""
|
||||
self._save()
|
||||
self.decide_version(row, candidates[int(answer) - 1].get("version_id"))
|
||||
return
|
||||
self.console.print("[yellow]didn't understand that — try again[/yellow]")
|
||||
|
||||
# -- entry point ----------------------------------------------------
|
||||
|
||||
def run(self) -> None:
|
||||
pending = [
|
||||
r for r in self.rows if r["match_status"] in ("ambiguous", "unmatched")
|
||||
]
|
||||
try:
|
||||
for row in pending:
|
||||
for row in self.pending_rows():
|
||||
self._review_match_row(row)
|
||||
versions = [
|
||||
r for r in self.rows if r["version_status"] == "version_ambiguous"
|
||||
]
|
||||
versions = self.version_rows()
|
||||
if versions:
|
||||
answer = self._ask(
|
||||
f"{len(versions)} game(s) have ambiguous editions. "
|
||||
@@ -252,7 +680,7 @@ class ReviewSession:
|
||||
self.console.print("[dim]stopping — progress is saved[/dim]")
|
||||
|
||||
remaining = sum(
|
||||
1 for r in self.rows if r["match_status"] in ("ambiguous", "unmatched")
|
||||
1 for r in self.rows if r["match_status"] in PENDING_MATCH_STATUSES
|
||||
)
|
||||
self.console.print(
|
||||
f"Recorded {self.decisions} decision(s); "
|
||||
@@ -272,6 +700,6 @@ def run_review(
|
||||
session.console.print(
|
||||
f"{cfg.matches_path} is empty — run `bggpipe resolve` first."
|
||||
)
|
||||
return session
|
||||
raise typer.Exit(code=1)
|
||||
session.run()
|
||||
return session
|
||||
|
||||
@@ -0,0 +1,623 @@
|
||||
/* bggpipe design system — one stylesheet for the whole app.
|
||||
*
|
||||
* The visual world is Juniper's mascot drawing: a sky-blue day, flat cel
|
||||
* color inside confident dark outlines, a cream game board with a rainbow
|
||||
* path, gold pipe fittings, purple hair, a spring-green shirt, an orange
|
||||
* bow tie, navy jeans. Tokens map those to roles — purple is brand and
|
||||
* action, green means go, bow-tie orange means danger, gold is trim, navy
|
||||
* is ink and dark chrome. The one loud element is the rainbow path stripe
|
||||
* under the header; everything else stays flat and outlined.
|
||||
*
|
||||
* Contrast rule: --accent/--go/--stop have *-ink variants for text on
|
||||
* light surfaces; the base variants fill objects.
|
||||
*/
|
||||
|
||||
:root {
|
||||
--sky: #cbe7f7; /* the drawing's daytime background */
|
||||
--sky-deep: #b3d9ef;
|
||||
--board: #fbf5e6; /* cream game-board card */
|
||||
--board-edge: #e9dfc6;
|
||||
--ink: #1f2433; /* outline navy-black */
|
||||
--ink-soft: #52607a;
|
||||
--navy: #2c3a5c; /* jeans: dark chrome, header, log */
|
||||
--navy-deep: #232e49;
|
||||
--accent: #8330c2; /* hair purple: brand + actions */
|
||||
--accent-ink: #6b27a0;
|
||||
--go: #2f8a45; /* shirt green, darkened for white text */
|
||||
--go-ink: #256e37;
|
||||
--stop: #d94a26; /* bow-tie orange-red */
|
||||
--stop-ink: #b03a1c;
|
||||
--gold: #f2c04b; /* pipe fittings: rings, trim, hovers */
|
||||
--gold-ink: #8a6414;
|
||||
--ticket: #fdf3d2; /* reshoot work-orders */
|
||||
--go-tint: #e2f2e4; /* pale state washes of the role colors */
|
||||
--stop-tint: #fbe3da;
|
||||
--gold-tint: #fdeebb;
|
||||
--accent-tint: #ece5f7;
|
||||
--navy-tint: #e3ecf3;
|
||||
--focus: var(--accent);
|
||||
--path: linear-gradient(90deg,
|
||||
#f767b8, #f79a3e, #f2c04b, #6fce6f, #5aa7f0, #9a5be0);
|
||||
--path-v: linear-gradient(180deg,
|
||||
#f767b8, #f79a3e, #f2c04b, #6fce6f, #5aa7f0, #9a5be0);
|
||||
--font-display: ui-rounded, "Hiragino Maru Gothic ProN", "Arial Rounded MT Bold", var(--font-body);
|
||||
--font-body: system-ui, -apple-system, "Segoe UI", sans-serif;
|
||||
--font-mono: ui-monospace, "SF Mono", Menlo, monospace;
|
||||
--radius: 8px;
|
||||
--radius-lg: 12px;
|
||||
--line: 2px solid var(--ink);
|
||||
--shadow-card: 3px 3px 0 rgba(31, 36, 51, .18);
|
||||
--shadow-raised: 5px 5px 0 rgba(31, 36, 51, .22);
|
||||
}
|
||||
|
||||
/* -- base ------------------------------------------------------------- */
|
||||
* { box-sizing: border-box; }
|
||||
html { background: var(--sky); }
|
||||
body {
|
||||
margin: 0;
|
||||
font-family: var(--font-body);
|
||||
color: var(--ink);
|
||||
background:
|
||||
radial-gradient(ellipse at 50% -20%, rgba(255,255,255,.5), transparent 55%),
|
||||
var(--sky);
|
||||
min-height: 100vh;
|
||||
}
|
||||
main { max-width: 62rem; margin: 0 auto; padding: 1.4rem 1.2rem 6rem; }
|
||||
h2 {
|
||||
font-family: var(--font-display);
|
||||
font-weight: 700; font-size: 1.1rem; letter-spacing: .02em;
|
||||
margin: 2rem 0 .8rem;
|
||||
}
|
||||
h2 .count { color: var(--ink-soft); font-size: .85rem; font-weight: 400; }
|
||||
:focus-visible { outline: 3px solid var(--focus); outline-offset: 2px; }
|
||||
|
||||
/* -- layout: sidebar rail + content column ----------------------------- */
|
||||
body { display: grid; grid-template-columns: 15.5rem minmax(0, 1fr); }
|
||||
.sidebar {
|
||||
position: sticky; top: 0; height: 100vh;
|
||||
background: var(--navy); color: #fff;
|
||||
display: flex; flex-direction: column;
|
||||
border-right: 4px solid transparent;
|
||||
/* the rainbow game path, running the rail top to bottom */
|
||||
border-image: var(--path-v) 1;
|
||||
}
|
||||
.brand {
|
||||
display: flex; align-items: center; gap: .6rem;
|
||||
text-decoration: none; color: inherit;
|
||||
padding: .9rem 1rem .7rem;
|
||||
}
|
||||
.brand img {
|
||||
width: 38px; height: 38px; border-radius: 50%;
|
||||
border: 2px solid var(--gold); object-fit: cover; display: block;
|
||||
background: var(--sky);
|
||||
}
|
||||
.wordmark {
|
||||
font-family: var(--font-display);
|
||||
font-size: 1.3rem; font-weight: 700; letter-spacing: .01em;
|
||||
}
|
||||
.wordmark small { display: block; opacity: .8; font-family: var(--font-body); font-weight: 400; font-size: .72rem; }
|
||||
nav[aria-label="Primary"] { display: flex; flex-direction: column; padding: .4rem 0; }
|
||||
nav[aria-label="Primary"] a {
|
||||
color: #fff; text-decoration: none; opacity: .85;
|
||||
padding: .45rem 1rem;
|
||||
display: flex; align-items: center; gap: .6rem;
|
||||
border-left: 4px solid transparent;
|
||||
font-size: .95rem;
|
||||
}
|
||||
nav[aria-label="Primary"] a:hover { opacity: 1; background: rgba(255,255,255,.07); }
|
||||
nav[aria-label="Primary"] a[aria-current="page"] {
|
||||
opacity: 1; border-left-color: var(--gold); font-weight: 600;
|
||||
background: rgba(255,255,255,.1);
|
||||
}
|
||||
.navbadge {
|
||||
margin-left: auto;
|
||||
font-size: .72rem; font-weight: 700;
|
||||
background: var(--gold); color: var(--navy-deep);
|
||||
border-radius: 999px; padding: .05rem .5rem;
|
||||
min-width: 1.5em; text-align: center;
|
||||
}
|
||||
.navbadge:empty { display: none; }
|
||||
.piperbox { margin-top: auto; padding: 1rem 1rem 1.1rem; text-align: center; }
|
||||
.piperbox img {
|
||||
width: 100%; max-width: 11.5rem;
|
||||
border-radius: var(--radius-lg);
|
||||
border: 2px solid var(--gold);
|
||||
display: block; margin: 0 auto;
|
||||
}
|
||||
.piperbox figcaption { font-size: .72rem; opacity: .8; margin-top: .45rem; }
|
||||
.navtoggle { display: none; }
|
||||
.piperbox .legal { font-size: .62rem; opacity: .6; line-height: 1.45; text-align: left; }
|
||||
.skip {
|
||||
position: absolute; left: -999px; top: 0; z-index: 10;
|
||||
background: var(--board); color: var(--ink);
|
||||
padding: .5rem 1rem; border-radius: 0 0 var(--radius) 0;
|
||||
}
|
||||
.skip:focus { left: 0; }
|
||||
|
||||
/* per-page strip under the page title: section links, shortcuts, counts */
|
||||
.pagebar {
|
||||
display: flex; gap: 1rem; align-items: center; flex-wrap: wrap;
|
||||
font-size: .85rem; color: var(--ink-soft); margin: -.4rem 0 1rem;
|
||||
}
|
||||
.pagebar a { color: inherit; text-decoration: underline dotted; text-underline-offset: 3px; }
|
||||
.pagebar a:hover { color: var(--accent-ink); }
|
||||
.pagebar b { color: var(--ink); }
|
||||
.keyhelp { margin-left: auto; font-size: .75rem; }
|
||||
h1 {
|
||||
font-family: var(--font-display); font-weight: 700;
|
||||
font-size: 1.45rem; margin: 1.2rem 0 .9rem;
|
||||
}
|
||||
kbd {
|
||||
font-family: var(--font-mono);
|
||||
font-size: .72rem;
|
||||
background: var(--board);
|
||||
color: var(--ink);
|
||||
border: 1px solid var(--board-edge);
|
||||
border-bottom-width: 2px;
|
||||
border-radius: 4px;
|
||||
padding: 0 .35em;
|
||||
display: inline-block; min-width: 1.4em; text-align: center;
|
||||
}
|
||||
|
||||
|
||||
/* -- banners ----------------------------------------------------------- */
|
||||
#banner { max-width: 62rem; margin: 0 auto; padding: 0 1.2rem; }
|
||||
.banner {
|
||||
border-radius: var(--radius); padding: .6rem .9rem; margin-top: .9rem;
|
||||
font-size: .85rem; line-height: 1.4;
|
||||
border: var(--line);
|
||||
background: var(--board);
|
||||
}
|
||||
.banner.error { border-color: var(--stop); color: var(--stop-ink); background: var(--stop-tint); }
|
||||
.banner.warn { border-color: var(--gold-ink); color: var(--gold-ink); background: var(--ticket); }
|
||||
|
||||
/* -- buttons ----------------------------------------------------------- */
|
||||
button, .linkbtn {
|
||||
font: inherit; font-size: .85rem;
|
||||
border: var(--line); background: #fff; color: var(--ink);
|
||||
border-radius: var(--radius); padding: .3rem .8rem; cursor: pointer;
|
||||
text-decoration: none; display: inline-block;
|
||||
box-shadow: 2px 2px 0 rgba(31, 36, 51, .18);
|
||||
}
|
||||
button:hover, .linkbtn:hover { background: var(--board); }
|
||||
button:active, .linkbtn:active { transform: translate(1px, 1px); box-shadow: 1px 1px 0 rgba(31,36,51,.18); }
|
||||
button:disabled { opacity: .45; cursor: default; box-shadow: none; }
|
||||
button.primary { background: var(--go); border-color: var(--ink); color: #fff; }
|
||||
button.primary:hover { background: var(--go-ink); }
|
||||
button.danger { color: var(--stop-ink); border-color: var(--stop); background: #fff; }
|
||||
|
||||
/* -- decision cards (review) ------------------------------------------ */
|
||||
.card {
|
||||
background: var(--board);
|
||||
border: var(--line);
|
||||
border-radius: var(--radius-lg);
|
||||
box-shadow: var(--shadow-card);
|
||||
padding: 1rem 1.1rem;
|
||||
margin-bottom: 1rem;
|
||||
display: flex; gap: 1.1rem;
|
||||
scroll-margin-top: 5rem;
|
||||
}
|
||||
.card.prose { display: block; }
|
||||
.card.prose p, .card.prose ol { margin: .55rem 0; line-height: 1.55; }
|
||||
.card.prose > :first-child { margin-top: 0; }
|
||||
.card.prose > :last-child { margin-bottom: 0; }
|
||||
.artcard { text-align: center; }
|
||||
.artcard img {
|
||||
width: 100%; max-width: 15rem;
|
||||
border: var(--line); border-radius: var(--radius-lg);
|
||||
background: var(--sky);
|
||||
}
|
||||
.artcard .legal { font-size: .74rem; color: var(--ink-soft); max-width: 30rem; margin: .6rem auto 0; }
|
||||
.card.active {
|
||||
border-color: var(--accent);
|
||||
box-shadow: var(--shadow-raised);
|
||||
}
|
||||
@media (prefers-reduced-motion: no-preference) {
|
||||
.card { transition: box-shadow .15s ease, border-color .15s ease; }
|
||||
button, .linkbtn { transition: transform .06s ease, box-shadow .06s ease; }
|
||||
}
|
||||
.shots { flex: 0 0 180px; display: flex; flex-direction: column; gap: .5rem; }
|
||||
.shots img { width: 100%; border-radius: 6px; border: var(--line); display: block; }
|
||||
.shots .noshot {
|
||||
color: var(--ink-soft); font-size: .8rem; border: 2px dashed var(--board-edge);
|
||||
border-radius: 6px; padding: 1.2rem .6rem; text-align: center;
|
||||
}
|
||||
.body { flex: 1; min-width: 0; }
|
||||
.title { font-size: 1.15rem; font-weight: 700; font-family: var(--font-display); margin: 0 0 .15rem; }
|
||||
.status { font-size: .72rem; text-transform: uppercase; letter-spacing: .08em; color: var(--ink-soft); }
|
||||
.cues { margin: .5rem 0 .7rem; display: flex; flex-wrap: wrap; gap: .35rem; }
|
||||
.cue {
|
||||
font-size: .74rem; background: #fff; border: 1px solid var(--board-edge);
|
||||
border-radius: 999px; padding: .1rem .6rem; color: var(--ink);
|
||||
}
|
||||
.cue b { font-weight: 600; color: var(--ink-soft); }
|
||||
.cands { list-style: none; margin: 0; padding: 0; }
|
||||
.cands li {
|
||||
display: flex; align-items: center; gap: .6rem;
|
||||
padding: .4rem .5rem; border-radius: var(--radius); cursor: pointer;
|
||||
border: 2px solid transparent;
|
||||
}
|
||||
.cands li:hover, .cands li:focus-visible { background: #fff; border-color: var(--accent); }
|
||||
.thumb { width: 42px; height: 42px; border-radius: 6px; object-fit: cover; border: 1px solid var(--board-edge); flex: 0 0 42px; }
|
||||
.thumb.ph {
|
||||
display: flex; align-items: center; justify-content: center;
|
||||
background: var(--accent); color: #fff;
|
||||
font-family: var(--font-display); font-size: 1.2rem;
|
||||
}
|
||||
.cname { font-weight: 600; }
|
||||
.cmeta { color: var(--ink-soft); font-size: .8rem; margin-left: .4rem; }
|
||||
.rowactions { margin-top: .7rem; display: flex; gap: 1rem; align-items: center; flex-wrap: wrap; font-size: .85rem; }
|
||||
.rowactions button { padding: .25rem .7rem; box-shadow: none; }
|
||||
.rowactions button.reject { color: var(--stop-ink); border-color: var(--stop); }
|
||||
.research { display: flex; gap: .35rem; align-items: center; flex-wrap: wrap; }
|
||||
.research input[type=text] { width: 12em; }
|
||||
.research button { font-size: .78rem; }
|
||||
.rowactions input[type=text] {
|
||||
font: inherit; width: 8.5em; padding: .25rem .5rem;
|
||||
border: 2px solid var(--board-edge); border-radius: var(--radius);
|
||||
}
|
||||
|
||||
/* -- reshoot work-orders: tickets, not cards --------------------------- */
|
||||
.ticket {
|
||||
background: var(--ticket);
|
||||
border: 2px dashed var(--gold-ink);
|
||||
border-radius: var(--radius);
|
||||
padding: .8rem 1rem;
|
||||
margin-bottom: .8rem;
|
||||
display: flex; gap: 1rem; align-items: flex-start;
|
||||
scroll-margin-top: 5rem;
|
||||
}
|
||||
.ticket.active { border-style: solid; box-shadow: var(--shadow-raised); }
|
||||
.ticket .stencil {
|
||||
writing-mode: vertical-rl; text-orientation: mixed;
|
||||
font-family: var(--font-mono);
|
||||
font-size: .7rem; letter-spacing: .35em; font-weight: 700;
|
||||
color: var(--gold-ink); text-transform: uppercase;
|
||||
border-right: 1px solid var(--gold-ink); padding-right: .5rem;
|
||||
}
|
||||
.ticket img { width: 130px; border-radius: 4px; border: 1px solid var(--gold-ink); }
|
||||
.ticket .loc { font-weight: 600; margin-bottom: .25rem; }
|
||||
.ticket .partial { font-family: var(--font-mono); font-size: .85rem; }
|
||||
.ticket .notes { color: var(--gold-ink); font-size: .85rem; margin-top: .2rem; }
|
||||
.ticket button {
|
||||
font-size: .8rem; margin-top: .5rem;
|
||||
background: none; border: 1px solid var(--gold-ink); color: var(--gold-ink);
|
||||
padding: .2rem .6rem; box-shadow: none;
|
||||
}
|
||||
|
||||
/* -- all-done celebration ---------------------------------------------- */
|
||||
.done {
|
||||
background: var(--board); border: var(--line); border-radius: var(--radius-lg);
|
||||
padding: 1.6rem; text-align: center; box-shadow: var(--shadow-raised);
|
||||
}
|
||||
.done .piper {
|
||||
height: 165px; margin-bottom: .4rem;
|
||||
border-radius: var(--radius-lg); border: var(--line);
|
||||
background: var(--sky);
|
||||
}
|
||||
.done h2 { margin-top: 0; }
|
||||
.done .nums { display: flex; justify-content: center; gap: 2rem; margin: 1rem 0; }
|
||||
.done .nums div { font-size: 1.6rem; font-weight: 700; font-family: var(--font-display); }
|
||||
.done .nums span { display: block; font-size: .72rem; text-transform: uppercase; letter-spacing: .08em; color: var(--ink-soft); }
|
||||
.done code {
|
||||
font-family: var(--font-mono);
|
||||
background: var(--navy); color: #fff;
|
||||
padding: .3rem .8rem; border-radius: var(--radius);
|
||||
display: inline-block; white-space: nowrap;
|
||||
}
|
||||
.done .waiting {
|
||||
color: var(--gold-ink);
|
||||
max-width: 34rem; margin: 1rem auto .2rem; line-height: 1.5;
|
||||
}
|
||||
.done .next { margin: .8rem 0 .2rem; }
|
||||
|
||||
/* -- catalog: read-only status ledger ---------------------------------- */
|
||||
.catalog { background: var(--board); border: var(--line); border-radius: var(--radius-lg); padding: .4rem 1rem; box-shadow: var(--shadow-card); }
|
||||
.catalog table { width: 100%; border-collapse: collapse; font-size: .85rem; }
|
||||
.catalog td { padding: .38rem .5rem; border-top: 1px solid var(--board-edge); vertical-align: top; }
|
||||
.catalog tr:first-child td { border-top: none; }
|
||||
.catalog tr:hover td { background: #fff; }
|
||||
.catalog .t { font-weight: 600; max-width: 22rem; }
|
||||
.catalog .meta { color: var(--ink-soft); }
|
||||
.catalog a {
|
||||
color: inherit;
|
||||
text-decoration: underline dotted;
|
||||
text-underline-offset: 2px;
|
||||
}
|
||||
.catalog a:hover { color: var(--accent-ink); }
|
||||
.catalog td.actions { text-align: right; }
|
||||
.catalog td.actions button { margin: .1rem 0 .1rem .3rem; white-space: nowrap; }
|
||||
.catalog td.actions .ballotlink {
|
||||
display: inline-block; margin: .1rem 0 .1rem .3rem; white-space: nowrap;
|
||||
font-size: .78rem; color: var(--accent-ink);
|
||||
text-decoration: underline dotted; text-underline-offset: 3px;
|
||||
}
|
||||
.catalog td.actions button { font-size: .78rem; padding: .2rem .6rem; box-shadow: none; }
|
||||
.catalog td.actions button:hover { background: var(--board); }
|
||||
.catalog tr.editrow td { background: #fff; border-top: none; padding: .2rem .5rem .7rem; }
|
||||
.editform { display: flex; gap: .7rem; align-items: end; flex-wrap: wrap; }
|
||||
.editform label {
|
||||
display: flex; flex-direction: column; gap: .15rem;
|
||||
font-size: .72rem; text-transform: uppercase; letter-spacing: .06em;
|
||||
color: var(--ink-soft);
|
||||
}
|
||||
.editform input {
|
||||
font: inherit; font-size: .85rem; color: var(--ink);
|
||||
border: 2px solid var(--board-edge); border-radius: var(--radius);
|
||||
padding: .25rem .45rem; background: #fff;
|
||||
}
|
||||
.editactions { display: flex; gap: .5rem; }
|
||||
.edithint { font-size: .78rem; color: var(--ink-soft); align-self: center; }
|
||||
.chip {
|
||||
font-size: .68rem; text-transform: uppercase; letter-spacing: .06em;
|
||||
border-radius: 999px; padding: .1rem .55rem; white-space: nowrap;
|
||||
border: 1px solid currentColor;
|
||||
}
|
||||
.card code {
|
||||
font-family: var(--font-mono); font-size: .82em;
|
||||
background: #fff; border: 1px solid var(--board-edge);
|
||||
border-radius: 4px; padding: 0 .3em;
|
||||
}
|
||||
.card ol { padding-left: 1.3rem; }
|
||||
.card ol li { margin: .35rem 0; }
|
||||
.chip.ok { background: var(--go-tint); color: var(--go-ink); }
|
||||
.chip.wait { background: var(--ticket); color: var(--gold-ink); }
|
||||
.chip.no { background: var(--stop-tint); color: var(--stop-ink); }
|
||||
.chip.open { background: var(--accent-tint); color: var(--accent-ink); }
|
||||
.chip.merged { background: var(--navy-tint); color: var(--navy); }
|
||||
.chip.shaky { background: var(--ticket); color: var(--gold-ink); border: 1px dashed var(--gold-ink); }
|
||||
|
||||
/* -- merge notices: slim, undoable ------------------------------------- */
|
||||
.card.merge { padding: .55rem 1rem; align-items: center; }
|
||||
.card.merge .body { display: flex; align-items: center; gap: .8rem; flex-wrap: wrap; }
|
||||
.card.merge .arrow { color: var(--ink-soft); }
|
||||
.card.merge button {
|
||||
font-size: .8rem; margin-left: auto;
|
||||
background: none; border: 1px solid var(--board-edge);
|
||||
padding: .2rem .6rem; box-shadow: none;
|
||||
}
|
||||
.card.merge button:hover { border-color: var(--stop); color: var(--stop-ink); }
|
||||
|
||||
/* -- dashboard: stage cards, dropzone, activity log --------------------- */
|
||||
.stages { display: grid; grid-template-columns: repeat(auto-fit, minmax(17rem, 1fr)); gap: .9rem; }
|
||||
.stage {
|
||||
background: var(--board); border: var(--line); border-radius: var(--radius-lg);
|
||||
box-shadow: var(--shadow-card);
|
||||
padding: .9rem 1rem;
|
||||
display: flex; flex-direction: column; gap: .5rem;
|
||||
}
|
||||
.stage .top { display: flex; align-items: baseline; gap: .55rem; }
|
||||
.stage .num {
|
||||
font-family: var(--font-mono);
|
||||
color: var(--accent-ink); font-size: .8rem; font-weight: 700;
|
||||
}
|
||||
.stage .name { font-weight: 700; font-family: var(--font-display); }
|
||||
.stage .facts { color: var(--ink-soft); font-size: .84rem; line-height: 1.45; flex: 1; }
|
||||
.stage .facts b { color: var(--ink); }
|
||||
.stage .act { display: flex; gap: .5rem; align-items: center; flex-wrap: wrap; }
|
||||
.stage input[type=number] {
|
||||
font: inherit; width: 4.5em; padding: .25rem .4rem;
|
||||
border: 2px solid var(--board-edge); border-radius: var(--radius);
|
||||
}
|
||||
#dropzone {
|
||||
display: block; width: 100%;
|
||||
background: var(--ticket);
|
||||
border: 2px dashed var(--gold-ink);
|
||||
border-radius: var(--radius);
|
||||
padding: 1.4rem;
|
||||
text-align: center; color: var(--gold-ink);
|
||||
font: inherit; cursor: pointer;
|
||||
box-shadow: none;
|
||||
}
|
||||
#dropzone.hot, #dropzone:hover { border-style: solid; background: var(--gold-tint); }
|
||||
#dropzone[aria-busy="true"] { cursor: progress; opacity: .8; }
|
||||
#dropzone.ok { border-style: solid; border-color: var(--go-ink); color: var(--go-ink); background: var(--go-tint); }
|
||||
#joblog {
|
||||
background: var(--navy); color: #eaf1fb;
|
||||
font-family: var(--font-mono); font-size: .78rem;
|
||||
border: var(--line); border-radius: var(--radius-lg);
|
||||
padding: .8rem 1rem;
|
||||
max-height: 20rem; overflow-y: auto; white-space: pre-wrap;
|
||||
}
|
||||
#jobstate { font-size: .85rem; color: var(--ink); margin-bottom: .4rem; }
|
||||
#jobstate .s-running { color: var(--accent-ink); font-weight: 600; }
|
||||
#jobstate .s-done { color: var(--go-ink); font-weight: 600; }
|
||||
#jobstate .s-failed { color: var(--stop-ink); font-weight: 600; }
|
||||
|
||||
/* (phone-width rules live in the responsive section at the end) */
|
||||
|
||||
/* -- photos page: gallery + status ------------------------------------- */
|
||||
.gallery { display: grid; grid-template-columns: repeat(auto-fill, minmax(11rem, 1fr)); gap: .9rem; }
|
||||
.shot {
|
||||
background: var(--board); border: var(--line); border-radius: var(--radius);
|
||||
box-shadow: var(--shadow-card); overflow: hidden;
|
||||
display: flex; flex-direction: column;
|
||||
}
|
||||
.shot img { width: 100%; aspect-ratio: 4/3; object-fit: cover; display: block; }
|
||||
.shot .meta { padding: .4rem .6rem; font-size: .78rem; color: var(--ink-soft); }
|
||||
.shot .meta b { color: var(--ink); }
|
||||
.fullshot {
|
||||
width: 100%; max-height: 60vh; object-fit: contain;
|
||||
background: var(--navy); border: var(--line); border-radius: var(--radius-lg);
|
||||
display: block; margin-bottom: 1rem;
|
||||
}
|
||||
|
||||
/* -- queue + library --------------------------------------------------- */
|
||||
.ledger { background: var(--board); border: var(--line); border-radius: var(--radius-lg); padding: .4rem 1rem; box-shadow: var(--shadow-card); }
|
||||
.ledger table { width: 100%; border-collapse: collapse; font-size: .85rem; }
|
||||
.ledger td, .ledger th { padding: .38rem .5rem; border-top: 1px solid var(--board-edge); vertical-align: top; text-align: left; }
|
||||
.ledger th { font-size: .72rem; text-transform: uppercase; letter-spacing: .06em; color: var(--ink-soft); border-top: none; }
|
||||
.ledger .meta { color: var(--ink-soft); }
|
||||
.shelfgrid { display: grid; grid-template-columns: repeat(auto-fill, minmax(13rem, 1fr)); gap: .9rem; }
|
||||
.game {
|
||||
background: var(--board); border: var(--line); border-radius: var(--radius-lg);
|
||||
box-shadow: var(--shadow-card); overflow: hidden;
|
||||
display: flex; flex-direction: column;
|
||||
}
|
||||
.game img { width: 100%; aspect-ratio: 1; object-fit: cover; display: block; background: var(--sky-deep); }
|
||||
.game .noart {
|
||||
width: 100%; aspect-ratio: 1; display: flex; align-items: center; justify-content: center;
|
||||
background: var(--sky-deep); color: var(--navy);
|
||||
font-family: var(--font-display); font-size: 2.2rem; font-weight: 700;
|
||||
}
|
||||
.game .info { padding: .55rem .7rem .7rem; }
|
||||
.game .gname { font-weight: 700; font-family: var(--font-display); }
|
||||
.game .gmeta { font-size: .78rem; color: var(--ink-soft); margin-top: .2rem; line-height: 1.5; }
|
||||
/* -- library: card grid links + one game's detail ---------------------- */
|
||||
a.game { text-decoration: none; color: inherit; }
|
||||
a.game:hover { border-color: var(--accent); box-shadow: var(--shadow-raised); }
|
||||
a.game:focus-visible { outline: 3px solid var(--focus); outline-offset: 2px; }
|
||||
.gamedetail { display: flex; gap: 1.4rem; align-items: flex-start; flex-wrap: wrap; }
|
||||
.gameartcol { flex: 0 0 clamp(12rem, 28vw, 18rem); }
|
||||
.gameart {
|
||||
width: 100%; border: var(--line); border-radius: var(--radius-lg);
|
||||
box-shadow: var(--shadow-card); display: block; background: var(--board);
|
||||
}
|
||||
.gameart.noart {
|
||||
aspect-ratio: 1; display: flex; align-items: center; justify-content: center;
|
||||
background: var(--sky-deep); color: var(--navy);
|
||||
font-family: var(--font-display); font-size: 3rem; font-weight: 700;
|
||||
}
|
||||
.gamefacts {
|
||||
flex: 1 1 18rem; background: var(--board); border: var(--line);
|
||||
border-radius: var(--radius-lg); box-shadow: var(--shadow-card);
|
||||
padding: .9rem 1.1rem;
|
||||
}
|
||||
.factrow { margin: .45rem 0; line-height: 1.5; }
|
||||
.factrow b {
|
||||
font-size: .72rem; text-transform: uppercase; letter-spacing: .06em;
|
||||
color: var(--ink-soft); margin-right: .4rem;
|
||||
}
|
||||
.chiplist { display: inline-flex; flex-wrap: wrap; gap: .3rem; vertical-align: middle; }
|
||||
.gdesc { white-space: pre-wrap; line-height: 1.6; }
|
||||
.artbtn { margin-top: .6rem; width: 100%; font-size: .8rem; }
|
||||
.editform label.wide { flex: 1 1 100%; }
|
||||
.editform textarea {
|
||||
font: inherit; font-size: .85rem; color: var(--ink); width: 100%;
|
||||
border: 2px solid var(--board-edge); border-radius: var(--radius);
|
||||
padding: .35rem .45rem; background: #fff; resize: vertical;
|
||||
}
|
||||
|
||||
.empty {
|
||||
background: var(--board); border: 2px dashed var(--board-edge);
|
||||
border-radius: var(--radius-lg); padding: 2rem; text-align: center;
|
||||
color: var(--ink-soft); line-height: 1.6;
|
||||
}
|
||||
.empty code {
|
||||
font-family: var(--font-mono); background: var(--navy); color: #fff;
|
||||
padding: .15rem .5rem; border-radius: 6px;
|
||||
}
|
||||
.filterbar { display: flex; gap: .6rem; margin-bottom: 1rem; flex-wrap: wrap; }
|
||||
.filterbar button[aria-pressed="true"] {
|
||||
background: var(--accent); border-color: var(--ink); color: #fff;
|
||||
}
|
||||
.filterbar input[type=search] {
|
||||
font: inherit; padding: .35rem .7rem; min-width: 16rem;
|
||||
border: var(--line); border-radius: var(--radius); background: #fff;
|
||||
}
|
||||
|
||||
/* -- toast: viewport-pinned transient confirmations --------------------- */
|
||||
#toast {
|
||||
position: fixed; bottom: 1.1rem; left: 50%; transform: translateX(-50%);
|
||||
z-index: 20; max-width: min(92vw, 34rem);
|
||||
background: var(--navy); color: #fff;
|
||||
border: var(--line); border-radius: var(--radius-lg);
|
||||
box-shadow: var(--shadow-raised);
|
||||
padding: .7rem 1rem; font-size: .9rem; line-height: 1.45;
|
||||
}
|
||||
#toast a { color: var(--gold); }
|
||||
#toast button {
|
||||
font-size: .8rem; padding: .25rem .7rem; margin-left: .5rem;
|
||||
box-shadow: none;
|
||||
}
|
||||
|
||||
/* -- responsive -------------------------------------------------------- */
|
||||
|
||||
/* The rail collapses to a top bar: brand + hamburger; the nav drops
|
||||
* down as a stacked menu (a scrolling strip lost its position on every
|
||||
* page load and hid the far entries). */
|
||||
@media (max-width: 900px) {
|
||||
body { display: block; }
|
||||
.sidebar {
|
||||
position: sticky; top: 0; height: auto; z-index: 5;
|
||||
display: flex; flex-direction: row; align-items: center; flex-wrap: wrap;
|
||||
border-right: none; border-bottom: 4px solid transparent;
|
||||
border-image: var(--path) 1;
|
||||
}
|
||||
.brand { padding: .5rem .8rem; flex: 1; }
|
||||
.brand img { width: 30px; height: 30px; }
|
||||
.wordmark { font-size: 1.1rem; }
|
||||
.wordmark small { display: inline; margin-left: .5rem; font-size: .68rem; }
|
||||
.navtoggle {
|
||||
display: inline-flex; align-items: center; justify-content: center;
|
||||
width: 44px; height: 40px; margin-right: .6rem;
|
||||
background: none; border: 2px solid rgba(255, 255, 255, .75);
|
||||
border-radius: var(--radius); box-shadow: none; padding: 0;
|
||||
}
|
||||
.navtoggle .bars {
|
||||
display: block; position: relative;
|
||||
width: 18px; height: 2px; background: #fff; border-radius: 2px;
|
||||
}
|
||||
.navtoggle .bars::before, .navtoggle .bars::after {
|
||||
content: ""; position: absolute; left: 0;
|
||||
width: 18px; height: 2px; background: #fff; border-radius: 2px;
|
||||
}
|
||||
.navtoggle .bars::before { top: -6px; }
|
||||
.navtoggle .bars::after { top: 6px; }
|
||||
body.navopen .navtoggle { background: rgba(255, 255, 255, .15); }
|
||||
nav[aria-label="Primary"] {
|
||||
display: none; flex-basis: 100%; flex-direction: column;
|
||||
padding: 0 0 .4rem;
|
||||
}
|
||||
body.navopen nav[aria-label="Primary"] { display: flex; }
|
||||
nav[aria-label="Primary"] a { padding: .6rem 1rem; }
|
||||
.piperbox { display: none; }
|
||||
}
|
||||
|
||||
/* Phone-width layout: stacked cards, stacked tables, touch targets. */
|
||||
@media (max-width: 700px) {
|
||||
main { padding: 1rem .9rem 4rem; }
|
||||
h1 { font-size: 1.3rem; margin: .9rem 0 .7rem; }
|
||||
.keyhelp { display: none; } /* no keyboard on a phone */
|
||||
button, .linkbtn { padding: .45rem .9rem; } /* finger-sized */
|
||||
/* component buttons set their own padding at higher specificity —
|
||||
* they need the touch bump spelled out */
|
||||
.rowactions button, .ticket button, .card.merge button { padding: .4rem .8rem; }
|
||||
|
||||
.card, .ticket { flex-direction: column; }
|
||||
.shots { flex-basis: auto; }
|
||||
.ticket .stencil {
|
||||
writing-mode: horizontal-tb; letter-spacing: .35em;
|
||||
border-right: none; border-bottom: 1px solid var(--gold-ink);
|
||||
padding: 0 0 .3rem;
|
||||
}
|
||||
.done { padding: 1.1rem .9rem; }
|
||||
.done .nums { flex-wrap: wrap; gap: .8rem 1.6rem; }
|
||||
|
||||
.filterbar input[type=search] { min-width: 0; flex: 1 1 12rem; }
|
||||
.editform { flex-direction: column; align-items: stretch; }
|
||||
.editform input { width: 100%; }
|
||||
.editactions { flex-wrap: wrap; }
|
||||
.editactions button { flex: 1 1 auto; }
|
||||
|
||||
/* the titles/photo tables become stacked line-cards: one bordered
|
||||
* block per game, cells flowing top to bottom, empty cells gone */
|
||||
.catalog table, .catalog tbody, .catalog tr, .catalog td { display: block; }
|
||||
.catalog tr { padding: .6rem 0; border-top: 1px solid var(--board-edge); }
|
||||
.catalog tr:first-child { border-top: none; }
|
||||
.catalog tr:hover td { background: none; }
|
||||
.catalog td { border-top: none; padding: .12rem 0; }
|
||||
.catalog td:empty { display: none; }
|
||||
.catalog .t { max-width: none; font-size: .95rem; }
|
||||
.catalog td.actions {
|
||||
text-align: left; white-space: normal;
|
||||
display: flex; gap: .5rem; flex-wrap: wrap; padding-top: .4rem;
|
||||
}
|
||||
.catalog td.actions button { padding: .4rem .8rem; }
|
||||
.catalog tr.editrow { padding-top: 0; border-top: none; }
|
||||
|
||||
/* queue/library ledgers keep their columns but scroll inside their
|
||||
* own card; long photo lists wrap within their cell */
|
||||
.ledger { overflow-x: auto; }
|
||||
.ledger td { overflow-wrap: anywhere; min-width: 6rem; }
|
||||
}
|
||||
@@ -0,0 +1,192 @@
|
||||
/* Shared client plumbing for every bggpipe page: escaping, banners,
|
||||
* fetch helpers, and the sidebar's live count badges. Loaded as a
|
||||
* blocking script before each page's own script. */
|
||||
"use strict";
|
||||
|
||||
const esc = s => String(s ?? "").replace(/[&<>"']/g,
|
||||
c => ({"&": "&", "<": "<", ">": ">", '"': """, "'": "'"}[c]));
|
||||
|
||||
/* Contract for every innerHTML sink in this app: interpolated values MUST
|
||||
* pass through esc() at the call site. The API serves only local pipeline
|
||||
* data, but photo names and BGG titles still count as untrusted. */
|
||||
function showBanner(html) {
|
||||
document.getElementById("banner").innerHTML = html;
|
||||
}
|
||||
|
||||
/* Transient confirmation pinned to the viewport: the #banner region sits
|
||||
* at the top of the page, which a user deep in a long list never sees.
|
||||
* The toast may carry buttons — wire them before the timer clears it. */
|
||||
function showToast(html, ms = 8000) {
|
||||
let el = document.getElementById("toast");
|
||||
if (!el) {
|
||||
el = document.createElement("div");
|
||||
el.id = "toast";
|
||||
el.setAttribute("role", "status");
|
||||
document.body.appendChild(el);
|
||||
}
|
||||
el.innerHTML = html;
|
||||
el.hidden = false;
|
||||
clearTimeout(showToast._timer);
|
||||
showToast._timer = setTimeout(() => { el.hidden = true; }, ms);
|
||||
return el;
|
||||
}
|
||||
|
||||
function errorBanner(detail) {
|
||||
// an HTTP status means the server answered — the problem is its data
|
||||
const httpish = /^\d{3}$/.test(String(detail));
|
||||
showBanner(`<div class="banner error">${httpish
|
||||
? `the server hit an error (HTTP ${esc(detail)}) — a data file may be broken`
|
||||
: `lost contact with the server (${esc(detail)})`} — check its terminal</div>`);
|
||||
}
|
||||
|
||||
async function fetchJSON(url) {
|
||||
const res = await fetch(url);
|
||||
if (!res.ok) throw new Error(`${res.status}`);
|
||||
return res.json();
|
||||
}
|
||||
|
||||
async function apiPost(url, body) {
|
||||
let res;
|
||||
try {
|
||||
res = await fetch(url, {
|
||||
method: "POST",
|
||||
headers: {"Content-Type": "application/json"},
|
||||
body: JSON.stringify(body ?? {}),
|
||||
});
|
||||
} catch (err) {
|
||||
alert("No response from the server: " + err);
|
||||
return null;
|
||||
}
|
||||
if (!res.ok) {
|
||||
const detail = await res.json().then(d => d.detail).catch(() => null);
|
||||
alert("That didn't work: " + (detail ?? res.statusText));
|
||||
return null;
|
||||
}
|
||||
return res;
|
||||
}
|
||||
|
||||
/* Per-page change gate for poll loops: render only when the payload
|
||||
* changed, and record it only after render returns — a throw leaves the
|
||||
* page stale, so the next poll retries instead of freezing on "current". */
|
||||
function changeGate() {
|
||||
let seen = null;
|
||||
const gate = (value, render) => {
|
||||
const key = JSON.stringify(value);
|
||||
if (key === seen) return;
|
||||
render();
|
||||
seen = key;
|
||||
};
|
||||
gate.reset = () => { seen = null; };
|
||||
return gate;
|
||||
}
|
||||
|
||||
/* Poll `fn` every `ms`; after 3 consecutive failures show the lost-contact
|
||||
* banner, and clear it (via `recovered`) on the next success. */
|
||||
function pollLoop(fn, ms, recovered) {
|
||||
let misses = 0;
|
||||
setInterval(async () => {
|
||||
const el = document.activeElement;
|
||||
if (el && (el.tagName === "INPUT" || el.tagName === "TEXTAREA")) return;
|
||||
try {
|
||||
await fn();
|
||||
if (misses >= 3 && recovered) recovered();
|
||||
misses = 0;
|
||||
} catch (err) {
|
||||
if (++misses >= 3) errorBanner(err.message || err);
|
||||
}
|
||||
}, ms);
|
||||
}
|
||||
|
||||
/* Mobile: the top bar's hamburger opens the nav; navigation reloads the
|
||||
* page, so each page starts with the menu closed. */
|
||||
{
|
||||
const toggle = document.querySelector(".navtoggle");
|
||||
if (toggle) toggle.addEventListener("click", () => {
|
||||
const open = document.body.classList.toggle("navopen");
|
||||
toggle.setAttribute("aria-expanded", String(open));
|
||||
});
|
||||
}
|
||||
|
||||
/* Sidebar badges: the counts that mean "something wants your attention". */
|
||||
async function refreshBadges() {
|
||||
const p = await fetchJSON("/api/pipeline");
|
||||
const set = (name, n) => {
|
||||
const el = document.querySelector(`[data-badge="${name}"]`);
|
||||
if (el) el.textContent = n > 0 ? String(n) : "";
|
||||
};
|
||||
set("photos", p.reshoot);
|
||||
set("titles", p.shaky_reads);
|
||||
set("review", p.pending_review);
|
||||
set("queue", p.to_add + p.to_update);
|
||||
return p;
|
||||
}
|
||||
refreshBadges().catch(() => {});
|
||||
setInterval(() => refreshBadges().catch(() => {}), 5000);
|
||||
|
||||
/* One-line match summary for a titles/photo table row. Empty when the
|
||||
* row has no BGG data yet, so the mobile stacker can hide the cell. */
|
||||
/* Canonical outbound URL for a matched thing. RPG items live on RPGGeek
|
||||
* (same database, different site) — a /boardgame/ URL is the wrong home
|
||||
* for them. */
|
||||
function bggUrl(id, type) {
|
||||
return type === "rpgitem"
|
||||
? `https://rpggeek.com/rpgitem/${encodeURIComponent(id)}`
|
||||
: `https://boardgamegeek.com/boardgame/${encodeURIComponent(id)}`;
|
||||
}
|
||||
|
||||
function metaLine(c) {
|
||||
const name = !c.bgg_name ? "" : c.bgg_id
|
||||
? `<a href="${bggUrl(c.bgg_id, c.type)}" target="_blank" rel="noopener"
|
||||
title="open on ${c.type === "rpgitem" ? "RPGGeek" : "BGG"}">${esc(c.bgg_name)} ↗</a>`
|
||||
: esc(c.bgg_name);
|
||||
return [
|
||||
name,
|
||||
c.version_name ? esc(c.version_name) : "",
|
||||
c.type === "rpgitem" ? `<span class="chip open">RPG · local only</span>` : "",
|
||||
].filter(Boolean).join(" · ");
|
||||
}
|
||||
|
||||
/* Reshoot work-order card, shared by the photos gallery and the photo
|
||||
* detail page (which drops the thumbnail and source line — the photo is
|
||||
* right there). Dismiss buttons are wired by wireDismiss below. */
|
||||
function ticketCard(s, {showPhoto = true} = {}) {
|
||||
const img = showPhoto && s.photo_exists
|
||||
? `<a href="/photos/${encodeURIComponent(s.photo)}" target="_blank" tabindex="-1">
|
||||
<img src="/photos/${encodeURIComponent(s.photo)}" alt="photo ${esc(s.photo)}"></a>`
|
||||
: "";
|
||||
return `
|
||||
<section class="ticket"
|
||||
data-photo="${esc(s.photo)}" data-location="${esc(s.location)}"
|
||||
data-partial="${esc(s.partial_text)}" data-art="${esc(s.art_notes)}">
|
||||
<span class="stencil">reshoot</span>
|
||||
${img}
|
||||
<div>
|
||||
<div class="loc">${esc(s.location) || "somewhere in " + esc(s.photo)}</div>
|
||||
${s.partial_text ? `<div class="partial">text visible: ${esc(s.partial_text)}</div>` : ""}
|
||||
${s.art_notes ? `<div class="notes">${esc(s.art_notes)}</div>` : ""}
|
||||
${showPhoto ? `<div class="notes">from ${esc(s.photo)} — take a closer shot and drop it above</div>` : ""}
|
||||
<button class="dismiss">dismiss — found it / not a game</button>
|
||||
</div>
|
||||
</section>`;
|
||||
}
|
||||
|
||||
function wireDismiss(root, done) {
|
||||
root.querySelectorAll(".dismiss").forEach(b => b.addEventListener("click", async () => {
|
||||
const t = b.closest(".ticket");
|
||||
const res = await apiPost("/api/dismiss", {
|
||||
photo: t.dataset.photo, location: t.dataset.location,
|
||||
partial_text: t.dataset.partial, art_notes: t.dataset.art,
|
||||
});
|
||||
if (res) done();
|
||||
}));
|
||||
}
|
||||
|
||||
/* Status chip for a catalog entry — shared by the catalog and photo pages. */
|
||||
function statusChip(c) {
|
||||
if (c.status === "awaiting_resolve") return `<span class="chip wait">awaiting resolve</span>`;
|
||||
if (c.status === "auto" || c.status === "approved") return `<span class="chip ok">${esc(c.status)}</span>`;
|
||||
if (c.status === "rejected") return `<span class="chip no">rejected</span>`;
|
||||
if (c.status === "local") return `<span class="chip open">local — not on BGG</span>`;
|
||||
if (c.status === "merged") return `<span class="chip merged" title="merged into ${esc(c.merged_into)}">merged → ${esc(c.merged_into)}</span>`;
|
||||
return `<span class="chip open">${esc(c.status)}</span>`;
|
||||
}
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 39 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 32 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 37 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 16 KiB |
@@ -0,0 +1,80 @@
|
||||
<h1>Help</h1>
|
||||
<div class="pagebar">
|
||||
<a href="#flow">The flow</a>
|
||||
<a href="#pages">Pages</a>
|
||||
<a href="#curation">Fixing the titles</a>
|
||||
<a href="#statuses">Statuses</a>
|
||||
<a href="#keys">Keyboard</a>
|
||||
<a href="#files">Your data</a>
|
||||
</div>
|
||||
|
||||
<h2 id="flow">The flow: shelves → collection</h2>
|
||||
<div class="card prose">
|
||||
<p>Six pipeline stages and two checkpoints that are yours. Stages run from the <a href="/">Pipeline</a> page (or the CLI — both share all state and either can pick up where the other left off):</p>
|
||||
<ol>
|
||||
<li><b>extract</b> — every photo goes to the vision model once (Claude by default; a local model works too — see the README); what it reads (titles plus edition cues: publisher, edition wording, year, language) lands on the Titles page. Boxes it can see but can't read become <b>reshoot tickets</b> on the Photos page.</li>
|
||||
<li><b>proofread</b> <i>(you, on <a href="/titles">Titles</a>)</i> — fix misread titles, add cues you know, split multi-box lines, remove non-games, <a href="#curation">details below</a>. Worth doing <i>before</i> resolve: a fix made now is one BGG search done right; a fix made later sends the title back through resolve again.</li>
|
||||
<li><b>resolve</b> — titles are matched to BoardGameGeek games and editions. Anything uncertain is flagged, never guessed. Without a BGG API token, titles wait as <i>awaiting BGG</i> and are picked up automatically once the token exists.</li>
|
||||
<li><b>review</b> <i>(you, on <a href="/review">Review</a>)</i> — decide the flagged ones: which game, which edition, whether two reads are one box. Every decision saves immediately.</li>
|
||||
<li><b>diff</b> — your existing BGG collection is fetched and compared, per copy. What's genuinely new lands in the <a href="/queue">Queue</a>.</li>
|
||||
<li><b>upload</b> — a real browser logs into BGG and adds each queued game, slowly and politely. Always dry-run first; the buttons enforce that order.</li>
|
||||
<li><b>enrich</b> — full metadata (players, weight, rank, artwork) fills the <a href="/library">Library</a>.</li>
|
||||
</ol>
|
||||
<p>It's a loop, not a line: new photos, edits, and splits feed both checkpoints again, and the sidebar badges show when a page has work for you. Every stage is resumable — stop anything mid-run and nothing is lost; re-runs skip work already done.</p>
|
||||
</div>
|
||||
|
||||
<h2 id="pages">What each page is for</h2>
|
||||
<div class="card prose">
|
||||
<p><b><a href="/">Pipeline</a></b> — run stages one at a time and watch their live output. Shows what's blocking (missing keys, stub data) and the counts at every step.</p>
|
||||
<p><b><a href="/photos">Photos</a></b> — drag photos in, drop them in the <code>photos/</code> folder, or (on a paired phone) tap the drop zone and shoot straight from the camera; camera captures get unique <code>shelf-…</code> names so they can never overwrite each other. Each photo has its own page listing every title read from it and any reshoot tickets — boxes seen but not identified. Photograph those up close, drop the new shot in, and extract again. Re-uploading a photo under the same <i>file name</i> deliberately replaces it, and the next extract run re-reads it.</p>
|
||||
<p><b><a href="/titles">Titles</a></b> — every read off your shelves, alphabetized, with its status and photos. This is the proofread checkpoint: <a href="#curation">edit, split, remove</a>. Its badge counts <span class="chip shaky">shaky read</span> lines — the model wasn't sure and nothing has verified them; filter to them, then press <b>✓ looks right</b> or edit each one.</p>
|
||||
<p><b><a href="/review">Review</a></b> — the decisions only you can make: which game a title is, which edition a copy is, whether two same-game reads are really one box (merges show a veto), and whether an unmatched title is a real game BGG simply doesn't have (<b>keep locally</b>: it joins the Library, never uploads). Keyboard-first; see <a href="#keys">shortcuts</a>.</p>
|
||||
<p><b><a href="/queue">Queue</a></b> — exactly what upload will do (new entries and version upgrades) and the log of everything it has done. Nothing reaches BGG that isn't visible here first. A job that fails is skipped by later runs (so one broken game can't loop forever); when any exist, the Pipeline's upload card offers a <b>retry N failed</b> checkbox. Each queued row shows what upload did with it — <span class="chip open">pending</span>, <span class="chip ok">done</span>, <span class="chip no">failed</span>, or <span class="chip no">retired</span> (a review decision since the last diff withdrew it). Finished rows stay listed until the next <b>diff</b> rebuilds the queue; the log below them is the permanent record.</p>
|
||||
<p><b><a href="/library">Library</a></b> — your enriched collection. Search titles, designers, mechanics and categories at once; filter by kind (board games, RPGs, off-BGG) or by how many people are playing tonight; sort by name, year, BGG rank, weight, or playing time. Click any game for its full detail: art, the usual stats, designers and mechanics, <b>your</b> edition, the shelf photos it was read from, and a link to its BGG page. RPG and off-BGG games live here too — identified and enriched, never uploaded. RPGs pull their designers, publishers and genres from RPGGeek; an off-BGG game's detail page lets you write its facts yourself and add a cover photo, since nothing else will ever have them (both are saved under <code>data/</code> and folded in by the next <b>enrich</b>).</p>
|
||||
</div>
|
||||
|
||||
<h2 id="curation">Fixing the titles: edit, split, remove</h2>
|
||||
<div class="card prose">
|
||||
<p>Vision reads aren't perfect, and you know things the photos don't show. Every line on the Titles page has curation actions, and every one of them is <b>durable</b>: the decision is saved in a small committed file and replayed on every rebuild, so re-running extract or resolve can never undo it.</p>
|
||||
<p><b>edit</b> — fix a misread title or add cues you already know (publisher, edition, year, language). A corrected misspelling automatically merges with a correctly-read sighting of the same game from another photo. If the line already had a BGG match, saving re-queues it so resolve searches again with the corrected data.</p>
|
||||
<p><b>split into copies</b> — one line, several physical boxes? Splitting makes each photo its own copy, and each copy picks its own edition afterward. Appears on any line whose title was seen in more than one photo. Splitting one game never affects a same-named different edition.</p>
|
||||
<p><b>pick edition</b> — a matched game with no legible edition cues stays version-less by design (never guess) — but you know which printing your box is. This fetches the game's complete version list into a Review ballot; pick yours there.</p>
|
||||
<p><b>wrong match</b> (inside the edit panel) — an auto-match landed on the wrong game (same-name impostors happen). This clears the match, re-searches immediately, and returns the title to Review as a fresh ballot of candidates (including same-named sibling editions); manual-id entry is there too for games BGG's search can't find. Note BGG sometimes splits one game's lineage across entries — Wiz-War's early editions and its FFG remake are separate games — so a copy whose edition isn't on the ballot may belong to the sibling entry.</p>
|
||||
<p><b>add a game</b> (top of the Titles page) — a game no photo shows: an expansion stored inside a base box, a game away from the shelves. It joins the list like any read (BGG wants base game and expansion as separate collection entries, so boxes that hold both need this for the hidden half), and if a later photo shows it, the sighting merges instead of duplicating.</p>
|
||||
<p><b>remove</b> (inside the edit panel) — for lines that shouldn't exist at all: a book read as a game, box art misread as a title, or a <i>duplicate read</i> — the same physical copy read differently from two photos, leaving two lines for one box (remove the worse read; the survivor keeps its own photos). This is different from <i>reject</i> on the Review page, which keeps the line visible as "no BGG match" — right for real games BGG doesn't know. And the mirror case — ONE line that's really several physical copies — wants <b>split</b>, not remove.</p>
|
||||
<p>Undo: each decision is one record in <code>data/title_edits.json</code>, <code>data/title_splits.json</code>, or <code>data/title_removals.json</code> — delete the record and the next rebuild restores the old state.</p>
|
||||
</div>
|
||||
|
||||
<h2 id="statuses">What the status chips mean</h2>
|
||||
<div class="card prose">
|
||||
<p><span class="chip wait">awaiting resolve</span> extracted (or re-queued by an edit) but not yet matched — run <b>resolve</b> from the Pipeline page.</p>
|
||||
<p><span class="chip ok">auto</span> matched confidently, no review needed. <span class="chip ok">approved</span> you picked the match yourself.</p>
|
||||
<p><span class="chip open">ambiguous</span> several plausible games — needs your pick on Review. <span class="chip open">unmatched</span> nothing plausible found — enter a BGG id or re-search on Review.</p>
|
||||
<p><span class="chip merged">merged</span> two reads judged to be the same physical box; the merge is veto-able on Review. <span class="chip merged">copy</span> one copy of a title you split.</p>
|
||||
<p><span class="chip open">local — not on BGG</span> you ruled it's a real game BGG doesn't have: it joins the Library from its own photo reads (fill in its facts and add a cover photo on its Library page), and never uploads. Changed your mind — or suspect it's on RPGGeek after all? <b>look it up</b> on its Titles row searches again, RPGGeek included.</p>
|
||||
<p><span class="chip no">rejected</span> you ruled it's a bad read or not worth matching; it stays listed but goes no further.</p>
|
||||
<p><span class="chip shaky">shaky read</span> the vision model wasn't sure of this transcription and nothing has verified it yet — these are what the Titles badge counts. Clear one by pressing its <b>✓ looks right</b> (the read is fine as-is) or by editing it (you fixed it). A BGG match also clears it: a wrong read wouldn't have matched.</p>
|
||||
</div>
|
||||
|
||||
<h2 id="keys">Keyboard shortcuts</h2>
|
||||
<div class="card prose">
|
||||
<p><b>Review:</b> <kbd>j</kbd>/<kbd>k</kbd> move between cards · <kbd>1</kbd>–<kbd>9</kbd> pick a candidate · <kbd>r</kbd> reject · <kbd>l</kbd> keep local (not on BGG) · <kbd>m</kbd> manual BGG id · <kbd>u</kbd> edition unknown · <kbd>v</kbd> veto a merge.</p>
|
||||
<p><b>Photo pages:</b> <kbd>←</kbd>/<kbd>→</kbd> move between photos.</p>
|
||||
</div>
|
||||
|
||||
<h2 id="files">Your data, on disk</h2>
|
||||
<div class="card prose">
|
||||
<p>Everything lives in flat files under <code>data/</code> — inspectable, hand-editable, and git-friendly. The pipeline artifacts: <code>titles.json</code> (what was read), <code>matches.csv</code> (what it matched), <code>to_add.csv</code>/<code>to_update.csv</code> (what upload will do), <code>upload_log.csv</code> (what it did), <code>games.json</code> (the library). Your curation: <code>title_edits.json</code>, <code>title_splits.json</code>, <code>title_removals.json</code>, <code>unidentified_dismissed.json</code>.</p>
|
||||
<p>Credentials never live in files — only environment variables, set up by <code>bggpipe init</code>. The app serves localhost only, unless started with <code>--lan</code> — that opens it to your network behind an access key: scan the QR code the server prints (or open the printed link) once per device, and a year-long cookie keeps it paired. Delete <code>data/.lan_key</code> to revoke every device. No login beyond the key; trusted networks only.</p>
|
||||
<p>More depth: the README covers setup and photo technique; <code>docs/bgg-upload-flow.md</code> documents the upload automation.</p>
|
||||
</div>
|
||||
|
||||
<h2 id="mascot">The piper</h2>
|
||||
<div class="card prose artcard">
|
||||
<img src="/static/logo-full.jpg"
|
||||
alt="the bggpipe piper — a bagpiper whose bag is a board game box">
|
||||
<p>Mascot art by Juniper, used with pride.</p>
|
||||
<p class="legal">BoardGameGeek and BGG are trademarks of BoardGameGeek, LLC.
|
||||
bggpipe is an independent project, not affiliated with or endorsed by
|
||||
BoardGameGeek.</p>
|
||||
</div>
|
||||
@@ -0,0 +1,124 @@
|
||||
<h1>Library</h1>
|
||||
<div class="pagebar"><span id="libcount"></span></div>
|
||||
<div class="filterbar" role="group" aria-label="filter by kind">
|
||||
<input type="search" id="libsearch" placeholder="search titles, designers, mechanics…"
|
||||
aria-label="search library">
|
||||
<button type="button" data-kind="" aria-pressed="true">All</button>
|
||||
<button type="button" data-kind="boardgame" aria-pressed="false">Board games</button>
|
||||
<button type="button" data-kind="rpgitem" aria-pressed="false">RPGs</button>
|
||||
<button type="button" data-kind="localgame" aria-pressed="false">Off BGG</button>
|
||||
</div>
|
||||
<div class="filterbar">
|
||||
<label>plays with
|
||||
<input type="number" id="libplayers" min="1" max="20" placeholder="any"
|
||||
style="width:5em" aria-label="filter by player count"></label>
|
||||
<label>sort
|
||||
<select id="libsort" aria-label="sort library">
|
||||
<option value="name">name</option>
|
||||
<option value="year">year (newest)</option>
|
||||
<option value="rank">BGG rank</option>
|
||||
<option value="weight">weight (lightest)</option>
|
||||
<option value="playtime">playing time</option>
|
||||
</select></label>
|
||||
</div>
|
||||
<div id="libbody"></div>
|
||||
<script>
|
||||
"use strict";
|
||||
let GAMES = [];
|
||||
let KIND = ""; // "" = all; "boardgame" also covers expansions
|
||||
|
||||
function gameCard(g) {
|
||||
const art = g.thumbnail || g.image;
|
||||
const lo = g.min_players ?? g.max_players, hi = g.max_players ?? g.min_players;
|
||||
const players = lo
|
||||
? (lo === hi ? `${lo}` : `${lo}–${hi}`) + " players"
|
||||
: "";
|
||||
const time = g.playtime ? `${g.playtime} min` : "";
|
||||
const weight = g.weight ? `weight ${g.weight.toFixed(1)}` : "";
|
||||
const rank = g.rank ? `BGG rank ${g.rank}` : "";
|
||||
return `
|
||||
<a class="game" href="/library/game/${encodeURIComponent(g.key)}">
|
||||
${art
|
||||
? `<img src="${esc(art)}" alt="" loading="lazy">`
|
||||
: `<div class="noart" aria-hidden="true">${esc((g.name || "?").charAt(0).toUpperCase())}</div>`}
|
||||
<div class="info">
|
||||
<div class="gname">${esc(g.name)}${g.year ? ` <span class="meta">(${esc(g.year)})</span>` : ""}
|
||||
${g.type === "rpgitem" ? `<span class="chip open">RPG · local only</span>` : ""}
|
||||
${g.type === "localgame" ? `<span class="chip open">not on BGG · local</span>` : ""}</div>
|
||||
<div class="gmeta">${[players, time, weight, rank].filter(Boolean).map(esc).join(" · ")}</div>
|
||||
${g.version ? `<div class="gmeta">${esc(g.version.name || "")}</div>` : ""}
|
||||
</div>
|
||||
</a>`;
|
||||
}
|
||||
|
||||
function matchesKind(g) {
|
||||
if (!KIND) return true;
|
||||
if (KIND === "boardgame") return !["rpgitem", "localgame"].includes(g.type);
|
||||
return g.type === KIND;
|
||||
}
|
||||
|
||||
// null sorts last whichever way the column runs: an unranked game is not
|
||||
// "rank 0", and a game with no weight is not the lightest
|
||||
function by(field, dir = 1) {
|
||||
return (a, b) => {
|
||||
const x = a[field], y = b[field];
|
||||
if (x == null && y == null) return 0;
|
||||
if (x == null) return 1;
|
||||
if (y == null) return -1;
|
||||
return (x - y) * dir;
|
||||
};
|
||||
}
|
||||
|
||||
const SORTS = {
|
||||
name: (a, b) => (a.name || "").localeCompare(b.name || "", undefined, {sensitivity: "base"}),
|
||||
year: by("year", -1),
|
||||
rank: by("rank"),
|
||||
weight: by("weight"),
|
||||
playtime: by("playtime"),
|
||||
};
|
||||
|
||||
function render() {
|
||||
const q = document.getElementById("libsearch").value.trim().toLowerCase();
|
||||
const seats = Number(document.getElementById("libplayers").value) || 0;
|
||||
let rows = GAMES.filter(matchesKind);
|
||||
if (q) rows = rows.filter(g => [
|
||||
g.name,
|
||||
(g.designers || []).join(" "),
|
||||
(g.mechanics || []).join(" "),
|
||||
(g.categories || []).join(" "),
|
||||
g.version && g.version.name,
|
||||
].filter(Boolean).join(" ").toLowerCase().includes(q));
|
||||
if (seats) rows = rows.filter(g =>
|
||||
(g.min_players || g.max_players || 0) <= seats
|
||||
&& seats <= (g.max_players || g.min_players || 0));
|
||||
rows = [...rows].sort(SORTS[document.getElementById("libsort").value] || SORTS.name);
|
||||
document.getElementById("libcount").innerHTML =
|
||||
`<b>${rows.length}</b> of <b>${GAMES.length}</b> game(s)`
|
||||
+ (seats ? ` playable with <b>${seats}</b>` : "");
|
||||
document.getElementById("libbody").innerHTML = rows.length
|
||||
? `<div class="shelfgrid">${rows.map(gameCard).join("")}</div>`
|
||||
: `<p class="empty">${GAMES.length
|
||||
? "No games match that search."
|
||||
: `Your library appears here after <b>enrich</b> runs — full metadata for
|
||||
every cataloged game: art, player counts, playtime, and more.<br>
|
||||
Run the pipeline through <code>enrich</code> to fill these shelves.`}</p>`;
|
||||
}
|
||||
|
||||
const GATE = changeGate();
|
||||
async function refresh() {
|
||||
const games = await fetchJSON("/api/library");
|
||||
GATE(games, () => { GAMES = games; render(); });
|
||||
}
|
||||
|
||||
document.getElementById("libsearch").addEventListener("input", render);
|
||||
document.getElementById("libplayers").addEventListener("input", render);
|
||||
document.getElementById("libsort").addEventListener("change", render);
|
||||
document.querySelectorAll("[data-kind]").forEach(b => b.addEventListener("click", () => {
|
||||
KIND = b.dataset.kind;
|
||||
document.querySelectorAll("[data-kind]").forEach(o =>
|
||||
o.setAttribute("aria-pressed", String(o === b)));
|
||||
render();
|
||||
}));
|
||||
refresh().catch(err => errorBanner(err.message || err));
|
||||
pollLoop(refresh, 10000, () => showBanner(""));
|
||||
</script>
|
||||
@@ -0,0 +1,185 @@
|
||||
<h1 id="gname">Loading…</h1>
|
||||
<div class="pagebar">
|
||||
<a href="/library">← library</a>
|
||||
<span id="gcrumb"></span>
|
||||
</div>
|
||||
<div id="gbody"><p class="empty">Loading…</p></div>
|
||||
<script>
|
||||
"use strict";
|
||||
const KEY = decodeURIComponent(location.pathname.replace(/^\/library\/game\//, ""));
|
||||
|
||||
function chips(label, values) {
|
||||
if (!values || !values.length) return "";
|
||||
return `<div class="factrow"><b>${esc(label)}</b>
|
||||
<span class="chiplist">${values.map(v =>
|
||||
`<span class="cue">${esc(v)}</span>`).join("")}</span></div>`;
|
||||
}
|
||||
|
||||
function fact(label, value) {
|
||||
return value === null || value === undefined || value === ""
|
||||
? ""
|
||||
: `<div class="factrow"><b>${esc(label)}</b> ${esc(value)}</div>`;
|
||||
}
|
||||
|
||||
function players(g) {
|
||||
const lo = g.min_players, hi = g.max_players;
|
||||
if (!lo && !hi) return "";
|
||||
const range = lo === hi ? `${lo}` : `${lo ?? "?"}–${hi ?? "?"}`;
|
||||
const best = (g.best_player_counts || []).length
|
||||
? ` (best at ${esc(g.best_player_counts.join(", "))})`
|
||||
: "";
|
||||
return `${range} players${best}`;
|
||||
}
|
||||
|
||||
function playtime(g) {
|
||||
if (!g.playtime && !g.min_playtime) return "";
|
||||
if (g.min_playtime && g.max_playtime && g.min_playtime !== g.max_playtime)
|
||||
return `${g.min_playtime}–${g.max_playtime} min`;
|
||||
return `${g.playtime || g.min_playtime} min`;
|
||||
}
|
||||
|
||||
function localForm(g) {
|
||||
const v = (x) => (x === null || x === undefined ? "" : x);
|
||||
return `
|
||||
<h2>Your notes</h2>
|
||||
<div class="card prose">
|
||||
<p class="meta">BGG has no entry for this game, so what you type here is
|
||||
all it will ever know. Saved to <code>data/local_games.json</code>.</p>
|
||||
<form class="editform" id="localform">
|
||||
<label>Title <input name="name" value="${esc(v(g.name))}" required></label>
|
||||
<label>Year <input name="year" value="${esc(v(g.year))}" inputmode="numeric" size="6"></label>
|
||||
<label>Players from <input name="min_players" value="${esc(v(g.min_players))}" size="3"></label>
|
||||
<label>to <input name="max_players" value="${esc(v(g.max_players))}" size="3"></label>
|
||||
<label>Minutes <input name="playtime" value="${esc(v(g.playtime))}" size="5"></label>
|
||||
<label>Publishers <input name="publishers" value="${esc((g.publishers || []).join(", "))}"></label>
|
||||
<label>Designers <input name="designers" value="${esc((g.designers || []).join(", "))}"></label>
|
||||
<label class="wide">Notes
|
||||
<textarea name="description" rows="4">${esc(v(g.description))}</textarea></label>
|
||||
<span class="editactions"><button type="submit" class="primary">save</button></span>
|
||||
</form>
|
||||
</div>`;
|
||||
}
|
||||
|
||||
function wireLocal(g) {
|
||||
const form = document.getElementById("localform");
|
||||
if (form) form.addEventListener("submit", async e => {
|
||||
e.preventDefault();
|
||||
const f = Object.fromEntries(new FormData(form).entries());
|
||||
const res = await apiPost(`/api/local-game/${encodeURIComponent(KEY)}`, f);
|
||||
if (res) {
|
||||
showToast("saved — run <b>enrich</b> to fold this into the library");
|
||||
refresh();
|
||||
}
|
||||
});
|
||||
const btn = document.getElementById("artbtn");
|
||||
const file = document.getElementById("artfile");
|
||||
if (btn) btn.addEventListener("click", () => file.click());
|
||||
if (file) file.addEventListener("change", async () => {
|
||||
if (!file.files.length) return;
|
||||
const body = new FormData();
|
||||
body.append("file", file.files[0]);
|
||||
btn.disabled = true;
|
||||
btn.textContent = "uploading…";
|
||||
let res = null;
|
||||
try {
|
||||
res = await fetch(`/api/local-art/${encodeURIComponent(KEY)}`, {method: "POST", body});
|
||||
} catch (err) {
|
||||
alert("Upload failed: " + err);
|
||||
}
|
||||
btn.disabled = false;
|
||||
if (res && res.ok) {
|
||||
showToast("photo saved — run <b>enrich</b> to fold it into the library");
|
||||
refresh();
|
||||
} else if (res) {
|
||||
const detail = await res.json().then(d => d.detail).catch(() => null);
|
||||
alert("Upload failed: " + (detail ?? res.statusText));
|
||||
refresh();
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
function render(g) {
|
||||
document.getElementById("gname").textContent = g.name || "(unnamed)";
|
||||
document.title = `${g.name} · bggpipe`;
|
||||
const local = g.type === "localgame";
|
||||
const rpg = g.type === "rpgitem";
|
||||
document.getElementById("gcrumb").innerHTML = [
|
||||
g.year ? `<b>${esc(g.year)}</b>` : "",
|
||||
local ? `<span class="chip open">not on BGG · local</span>` : "",
|
||||
rpg ? `<span class="chip open">RPG · local only</span>` : "",
|
||||
g.bgg_id
|
||||
? `<a href="${bggUrl(g.bgg_id, g.type)}" target="_blank" rel="noopener">
|
||||
view on ${rpg ? "RPGGeek" : "BGG"} ↗</a>`
|
||||
: "",
|
||||
].filter(Boolean).join(" · ");
|
||||
|
||||
const art = g.image
|
||||
? `<img class="gameart" src="${esc(g.image)}" alt="box art for ${esc(g.name)}">`
|
||||
: `<div class="gameart noart">${esc((g.name || "?")[0])}</div>`;
|
||||
// an off-BGG game has no publisher art and no API to fetch any: the
|
||||
// owner's own photo is the only cover it will ever have
|
||||
const artAdd = local
|
||||
? `<button id="artbtn" class="artbtn">${g.image ? "replace" : "add"} a photo</button>
|
||||
<input id="artfile" type="file" accept=".jpg,.jpeg,.png,.heic" hidden
|
||||
aria-label="cover photo for ${esc(g.name)}">`
|
||||
: "";
|
||||
|
||||
const facts = [
|
||||
fact("players", players(g)),
|
||||
fact("playing time", playtime(g)),
|
||||
fact("ages", g.min_age ? `${g.min_age}+` : ""),
|
||||
fact("weight", g.weight ? `${g.weight.toFixed(2)} / 5` : ""),
|
||||
fact("BGG rank", g.rank || ""),
|
||||
fact("BGG rating", g.rating ? g.rating.toFixed(2) : ""),
|
||||
fact("owned by", g.users_owned ? `${g.users_owned.toLocaleString()} people` : ""),
|
||||
chips("designers", g.designers),
|
||||
chips("artists", (g.artists || []).slice(0, 8)),
|
||||
chips("publishers", (g.publishers || []).slice(0, 6)),
|
||||
chips("categories", g.categories),
|
||||
chips("mechanics", g.mechanics),
|
||||
].filter(Boolean).join("");
|
||||
|
||||
const version = g.version
|
||||
? `<h2>Your edition</h2>
|
||||
<div class="card prose">
|
||||
<div class="factrow"><b>${esc(g.version.name || "—")}</b>
|
||||
${g.version.year ? ` · ${esc(g.version.year)}` : ""}</div>
|
||||
${chips("publishers", g.version.publishers)}
|
||||
${chips("languages", g.version.languages)}
|
||||
</div>`
|
||||
: `<h2>Your edition</h2>
|
||||
<div class="card prose"><p class="meta">No specific edition recorded —
|
||||
set one from the <a href="/titles">Titles</a> page if you know it.</p></div>`;
|
||||
|
||||
const photos = (g.photos || []).length
|
||||
? `<h2>Seen on your shelves</h2>
|
||||
<div class="card prose"><p>${g.photos.map(p =>
|
||||
`<a href="/photos/view/${encodeURIComponent(p)}">${esc(p)}</a>`
|
||||
).join(" · ")}</p></div>`
|
||||
: "";
|
||||
|
||||
document.getElementById("gbody").innerHTML = `
|
||||
<div class="gamedetail">
|
||||
<div class="gameartcol">${art}${artAdd}</div>
|
||||
<div class="gamefacts">${facts}</div>
|
||||
</div>
|
||||
${local ? localForm(g) : ""}
|
||||
${version}
|
||||
${photos}
|
||||
${g.description && !local ? `<h2>About</h2>
|
||||
<div class="card prose"><p class="gdesc">${esc(g.description)}</p></div>` : ""}`;
|
||||
wireLocal(g);
|
||||
}
|
||||
|
||||
async function refresh() {
|
||||
try {
|
||||
render(await fetchJSON(`/api/library/${encodeURIComponent(KEY)}`));
|
||||
} catch (err) {
|
||||
document.getElementById("gbody").innerHTML =
|
||||
`<p class="empty">This game isn't in the library
|
||||
(${esc(err.message || err)}) — it may need <b>enrich</b> to run.
|
||||
Back to the <a href="/library">library</a>.</p>`;
|
||||
}
|
||||
}
|
||||
refresh();
|
||||
</script>
|
||||
@@ -0,0 +1,85 @@
|
||||
<h1 id="photoname">Photo</h1>
|
||||
<div class="pagebar">
|
||||
<a href="/photos">← all photos</a>
|
||||
<span id="position"></span>
|
||||
<a id="prevlink" hidden>previous</a>
|
||||
<a id="nextlink" hidden>next</a>
|
||||
<a id="rawlink" target="_blank">open full size</a>
|
||||
<span class="keyhelp"><kbd>←</kbd>/<kbd>→</kbd> move between photos</span>
|
||||
</div>
|
||||
<div id="photobody"><p class="empty">Loading…</p></div>
|
||||
<script>
|
||||
"use strict";
|
||||
const NAME = decodeURIComponent(location.pathname.split("/").pop());
|
||||
document.getElementById("photoname").textContent = NAME;
|
||||
document.getElementById("rawlink").href = `/photos/${encodeURIComponent(NAME)}`;
|
||||
document.title = `${NAME} · bggpipe`;
|
||||
|
||||
function render(state, photos) {
|
||||
const info = photos.find(p => p.name === NAME);
|
||||
const body = document.getElementById("photobody");
|
||||
if (!info) {
|
||||
body.innerHTML = `<p class="empty">No photo named <b>${esc(NAME)}</b> is on file —
|
||||
back to <a href="/photos">all photos</a>.</p>`;
|
||||
return;
|
||||
}
|
||||
|
||||
const ix = photos.findIndex(p => p.name === NAME);
|
||||
document.getElementById("position").innerHTML =
|
||||
`<b>${ix + 1}</b> of <b>${photos.length}</b>`;
|
||||
const prev = photos[ix - 1], next = photos[ix + 1];
|
||||
const prevEl = document.getElementById("prevlink");
|
||||
const nextEl = document.getElementById("nextlink");
|
||||
prevEl.hidden = !prev;
|
||||
nextEl.hidden = !next;
|
||||
if (prev) prevEl.href = `/photos/view/${encodeURIComponent(prev.name)}`;
|
||||
if (next) nextEl.href = `/photos/view/${encodeURIComponent(next.name)}`;
|
||||
|
||||
const titles = state.catalog.filter(c => c.photos.includes(NAME));
|
||||
const tickets = state.unidentified.filter(s => s.photo === NAME);
|
||||
|
||||
let html = `
|
||||
<a href="/photos/${encodeURIComponent(NAME)}" target="_blank">
|
||||
<img class="fullshot" src="/photos/${encodeURIComponent(NAME)}"
|
||||
alt="shelf photo ${esc(NAME)}"></a>`;
|
||||
|
||||
html += `<h2>Titles read from this photo <span class="count">— ${titles.length}</span></h2>`;
|
||||
html += titles.length
|
||||
? `<div class="catalog"><table>` + titles.map(c => `
|
||||
<tr>
|
||||
<td class="t">${esc(c.title_raw)}</td>
|
||||
<td>${statusChip(c)}</td>
|
||||
<td class="meta">${metaLine(c)}</td>
|
||||
</tr>`).join("") + `</table></div>`
|
||||
: `<p class="empty">${info.extracted
|
||||
? "No titles were read from this photo."
|
||||
: "Not extracted yet — run <b>extract</b> from the <a href='/'>pipeline</a>."}</p>`;
|
||||
|
||||
if (tickets.length) {
|
||||
html += `<h2>Reshoot tickets <span class="count">— boxes seen here but not identified</span></h2>`;
|
||||
html += tickets.map(s => ticketCard(s, {showPhoto: false})).join("");
|
||||
}
|
||||
body.innerHTML = html;
|
||||
wireDismiss(body, () => refresh().catch(() => {}));
|
||||
}
|
||||
|
||||
const GATE = changeGate();
|
||||
async function refresh() {
|
||||
const [state, photos] = await Promise.all([
|
||||
fetchJSON("/api/state"),
|
||||
fetchJSON("/api/photos-list"),
|
||||
]);
|
||||
GATE([state.catalog, state.unidentified, photos], () => render(state, photos));
|
||||
}
|
||||
|
||||
document.addEventListener("keydown", e => {
|
||||
if (e.target.tagName === "INPUT") return;
|
||||
if (e.key === "ArrowLeft" && !document.getElementById("prevlink").hidden)
|
||||
location.href = document.getElementById("prevlink").href;
|
||||
if (e.key === "ArrowRight" && !document.getElementById("nextlink").hidden)
|
||||
location.href = document.getElementById("nextlink").href;
|
||||
});
|
||||
|
||||
refresh().catch(err => errorBanner(err.message || err));
|
||||
pollLoop(refresh, 5000, () => showBanner(""));
|
||||
</script>
|
||||
@@ -0,0 +1,104 @@
|
||||
<h1>Photos</h1>
|
||||
<button id="dropzone" type="button">drop shelf photos here, or click to choose</button>
|
||||
<input id="filepick" type="file" accept=".jpg,.jpeg,.png,.heic" multiple hidden
|
||||
aria-label="choose shelf photos">
|
||||
|
||||
<h2 id="reshoot">Reshoot <span class="count">— boxes seen but not identified; nothing blocks on these</span></h2>
|
||||
<div id="tickets"><p class="empty">No open reshoot tickets.</p></div>
|
||||
|
||||
<h2 id="gallery">Shelf photos <span class="count" id="gallerycount"></span></h2>
|
||||
<div class="gallery" id="shots"></div>
|
||||
<script>
|
||||
"use strict";
|
||||
|
||||
function render(state, photos) {
|
||||
const tickets = document.getElementById("tickets");
|
||||
tickets.innerHTML = state.unidentified.length
|
||||
? state.unidentified.map(s => ticketCard(s)).join("")
|
||||
: `<p class="empty">No open reshoot tickets.</p>`;
|
||||
wireDismiss(tickets, () => refresh().catch(() => {}));
|
||||
|
||||
document.getElementById("gallerycount").textContent = `— ${photos.length} on file`;
|
||||
document.getElementById("shots").innerHTML = photos.map(p => `
|
||||
<figure class="shot">
|
||||
<a href="/photos/view/${encodeURIComponent(p.name)}" aria-label="open ${esc(p.name)} details">
|
||||
<img src="/photos/${encodeURIComponent(p.name)}" alt="shelf photo ${esc(p.name)}" loading="lazy"></a>
|
||||
<figcaption class="meta">${esc(p.name)}<br>
|
||||
${p.extracted
|
||||
? `<b>${p.titles}</b> title(s)${p.unidentified ? ` · ${p.unidentified} unidentified` : ""}`
|
||||
: `not extracted yet — run <b>extract</b>`}
|
||||
</figcaption>
|
||||
</figure>`).join("");
|
||||
}
|
||||
|
||||
const GATE = changeGate(); // rebuilding innerHTML re-renders every <img>
|
||||
async function refresh() {
|
||||
const [state, photos] = await Promise.all([
|
||||
fetchJSON("/api/state"),
|
||||
fetchJSON("/api/photos-list"),
|
||||
]);
|
||||
GATE([state.unidentified, state.warnings, photos], () => render(state, photos));
|
||||
}
|
||||
|
||||
const zone = document.getElementById("dropzone");
|
||||
const pick = document.getElementById("filepick");
|
||||
zone.addEventListener("click", () => pick.click());
|
||||
zone.addEventListener("dragover", e => { e.preventDefault(); zone.classList.add("hot"); });
|
||||
zone.addEventListener("dragleave", () => zone.classList.remove("hot"));
|
||||
zone.addEventListener("drop", e => {
|
||||
e.preventDefault(); zone.classList.remove("hot");
|
||||
sendPhotos(e.dataTransfer.files);
|
||||
});
|
||||
pick.addEventListener("change", () => sendPhotos(pick.files));
|
||||
|
||||
const ZONE_IDLE = "drop shelf photos here, or click to choose";
|
||||
let UPLOADING = false;
|
||||
|
||||
async function sendPhotos(files) {
|
||||
if (!files.length || UPLOADING) return;
|
||||
const form = new FormData();
|
||||
for (const f of files) form.append("files", f);
|
||||
const mb = ([...files].reduce((total, f) => total + f.size, 0) / 1048576).toFixed(1);
|
||||
UPLOADING = true;
|
||||
zone.disabled = true;
|
||||
zone.setAttribute("aria-busy", "true");
|
||||
zone.classList.remove("ok");
|
||||
zone.textContent = `uploading ${files.length} photo(s) — ${mb} MB… keep this page open`;
|
||||
let res = null;
|
||||
try {
|
||||
res = await fetch("/api/photos", {method: "POST", body: form});
|
||||
} catch (err) {
|
||||
alert("Upload failed (no response from the server): " + err);
|
||||
}
|
||||
UPLOADING = false;
|
||||
zone.disabled = false;
|
||||
zone.removeAttribute("aria-busy");
|
||||
pick.value = ""; // re-picking the same photo must fire change again
|
||||
if (!res) { zone.textContent = ZONE_IDLE; return; }
|
||||
if (!res.ok) {
|
||||
const detail = await res.json().then(d => d.detail).catch(() => null);
|
||||
zone.textContent = ZONE_IDLE;
|
||||
alert("Upload failed: " + (detail ?? res.statusText));
|
||||
return;
|
||||
}
|
||||
const saved = await res.json().then(d => d.saved || []).catch(() => []);
|
||||
zone.classList.add("ok");
|
||||
zone.textContent = `✓ saved ${saved.join(", ")} — add another?`;
|
||||
setTimeout(() => {
|
||||
if (!UPLOADING && zone.classList.contains("ok")) {
|
||||
zone.classList.remove("ok");
|
||||
zone.textContent = ZONE_IDLE;
|
||||
}
|
||||
}, 6000);
|
||||
GATE.reset(); // the new photo must appear even if nothing else changed
|
||||
refresh().catch(() => {}); // the next poll self-heals a refresh hiccup
|
||||
}
|
||||
|
||||
// leaving mid-upload silently loses the photo — make the browser ask
|
||||
window.addEventListener("beforeunload", e => {
|
||||
if (UPLOADING) e.preventDefault();
|
||||
});
|
||||
|
||||
refresh().catch(err => errorBanner(err.message || err));
|
||||
pollLoop(refresh, 5000, () => showBanner(""));
|
||||
</script>
|
||||
@@ -0,0 +1,110 @@
|
||||
<h1>Pipeline</h1>
|
||||
<div class="stages" id="stages"></div>
|
||||
|
||||
<h2 id="activity">Activity</h2>
|
||||
<div id="jobstate" aria-live="polite">idle</div>
|
||||
<div id="joblog" role="log" aria-label="stage output">(stage output appears here)</div>
|
||||
<script>
|
||||
"use strict";
|
||||
let P = null;
|
||||
let RUNNING = false;
|
||||
|
||||
async function runStage(stage, body) {
|
||||
const res = await apiPost(`/api/run/${stage}`, body);
|
||||
if (res) refresh().catch(() => {}); // the poll self-heals a hiccup
|
||||
}
|
||||
|
||||
function stageCard(num, name, facts, actions) {
|
||||
return `<section class="stage">
|
||||
<div class="top"><span class="num">${num}</span><span class="name">${name}</span></div>
|
||||
<div class="facts">${facts}</div>
|
||||
<div class="act">${actions}</div>
|
||||
</section>`;
|
||||
}
|
||||
|
||||
function runBtn(stage, label) {
|
||||
return `<button class="primary" data-run="${stage}" ${RUNNING ? "disabled" : ""}>${esc(label ?? "Run")}</button>`;
|
||||
}
|
||||
|
||||
function render() {
|
||||
const m = P.matches, log = P.upload_log;
|
||||
RUNNING = P.job.status === "running"; // buttons below depend on this
|
||||
|
||||
const banners = [];
|
||||
if (P.stub_data) banners.push(
|
||||
`<div class="banner warn">Stub fixtures active: resolved ids are placeholders and the
|
||||
real upload stays locked until real BGG data replaces them.</div>`);
|
||||
const missing = Object.entries(P.env).filter(([, ok]) => !ok).map(([k]) => k);
|
||||
if (missing.length) banners.push(
|
||||
`<div class="banner warn">Credentials not loaded in this server's environment:
|
||||
${missing.map(esc).join(", ")} — run <code>bggpipe init</code> or load .env, then restart.</div>`);
|
||||
showBanner(banners.join(""));
|
||||
|
||||
const uploadFacts = `<b>${P.to_add}</b> to add · <b>${P.to_update}</b> version updates
|
||||
${log.added || log.added_no_version ? `· <b>${(log.added ?? 0) + (log.added_no_version ?? 0)}</b> added` : ""}
|
||||
${P.upload_failed ? `· <b>${P.upload_failed}</b> failed` : ""}
|
||||
· <a href="/queue">inspect the queue</a>
|
||||
${!P.to_add && !P.to_update && P.queued_total
|
||||
? `<br><span class="meta">${P.queued_total} queued row(s) already applied —
|
||||
the next <b>diff</b> clears them (BGG's collection export can lag the
|
||||
site by a while)</span>`
|
||||
: ""}`;
|
||||
|
||||
document.getElementById("stages").innerHTML = [
|
||||
stageCard(1, "extract", `read titles off <b>${P.photos}</b> <a href="/photos">photo(s)</a> — <b>${P.titles}</b> so far`
|
||||
+ (P.titles > 0
|
||||
? P.shaky_reads > 0
|
||||
? ` · <b>${P.shaky_reads}</b> shaky read(s) worth a <a href="/titles">proofread</a> before resolving`
|
||||
: ` · <a href="/titles">proofread the titles</a> before resolving`
|
||||
: ""), runBtn("extract")),
|
||||
stageCard(2, "resolve", `match titles to BGG ids — <b>${(m.auto ?? 0)}</b> auto · <b>${m.ambiguous ?? 0}</b> ambiguous · <b>${m.unmatched ?? 0}</b> unmatched`, runBtn("resolve")),
|
||||
stageCard(3, "review", `<b>${P.pending_review}</b> item(s) waiting for your call`, `<a class="linkbtn" href="/review">Open review</a>`),
|
||||
stageCard(4, "diff", `compare against your BGG collection`, runBtn("diff")),
|
||||
stageCard(5, "upload", uploadFacts,
|
||||
`${runBtn("upload", "Dry run")}
|
||||
<button class="danger" data-upload-real ${RUNNING || P.stub_data ? "disabled" : ""}>Upload</button>
|
||||
<label>limit <input type="number" id="uplimit" min="1" placeholder="all"></label>
|
||||
${P.upload_failed
|
||||
? `<label title="failed jobs are skipped on normal runs so a broken one can't loop">
|
||||
<input type="checkbox" id="upretry"> retry ${P.upload_failed} failed</label>`
|
||||
: ""}`),
|
||||
stageCard(6, "enrich", `<b>${P.games}</b> game(s) in the <a href="/library">library</a>`, runBtn("enrich")),
|
||||
].join("");
|
||||
|
||||
document.querySelectorAll("[data-run]").forEach(b =>
|
||||
b.addEventListener("click", () => runStage(b.dataset.run,
|
||||
b.dataset.run === "upload" ? {dry_run: true, ...uploadOpts()} : {})));
|
||||
const uploadOpts = () => ({
|
||||
limit: Number(document.getElementById("uplimit").value) || null,
|
||||
retry_failed: !!document.getElementById("upretry")?.checked,
|
||||
});
|
||||
const real = document.querySelector("[data-upload-real]");
|
||||
if (real) real.addEventListener("click", () => {
|
||||
const {limit, retry_failed} = uploadOpts();
|
||||
if (!confirm(`Really add games to your live BGG collection${limit ? ` (limit ${limit})` : ""}` +
|
||||
`${retry_failed ? ", retrying previously failed jobs" : ""}? A browser window will open.`)) return;
|
||||
runStage("upload", {dry_run: false, limit, retry_failed});
|
||||
});
|
||||
|
||||
const job = P.job;
|
||||
const state = document.getElementById("jobstate");
|
||||
if (job.status === "idle") state.textContent = "idle";
|
||||
// s- prefix: a bare status class collides with page classes (.done is
|
||||
// the review celebration card)
|
||||
else state.innerHTML = `<span class="s-${esc(job.status)}">${esc(job.stage)}: ${esc(job.status)}</span>` +
|
||||
(job.error ? ` — ${esc(job.error)}` : "");
|
||||
const logEl = document.getElementById("joblog");
|
||||
const atBottom = logEl.scrollTop + logEl.clientHeight >= logEl.scrollHeight - 4;
|
||||
logEl.textContent = job.log.length ? job.log.join("\n") : "(stage output appears here)";
|
||||
if (atBottom) logEl.scrollTop = logEl.scrollHeight;
|
||||
}
|
||||
|
||||
const GATE = changeGate();
|
||||
async function refresh() {
|
||||
const p = await fetchJSON("/api/pipeline");
|
||||
GATE(p, () => { P = p; render(); });
|
||||
}
|
||||
|
||||
refresh().catch(err => errorBanner(err.message || err));
|
||||
pollLoop(refresh, 2000, () => render());
|
||||
</script>
|
||||
@@ -0,0 +1,80 @@
|
||||
<h1>Upload queue</h1>
|
||||
<p class="pagebar">Exactly what the upload stage will do, and what it has already done —
|
||||
inspect here before any real run.</p>
|
||||
<div id="queuebody"></div>
|
||||
<script>
|
||||
"use strict";
|
||||
|
||||
function table(headers, rows) {
|
||||
return `<div class="ledger"><table>
|
||||
<tr>${headers.map(h => `<th scope="col">${esc(h)}</th>`).join("")}</tr>
|
||||
${rows.join("")}</table></div>`;
|
||||
}
|
||||
|
||||
// a queue row's state comes from the upload log: the CSVs are diff-time
|
||||
// snapshots and never shrink as work completes
|
||||
function stateChip(r) {
|
||||
if (r.stale) return `<span class="chip no" title="${esc(r.stale)}">retired</span>`;
|
||||
if (r.state === "done" && r.last_status === "added_no_version")
|
||||
return `<span class="chip ok" title="the add saved without an edition — set it on BGG by hand if it matters">done · no version</span>`;
|
||||
if (r.state === "done") return `<span class="chip ok">done</span>`;
|
||||
if (r.state === "failed") return `<span class="chip no">failed</span>`;
|
||||
return `<span class="chip open">pending</span>`;
|
||||
}
|
||||
|
||||
function tally(rows) {
|
||||
const n = s => rows.filter(r => !r.stale && r.state === s).length;
|
||||
const parts = [`<b>${n("")}</b> pending`];
|
||||
if (n("done")) parts.push(`<b>${n("done")}</b> done`);
|
||||
if (n("failed")) parts.push(`<b>${n("failed")}</b> failed`);
|
||||
const retired = rows.filter(r => r.stale).length;
|
||||
if (retired) parts.push(`<b>${retired}</b> retired by review`);
|
||||
return parts.join(" · ");
|
||||
}
|
||||
|
||||
function render(q) {
|
||||
let html = "";
|
||||
|
||||
html += `<h2>To add <span class="count">— ${tally(q.to_add)}</span></h2>`;
|
||||
html += q.to_add.length
|
||||
? table(["game", "version", "seen in", ""], q.to_add.map(r => `
|
||||
<tr><td class="t">${esc(r.bgg_name)} <span class="meta">· ${esc(r.bgg_id)}</span></td>
|
||||
<td>${r.version_name ? esc(r.version_name) : `<span class="meta">no version</span>`}</td>
|
||||
<td class="meta">${esc((r.source_photos ?? "").split(";").join(", "))}</td>
|
||||
<td>${stateChip(r)}</td></tr>`))
|
||||
: `<p class="empty">Nothing queued — run <b>diff</b> from the <a href="/">pipeline</a> first.</p>`;
|
||||
|
||||
html += `<h2>Version updates <span class="count">— ${tally(q.to_update)}</span></h2>`;
|
||||
html += q.to_update.length
|
||||
? table(["game", "version to set", "collection id", ""], q.to_update.map(r => `
|
||||
<tr><td class="t">${esc(r.bgg_name)} <span class="meta">· ${esc(r.bgg_id)}</span></td>
|
||||
<td>${esc(r.version_name)}</td>
|
||||
<td class="meta">${esc(r.collid)}</td>
|
||||
<td>${stateChip(r)}</td></tr>`))
|
||||
: `<p class="empty">No version updates pending.</p>`;
|
||||
if (q.to_add.concat(q.to_update).some(r => r.state === "done"))
|
||||
html += `<p class="empty">Finished jobs stay listed until the next <b>diff</b>
|
||||
rebuilds the queue — the log below is the permanent record.</p>`;
|
||||
|
||||
html += `<h2>Upload log <span class="count">— every attempt ever made (${q.log.length})</span></h2>`;
|
||||
html += q.log.length
|
||||
? table(["when", "action", "game", "status", "note"], q.log.slice().reverse().map(r => `
|
||||
<tr><td class="meta">${esc(r.timestamp)}</td>
|
||||
<td class="meta">${esc(r.action)}</td>
|
||||
<td class="t">${esc(r.name)}</td>
|
||||
<td>${esc(r.status)}</td>
|
||||
<td class="meta">${esc(r.error)}</td></tr>`))
|
||||
: `<p class="empty">No uploads attempted yet.</p>`;
|
||||
|
||||
document.getElementById("queuebody").innerHTML = html;
|
||||
}
|
||||
|
||||
const GATE = changeGate();
|
||||
async function refresh() {
|
||||
const q = await fetchJSON("/api/queue");
|
||||
GATE(q, () => render(q));
|
||||
}
|
||||
|
||||
refresh().catch(err => errorBanner(err.message || err));
|
||||
pollLoop(refresh, 5000, () => showBanner(""));
|
||||
</script>
|
||||
@@ -0,0 +1,324 @@
|
||||
<h1>Review</h1>
|
||||
<div class="pagebar">
|
||||
<span id="sectionlinks"></span>
|
||||
<span class="keyhelp">
|
||||
<kbd>j</kbd>/<kbd>k</kbd> move · <kbd>1</kbd>–<kbd>9</kbd> pick ·
|
||||
<kbd>r</kbd> reject · <kbd>l</kbd> keep local · <kbd>m</kbd> manual id · <kbd>u</kbd> unknown ·
|
||||
<kbd>v</kbd> veto merge
|
||||
</span>
|
||||
</div>
|
||||
<div id="cards"></div>
|
||||
<script>
|
||||
"use strict";
|
||||
let STATE = null;
|
||||
let ACTIVE = 0;
|
||||
|
||||
async function refresh() {
|
||||
STATE = await fetchJSON("/api/state");
|
||||
render();
|
||||
}
|
||||
|
||||
async function post(url, body) {
|
||||
const res = await apiPost(url, body);
|
||||
if (!res) return;
|
||||
try {
|
||||
STATE = await res.json();
|
||||
render();
|
||||
} catch (err) {
|
||||
// the decision saved server-side; only the re-render failed
|
||||
errorBanner(`saved, but the page failed to refresh (${err.message || err}) — reload the page`);
|
||||
}
|
||||
}
|
||||
|
||||
function cueChips(cues) {
|
||||
const parts = [];
|
||||
if (cues.publisher) parts.push(`<span class="cue"><b>publisher</b> ${esc(cues.publisher)}</span>`);
|
||||
if (cues.edition) parts.push(`<span class="cue"><b>edition</b> ${esc(cues.edition)}</span>`);
|
||||
if (cues.year) parts.push(`<span class="cue"><b>year</b> ${esc(cues.year)}</span>`);
|
||||
if (cues.language) parts.push(`<span class="cue"><b>language</b> ${esc(cues.language)}</span>`);
|
||||
if (cues.art_notes) parts.push(`<span class="cue"><b>art</b> ${esc(cues.art_notes)}</span>`);
|
||||
return parts.length ? `<div class="cues">${parts.join("")}</div>` : "";
|
||||
}
|
||||
|
||||
function shots(photos) {
|
||||
if (!photos.length) return `<div class="shots"><div class="noshot">photo not on disk</div></div>`;
|
||||
return `<div class="shots">` + photos.map(p =>
|
||||
`<a href="/photos/${encodeURIComponent(p)}" target="_blank" tabindex="-1">
|
||||
<img src="/photos/${encodeURIComponent(p)}" alt="source photo ${esc(p)}"></a>`
|
||||
).join("") + `</div>`;
|
||||
}
|
||||
|
||||
function thumbHtml(c) {
|
||||
if (c.thumbnail) return `<img class="thumb" src="${esc(c.thumbnail)}" alt="">`;
|
||||
const initial = (c.name || "?").trim().charAt(0).toUpperCase();
|
||||
return `<div class="thumb ph" aria-hidden="true">${esc(initial)}</div>`;
|
||||
}
|
||||
|
||||
function matchCard(row, idx) {
|
||||
const cands = row.candidates.map((c, i) => `
|
||||
<li data-pick="${c.bgg_id}" title="press ${i + 1}" tabindex="0" role="button"
|
||||
aria-label="pick ${esc(c.name)} (${esc(c.year ?? "year unknown")})">
|
||||
<kbd>${i + 1}</kbd> ${thumbHtml(c)}
|
||||
<span><span class="cname">${esc(c.name)}</span>
|
||||
<span class="cmeta">${esc(c.year ?? "—")} · ${esc(c.type || "?")}
|
||||
· rank ${esc(c.rank ?? "—")} · owned ${esc(c.owned ?? "—")}
|
||||
· <a href="${bggUrl(c.bgg_id, c.type)}" target="_blank" rel="noopener"
|
||||
title="check this candidate before picking">view ↗</a></span></span>
|
||||
</li>`).join("");
|
||||
return `
|
||||
<section class="card actionable" data-kind="match" data-rowix="${row.row_ix}"
|
||||
data-title="${esc(row.title_raw)}" data-photos="${esc(row.source_photos)}">
|
||||
${shots(row.photos)}
|
||||
<div class="body">
|
||||
<p class="title">${esc(row.title_raw)}</p>
|
||||
<p class="status">${esc(row.match_status)} · ${idx + 1} of ${STATE.pending.length + STATE.versions.length} to review</p>
|
||||
${cueChips(row.cues)}
|
||||
<ol class="cands">${cands || "<li class='cmeta'>no candidates — enter a BGG id or reject</li>"}</ol>
|
||||
<div class="rowactions">
|
||||
<span><kbd>m</kbd> <input type="text" inputmode="numeric" placeholder="BGG id, then ⏎"
|
||||
aria-label="manual BGG id"></span>
|
||||
<span class="research">
|
||||
<input type="text" class="rq" value="${esc(row.title_raw)}"
|
||||
aria-label="search text for ${esc(row.title_raw)}">
|
||||
<button class="dosearch" data-types="">search BGG</button>
|
||||
<button class="dosearch" data-types="rpgitem"
|
||||
title="BGG's board-game entries hide same-named RPGs from the automatic search">
|
||||
search RPGGeek</button>
|
||||
</span>
|
||||
<button class="reject" title="press r"><kbd>r</kbd> reject — not a game / bad read</button>
|
||||
<button class="golocal" title="press l"><kbd>l</kbd> not on BGG — keep locally</button>
|
||||
</div>
|
||||
</div>
|
||||
</section>`;
|
||||
}
|
||||
|
||||
function versionCard(row, idx) {
|
||||
const cands = row.candidates.map((v, i) => `
|
||||
<li data-pickver="${v.version_id}" title="press ${i + 1}" tabindex="0" role="button"
|
||||
aria-label="pick ${esc(v.name)} (${esc(v.year ?? "year unknown")})">
|
||||
<kbd>${i + 1}</kbd>
|
||||
<span><span class="cname">${esc(v.name)}</span>
|
||||
<span class="cmeta">${esc(v.year ?? "—")} · ${esc((v.publishers || []).join(", "))}
|
||||
· ${esc((v.languages || []).join(", "))} · score ${esc(v.score ?? "—")}</span></span>
|
||||
</li>`).join("");
|
||||
return `
|
||||
<section class="card actionable" data-kind="version" data-rowix="${row.row_ix}"
|
||||
data-title="${esc(row.title_raw)}" data-photos="${esc(row.source_photos)}">
|
||||
${shots(row.photos || [])}
|
||||
<div class="body">
|
||||
<p class="title">${esc(row.bgg_name || row.title_raw)}
|
||||
${row.bgg_id ? `<a class="cmeta" href="${bggUrl(row.bgg_id, row.type)}/versions"
|
||||
target="_blank" rel="noopener"
|
||||
title="BGG's own list of every printing">versions on BGG ↗</a>` : ""}</p>
|
||||
<p class="status">which edition${row.photos && row.photos.length
|
||||
? ` is the copy in ${esc(row.photos.join(", "))}`
|
||||
: ""}? (skippable — <kbd>u</kbd> records "unknown", or just move on)</p>
|
||||
<ol class="cands">${cands}</ol>
|
||||
<div class="rowactions">
|
||||
<button class="unknown" title="press u"><kbd>u</kbd> can't tell — leave version unset</button>
|
||||
<button class="allversions"
|
||||
title="the cues shortlisted these — fetch every published printing instead">
|
||||
list every printing</button>
|
||||
<button class="wronggame"
|
||||
title="none of these printings is your box: BGG may file it as a SEPARATE game — re-match it">
|
||||
wrong game — re-match</button>
|
||||
</div>
|
||||
</div>
|
||||
</section>`;
|
||||
}
|
||||
|
||||
function render() {
|
||||
const m = document.getElementById("cards");
|
||||
const s = STATE;
|
||||
showBanner((s.warnings || []).length
|
||||
? `<div class="banner warn">${s.warnings.map(esc).join("<br>")}</div>` : "");
|
||||
|
||||
const link = (n, label, anchor) => n ? `<a href="#${anchor}"><b>${n}</b> ${label}</a>` : "";
|
||||
document.getElementById("sectionlinks").innerHTML = [
|
||||
link(s.pending.length, "matches", "matches"),
|
||||
link(s.versions.length, "editions", "editions"),
|
||||
link(s.merges.length, "merged", "merges"),
|
||||
s.summary.unresolved ? `<span><b>${s.summary.unresolved}</b> awaiting resolve</span>` : "",
|
||||
`<span>${s.decisions} decided this sitting</span>`,
|
||||
].filter(Boolean).join(" ");
|
||||
|
||||
let html = "";
|
||||
if (!s.pending.length && !s.versions.length) {
|
||||
const waiting = s.summary.unresolved;
|
||||
html += `
|
||||
<div class="done">
|
||||
<h2>${waiting
|
||||
? "Resolved set fully reviewed"
|
||||
: "All reviewed — every title is diff-ready"}</h2>
|
||||
<div class="nums">
|
||||
<div>${s.summary.extracted}<span>extracted</span></div>
|
||||
<div>${s.summary.recognized}<span>recognized</span></div>
|
||||
<div>${s.summary.version_updates}<span>with versions</span></div>
|
||||
<div>${s.summary.rejected}<span>rejected</span></div>
|
||||
</div>
|
||||
${waiting
|
||||
? s.summary.token_present
|
||||
? `<p class="waiting">${waiting} title(s) are extracted but not yet matched
|
||||
to BGG.</p>
|
||||
<p class="next">Run <b>resolve</b> from the <a href="/">pipeline</a> to match them.</p>`
|
||||
: `<p class="waiting">${waiting} title(s) are extracted but not yet matched
|
||||
to BGG — they're waiting on your API token
|
||||
(<a href="https://boardgamegeek.com/applications" target="_blank"
|
||||
rel="noopener">boardgamegeek.com/applications</a>).</p>
|
||||
<p class="next">When it arrives, run <b>resolve</b> from the <a href="/">pipeline</a>.</p>`
|
||||
: `<p class="next">Next: run <b>diff</b> from the <a href="/">pipeline</a>, then check the <a href="/queue">queue</a>.</p>`}
|
||||
</div>`;
|
||||
}
|
||||
// csv order is extraction order — alphabetize so a long session has a
|
||||
// findable, predictable shape (stable within a payload, so the keyboard
|
||||
// cursor stays put between polls)
|
||||
const byTitle = (a, b) =>
|
||||
a.title_raw.localeCompare(b.title_raw, undefined, { sensitivity: "base" });
|
||||
if (s.pending.length) {
|
||||
html += `<h2 id="matches">Matches <span class="count">— pick the game each photo shows</span></h2>`;
|
||||
html += [...s.pending].sort(byTitle).map(matchCard).join("");
|
||||
}
|
||||
if (s.versions.length) {
|
||||
html += `<h2 id="editions">Editions <span class="count">— optional pass, never blocks uploads</span></h2>`;
|
||||
html += [...s.versions].sort(byTitle).map(versionCard).join("");
|
||||
}
|
||||
if (s.merges.length) {
|
||||
html += `<h2 id="merges">Merges <span class="count">— duplicate reads folded into one game; veto if wrong</span></h2>`;
|
||||
html += [...s.merges].sort(byTitle).map(mg => `
|
||||
<section class="card merge actionable" data-kind="merge"
|
||||
data-rowix="${mg.row_ix}"
|
||||
data-title="${esc(mg.title_raw)}" data-photos="${esc(mg.source_photos)}">
|
||||
<div class="body">
|
||||
<span class="cname">${esc(mg.title_raw)}</span>
|
||||
<span class="arrow">merged into</span>
|
||||
<span class="cname">${esc(mg.merged_into)}</span>
|
||||
<span class="cmeta">${esc(mg.bgg_name)} · ${esc(mg.bgg_id)}</span>
|
||||
<button class="veto" title="press v"><kbd>v</kbd> veto — these are different games</button>
|
||||
</div>
|
||||
</section>`).join("");
|
||||
}
|
||||
m.innerHTML = html;
|
||||
|
||||
const cards = actionables();
|
||||
if (ACTIVE >= cards.length) ACTIVE = Math.max(0, cards.length - 1);
|
||||
highlight();
|
||||
|
||||
// an outbound "view ↗" inside a pick row must not also cast the vote
|
||||
m.querySelectorAll("[data-pick] a, [data-pickver] a").forEach(a =>
|
||||
a.onclick = e => e.stopPropagation());
|
||||
m.querySelectorAll("[data-pick]").forEach(li => li.onclick = () => {
|
||||
const card = li.closest(".card");
|
||||
decide(card, "pick", Number(li.dataset.pick));
|
||||
});
|
||||
m.querySelectorAll("[data-pickver]").forEach(li => li.onclick = () => {
|
||||
const card = li.closest(".card");
|
||||
version(card, "pick", Number(li.dataset.pickver));
|
||||
});
|
||||
m.querySelectorAll(".reject").forEach(b => b.onclick = () =>
|
||||
decide(b.closest(".card"), "reject"));
|
||||
m.querySelectorAll(".golocal").forEach(b => b.onclick = () =>
|
||||
decide(b.closest(".card"), "local"));
|
||||
m.querySelectorAll(".dosearch").forEach(b => b.onclick = () => {
|
||||
const card = b.closest(".card");
|
||||
post("/api/research", {
|
||||
title_raw: card.dataset.title, source_photos: card.dataset.photos,
|
||||
row_ix: rowIx(card), query: card.querySelector(".rq").value,
|
||||
types: b.dataset.types || null,
|
||||
});
|
||||
});
|
||||
m.querySelectorAll(".wronggame").forEach(b => b.onclick = () =>
|
||||
post("/api/reopen-match", {
|
||||
title_raw: b.closest(".card").dataset.title,
|
||||
source_photos: b.closest(".card").dataset.photos,
|
||||
row_ix: rowIx(b.closest(".card")),
|
||||
}));
|
||||
m.querySelectorAll(".allversions").forEach(b => b.onclick = () =>
|
||||
post("/api/open-versions", {
|
||||
title_raw: b.closest(".card").dataset.title,
|
||||
source_photos: b.closest(".card").dataset.photos,
|
||||
row_ix: rowIx(b.closest(".card")),
|
||||
}));
|
||||
m.querySelectorAll(".unknown").forEach(b => b.onclick = () =>
|
||||
version(b.closest(".card"), "unknown"));
|
||||
m.querySelectorAll(".veto").forEach(b => b.onclick = () =>
|
||||
vetoMerge(b.closest(".card")));
|
||||
m.querySelectorAll(".rowactions input").forEach(inp => {
|
||||
inp.onkeydown = e => {
|
||||
if (e.key === "Enter" && inp.value.trim().match(/^\d+$/)) {
|
||||
decide(inp.closest(".card"), "manual", Number(inp.value.trim()));
|
||||
}
|
||||
if (e.key === "Escape") inp.blur();
|
||||
e.stopPropagation();
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
const actionables = () => [...document.querySelectorAll(".actionable")];
|
||||
|
||||
function highlight(scroll = false) {
|
||||
actionables().forEach((el, i) => el.classList.toggle("active", i === ACTIVE));
|
||||
// scroll only for KEYBOARD moves: a mouse user's cursor idles on card
|
||||
// one, and scrolling to it on every re-render yanks them to the top
|
||||
const el = actionables()[ACTIVE];
|
||||
if (el && scroll) el.scrollIntoView({block: "nearest", behavior: "auto"});
|
||||
}
|
||||
|
||||
const rowIx = card => Number(card.dataset.rowix); // every card template sets it
|
||||
const decide = (card, action, bgg_id = null) => post("/api/decision", {
|
||||
title_raw: card.dataset.title, source_photos: card.dataset.photos,
|
||||
row_ix: rowIx(card), action, bgg_id,
|
||||
});
|
||||
const version = (card, action, version_id = null) => post("/api/version", {
|
||||
title_raw: card.dataset.title, source_photos: card.dataset.photos,
|
||||
row_ix: rowIx(card), action, version_id,
|
||||
});
|
||||
const vetoMerge = card => post("/api/veto-merge", {
|
||||
title_raw: card.dataset.title, source_photos: card.dataset.photos,
|
||||
row_ix: rowIx(card),
|
||||
});
|
||||
|
||||
document.addEventListener("keydown", e => {
|
||||
if ((e.key === "Enter" || e.key === " ") && e.target.matches("[data-pick],[data-pickver]")) {
|
||||
e.preventDefault();
|
||||
e.target.click();
|
||||
return;
|
||||
}
|
||||
if (e.target.tagName === "INPUT") return;
|
||||
const cards = actionables();
|
||||
if (!cards.length) return;
|
||||
const card = cards[ACTIVE];
|
||||
const kind = card?.dataset.kind;
|
||||
if (e.key === "j" || e.key === "ArrowDown") { ACTIVE = Math.min(ACTIVE + 1, cards.length - 1); highlight(true); }
|
||||
else if (e.key === "k" || e.key === "ArrowUp") { ACTIVE = Math.max(ACTIVE - 1, 0); highlight(true); }
|
||||
else if (/^[1-9]$/.test(e.key) && kind === "match") {
|
||||
const li = card.querySelectorAll("[data-pick]")[Number(e.key) - 1];
|
||||
if (li) decide(card, "pick", Number(li.dataset.pick));
|
||||
}
|
||||
else if (/^[1-9]$/.test(e.key) && kind === "version") {
|
||||
const li = card.querySelectorAll("[data-pickver]")[Number(e.key) - 1];
|
||||
if (li) version(card, "pick", Number(li.dataset.pickver));
|
||||
}
|
||||
else if (e.key === "r" && kind === "match") decide(card, "reject");
|
||||
else if (e.key === "l" && kind === "match") decide(card, "local");
|
||||
else if (e.key === "u" && kind === "version") version(card, "unknown");
|
||||
else if (e.key === "v" && kind === "merge") vetoMerge(card);
|
||||
else if (e.key === "m" && kind === "match") { card.querySelector("input")?.focus(); e.preventDefault(); }
|
||||
});
|
||||
|
||||
refresh().catch(err => errorBanner(err.message || err));
|
||||
|
||||
// Live-follow the data files; only re-render on an actual change (keeps
|
||||
// the keyboard cursor stable) and never mid-typing. Stale poll responses
|
||||
// (answered before a decision landed) are discarded by revision.
|
||||
pollLoop(async () => {
|
||||
const fresh = await fetchJSON("/api/state");
|
||||
// discard stale responses — but only within one server lifetime: a
|
||||
// restart resets revision to 0 (boot changes), and refusing forever
|
||||
// would freeze the page
|
||||
if (STATE && fresh.boot === STATE.boot && fresh.revision < STATE.revision) return;
|
||||
if (JSON.stringify(fresh) !== JSON.stringify(STATE)) {
|
||||
STATE = fresh;
|
||||
render();
|
||||
}
|
||||
}, 3000, () => render());
|
||||
</script>
|
||||
@@ -0,0 +1,284 @@
|
||||
<h1>Titles</h1>
|
||||
<div class="pagebar"><span id="catcount"></span></div>
|
||||
<div class="filterbar">
|
||||
<input type="search" id="catsearch" placeholder="filter titles…" aria-label="filter titles">
|
||||
<button id="shakyfilter" aria-pressed="false" hidden
|
||||
title="reads the model wasn't sure of — confirm or fix each one">shaky reads</button>
|
||||
<button id="addtitle" aria-expanded="false"
|
||||
title="a game no photo shows — an expansion inside a base box, a game away from the shelves">add a game</button>
|
||||
</div>
|
||||
<form id="addform" class="editform card" hidden>
|
||||
<label>Title <input name="title" required></label>
|
||||
<label>Publisher <input name="publisher"></label>
|
||||
<label>Edition <input name="edition"></label>
|
||||
<label>Year <input name="year" inputmode="numeric" size="6"></label>
|
||||
<label>Language <input name="language"></label>
|
||||
<span class="editactions">
|
||||
<button type="submit" class="primary">add</button>
|
||||
<button type="button" id="addcancel">cancel</button>
|
||||
</span>
|
||||
<span class="edithint">joins the titles list like a photo read — resolve matches it next run</span>
|
||||
</form>
|
||||
<div id="catbody"><p class="empty">Nothing extracted yet — start on the <a href="/photos">photos page</a>.</p></div>
|
||||
<script>
|
||||
"use strict";
|
||||
let CATALOG = [];
|
||||
let EDITING = null; // lineKey of the row whose editor is open
|
||||
|
||||
function lineKey(c) { return c.title_raw + "|" + c.photos.join(";"); }
|
||||
|
||||
function editorRow(c) {
|
||||
const cue = c.cues || {};
|
||||
return `
|
||||
<tr class="editrow"><td colspan="5">
|
||||
<form class="editform" data-title="${esc(c.title_raw)}"
|
||||
data-photos="${esc(c.photos.join(";"))}" data-rowix="${c.row_ix ?? ""}">
|
||||
<label>Title <input name="title" value="${esc(c.title_raw)}" required></label>
|
||||
<label>Publisher <input name="publisher" value="${esc(cue.publisher || "")}"></label>
|
||||
<label>Edition <input name="edition" value="${esc(cue.edition || "")}"></label>
|
||||
<label>Year <input name="year" value="${esc(cue.year ?? "")}" inputmode="numeric" size="6"></label>
|
||||
<label>Language <input name="language" value="${esc(cue.language || "")}"></label>
|
||||
<span class="editactions">
|
||||
<button type="submit" class="primary">save</button>
|
||||
<button type="button" class="canceledit">cancel</button>
|
||||
${c.bgg_id && ["auto", "approved"].includes(c.status)
|
||||
? `<button type="button" class="wrongmatch danger">wrong match — re-search</button>`
|
||||
: ""}
|
||||
<button type="button" class="removetitle danger"
|
||||
title="not a game, a duplicate read of a copy already listed, or otherwise doesn't belong">
|
||||
remove this line</button>
|
||||
</span>
|
||||
<span class="edithint">saving re-queues this title for resolve with the corrected data</span>
|
||||
</form>
|
||||
</td></tr>`;
|
||||
}
|
||||
|
||||
let SHAKY_ONLY = false;
|
||||
|
||||
function render() {
|
||||
const q = document.getElementById("catsearch").value.trim().toLowerCase();
|
||||
const shakyCount = CATALOG.filter(c => c.shaky).length;
|
||||
const fbtn = document.getElementById("shakyfilter");
|
||||
fbtn.hidden = shakyCount === 0 && !SHAKY_ONLY;
|
||||
fbtn.textContent = `shaky reads — ${shakyCount}`;
|
||||
fbtn.setAttribute("aria-pressed", String(SHAKY_ONLY));
|
||||
const sorted = [...CATALOG].sort((a, b) =>
|
||||
a.title_raw.localeCompare(b.title_raw, undefined, { sensitivity: "base" }));
|
||||
const searched = q
|
||||
? sorted.filter(c => (c.title_raw + " " + c.bgg_name).toLowerCase().includes(q))
|
||||
: sorted;
|
||||
const rows = SHAKY_ONLY ? searched.filter(c => c.shaky) : searched;
|
||||
document.getElementById("catcount").innerHTML =
|
||||
`<b>${rows.length}</b> of <b>${CATALOG.length}</b> title(s)`;
|
||||
document.getElementById("catbody").innerHTML = rows.length
|
||||
? `<div class="catalog"><table>` + rows.map(c => `
|
||||
<tr>
|
||||
<td class="t">${esc(c.title_raw)}
|
||||
${c.split_copy ? `<span class="chip merged">copy</span>` : ""}
|
||||
${c.shaky ? `<span class="chip shaky"
|
||||
title="the model wasn't sure of this read — press ✓ if it's right, or edit it">shaky read</span>` : ""}</td>
|
||||
<td>${statusChip(c)}</td>
|
||||
<td class="meta">${metaLine(c)}</td>
|
||||
<td class="meta">${c.photos.length
|
||||
? c.photos.map(p =>
|
||||
`<a href="/photos/view/${encodeURIComponent(p)}">${esc(p)}</a>`
|
||||
).join(", ")
|
||||
: `<span class="chip merged">added by hand</span>`}</td>
|
||||
<td class="actions">${c.can_split
|
||||
? `<button class="split" data-title="${esc(c.title_raw)}"
|
||||
data-photos="${esc(c.photos.join(";"))}"
|
||||
data-rowix="${c.row_ix ?? ""}"
|
||||
title="one line, several boxes? make each photo its own copy">
|
||||
split into copies</button>`
|
||||
: ""}
|
||||
${c.status === "local"
|
||||
? `<button class="wrongmatch" data-title="${esc(c.title_raw)}"
|
||||
data-photos="${esc(c.photos.join(";"))}" data-rowix="${c.row_ix ?? ""}"
|
||||
title="search BGG and RPGGeek again — RPGs and expansions often are listed">
|
||||
look it up</button>`
|
||||
: ""}
|
||||
${c.version_status === "version_ambiguous"
|
||||
? `<a class="ballotlink" href="/review#editions"
|
||||
title="this copy's edition ballot is open">ballot in Review →</a>`
|
||||
: ""}
|
||||
${c.bgg_id && ["auto", "approved"].includes(c.status)
|
||||
&& ["version_unknown", "version_error", ""].includes(c.version_status || "")
|
||||
? `<button class="pickedition" data-title="${esc(c.title_raw)}"
|
||||
data-photos="${esc(c.photos.join(";"))}" data-rowix="${c.row_ix ?? ""}"
|
||||
title="you know which printing this box is — fetch the full edition list into Review">
|
||||
pick edition</button>`
|
||||
: ""}
|
||||
${c.shaky ? `<button class="confirmread" data-title="${esc(c.title_raw)}"
|
||||
data-photos="${esc(c.photos.join(";"))}"
|
||||
title="the read is correct as-is — mark it verified">✓ looks right</button>` : ""}
|
||||
<button class="edit" data-key="${esc(lineKey(c))}"
|
||||
title="fix a misread title or add cues you already know">edit</button>
|
||||
</td>
|
||||
</tr>` + (EDITING === lineKey(c) ? editorRow(c) : "")).join("") + `</table></div>`
|
||||
: `<p class="empty">${CATALOG.length
|
||||
? "No titles match that filter."
|
||||
: `Nothing extracted yet — start on the <a href="/photos">photos page</a>.`}</p>`;
|
||||
}
|
||||
|
||||
const GATE = changeGate();
|
||||
async function refresh() {
|
||||
const state = await fetchJSON("/api/state");
|
||||
if (EDITING) return; // never repaint under an open editor
|
||||
GATE(state.catalog, () => { CATALOG = state.catalog; render(); });
|
||||
}
|
||||
|
||||
document.getElementById("catbody").addEventListener("click", async e => {
|
||||
const cancel = e.target.closest("button.canceledit");
|
||||
if (cancel) { EDITING = null; GATE.reset(); render(); refresh().catch(() => {}); return; }
|
||||
const wm = e.target.closest("button.wrongmatch");
|
||||
if (wm) {
|
||||
// in the edit panel it means "wrong match"; on a local row it means
|
||||
// "look it up" — same reopen, different starting point
|
||||
const f = wm.closest("form.editform");
|
||||
const src = f ? f.dataset : wm.dataset;
|
||||
if (f && !confirm(`"${src.title}" matched the wrong game? This clears the ` +
|
||||
`match and sends it back to Review for a re-search.`)) return;
|
||||
const res = await apiPost("/api/reopen-match", {
|
||||
title_raw: src.title,
|
||||
source_photos: src.photos,
|
||||
row_ix: src.rowix === "" ? null : Number(src.rowix),
|
||||
});
|
||||
if (res) {
|
||||
EDITING = null; GATE.reset(); refresh().catch(() => {});
|
||||
showToast(`searching BGG and RPGGeek for <b>${esc(src.title)}</b> —
|
||||
<a href="/review#matches">pick a match in Review</a>`);
|
||||
}
|
||||
return;
|
||||
}
|
||||
const pe = e.target.closest("button.pickedition");
|
||||
if (pe) {
|
||||
const res = await apiPost("/api/open-versions", {
|
||||
title_raw: pe.dataset.title,
|
||||
source_photos: pe.dataset.photos,
|
||||
row_ix: pe.dataset.rowix === "" ? null : Number(pe.dataset.rowix),
|
||||
});
|
||||
if (res) {
|
||||
showToast(`edition ballot ready for <b>${esc(pe.dataset.title)}</b> —
|
||||
<a href="/review#editions">pick it in Review</a>`);
|
||||
GATE.reset(); refresh().catch(() => {});
|
||||
}
|
||||
return;
|
||||
}
|
||||
const ok = e.target.closest("button.confirmread");
|
||||
if (ok) {
|
||||
const res = await apiPost("/api/edit-title", {
|
||||
title_raw: ok.dataset.title,
|
||||
source_photos: ok.dataset.photos,
|
||||
confirm: true,
|
||||
});
|
||||
if (res) { GATE.reset(); refresh().catch(() => {}); }
|
||||
return;
|
||||
}
|
||||
const rm = e.target.closest("button.removetitle");
|
||||
if (rm) {
|
||||
const f = rm.closest("form.editform");
|
||||
if (!confirm(`Remove "${f.dataset.title}" from the titles list? ` +
|
||||
`Right for a line that shouldn't exist: not a game, or a duplicate ` +
|
||||
`read of a copy that's already its own line. ` +
|
||||
`(One line for several REAL boxes? Use split instead.) ` +
|
||||
`Re-running extract won't bring it back — the removal is saved ` +
|
||||
`in data/title_removals.json (delete its record there to undo).`)) return;
|
||||
const res = await apiPost("/api/remove-title", {
|
||||
title_raw: f.dataset.title,
|
||||
source_photos: f.dataset.photos,
|
||||
});
|
||||
if (res) { EDITING = null; GATE.reset(); refresh().catch(() => {}); }
|
||||
return;
|
||||
}
|
||||
const edit = e.target.closest("button.edit");
|
||||
if (edit) {
|
||||
EDITING = EDITING === edit.dataset.key ? null : edit.dataset.key;
|
||||
render();
|
||||
return;
|
||||
}
|
||||
const b = e.target.closest("button.split");
|
||||
if (!b) return;
|
||||
const n = b.dataset.photos.split(";").length;
|
||||
if (!confirm(`Split "${b.dataset.title}" into ${n} separate copies (one per photo)? ` +
|
||||
`Each picks its own edition afterward.`)) return;
|
||||
const res = await apiPost("/api/split", {
|
||||
title_raw: b.dataset.title,
|
||||
source_photos: b.dataset.photos,
|
||||
row_ix: b.dataset.rowix === "" ? null : Number(b.dataset.rowix),
|
||||
});
|
||||
if (res) refresh().catch(() => {});
|
||||
});
|
||||
|
||||
document.getElementById("catbody").addEventListener("submit", async e => {
|
||||
const f = e.target.closest("form.editform");
|
||||
if (!f) return;
|
||||
e.preventDefault();
|
||||
const orig = CATALOG.find(c => lineKey(c) === EDITING) || {};
|
||||
const cue = orig.cues || {};
|
||||
const v = name => f.elements[name].value;
|
||||
const body = { title_raw: f.dataset.title, source_photos: f.dataset.photos };
|
||||
if (v("title").trim() !== f.dataset.title) body.title_new = v("title").trim();
|
||||
if (v("publisher") !== (cue.publisher || "")) body.publisher = v("publisher");
|
||||
if (v("edition") !== (cue.edition || "")) body.edition = v("edition");
|
||||
if (v("year") !== String(cue.year ?? "")) body.year = v("year");
|
||||
if (v("language") !== (cue.language || "")) body.language = v("language");
|
||||
if (Object.keys(body).length <= 2) {
|
||||
// saving unchanged: on a shaky line that means "it's right as-is"
|
||||
if (orig.shaky) {
|
||||
const res = await apiPost("/api/edit-title", { ...body, confirm: true });
|
||||
if (res) { EDITING = null; GATE.reset(); refresh().catch(() => {}); return; }
|
||||
}
|
||||
EDITING = null; render(); return;
|
||||
}
|
||||
const res = await apiPost("/api/edit-title", body);
|
||||
if (res) {
|
||||
EDITING = null; GATE.reset(); refresh().catch(() => {});
|
||||
// an edit re-queues any matched row: without a resolve run the line
|
||||
// sits "awaiting resolve" — say so, and offer the run right here
|
||||
const toast = showToast(`saved — a previously matched title re-matches
|
||||
on the next resolve run. <button id="resolvenow">run resolve now</button>`);
|
||||
const btn = toast.querySelector("#resolvenow");
|
||||
if (btn) btn.addEventListener("click", async () => {
|
||||
const r = await apiPost("/api/run/resolve");
|
||||
if (r) showToast(`resolve running — progress on the
|
||||
<a href="/">Pipeline</a> page; this list updates itself`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
const addForm = document.getElementById("addform");
|
||||
const addBtn = document.getElementById("addtitle");
|
||||
addBtn.addEventListener("click", () => {
|
||||
addForm.hidden = !addForm.hidden;
|
||||
addBtn.setAttribute("aria-expanded", String(!addForm.hidden));
|
||||
if (!addForm.hidden) addForm.elements.title.focus();
|
||||
});
|
||||
document.getElementById("addcancel").addEventListener("click", () => {
|
||||
addForm.hidden = true;
|
||||
addBtn.setAttribute("aria-expanded", "false");
|
||||
addForm.reset();
|
||||
});
|
||||
addForm.addEventListener("submit", async e => {
|
||||
e.preventDefault();
|
||||
const v = name => addForm.elements[name].value;
|
||||
const res = await apiPost("/api/add-title", {
|
||||
title: v("title"), publisher: v("publisher"), edition: v("edition"),
|
||||
year: v("year"), language: v("language"),
|
||||
});
|
||||
if (res) {
|
||||
addForm.reset();
|
||||
addForm.hidden = true;
|
||||
addBtn.setAttribute("aria-expanded", "false");
|
||||
GATE.reset();
|
||||
refresh().catch(() => {});
|
||||
}
|
||||
});
|
||||
|
||||
document.getElementById("catsearch").addEventListener("input", render);
|
||||
document.getElementById("shakyfilter").addEventListener("click", () => {
|
||||
SHAKY_ONLY = !SHAKY_ONLY;
|
||||
render();
|
||||
});
|
||||
refresh().catch(err => errorBanner(err.message || err));
|
||||
pollLoop(refresh, 5000, () => showBanner(""));
|
||||
</script>
|
||||
@@ -0,0 +1,41 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title><!--TITLE--></title>
|
||||
<link rel="icon" type="image/png" href="/static/favicon.png">
|
||||
<link rel="apple-touch-icon" href="/static/apple-touch-icon.png">
|
||||
<meta name="apple-mobile-web-app-title" content="bggpipe">
|
||||
<link rel="stylesheet" href="/static/app.css">
|
||||
</head>
|
||||
<body>
|
||||
<a class="skip" href="#main">Skip to content</a>
|
||||
<div class="sidebar">
|
||||
<a class="brand" href="/">
|
||||
<img src="/static/logo.jpg" alt="">
|
||||
<span class="wordmark">bggpipe<small>shelf → BGG pipeline</small></span>
|
||||
</a>
|
||||
<button class="navtoggle" aria-expanded="false" aria-controls="primary-nav"
|
||||
aria-label="Menu"><span class="bars"></span></button>
|
||||
<nav id="primary-nav" aria-label="Primary">
|
||||
<!--NAV-->
|
||||
</nav>
|
||||
<figure class="piperbox">
|
||||
<img src="/static/logo-full.jpg"
|
||||
alt="the bggpipe piper — a bagpiper whose bag is a board game box">
|
||||
<figcaption>art by Juniper</figcaption>
|
||||
<figcaption class="legal">BoardGameGeek and BGG are trademarks of
|
||||
BoardGameGeek, LLC. bggpipe is an independent project, not
|
||||
affiliated with or endorsed by BoardGameGeek.</figcaption>
|
||||
</figure>
|
||||
</div>
|
||||
<div class="content">
|
||||
<div id="banner" role="status"></div>
|
||||
<script src="/static/app.js"></script>
|
||||
<main id="main">
|
||||
<!--PAGE-->
|
||||
</main>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,999 @@
|
||||
"""Stage 5 — upload: add games / set versions on boardgamegeek.com via Playwright.
|
||||
|
||||
BGG has no write API, so this stage drives the real website with a logged-in
|
||||
browser session. Etiquette (spec + bgg-api skill):
|
||||
- 2-4 s randomized delay between games;
|
||||
- credentials only from BGG_USERNAME / BGG_PASSWORD env vars, never disk/logs;
|
||||
- browser storage state persists locally (gitignored) so login is rare;
|
||||
- every attempt is appended to data/upload_log.csv immediately, so a killed
|
||||
run loses nothing and re-runs skip completed work;
|
||||
- refuses to touch the site while either provenance marker exists —
|
||||
data/bgg_cache/STUB_FIXTURES.marker (gitignored) or data/STUB_DATA.marker
|
||||
(committed, so fresh clones stay guarded); --dry-run still works, loudly
|
||||
labeled as synthetic.
|
||||
|
||||
Cloudflare: BGG fronts the site with a Turnstile check that blocks headless
|
||||
browsers outright (verified 2026-08-01 — headless shell never gets past
|
||||
"Just a moment..."). The stage therefore runs HEADED by default; a first
|
||||
login may need one human click on the challenge widget. BGG pages also never
|
||||
reach Playwright's networkidle (ad/analytics polling), so navigation waits on
|
||||
domcontentloaded plus explicit element waits.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import csv
|
||||
import os
|
||||
import random
|
||||
import re
|
||||
import time
|
||||
from collections import Counter
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import Protocol
|
||||
|
||||
import typer
|
||||
|
||||
from bggpipe.bgg_client import BGGAuthError, BGGClient, client_for
|
||||
from bggpipe.config import Config
|
||||
from bggpipe.fsio import atomic_write_text
|
||||
from bggpipe.models import CollectionItem, is_recognized
|
||||
from bggpipe.resolve import read_matches
|
||||
|
||||
BGG = "https://boardgamegeek.com"
|
||||
UPLOAD_LOG_COLUMNS = [
|
||||
"action",
|
||||
"bgg_id",
|
||||
"collid",
|
||||
"name",
|
||||
"version_id",
|
||||
"second_copy",
|
||||
"status",
|
||||
"timestamp",
|
||||
"error",
|
||||
]
|
||||
DONE_STATUSES = {"added", "added_no_version", "updated", "already_present"}
|
||||
MAX_VERSION_PAGES = 40
|
||||
|
||||
|
||||
_YEARISH = re.compile(r"[\s(]*\d{4}(?:-\d+)?[\s)]*")
|
||||
|
||||
|
||||
def _loose_version_text(text: str) -> str:
|
||||
"""Version names as displayed differ from the API's by punctuation,
|
||||
parens and year qualifiers — compare only the words that name it."""
|
||||
text = _YEARISH.sub(" ", text or "")
|
||||
return " ".join(re.sub(r"[^\w\s]", " ", text.casefold()).split())
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class UploadJob:
|
||||
action: str # "add" | "update"
|
||||
bgg_id: str
|
||||
name: str
|
||||
version_id: str = ""
|
||||
version_name: str = ""
|
||||
collid: str = ""
|
||||
second_copy: bool = False # game had prior copies: verify can't confirm
|
||||
|
||||
@property
|
||||
def key(self) -> tuple[str, str, str]:
|
||||
return _key(self.action, self.bgg_id, self.collid, self.version_id)
|
||||
|
||||
|
||||
def _key(
|
||||
action: str, bgg_id: str, collid: str, version_id: str
|
||||
) -> tuple[str, str, str]:
|
||||
# A second copy of the same game (different version) is a distinct add;
|
||||
# updates are keyed by the physical copy they amend. NOTE: two copies
|
||||
# with the SAME (bgg_id, version) — vetoed duplicates — share a key, so
|
||||
# build_queue counts completions per key instead of treating the key as
|
||||
# unique.
|
||||
if action == "update":
|
||||
return ("update", collid, "")
|
||||
return ("add", bgg_id, version_id)
|
||||
|
||||
|
||||
def _job_key(row: dict) -> tuple[str, str, str]:
|
||||
return _key(row["action"], row["bgg_id"], row["collid"], row["version_id"])
|
||||
|
||||
|
||||
def annotate_queue(
|
||||
queue_rows: list[dict], action: str, log_rows: list[dict]
|
||||
) -> list[dict]:
|
||||
"""Each queue row plus the outcome of its LAST upload attempt, in a
|
||||
`state` field: "" (pending), "done", or "failed". to_add.csv and
|
||||
to_update.csv are diff-time snapshots — nothing removes a row once its
|
||||
job succeeds — so a reader without the log sees finished work as
|
||||
outstanding forever."""
|
||||
last: dict[tuple[str, str, str], str] = {}
|
||||
done_count: Counter[tuple[str, str, str]] = Counter()
|
||||
for row in log_rows:
|
||||
last[_job_key(row)] = row["status"]
|
||||
if row["status"] in DONE_STATUSES:
|
||||
done_count[_job_key(row)] += 1
|
||||
# vetoed duplicate copies share a key: one success must mark ONE row
|
||||
# done, not both, or the queue reports a copy uploaded that never was
|
||||
claimed: Counter[tuple[str, str, str]] = Counter()
|
||||
out = []
|
||||
for row in queue_rows:
|
||||
key = _key(
|
||||
action,
|
||||
row.get("bgg_id", ""),
|
||||
row.get("collid", ""),
|
||||
row.get("version_id", ""),
|
||||
)
|
||||
status = last.get(key, "")
|
||||
if status in DONE_STATUSES and claimed[key] >= done_count[key]:
|
||||
status = "" # completions exhausted: this copy is still pending
|
||||
if status in DONE_STATUSES:
|
||||
claimed[key] += 1
|
||||
out.append(
|
||||
{
|
||||
**row,
|
||||
"state": "done"
|
||||
if status in DONE_STATUSES
|
||||
else "failed"
|
||||
if status == "failed"
|
||||
else "",
|
||||
"last_status": status,
|
||||
}
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
def stale_jobs(queue_rows: list[dict], match_rows: list[dict]) -> dict[str, str]:
|
||||
"""bgg_id -> why, for queued games the CURRENT matches.csv no longer
|
||||
endorses. to_add.csv is a snapshot from the last diff; a review
|
||||
decision taken afterwards (marking a game local, rejecting it, calling
|
||||
a match wrong) must not still upload."""
|
||||
if not match_rows:
|
||||
# nothing to compare against (no matches.csv, or a test harness
|
||||
# driving the queue directly): absence is not a verdict
|
||||
return {}
|
||||
live: dict[str, list[dict]] = {}
|
||||
endorsed: Counter[tuple[str, str]] = Counter()
|
||||
for row in match_rows:
|
||||
if row.get("bgg_id"):
|
||||
live.setdefault(row["bgg_id"], []).append(row)
|
||||
if is_recognized(row):
|
||||
endorsed[(row["bgg_id"], row.get("version_id", ""))] += 1
|
||||
queued: Counter[tuple[str, str]] = Counter()
|
||||
for row in queue_rows:
|
||||
queued[(row.get("bgg_id") or "", row.get("version_id", ""))] += 1
|
||||
stale = {}
|
||||
for row in queue_rows:
|
||||
bgg_id = row.get("bgg_id") or ""
|
||||
rows = live.get(bgg_id, [])
|
||||
if not rows:
|
||||
stale[bgg_id] = "no longer matched to this game in matches.csv"
|
||||
continue
|
||||
if not any(is_recognized(r) for r in rows):
|
||||
statuses = sorted({r["match_status"] for r in rows})
|
||||
stale[bgg_id] = f"now {', '.join(statuses)} in matches.csv"
|
||||
continue
|
||||
# the game survives, but does THIS copy? Rejecting one of two
|
||||
# editions must not ride along on the other's endorsement.
|
||||
pair = (bgg_id, row.get("version_id", ""))
|
||||
if queued[pair] > endorsed[pair]:
|
||||
stale[bgg_id] = (
|
||||
f"queued with version {row.get('version_id') or '(none)'} "
|
||||
"but matches.csv no longer endorses that copy — re-run diff"
|
||||
)
|
||||
return stale
|
||||
|
||||
|
||||
def outstanding_failures(
|
||||
log_rows: list[dict],
|
||||
queue_rows: list[dict] | None = None,
|
||||
match_rows: list[dict] | None = None,
|
||||
) -> int:
|
||||
"""Failures still worth retrying: the job's LATEST attempt failed AND
|
||||
it is still queued and still endorsed by matches.csv.
|
||||
|
||||
Two ways to over-count. upload_log.csv is append-only, so a successful
|
||||
retry leaves its old 'failed' line behind; and a failed job whose
|
||||
review decision has since changed (marked local, rejected) is not
|
||||
pending work at all — it will never run again, so offering to retry it
|
||||
is a lie."""
|
||||
last: dict[tuple[str, str, str], tuple[str, str]] = {}
|
||||
for row in log_rows:
|
||||
last[_job_key(row)] = (row["status"], row.get("bgg_id", ""))
|
||||
failed = [bgg_id for status, bgg_id in last.values() if status == "failed"]
|
||||
if queue_rows is None:
|
||||
return len(failed)
|
||||
live = {r.get("bgg_id", "") for r in queue_rows} - set(
|
||||
stale_jobs(queue_rows, match_rows or [])
|
||||
)
|
||||
return sum(1 for bgg_id in failed if bgg_id in live)
|
||||
|
||||
|
||||
def _read_csv(path: Path) -> list[dict]:
|
||||
if not path.exists():
|
||||
return []
|
||||
with path.open(newline="") as f:
|
||||
return list(csv.DictReader(f))
|
||||
|
||||
|
||||
def append_log_row(path: Path, row: dict) -> None:
|
||||
"""Append one attempt. One row per attempt, flushed immediately — the
|
||||
log is the resume point, so its header is created ATOMICALLY first (a
|
||||
torn header line would become DictReader's fieldnames and misparse
|
||||
every logged success on the next run)."""
|
||||
if not path.exists():
|
||||
atomic_write_text(path, ",".join(UPLOAD_LOG_COLUMNS) + "\r\n")
|
||||
with path.open("a", newline="") as f:
|
||||
writer = csv.DictWriter(f, fieldnames=UPLOAD_LOG_COLUMNS, extrasaction="ignore")
|
||||
writer.writerow(row)
|
||||
|
||||
|
||||
def build_queue(
|
||||
to_add: list[dict],
|
||||
to_update: list[dict],
|
||||
log_rows: list[dict],
|
||||
*,
|
||||
retry_failed: bool = False,
|
||||
) -> tuple[list[UploadJob], int, int]:
|
||||
"""Turn the diff outputs into pending jobs, minus work the log says is
|
||||
done. Returns (jobs, skipped_done, skipped_failed).
|
||||
|
||||
Completions are COUNTED per key, not looked up: two vetoed duplicate
|
||||
copies share a key, and one logged success must complete exactly one
|
||||
of them."""
|
||||
done_count: Counter[tuple[str, str, str]] = Counter()
|
||||
last_status: dict[tuple[str, str, str], str] = {}
|
||||
for row in log_rows:
|
||||
k = _job_key(row)
|
||||
if row["status"] in DONE_STATUSES:
|
||||
done_count[k] += 1
|
||||
last_status[k] = row["status"]
|
||||
|
||||
candidates = [
|
||||
UploadJob(
|
||||
action="add",
|
||||
bgg_id=row["bgg_id"],
|
||||
name=row["bgg_name"],
|
||||
version_id=row.get("version_id", ""),
|
||||
version_name=row.get("version_name", ""),
|
||||
second_copy=bool(row.get("second_copy")),
|
||||
)
|
||||
for row in to_add
|
||||
] + [
|
||||
UploadJob(
|
||||
action="update",
|
||||
bgg_id=row["bgg_id"],
|
||||
name=row["bgg_name"],
|
||||
version_id=row["version_id"],
|
||||
version_name=row["version_name"],
|
||||
collid=row["collid"],
|
||||
)
|
||||
for row in to_update
|
||||
]
|
||||
|
||||
done_versions: dict[tuple[str, str], set[str]] = {}
|
||||
for row in log_rows:
|
||||
if row["status"] in DONE_STATUSES:
|
||||
done_versions.setdefault(
|
||||
(row["action"], row["collid"] or row["bgg_id"]), set()
|
||||
).add(row["version_id"])
|
||||
|
||||
jobs: list[UploadJob] = []
|
||||
skipped_done = skipped_failed = 0
|
||||
seen: Counter[tuple[str, str, str]] = Counter()
|
||||
queued_versions: dict[str, set[str]] = {}
|
||||
for job in candidates:
|
||||
if job.action == "add":
|
||||
queued_versions.setdefault(job.bgg_id, set()).add(job.version_id)
|
||||
for job in candidates:
|
||||
prior = done_versions.get(
|
||||
("update", job.collid) if job.action == "update" else ("add", job.bgg_id),
|
||||
set(),
|
||||
)
|
||||
if job.action == "add" and prior & queued_versions.get(job.bgg_id, set()):
|
||||
# the done version is still queued alongside this one: a
|
||||
# multi-edition game partway through, not a re-review drift
|
||||
prior = set()
|
||||
if prior and job.version_id not in prior:
|
||||
typer.echo(
|
||||
f" skipping {job.name}: previously {job.action}ed with a "
|
||||
f"different version ({', '.join(sorted(prior)) or 'none'}) — "
|
||||
"the entry exists on BGG, so re-adding can only duplicate "
|
||||
"it; correct the version by hand (additive-only rule)"
|
||||
)
|
||||
skipped_done += 1
|
||||
continue
|
||||
occurrence = seen[job.key]
|
||||
seen[job.key] += 1
|
||||
if not job.name:
|
||||
# an empty name (manual id whose lookup failed) would make the
|
||||
# name-driven selectors match ANY heading/row — refuse loudly
|
||||
typer.echo(
|
||||
f" refusing to queue bgg_id {job.bgg_id}: empty game name "
|
||||
"— re-review this match so the name resolves"
|
||||
)
|
||||
skipped_failed += 1
|
||||
elif occurrence < done_count[job.key]:
|
||||
skipped_done += 1
|
||||
elif last_status.get(job.key) == "failed" and not retry_failed:
|
||||
skipped_failed += 1
|
||||
else:
|
||||
# same-game updates coexist in one run: update_entry addresses
|
||||
# the copy by collid and the edition by radio value, so a
|
||||
# second update cannot reopen what the first just saved
|
||||
jobs.append(job)
|
||||
return jobs, skipped_done, skipped_failed
|
||||
|
||||
|
||||
def _scrub(text: str) -> str:
|
||||
"""Credentials must never reach the log, even via a selector error that
|
||||
echoes filled form values. Playwright errors carry multi-line call
|
||||
logs; a CSV cell keeps one line."""
|
||||
for key in ("BGG_PASSWORD", "BGG_USERNAME"):
|
||||
value = os.environ.get(key)
|
||||
if value:
|
||||
text = text.replace(value, "***")
|
||||
return " ".join(text.split())
|
||||
|
||||
|
||||
class LoginError(RuntimeError):
|
||||
"""Authentication is broken (bad credentials, Cloudflare block, changed
|
||||
login page). Systemic by definition: retrying per-game would hammer the
|
||||
login endpoint and poison upload_log.csv with misleading failures."""
|
||||
|
||||
|
||||
class Uploader(Protocol):
|
||||
def add_game(self, job: UploadJob) -> tuple[str, str]: ...
|
||||
|
||||
def update_entry(self, job: UploadJob) -> tuple[str, str]: ...
|
||||
|
||||
|
||||
class PlaywrightUploader:
|
||||
"""Drives the real site. Selector documentation: docs/bgg-upload-flow.md.
|
||||
|
||||
All flows verified against the live site 2026-08-06 (adds, the version
|
||||
picker with pagination, and the collection-row version edit). The
|
||||
remaining unknown is the second-copy add: BGG may edit the existing
|
||||
entry instead of creating one, which run_upload reports as a copy-count
|
||||
shortfall during --verify.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
username: str,
|
||||
storage_state: Path | None = None,
|
||||
headless: bool = False,
|
||||
) -> None:
|
||||
self._username = username
|
||||
self._storage_state = storage_state or Config().storage_state_path
|
||||
self._headless = headless
|
||||
self._authed = False
|
||||
|
||||
def __enter__(self) -> PlaywrightUploader:
|
||||
from playwright.sync_api import TimeoutError as PWTimeoutError
|
||||
from playwright.sync_api import sync_playwright
|
||||
|
||||
self._timeout_error = PWTimeoutError
|
||||
self._pw = sync_playwright().start()
|
||||
self._browser = self._pw.chromium.launch(headless=self._headless)
|
||||
state = str(self._storage_state) if self._storage_state.exists() else None
|
||||
self._context = self._browser.new_context(storage_state=state)
|
||||
self._page = self._context.new_page()
|
||||
return self
|
||||
|
||||
def __exit__(self, *exc: object) -> None:
|
||||
self._context.close()
|
||||
self._browser.close()
|
||||
self._pw.stop()
|
||||
|
||||
def _goto(self, url: str) -> None:
|
||||
# networkidle never arrives on BGG (ad polling) — domcontentloaded
|
||||
# plus per-element auto-waiting is the reliable pattern.
|
||||
self._page.goto(url, wait_until="domcontentloaded")
|
||||
|
||||
# BGG's Sign In control is an <a class="btn"> with NO href, so it has
|
||||
# no implicit link role — role-based queries cannot find it.
|
||||
_SIGN_IN = 'a:has-text("Sign In"), button:has-text("Sign In")'
|
||||
_SIGNED_IN = ':text-is("Sign Out"), :text-is("Log Out")'
|
||||
|
||||
def _signed_out(self) -> bool:
|
||||
"""True = signed out. Waits for the header to prove ONE state or
|
||||
the other: BGG hydrates it after domcontentloaded, so an early read
|
||||
finds NEITHER control — and any check resting on a single absence
|
||||
guesses, silently, in the unsafe direction. Undeterminable state
|
||||
raises rather than assumes."""
|
||||
page = self._page
|
||||
deadline = time.monotonic() + 30
|
||||
while time.monotonic() < deadline:
|
||||
out = page.locator(self._SIGN_IN).count()
|
||||
inside = page.locator(self._SIGNED_IN).count()
|
||||
if out and not inside:
|
||||
return True
|
||||
if inside and not out:
|
||||
return False
|
||||
page.wait_for_timeout(250)
|
||||
raise LoginError(
|
||||
"could not tell whether this browser is signed in (neither a "
|
||||
"Sign In nor a Sign Out control appeared in 30s) — BGG's header "
|
||||
"may have changed; see docs/bgg-upload-flow.md"
|
||||
)
|
||||
|
||||
def _ensure_logged_in(self) -> None:
|
||||
if self._authed:
|
||||
return
|
||||
page = self._page
|
||||
self._goto(f"{BGG}/")
|
||||
if "just a moment" in (page.title() or "").casefold():
|
||||
raise LoginError(
|
||||
"Cloudflare is challenging this browser ('Just a moment...') "
|
||||
"— run headed (drop --headless) and click the widget once"
|
||||
)
|
||||
if self._signed_out():
|
||||
user = os.environ.get("BGG_USERNAME", "")
|
||||
password = os.environ.get("BGG_PASSWORD", "")
|
||||
if not (user and password):
|
||||
raise LoginError(
|
||||
"BGG_USERNAME and BGG_PASSWORD env vars are required to log in"
|
||||
)
|
||||
self._goto(f"{BGG}/login")
|
||||
# Generous timeout: in headed mode a human may need to click the
|
||||
# Cloudflare Turnstile widget before the form renders.
|
||||
page.locator("#inputUsername").wait_for(timeout=120_000)
|
||||
page.locator("#inputUsername").fill(user)
|
||||
page.locator("#inputPassword").fill(password)
|
||||
page.get_by_role("button", name="Sign In").click()
|
||||
try:
|
||||
page.wait_for_url(lambda url: "/login" not in url, timeout=120_000)
|
||||
except self._timeout_error as err:
|
||||
raise LoginError(
|
||||
"login did not complete (wrong credentials, or the login "
|
||||
"page changed — see docs/bgg-upload-flow.md)"
|
||||
) from err
|
||||
# Verify the TRANSITION: the Sign In control was visible a
|
||||
# moment ago, so its disappearance is evidence, not inference.
|
||||
self._goto(f"{BGG}/")
|
||||
if self._signed_out():
|
||||
raise LoginError(
|
||||
"the login form was submitted but the site still offers "
|
||||
"Sign In — credentials rejected, or a challenge is "
|
||||
"pending in the browser window"
|
||||
)
|
||||
self._context.storage_state(path=str(self._storage_state))
|
||||
self._storage_state.chmod(0o600) # session cookies: owner-only
|
||||
self._authed = True
|
||||
|
||||
def _open_dialog(self, opener):
|
||||
"""Click an opener that may no-op right after page load (hydration
|
||||
race) and wait for the dialog to actually show. (Params/return are
|
||||
Playwright Locators; untyped because the import is lazy.)"""
|
||||
dialog = self._page.get_by_role("dialog")
|
||||
for attempt in (1, 2):
|
||||
opener.click()
|
||||
try:
|
||||
dialog.wait_for(state="visible", timeout=5_000)
|
||||
break
|
||||
except self._timeout_error:
|
||||
if attempt == 2:
|
||||
raise
|
||||
return dialog
|
||||
|
||||
# Verified 2026-08-06 against the live picker: version rows are the
|
||||
# <li>s carrying a thumbnail (the paging <li>s are not), and paging is
|
||||
# an AngularJS <ul class="pagination"> of anchors, not buttons.
|
||||
_VERSION_ROWS = "li:has(.summary-item-thumbnail)"
|
||||
# Paging controls render TWICE — a desktop set and a mobile set marked
|
||||
# visible-xs-* — so every one of these selectors matches hidden nodes
|
||||
# too; clicking one waits forever. Always pick the visible match.
|
||||
_NEXT_PAGE = 'ul.pagination a[title="Next Page"]'
|
||||
_PAGE_LINKS = "ul.pagination a"
|
||||
# Rows read "<game name> (<version name>) (<year>)", so the version name
|
||||
# is matched inside its parentheses — bare substrings would let
|
||||
# "English edition" match "English edition, second printing".
|
||||
_TRAILING_YEAR = re.compile(r"\s+\d{4}(?:-\d+)?$")
|
||||
|
||||
def _rows_on_page(self, dialog) -> list[str]:
|
||||
return [
|
||||
" ".join(t.split())
|
||||
for t in dialog.locator(self._VERSION_ROWS).all_text_contents()
|
||||
]
|
||||
|
||||
def _visible(self, dialog, selector):
|
||||
"""The first VISIBLE match, or None — see _NEXT_PAGE."""
|
||||
loc = dialog.locator(selector)
|
||||
for i in range(loc.count()):
|
||||
candidate = loc.nth(i)
|
||||
if candidate.is_visible():
|
||||
return candidate
|
||||
return None
|
||||
|
||||
def _has_next_page(self, dialog) -> bool:
|
||||
nxt = self._visible(dialog, self._NEXT_PAGE)
|
||||
if nxt is None:
|
||||
return False # single-page list: the container is ng-show'd off
|
||||
return "disabled" not in (nxt.evaluate("e => e.closest('li').className") or "")
|
||||
|
||||
def _select_version(self, dialog, version_name: str) -> tuple[bool, str]:
|
||||
"""Pick the version whose name matches, paging the whole list first
|
||||
so a decision is made against every candidate. Returns (picked,
|
||||
reason); on failure the sub-view is cancelled and the caller adds
|
||||
the game version-less — never a guessed edition (spec)."""
|
||||
dialog.get_by_role("button", name="Set version/edition").click()
|
||||
try:
|
||||
dialog.locator(self._VERSION_ROWS).first.wait_for(timeout=15_000)
|
||||
except self._timeout_error as err:
|
||||
# a version resolve found on BGG can't be missing from the
|
||||
# picker: an unrendered list means a slow page or changed markup
|
||||
raise RuntimeError(
|
||||
"version picker never rendered — site slow or markup "
|
||||
"changed; attempt is retryable"
|
||||
) from err
|
||||
|
||||
wanted = f"({version_name})".casefold()
|
||||
# BGG's API name can carry a printing qualifier the picker omits
|
||||
# ("English edition 2018-2" vs "(English edition) (2018)")
|
||||
relaxed = f"({self._TRAILING_YEAR.sub('', version_name).strip()})".casefold()
|
||||
self._goto_first_page(dialog) # a fresh sub-view opens on page 1,
|
||||
# but never assume it: paging state outlives a close/reopen
|
||||
exact: list[tuple[int, str]] = []
|
||||
loose: list[tuple[int, str]] = []
|
||||
pages = 0
|
||||
for page_ix in range(MAX_VERSION_PAGES):
|
||||
pages = page_ix + 1
|
||||
for text in self._rows_on_page(dialog):
|
||||
folded = text.casefold()
|
||||
if wanted in folded:
|
||||
exact.append((page_ix, text))
|
||||
elif relaxed != wanted and relaxed in folded:
|
||||
loose.append((page_ix, text))
|
||||
if not self._has_next_page(dialog):
|
||||
break
|
||||
self._visible(dialog, self._NEXT_PAGE).click()
|
||||
self._page.wait_for_timeout(400) # client-side paging: no request
|
||||
else:
|
||||
# never saw the end of the list: "not in picker" would be a false
|
||||
# verdict frozen into DONE_STATUSES
|
||||
raise RuntimeError(
|
||||
f"hit MAX_VERSION_PAGES ({MAX_VERSION_PAGES}) without "
|
||||
"finding the version or the end of the list — retryable"
|
||||
)
|
||||
|
||||
hits = exact or loose
|
||||
if len(hits) == 1:
|
||||
_, text = hits[0]
|
||||
if not self._click_row(dialog, text):
|
||||
raise RuntimeError(
|
||||
f"{text!r} was listed a moment ago but vanished on the "
|
||||
"second pass — retryable"
|
||||
)
|
||||
how = "" if exact else f" (matched loosely as {text!r})"
|
||||
return True, how
|
||||
|
||||
# Two-level dismissal: the sub-view has its own Cancel distinct from
|
||||
# the main dialog's.
|
||||
dialog.get_by_role("button", name="Cancel").first.click()
|
||||
if not hits:
|
||||
return False, f"not offered in the picker ({pages} page(s) scanned)"
|
||||
return False, (
|
||||
f"{len(hits)} versions match {version_name!r} "
|
||||
f"({', '.join(t for _, t in hits[:3])}…) — refusing to guess"
|
||||
)
|
||||
|
||||
def _goto_first_page(self, dialog) -> None:
|
||||
"""Back to page 1 via the numbered "1" anchor. Closing and
|
||||
reopening the sub-view does NOT reset Angular's paging state (it
|
||||
reopens wherever it was left), and First/Prev render in a
|
||||
mobile-only variant that a desktop viewport can never click."""
|
||||
pager = dialog.locator(self._PAGE_LINKS)
|
||||
for i in range(pager.count()):
|
||||
anchor = pager.nth(i)
|
||||
if anchor.is_visible() and (
|
||||
" ".join((anchor.text_content() or "").split()) == "1"
|
||||
):
|
||||
anchor.click()
|
||||
self._page.wait_for_timeout(400)
|
||||
return
|
||||
# no pagination rendered: a single-page list is already page 1
|
||||
|
||||
def _click_row(self, dialog, text: str) -> bool:
|
||||
"""Find the row again by its NORMALIZED text and click it by index.
|
||||
Playwright's has_text regex matches raw textContent — whose tabs and
|
||||
newlines a normalized capture will never equal — so the comparison
|
||||
happens in Python, and the click addresses a position."""
|
||||
self._goto_first_page(dialog)
|
||||
for _ in range(MAX_VERSION_PAGES):
|
||||
for i, row_text in enumerate(self._rows_on_page(dialog)):
|
||||
if row_text == text:
|
||||
dialog.locator(self._VERSION_ROWS).nth(i).click()
|
||||
return True
|
||||
if not self._has_next_page(dialog):
|
||||
return False
|
||||
self._visible(dialog, self._NEXT_PAGE).click()
|
||||
self._page.wait_for_timeout(400)
|
||||
return False
|
||||
|
||||
def _owned_button(self):
|
||||
"""The owned-game page replaces "Add To" with "In Collections" —
|
||||
positive evidence the collection already holds this game."""
|
||||
return self._page.get_by_role("button", name="In Collections")
|
||||
|
||||
def add_game(self, job: UploadJob) -> tuple[str, str]:
|
||||
self._ensure_logged_in()
|
||||
page = self._page
|
||||
# /boardgame/<id> redirects to the canonical slug for any subtype.
|
||||
self._goto(f"{BGG}/boardgame/{job.bgg_id}/")
|
||||
add_btn = page.get_by_role("button", name="Add To")
|
||||
for _ in range(60): # the header hydrates late: poll for EITHER state
|
||||
if add_btn.count() and add_btn.first.is_visible():
|
||||
break
|
||||
owned = self._owned_button()
|
||||
if owned.count() and owned.first.is_visible():
|
||||
if job.second_copy:
|
||||
# the second-copy path goes through the In Collections
|
||||
# dialog, which this code has never driven live
|
||||
raise RuntimeError(
|
||||
"second copy of an owned game — the In Collections "
|
||||
"add-a-copy flow is unverified; add this copy by hand"
|
||||
)
|
||||
return (
|
||||
"already_present",
|
||||
"the site already lists this game as owned — a previous "
|
||||
"attempt likely landed without reaching the log",
|
||||
)
|
||||
page.wait_for_timeout(500)
|
||||
dialog = self._open_dialog(add_btn.first)
|
||||
# Wait for the form itself, NOT for a heading matching our stored
|
||||
# name: a match made through an ALTERNATE name (BGG 140509 is
|
||||
# "Dungeons & Dragons" to search, "Dragones Y Mazmorras" on the
|
||||
# page) would never show it. The /boardgame/<id>/ URL already
|
||||
# guarantees which game this is.
|
||||
dialog.get_by_role("checkbox", name="Own", exact=True).wait_for(timeout=15_000)
|
||||
# exact: "Own" is a substring of "Prev. Owned", and a loose label
|
||||
# match resolves to both checkboxes (strict-mode violation)
|
||||
dialog.get_by_role("checkbox", name="Own", exact=True).check()
|
||||
status, note = "added", ""
|
||||
if job.version_name:
|
||||
picked, why = self._select_version(dialog, job.version_name)
|
||||
if picked:
|
||||
note = why.strip()
|
||||
else:
|
||||
status = "added_no_version"
|
||||
note = f"version {job.version_name!r}: {why}; added without version"
|
||||
dialog.get_by_role("button", name="Save").click()
|
||||
# The dialog is hidden after save, not removed from the DOM.
|
||||
try:
|
||||
dialog.wait_for(state="hidden", timeout=15_000)
|
||||
except self._timeout_error:
|
||||
# a slow hide is not a failed save: reload and ask the page.
|
||||
# Logging "failed" for a landed add would double-add on retry.
|
||||
self._goto(f"{BGG}/boardgame/{job.bgg_id}/")
|
||||
owned = self._owned_button()
|
||||
try:
|
||||
owned.first.wait_for(timeout=15_000)
|
||||
except self._timeout_error as err:
|
||||
raise RuntimeError(
|
||||
"the save dialog never closed and the page does not "
|
||||
"show the game as owned — the add may not have landed"
|
||||
) from err
|
||||
note = (note + "; " if note else "") + "save confirmed via page reload"
|
||||
return status, note
|
||||
|
||||
def update_entry(self, job: UploadJob) -> tuple[str, str]:
|
||||
"""Set the version on an EXISTING entry — strictly additive.
|
||||
|
||||
Verified 2026-08-06. The collection table's version cell carries its
|
||||
own collid in an onclick and opens an inline editor whose radios
|
||||
carry VERSION IDS as their values, so both the copy and the edition
|
||||
are addressed exactly — no name matching, no dialog, no pagination.
|
||||
Clicking a radio fires CE_SaveData itself; there is no Save button.
|
||||
"""
|
||||
self._ensure_logged_in()
|
||||
page = self._page
|
||||
self._goto(
|
||||
f"{BGG}/collection/user/{self._username}?objectid={job.bgg_id}&own=1"
|
||||
)
|
||||
cell = page.locator(f'td.collection_version[onclick*="{job.collid}"]')
|
||||
if cell.count() == 0:
|
||||
raise RuntimeError(
|
||||
f"collid {job.collid} has no version cell on the collection "
|
||||
"page — the entry may have been removed; re-run diff"
|
||||
)
|
||||
cell.first.click()
|
||||
radio = page.locator(
|
||||
f'form[id^="form_version"] input[type="radio"][value="{job.version_id}"]'
|
||||
)
|
||||
try:
|
||||
radio.first.wait_for(timeout=15_000)
|
||||
except self._timeout_error as err:
|
||||
# the editor lists every version of the game: an absent id means
|
||||
# the wrong game's editor opened, or BGG retired that version
|
||||
page.keyboard.press("Escape")
|
||||
raise RuntimeError(
|
||||
f"version {job.version_id} not offered for collid "
|
||||
f"{job.collid} — entry left untouched"
|
||||
) from err
|
||||
radio.first.click() # fires CE_SaveData: no separate Save button
|
||||
want = _loose_version_text(job.version_name)
|
||||
for _ in range(40):
|
||||
settled = " ".join((cell.first.text_content() or "").split())
|
||||
if settled and "editing" not in settled.casefold():
|
||||
got = _loose_version_text(settled)
|
||||
if want and want not in got and got not in want:
|
||||
# the editor closed but re-rendered its OLD content:
|
||||
# the AJAX save failed server-side. "updated" here
|
||||
# would mark the job done forever without evidence.
|
||||
raise RuntimeError(
|
||||
f"the version cell settled on {settled!r}, not the "
|
||||
f"chosen {job.version_name!r} — the save did not "
|
||||
"land; safe to retry (same radio, same result)"
|
||||
)
|
||||
return "updated", ""
|
||||
page.wait_for_timeout(500)
|
||||
raise RuntimeError(
|
||||
"the version cell never left its editing state — the save may "
|
||||
"not have landed; re-run with --verify"
|
||||
)
|
||||
|
||||
|
||||
def _process(
|
||||
uploader: Uploader,
|
||||
jobs: list[UploadJob],
|
||||
log_path: Path,
|
||||
*,
|
||||
sleep: Callable[[float], None],
|
||||
rng: random.Random,
|
||||
now: Callable[[], str],
|
||||
) -> list[dict]:
|
||||
results = []
|
||||
consecutive: tuple[str, int] = ("", 0)
|
||||
for i, job in enumerate(jobs):
|
||||
if i:
|
||||
sleep(rng.uniform(2.0, 4.0)) # polite pacing between games (spec)
|
||||
try:
|
||||
if job.action == "add":
|
||||
status, note = uploader.add_game(job)
|
||||
else:
|
||||
status, note = uploader.update_entry(job)
|
||||
except LoginError as exc:
|
||||
# Systemic: every remaining job would fail identically. Abort
|
||||
# WITHOUT logging failures, so the next run just retries.
|
||||
typer.echo(f" aborting — login is broken: {_scrub(str(exc))}")
|
||||
typer.echo(f" {len(jobs) - i} job(s) left untouched for the next run.")
|
||||
break
|
||||
except Exception as exc: # per-game isolation: log it, keep going
|
||||
status, note = "failed", _scrub(f"{type(exc).__name__}: {exc}")
|
||||
row = {
|
||||
"action": job.action,
|
||||
"bgg_id": job.bgg_id,
|
||||
"collid": job.collid,
|
||||
"name": job.name,
|
||||
"version_id": job.version_id,
|
||||
"second_copy": "1" if job.second_copy else "",
|
||||
"status": status,
|
||||
"timestamp": now(),
|
||||
# the column carries degradation notes on successes too
|
||||
# (e.g. added_no_version), not just failure text
|
||||
"error": note,
|
||||
}
|
||||
append_log_row(log_path, row)
|
||||
results.append(row)
|
||||
suffix = f" — {note}" if note else ""
|
||||
typer.echo(f" {job.name}: {status}{suffix}")
|
||||
if status == "failed":
|
||||
kind = note # IDENTICAL means the whole message, not the class
|
||||
consecutive = (kind, consecutive[1] + 1 if kind == consecutive[0] else 1)
|
||||
if consecutive[1] >= 3:
|
||||
typer.echo(
|
||||
" aborting — 3 identical consecutive failures look "
|
||||
"systemic, not per-game; remaining jobs left for the "
|
||||
"next run"
|
||||
)
|
||||
break
|
||||
else:
|
||||
consecutive = ("", 0)
|
||||
return results
|
||||
|
||||
|
||||
def verify_uploads(log_rows: list[dict], collection: list[CollectionItem]) -> list[str]:
|
||||
"""Cross-check the log's successes against a fresh collection fetch."""
|
||||
by_object: dict[int, list[CollectionItem]] = {}
|
||||
by_collid: dict[int, CollectionItem] = {}
|
||||
for item in collection:
|
||||
by_object.setdefault(item.object_id, []).append(item)
|
||||
by_collid[item.coll_id] = item
|
||||
|
||||
problems = []
|
||||
added_copies: Counter[int] = Counter()
|
||||
shortfall_reported: set[int] = set()
|
||||
latest: dict[tuple[str, str, str], dict] = {}
|
||||
for row in log_rows:
|
||||
k = _job_key(row)
|
||||
latest[k] = row
|
||||
if row["action"] == "add" and row["status"] in DONE_STATUSES:
|
||||
added_copies[int(row["bgg_id"])] += 1
|
||||
for row in latest.values():
|
||||
if row["status"] in ("added", "added_no_version"):
|
||||
if row.get("second_copy"):
|
||||
# the game had copies before this add: collection state can't
|
||||
# distinguish "new entry created" from "existing entry
|
||||
# edited" (the failure the unverified dialog could produce)
|
||||
problems.append(
|
||||
f"{row['name']}: second-copy add can't be verified from "
|
||||
"collection counts — confirm by eye on BGG"
|
||||
)
|
||||
continue
|
||||
copies = by_object.get(int(row["bgg_id"]), [])
|
||||
if not copies:
|
||||
problems.append(f"{row['name']}: logged added but not in collection")
|
||||
elif (
|
||||
int(row["bgg_id"]) not in shortfall_reported
|
||||
and len(copies) < added_copies[int(row["bgg_id"])]
|
||||
):
|
||||
# the unverified second-copy dialog may EDIT the existing
|
||||
# entry instead of creating one — a count shortfall is the
|
||||
# only externally visible symptom; report once per GAME
|
||||
shortfall_reported.add(int(row["bgg_id"]))
|
||||
problems.append(
|
||||
f"{row['name']}: {added_copies[int(row['bgg_id'])]} "
|
||||
f"add(s) logged but only {len(copies)} cop"
|
||||
f"{'y' if len(copies) == 1 else 'ies'} in the collection"
|
||||
" — a second-copy add may have edited an existing entry"
|
||||
)
|
||||
elif (
|
||||
row["status"] == "added" # no_version: absence is expected
|
||||
and row["version_id"]
|
||||
and not any(
|
||||
str(c.version_id or "") == row["version_id"] for c in copies
|
||||
)
|
||||
):
|
||||
problems.append(
|
||||
f"{row['name']}: in collection but no copy has "
|
||||
f"version {row['version_id']}"
|
||||
)
|
||||
elif row["status"] == "updated":
|
||||
item = by_collid.get(int(row["collid"]))
|
||||
if item is None:
|
||||
problems.append(
|
||||
f"{row['name']}: collid {row['collid']} not in collection"
|
||||
)
|
||||
elif str(item.version_id or "") != row["version_id"]:
|
||||
problems.append(
|
||||
f"{row['name']}: collid {row['collid']} does not carry "
|
||||
f"version {row['version_id']}"
|
||||
)
|
||||
return problems
|
||||
|
||||
|
||||
def run_upload(
|
||||
cfg: Config,
|
||||
*,
|
||||
dry_run: bool = False,
|
||||
verify: bool = False,
|
||||
retry_failed: bool = False,
|
||||
limit: int | None = None,
|
||||
headless: bool = False,
|
||||
uploader: Uploader | None = None,
|
||||
client: BGGClient | None = None,
|
||||
storage_state: Path | None = None,
|
||||
sleep: Callable[[float], None] = time.sleep,
|
||||
rng: random.Random | None = None,
|
||||
now: Callable[[], str] | None = None,
|
||||
) -> list[dict]:
|
||||
rng = rng or random.Random()
|
||||
now = now or (lambda: datetime.now(UTC).isoformat(timespec="seconds"))
|
||||
|
||||
# Two provenance markers guard the same fact from different angles: the
|
||||
# cache marker travels with the stub XML (gitignored, so a fresh clone
|
||||
# loses it), while data/STUB_DATA.marker is COMMITTED alongside the
|
||||
# stub-derived CSVs — so a clone can never upload placeholder ids.
|
||||
marker = next((m for m in cfg.stub_marker_paths if m.exists()), None)
|
||||
if marker is not None:
|
||||
if not dry_run:
|
||||
typer.echo(
|
||||
f"Refusing to upload: {marker} exists — every resolved "
|
||||
"version_id is a synthetic stub placeholder. Once "
|
||||
"BGG_API_TOKEN arrives: delete both cache dirs, re-record "
|
||||
"fixtures, `resolve --force`, re-review, re-diff, then "
|
||||
"delete the marker(s)."
|
||||
)
|
||||
raise typer.Exit(code=1)
|
||||
typer.echo(
|
||||
"WARNING: stub fixtures active — the ids below are SYNTHETIC "
|
||||
"placeholders, not real BGG data.\n"
|
||||
)
|
||||
|
||||
log_path = cfg.upload_log_path
|
||||
if not cfg.to_add_path.exists():
|
||||
typer.echo(f"{cfg.to_add_path} not found — run `bggpipe diff` first.")
|
||||
raise typer.Exit(code=1)
|
||||
to_add = _read_csv(cfg.to_add_path)
|
||||
if not cfg.to_update_path.exists():
|
||||
typer.echo(
|
||||
f"note: {cfg.to_update_path} not found — no version updates "
|
||||
"queued (re-run `bggpipe diff` if that's unexpected)"
|
||||
)
|
||||
to_update = _read_csv(cfg.to_update_path)
|
||||
log_rows = _read_csv(log_path)
|
||||
|
||||
# The queue is a snapshot; review decisions since the last diff win.
|
||||
stale = stale_jobs(to_add + to_update, read_matches(cfg.matches_path))
|
||||
if stale:
|
||||
to_add = [r for r in to_add if r.get("bgg_id") not in stale]
|
||||
to_update = [r for r in to_update if r.get("bgg_id") not in stale]
|
||||
typer.echo(
|
||||
f"Skipping {len(stale)} queued game(s) whose review decision "
|
||||
"changed since the last diff — re-run diff to refresh the queue:"
|
||||
)
|
||||
for bgg_id, why in stale.items():
|
||||
typer.echo(f" {bgg_id}: {why}")
|
||||
|
||||
jobs, skipped_done, skipped_failed = build_queue(
|
||||
to_add, to_update, log_rows, retry_failed=retry_failed
|
||||
)
|
||||
if limit is not None:
|
||||
jobs = jobs[:limit]
|
||||
|
||||
skip_note = ""
|
||||
if skipped_done or skipped_failed:
|
||||
skip_note = (
|
||||
f" (skipping {skipped_done} already done, {skipped_failed} "
|
||||
"previously failed — use --retry-failed)"
|
||||
)
|
||||
|
||||
results: list[dict] = []
|
||||
if dry_run:
|
||||
typer.echo(f"Dry run: {len(jobs)} job(s) pending{skip_note}")
|
||||
for job in jobs:
|
||||
if job.action == "add":
|
||||
version = f" [version {job.version_name}]" if job.version_name else ""
|
||||
typer.echo(f" would add {job.name} ({job.bgg_id}){version}")
|
||||
else:
|
||||
typer.echo(
|
||||
f" would set version {job.version_name!r} on existing "
|
||||
f"entry collid {job.collid} ({job.name})"
|
||||
)
|
||||
elif jobs:
|
||||
typer.echo(f"Uploading {len(jobs)} job(s){skip_note}")
|
||||
if uploader is None:
|
||||
if not (os.environ.get("BGG_USERNAME") and os.environ.get("BGG_PASSWORD")):
|
||||
typer.echo(
|
||||
"BGG_USERNAME and BGG_PASSWORD env vars are required "
|
||||
"(direnv loads them from .env)."
|
||||
)
|
||||
raise typer.Exit(code=1)
|
||||
with PlaywrightUploader(
|
||||
cfg.bgg_username,
|
||||
storage_state=storage_state or cfg.storage_state_path,
|
||||
headless=headless,
|
||||
) as real:
|
||||
results = _process(real, jobs, log_path, sleep=sleep, rng=rng, now=now)
|
||||
else:
|
||||
results = _process(uploader, jobs, log_path, sleep=sleep, rng=rng, now=now)
|
||||
counts = Counter(row["status"] for row in results)
|
||||
summary = " · ".join(f"{n} {status}" for status, n in sorted(counts.items()))
|
||||
typer.echo(f"\n{summary or 'nothing to do'}")
|
||||
else:
|
||||
typer.echo(f"Nothing to upload{skip_note}.")
|
||||
|
||||
if verify and not dry_run:
|
||||
_run_verify(cfg, client, log_path)
|
||||
return results
|
||||
|
||||
|
||||
def _run_verify(cfg: Config, client: BGGClient | None, log_path: Path) -> None:
|
||||
if not cfg.bgg_username:
|
||||
typer.echo("--verify needs BGG_USERNAME in the environment.")
|
||||
return
|
||||
client = client or client_for(cfg)
|
||||
try:
|
||||
collection = client.collection_full(cfg.bgg_username, refresh=True)
|
||||
except BGGAuthError as exc:
|
||||
typer.echo(f"--verify skipped: {exc}")
|
||||
return
|
||||
problems = verify_uploads(_read_csv(log_path), collection)
|
||||
if problems:
|
||||
typer.echo("\nVerification problems:")
|
||||
for line in problems:
|
||||
typer.echo(f" - {line}")
|
||||
else:
|
||||
typer.echo("\nVerification OK: every logged success is in the collection.")
|
||||
File diff suppressed because it is too large
Load Diff
+91
-7
@@ -2,7 +2,9 @@
|
||||
{
|
||||
"title_raw": "Catan",
|
||||
"confidence": "high",
|
||||
"source_photos": ["hand-typed-test-list"]
|
||||
"source_photos": [
|
||||
"hand-typed-test-list"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title_raw": "Wingspan",
|
||||
@@ -10,31 +12,113 @@
|
||||
"publisher_hint": "Stonemaier Games",
|
||||
"year_hint": 2019,
|
||||
"language_hint": "English",
|
||||
"source_photos": ["hand-typed-test-list"]
|
||||
"source_photos": [
|
||||
"hand-typed-test-list"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title_raw": "Wingspan: European Expansion",
|
||||
"confidence": "high",
|
||||
"source_photos": ["hand-typed-test-list"]
|
||||
"source_photos": [
|
||||
"hand-typed-test-list"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title_raw": "Wingspan Europe",
|
||||
"confidence": "medium",
|
||||
"source_photos": ["hand-typed-test-list"]
|
||||
"source_photos": [
|
||||
"hand-typed-test-list"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title_raw": "Café International",
|
||||
"confidence": "high",
|
||||
"source_photos": ["hand-typed-test-list"]
|
||||
"source_photos": [
|
||||
"hand-typed-test-list"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title_raw": "Citadels",
|
||||
"confidence": "high",
|
||||
"source_photos": ["hand-typed-test-list"]
|
||||
"source_photos": [
|
||||
"hand-typed-test-list"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title_raw": "Blorvath: Quest of the Zzyzx",
|
||||
"confidence": "low",
|
||||
"source_photos": ["hand-typed-test-list"]
|
||||
"source_photos": [
|
||||
"hand-typed-test-list"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title_raw": "CIVILIZATION Game of the Heroic Age - The Dawn of History 8000 BC to 250 BC",
|
||||
"confidence": "high",
|
||||
"publisher_hint": "The Avalon Hill Game Company, Baltimore, Maryland",
|
||||
"edition_hint": "Bookcase Game",
|
||||
"language_hint": "English",
|
||||
"source_photos": [
|
||||
"hand-typed-test-list"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title_raw": "ADVANCED CIVILIZATION Game Expansion of the Heroic Age - Featuring New Civilization, Commodity, and Calamity Cards",
|
||||
"confidence": "high",
|
||||
"publisher_hint": "The Avalon Hill Game Company, Baltimore, Maryland",
|
||||
"edition_hint": "Bookcase Game",
|
||||
"language_hint": "English",
|
||||
"source_photos": [
|
||||
"hand-typed-test-list"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title_raw": "SORCERER The Game of Magical Conflict",
|
||||
"confidence": "high",
|
||||
"publisher_hint": "Simulations Publications Incorporated (SPI)",
|
||||
"edition_hint": "Designer's Edition",
|
||||
"language_hint": "English",
|
||||
"source_photos": [
|
||||
"hand-typed-test-list"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title_raw": "STARFORCE ALPHA CENTAURI Interstellar Conflict in the 25th Century",
|
||||
"confidence": "high",
|
||||
"publisher_hint": "Simulations Publications Incorporated (SPI)",
|
||||
"edition_hint": "Designer's Edition",
|
||||
"language_hint": "English",
|
||||
"source_photos": [
|
||||
"hand-typed-test-list"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title_raw": "FLAT TOP",
|
||||
"confidence": "high",
|
||||
"year_hint": 1942,
|
||||
"language_hint": "English",
|
||||
"source_photos": [
|
||||
"hand-typed-test-list"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title_raw": "ALICE IS MISSING: A SILENT ROLE PLAYING GAME",
|
||||
"confidence": "high",
|
||||
"source_photos": [
|
||||
"hand-typed-test-list"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title_raw": "Dungeon!",
|
||||
"confidence": "high",
|
||||
"source_photos": [
|
||||
"hand-typed-test-list"
|
||||
]
|
||||
},
|
||||
{
|
||||
"title_raw": "WIZ-WAR",
|
||||
"confidence": "high",
|
||||
"source_photos": [
|
||||
"hand-typed-test-list"
|
||||
]
|
||||
}
|
||||
]
|
||||
|
||||
-1
@@ -1 +0,0 @@
|
||||
<items total="1"><item type="boardgame" id="207830"><name type="primary" value="5-Minute Dungeon"/><yearpublished value="2017"/></item></items>
|
||||
+1
-1
@@ -1 +1 @@
|
||||
<items total="0"></items>
|
||||
<?xml version="1.0" encoding="utf-8"?><items total="0" termsofuse="https://boardgamegeek.com/xmlapi/termsofuse"> </items>
|
||||
+1
@@ -0,0 +1 @@
|
||||
<?xml version="1.0" encoding="utf-8"?><items total="0" termsofuse="https://boardgamegeek.com/xmlapi/termsofuse"> </items>
|
||||
+1
-1
@@ -1 +1 @@
|
||||
<items total="0"></items>
|
||||
<?xml version="1.0" encoding="utf-8"?><items total="0" termsofuse="https://boardgamegeek.com/xmlapi/termsofuse"> </items>
|
||||
+9
-1
@@ -1 +1,9 @@
|
||||
<items total="1"><item type="boardgameexpansion" id="177"><name type="primary" value="Advanced Civilization"/><yearpublished value="1991"/></item></items>
|
||||
<?xml version="1.0" encoding="utf-8"?><items total="2" termsofuse="https://boardgamegeek.com/xmlapi/termsofuse"> <item type="boardgame" id="177">
|
||||
<name type="primary" value="Advanced Civilization"/>
|
||||
<yearpublished value="1991" />
|
||||
</item>
|
||||
<item type="boardgameexpansion" id="177">
|
||||
<name type="primary" value="Advanced Civilization"/>
|
||||
<yearpublished value="1991" />
|
||||
</item>
|
||||
</items>
|
||||
+1
@@ -0,0 +1 @@
|
||||
<?xml version="1.0" encoding="utf-8"?><items total="0" termsofuse="https://boardgamegeek.com/xmlapi/termsofuse"> </items>
|
||||
+1
@@ -0,0 +1 @@
|
||||
<?xml version="1.0" encoding="utf-8"?><items total="0" termsofuse="https://boardgamegeek.com/xmlapi/termsofuse"> </items>
|
||||
+1
@@ -0,0 +1 @@
|
||||
<?xml version="1.0" encoding="utf-8"?><items total="0" termsofuse="https://boardgamegeek.com/xmlapi/termsofuse"> </items>
|
||||
+1
@@ -0,0 +1 @@
|
||||
<?xml version="1.0" encoding="utf-8"?><items total="0" termsofuse="https://boardgamegeek.com/xmlapi/termsofuse"> </items>
|
||||
+1
@@ -0,0 +1 @@
|
||||
<?xml version="1.0" encoding="utf-8"?><items total="0" termsofuse="https://boardgamegeek.com/xmlapi/termsofuse"> </items>
|
||||
+1
@@ -0,0 +1 @@
|
||||
<?xml version="1.0" encoding="utf-8"?><items total="0" termsofuse="https://boardgamegeek.com/xmlapi/termsofuse"> </items>
|
||||
+9
@@ -0,0 +1,9 @@
|
||||
<?xml version="1.0" encoding="utf-8"?><items total="2" termsofuse="https://boardgamegeek.com/xmlapi/termsofuse"> <item type="rpgitem" id="311654">
|
||||
<name type="primary" value="Alice is Missing"/>
|
||||
<yearpublished value="2020" />
|
||||
</item>
|
||||
<item type="rpgitem" id="380459">
|
||||
<name type="primary" value="Alice is Missing: Silent Falls"/>
|
||||
<yearpublished value="2023" />
|
||||
</item>
|
||||
</items>
|
||||
Vendored
+1
@@ -0,0 +1 @@
|
||||
<?xml version="1.0" encoding="utf-8"?><items total="0" termsofuse="https://boardgamegeek.com/xmlapi/termsofuse"> </items>
|
||||
-1
@@ -1 +0,0 @@
|
||||
<items total="1"><item type="boardgame" id="26"><name type="primary" value="Age of Renaissance"/><yearpublished value="1996"/></item></items>
|
||||
+1
-1
@@ -1 +1 @@
|
||||
<items total="0"></items>
|
||||
<?xml version="1.0" encoding="utf-8"?><items total="0" termsofuse="https://boardgamegeek.com/xmlapi/termsofuse"> </items>
|
||||
Vendored
+1
@@ -0,0 +1 @@
|
||||
<?xml version="1.0" encoding="utf-8"?><items total="0" termsofuse="https://boardgamegeek.com/xmlapi/termsofuse"> </items>
|
||||
+1
@@ -0,0 +1 @@
|
||||
<?xml version="1.0" encoding="utf-8"?><items total="0" termsofuse="https://boardgamegeek.com/xmlapi/termsofuse"> </items>
|
||||
Vendored
+1
@@ -0,0 +1 @@
|
||||
<?xml version="1.0" encoding="utf-8"?><items total="0" termsofuse="https://boardgamegeek.com/xmlapi/termsofuse"> </items>
|
||||
Vendored
+1
-1
@@ -1 +1 @@
|
||||
<items total="0"></items>
|
||||
<?xml version="1.0" encoding="utf-8"?><items total="0" termsofuse="https://boardgamegeek.com/xmlapi/termsofuse"> </items>
|
||||
@@ -0,0 +1 @@
|
||||
<?xml version="1.0" encoding="utf-8"?><items total="0" termsofuse="https://boardgamegeek.com/xmlapi/termsofuse"> </items>
|
||||
Vendored
-1
@@ -1 +0,0 @@
|
||||
<items total="1"><item type="boardgame" id="240"><name type="primary" value="Britannia"/><yearpublished value="1986"/></item></items>
|
||||
Vendored
-1
@@ -1 +0,0 @@
|
||||
<items total="1"><item type="boardgame" id="235096"><name type="primary" value="Cat Crimes"/><yearpublished value="2017"/></item></items>
|
||||
+1
-1
@@ -1 +1 @@
|
||||
<items total="0"></items>
|
||||
<?xml version="1.0" encoding="utf-8"?><items total="0" termsofuse="https://boardgamegeek.com/xmlapi/termsofuse"> </items>
|
||||
+1
@@ -0,0 +1 @@
|
||||
<?xml version="1.0" encoding="utf-8"?><items total="0" termsofuse="https://boardgamegeek.com/xmlapi/termsofuse"> </items>
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user