RPGs pull real RPGGeek data; off-BGG games get facts and a cover photo

Two gaps at the edges of the library, both closed.

RPGGeek items live in the same database but use their own link types —
rpgdesigner, rpgpublisher, rpggenre, rpgcategory, rpgmechanic — so a
board-game-only parser found none of them and both RPG entries showed
just a year and a description. parse_things_full now reads both
vocabularies (plus rpgproducer/rpgseries): .dungeon gains John Battle
and Project Nerves, Parsely gains Jared A. Sorensen and its genres.

An off-BGG game has no API to enrich it and no publisher art to fetch,
so its detail page now hosts the only source it will ever have: a form
for title, year, players, playing time, publishers, designers and
notes, plus a cover photo upload. Both persist in data/local_games.json
and data/local_art/ (committed, like every other curation store) and
enrich merges them over the photo reads, so a rebuild can't erase them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016jXZFSTZQKzAC8fqpWSz9g

CLAUDE.md

@@ -47,11 +47,11 @@ Full design lives in `bgg-shelf-pipeline-spec.md` (read it before changing pipel
 - Base game vs. expansion vs. new edition is the top failure mode — bias matching toward `ambiguous` over auto-match ("Wingspan Europe" must not match base Wingspan).
 - Editions/versions matter: Eric owns multiple editions of some games — each is a separate collection entry (keyed by `collid` on BGG). Never guess a version: no legible cues → `version_unknown` and a version-less collection entry.
 - Normalize titles (casefold, strip punctuation/articles, special chars like é/&/:) identically on both sides of a match; dedupe across photos but keep `source_photos` provenance.
-- **Human curation is durable**: `data/title_splits.json` (photo-scoped split-into-copies decisions, honored by extract's dedupe AND resolve's dedupe), `data/title_edits.json` (corrected reads/cues, applied before dedupe on every titles.json rebuild), `data/title_removals.json` (lines removed from the catalog — filtered out of every rebuild; delete the record to undo), and `data/title_additions.json` (games added without a photo — joined into every rebuild; a later photo sighting dedupe-merges with them) persist forever. Row-level decisions persist via the `dedupe_veto` column — edits never drop veto'd rows (a rename retitles them in place); removal drops them (explicitly discarding the line).
-- **RPGs are local-only citizens**: when the board-game search runs dry, resolve falls back to `type=rpgitem` (same geekdo API/token). Matched rpgitems enrich into the library but diff routes them to `local_only` — they must never reach `to_add.csv`/upload (their collection lives on RPGGeek, out of scope).
+- **Human curation is durable**: `data/title_splits.json` (photo-scoped split-into-copies decisions, honored by extract's dedupe AND resolve's dedupe), `data/title_edits.json` (corrected reads/cues, applied before dedupe on every titles.json rebuild), `data/title_removals.json` (lines removed from the catalog — filtered out of every rebuild; delete the record to undo), `data/title_additions.json` (games added without a photo — joined into every rebuild; a later photo sighting dedupe-merges with them), and `data/local_games.json` + `data/local_art/` (hand-written facts and a cover photo for off-BGG games — the ONLY source for them, merged over the photo reads by enrich) persist forever. Row-level decisions persist via the `dedupe_veto` column — edits never drop veto'd rows (a rename retitles them in place); removal drops them (explicitly discarding the line).
+- **RPGs are local-only citizens**: when the board-game search runs dry, resolve falls back to `type=rpgitem` (same geekdo API/token). RPGGeek items carry their OWN link types (`rpgdesigner`, `rpgpublisher`, `rpggenre`, `rpgcategory`, `rpgmechanic`) — a board-game-only parser silently returns nothing for them. Matched rpgitems enrich into the library but diff routes them to `local_only` — they must never reach `to_add.csv`/upload (their collection lives on RPGGeek, out of scope).
 - Detailed BGG API behavior (202 queueing, collection-endpoint quirks, endpoints): use the `bgg-api` skill. **If the spec's BGG behavior changes, update the `bgg-api` skill to match** — they must not drift.
 
 ## Git
 
 - Remote is self-hosted Gitea 1.26 (`git.kestrelsnest.social/eric/bggpipe`), **not GitHub** — `gh` CLI does not work here.
-- Commit `data/matches.csv`, `data/to_add.csv`, `data/to_update.csv`, `data/upload_log.csv`, `data/titles.json`, `data/unidentified.json`, `data/unidentified_dismissed.json`, `data/title_splits.json`, `data/title_edits.json`, `data/title_removals.json`, `data/title_additions.json`, `data/games.json`, `data/STUB_DATA.marker` (while it applies), and the collection snapshot XMLs. Never commit `data/bgg_cache/`, `data/extract_raw/`, `photos/`, `data/.lan_key`, Playwright storage state, or `.env`.
+- Commit `data/matches.csv`, `data/to_add.csv`, `data/to_update.csv`, `data/upload_log.csv`, `data/titles.json`, `data/unidentified.json`, `data/unidentified_dismissed.json`, `data/title_splits.json`, `data/title_edits.json`, `data/title_removals.json`, `data/title_additions.json`, `data/local_games.json`, `data/local_art/`, `data/games.json`, `data/STUB_DATA.marker` (while it applies), and the collection snapshot XMLs. Never commit `data/bgg_cache/`, `data/extract_raw/`, `photos/`, `data/.lan_key`, Playwright storage state, or `.env`.

src/bggpipe/config.py

@@ -74,6 +74,17 @@ class Config:
     def games_path(self) -> Path:
         return self.data_dir / "games.json"
 
+    @property
+    def local_games_path(self) -> Path:
+        # hand-written metadata for games BGG doesn't have — the only
+        # source of truth for them, so it is committed like the other stores
+        return self.data_dir / "local_games.json"
+
+    @property
+    def local_art_dir(self) -> Path:
+        # cover photos for off-BGG games (committed: nothing else has them)
+        return self.data_dir / "local_art"
+
     @property
     def title_additions_path(self) -> Path:
         # games the human added without a photo (expansions stored inside

src/bggpipe/enrich.py

@@ -100,6 +100,17 @@ def run_enrich(
     # photo reads — no API involved, so a blocked run still lands them
     local_keys: set[str] = set()
     local_rows = [r for r in rows if r["match_status"] == "local"]
+    # hand-written metadata wins over the photo reads: for an off-BGG game
+    # it is the only real source there is
+    hand: dict = {}
+    if cfg.local_games_path.exists():
+        try:
+            hand = json.loads(cfg.local_games_path.read_text())
+        except json.JSONDecodeError as err:
+            raise ValueError(
+                f"{cfg.local_games_path} is corrupt ({err}) — it holds "
+                "hand-written game data, so check git history before deleting"
+            ) from err
     if local_rows:
         try:
             cues = {
@@ -122,6 +133,9 @@ def run_enrich(
                 else [],
                 "source_photos": [p for p in row["source_photos"].split(";") if p],
             }
+            games[key].update(
+                {k: v for k, v in (hand.get(key) or {}).items() if v not in (None, "")}
+            )
 
     # prune keys no current target claims: a row whose version was approved
     # after a bare-key run (or was later rejected) must not leave an orphan

src/bggpipe/models.py

@@ -232,11 +232,17 @@ def parse_things_full(xml_text: str) -> list[dict]:
                 "min_playtime": _attr_int(item.find("minplaytime")),
                 "max_playtime": _attr_int(item.find("maxplaytime")),
                 "min_age": _attr_int(item.find("minage")),
-                "designers": links("boardgamedesigner"),
-                "artists": links("boardgameartist"),
-                "publishers": links("boardgamepublisher"),
-                "categories": links("boardgamecategory"),
-                "mechanics": links("boardgamemechanic"),
+                # RPGGeek items live in the same database but use their own
+                # link types, so a board-game-only reader finds none of them
+                "designers": links("boardgamedesigner") + links("rpgdesigner"),
+                "artists": links("boardgameartist") + links("rpgartist"),
+                "publishers": links("boardgamepublisher") + links("rpgpublisher"),
+                "categories": links("boardgamecategory")
+                + links("rpggenre")
+                + links("rpgcategory"),
+                "mechanics": links("boardgamemechanic") + links("rpgmechanic"),
+                "producers": links("rpgproducer"),
+                "series": links("rpgseries"),
                 "rating": _attr_float(ratings.find("average"))
                 if ratings is not None
                 else None,

src/bggpipe/static/app.css

@@ -487,6 +487,13 @@ a.game:focus-visible { outline: 3px solid var(--focus); outline-offset: 2px; }
 }
 .chiplist { display: inline-flex; flex-wrap: wrap; gap: .3rem; vertical-align: middle; }
 .gdesc { white-space: pre-wrap; line-height: 1.6; }
+.artbtn { margin-top: .6rem; width: 100%; font-size: .8rem; }
+.editform label.wide { flex: 1 1 100%; }
+.editform textarea {
+  font: inherit; font-size: .85rem; color: var(--ink); width: 100%;
+  border: 2px solid var(--board-edge); border-radius: var(--radius);
+  padding: .35rem .45rem; background: #fff; resize: vertical;
+}
 
 .empty {
   background: var(--board); border: 2px dashed var(--board-edge);

src/bggpipe/templates/pages/help.html

@@ -30,7 +30,7 @@
   <p><b><a href="/titles">Titles</a></b> — every read off your shelves, alphabetized, with its status and photos. This is the proofread checkpoint: <a href="#curation">edit, split, remove</a>. Its badge counts <span class="chip shaky">shaky read</span> lines — the model wasn't sure and nothing has verified them; filter to them, then press <b>✓ looks right</b> or edit each one.</p>
   <p><b><a href="/review">Review</a></b> — the decisions only you can make: which game a title is, which edition a copy is, whether two same-game reads are really one box (merges show a veto), and whether an unmatched title is a real game BGG simply doesn't have (<b>keep locally</b>: it joins the Library, never uploads). Keyboard-first; see <a href="#keys">shortcuts</a>.</p>
   <p><b><a href="/queue">Queue</a></b> — exactly what upload will do (new entries and version upgrades) and the log of everything it has done. Nothing reaches BGG that isn't visible here first. A job that fails is skipped by later runs (so one broken game can't loop forever); when any exist, the Pipeline's upload card offers a <b>retry N failed</b> checkbox. Each queued row shows what upload did with it — <span class="chip open">pending</span>, <span class="chip ok">done</span>, <span class="chip no">failed</span>, or <span class="chip no">retired</span> (a review decision since the last diff withdrew it). Finished rows stay listed until the next <b>diff</b> rebuilds the queue; the log below them is the permanent record.</p>
-  <p><b><a href="/library">Library</a></b> — your enriched collection. Search titles, designers, mechanics and categories at once; filter by kind (board games, RPGs, off-BGG) or by how many people are playing tonight; sort by name, year, BGG rank, weight, or playing time. Click any game for its full detail: art, the usual stats, designers and mechanics, <b>your</b> edition, the shelf photos it was read from, and a link to its BGG page. RPG and off-BGG games live here too — identified and enriched, never uploaded.</p>
+  <p><b><a href="/library">Library</a></b> — your enriched collection. Search titles, designers, mechanics and categories at once; filter by kind (board games, RPGs, off-BGG) or by how many people are playing tonight; sort by name, year, BGG rank, weight, or playing time. Click any game for its full detail: art, the usual stats, designers and mechanics, <b>your</b> edition, the shelf photos it was read from, and a link to its BGG page. RPG and off-BGG games live here too — identified and enriched, never uploaded. RPGs pull their designers, publishers and genres from RPGGeek; an off-BGG game's detail page lets you write its facts yourself and add a cover photo, since nothing else will ever have them (both are saved under <code>data/</code> and folded in by the next <b>enrich</b>).</p>
 </div>
 
 <h2 id="curation">Fixing the titles: edit, split, remove</h2>

src/bggpipe/templates/pages/librarygame.html

@@ -38,6 +38,66 @@ function playtime(g) {
   return `${g.playtime || g.min_playtime} min`;
 }
 
+function localForm(g) {
+  const v = (x) => (x === null || x === undefined ? "" : x);
+  return `
+  <h2>Your notes</h2>
+  <div class="card prose">
+    <p class="meta">BGG has no entry for this game, so what you type here is
+      all it will ever know. Saved to <code>data/local_games.json</code>.</p>
+    <form class="editform" id="localform">
+      <label>Title <input name="name" value="${esc(v(g.name))}" required></label>
+      <label>Year <input name="year" value="${esc(v(g.year))}" inputmode="numeric" size="6"></label>
+      <label>Players from <input name="min_players" value="${esc(v(g.min_players))}" size="3"></label>
+      <label>to <input name="max_players" value="${esc(v(g.max_players))}" size="3"></label>
+      <label>Minutes <input name="playtime" value="${esc(v(g.playtime))}" size="5"></label>
+      <label>Publishers <input name="publishers" value="${esc((g.publishers || []).join(", "))}"></label>
+      <label>Designers <input name="designers" value="${esc((g.designers || []).join(", "))}"></label>
+      <label class="wide">Notes
+        <textarea name="description" rows="4">${esc(v(g.description))}</textarea></label>
+      <span class="editactions"><button type="submit" class="primary">save</button></span>
+    </form>
+  </div>`;
+}
+
+function wireLocal(g) {
+  const form = document.getElementById("localform");
+  if (form) form.addEventListener("submit", async e => {
+    e.preventDefault();
+    const f = Object.fromEntries(new FormData(form).entries());
+    const res = await apiPost(`/api/local-game/${encodeURIComponent(KEY)}`, f);
+    if (res) {
+      showToast("saved — run <b>enrich</b> to fold this into the library");
+      refresh();
+    }
+  });
+  const btn = document.getElementById("artbtn");
+  const file = document.getElementById("artfile");
+  if (btn) btn.addEventListener("click", () => file.click());
+  if (file) file.addEventListener("change", async () => {
+    if (!file.files.length) return;
+    const body = new FormData();
+    body.append("file", file.files[0]);
+    btn.disabled = true;
+    btn.textContent = "uploading…";
+    let res = null;
+    try {
+      res = await fetch(`/api/local-art/${encodeURIComponent(KEY)}`, {method: "POST", body});
+    } catch (err) {
+      alert("Upload failed: " + err);
+    }
+    btn.disabled = false;
+    if (res && res.ok) {
+      showToast("photo saved — run <b>enrich</b> to fold it into the library");
+      refresh();
+    } else if (res) {
+      const detail = await res.json().then(d => d.detail).catch(() => null);
+      alert("Upload failed: " + (detail ?? res.statusText));
+      refresh();
+    }
+  });
+}
+
 function render(g) {
   document.getElementById("gname").textContent = g.name || "(unnamed)";
   document.title = `${g.name} · bggpipe`;
@@ -56,6 +116,13 @@ function render(g) {
   const art = g.image
     ? `<img class="gameart" src="${esc(g.image)}" alt="box art for ${esc(g.name)}">`
     : `<div class="gameart noart">${esc((g.name || "?")[0])}</div>`;
+  // an off-BGG game has no publisher art and no API to fetch any: the
+  // owner's own photo is the only cover it will ever have
+  const artAdd = local
+    ? `<button id="artbtn" class="artbtn">${g.image ? "replace" : "add"} a photo</button>
+       <input id="artfile" type="file" accept=".jpg,.jpeg,.png,.heic" hidden
+              aria-label="cover photo for ${esc(g.name)}">`
+    : "";
 
   const facts = [
     fact("players", players(g)),
@@ -93,13 +160,15 @@ function render(g) {
 
   document.getElementById("gbody").innerHTML = `
     <div class="gamedetail">
-      <div class="gameartcol">${art}</div>
+      <div class="gameartcol">${art}${artAdd}</div>
       <div class="gamefacts">${facts}</div>
     </div>
+    ${local ? localForm(g) : ""}
     ${version}
     ${photos}
-    ${g.description ? `<h2>About</h2>
+    ${g.description && !local ? `<h2>About</h2>
       <div class="card prose"><p class="gdesc">${esc(g.description)}</p></div>` : ""}`;
+  wireLocal(g);
 }
 
 async function refresh() {

src/bggpipe/webreview.py

@@ -15,6 +15,7 @@ execute one at a time in a JobRunner.
 from __future__ import annotations
 
 import csv
+import hashlib
 import io
 import json
 import os
@@ -186,6 +187,20 @@ class AddBody(BaseModel):
     language: str = ""
 
 
+class LocalGameBody(BaseModel):
+    """Hand-written facts for a game BGG doesn't have. Blank clears a
+    field; absent leaves it alone."""
+
+    name: str | None = None
+    year: str | None = None
+    publishers: str | None = None  # comma-separated, like the UI shows them
+    designers: str | None = None
+    min_players: str | None = None
+    max_players: str | None = None
+    playtime: str | None = None
+    description: str | None = None
+
+
 class RemoveBody(BaseModel):
     title_raw: str
     source_photos: str = ""
@@ -825,6 +840,85 @@ def create_app(
             key=lambda g: (g.get("name") or "").casefold(),
         )
 
+    LOCAL_ART_SUFFIXES = PHOTO_SUFFIXES
+
+    def _load_local_games() -> dict:
+        if not cfg.local_games_path.exists():
+            return {}
+        try:
+            return json.loads(cfg.local_games_path.read_text())
+        except json.JSONDecodeError as err:
+            raise HTTPException(
+                500, f"{cfg.local_games_path.name} is corrupt ({err})"
+            ) from err
+
+    def _int_or_none(value: str | None, field: str) -> int | None:
+        if value is None or not value.strip():
+            return None
+        if not value.strip().isdigit():
+            raise HTTPException(400, f"{field} must be a number")
+        return int(value)
+
+    @app.post("/api/local-game/{key:path}")
+    def api_local_game(key: str, body: LocalGameBody) -> dict:
+        with lock:
+            revision["n"] += 1
+            freshen()
+            if not key.startswith("local:"):
+                raise HTTPException(400, "only off-BGG games are hand-editable")
+            store = _load_local_games()
+            entry = dict(store.get(key) or {})
+            if body.name is not None and body.name.strip():
+                entry["name"] = body.name.strip()
+            for field in ("publishers", "designers"):
+                value = getattr(body, field)
+                if value is not None:
+                    entry[field] = [
+                        part.strip() for part in value.split(",") if part.strip()
+                    ]
+            for field in ("year", "min_players", "max_players", "playtime"):
+                value = getattr(body, field)
+                if value is not None:
+                    entry[field] = _int_or_none(value, field)
+            if body.description is not None:
+                entry["description"] = body.description.strip()
+            store[key] = {k: v for k, v in entry.items() if v not in (None, "", [])}
+            atomic_write_text(
+                cfg.local_games_path,
+                json.dumps(store, indent=2, ensure_ascii=False, sort_keys=True) + "\n",
+            )
+            return {"saved": store[key]}
+
+    @app.post("/api/local-art/{key:path}")
+    async def api_local_art(key: str, file: UploadFile) -> dict:
+        if not key.startswith("local:"):
+            raise HTTPException(400, "only off-BGG games take a hand-added photo")
+        suffix = Path(file.filename or "").suffix.lower()
+        if suffix not in LOCAL_ART_SUFFIXES:
+            raise HTTPException(400, f"not a photo: {file.filename or '(unnamed)'}")
+        # the key is arbitrary text; hash it into a safe, stable filename
+        name = hashlib.sha1(key.encode()).hexdigest()[:16] + suffix  # noqa: S324
+        cfg.local_art_dir.mkdir(parents=True, exist_ok=True)
+        atomic_write_bytes(cfg.local_art_dir / name, await file.read())
+        with lock:
+            revision["n"] += 1
+            store = _load_local_games()
+            entry = dict(store.get(key) or {})
+            entry["image"] = f"/local-art/{name}"
+            store[key] = entry
+            atomic_write_text(
+                cfg.local_games_path,
+                json.dumps(store, indent=2, ensure_ascii=False, sort_keys=True) + "\n",
+            )
+        return {"image": entry["image"]}
+
+    @app.get("/local-art/{name}")
+    def local_art(name: str) -> FileResponse:
+        target = cfg.local_art_dir / Path(name).name  # no traversal
+        if not target.exists():
+            raise HTTPException(404, "no such image")
+        return FileResponse(target)
+
     @app.get("/api/library/{key:path}")
     def api_library_game(key: str) -> dict:
         game = library_entries().get(key)

tests/test_webreview.py

@@ -1288,3 +1288,59 @@ def test_library_detail_serves_one_game_with_provenance(tmp_path):
     assert web.get("/api/library/nope").status_code == 404
     page = web.get("/library/game/240:24621")
     assert page.status_code == 200 and 'href="/library"' in page.text
+
+
+def test_local_game_notes_and_art_round_trip(tmp_path):
+    """BGG has nothing for an off-BGG game, so the owner's own words and
+    photo are its only metadata — and must survive enrich rebuilding
+    games.json from titles.json."""
+    from bggpipe.enrich import run_enrich
+
+    cfg = make_cfg(tmp_path)
+    rows = read_matches(cfg.matches_path)
+    rows.append(
+        _row(title_raw="Homebrew Game", match_status="local", source_photos="shelf.jpg")
+    )
+    write_matches(cfg.matches_path, rows)
+    web = TestClient(create_app(cfg, client=unauthorized_client(tmp_path)))
+    key = "local:homebrew game:shelf.jpg"
+
+    saved = web.post(
+        f"/api/local-game/{key}",
+        json={
+            "name": "Homebrew Game",
+            "year": "1998",
+            "min_players": "2",
+            "max_players": "6",
+            "publishers": "Basement Press, Friend's Garage",
+            "description": "  Made by a friend.  ",
+        },
+    ).json()["saved"]
+    assert saved["year"] == 1998
+    assert saved["publishers"] == ["Basement Press", "Friend's Garage"]
+    assert saved["description"] == "Made by a friend."
+
+    art = web.post(
+        f"/api/local-art/{key}",
+        files={"file": ("box.jpg", b"\xff\xd8jpeg", "image/jpeg")},
+    ).json()
+    assert art["image"].startswith("/local-art/")
+    assert web.get(art["image"]).status_code == 200
+
+    # enrich folds both into the library entry
+    games = run_enrich(cfg, client=unauthorized_client(tmp_path))
+    entry = games[key]
+    assert entry["year"] == 1998 and entry["max_players"] == 6
+    assert entry["image"] == art["image"]
+    assert entry["publishers"] == ["Basement Press", "Friend's Garage"]
+
+    # guards: BGG-matched games and non-photos are refused
+    assert web.post("/api/local-game/13", json={"name": "Catan"}).status_code == 400
+    assert (
+        web.post(
+            f"/api/local-art/{key}",
+            files={"file": ("notes.txt", b"hi", "text/plain")},
+        ).status_code
+        == 400
+    )
+    assert web.post(f"/api/local-game/{key}", json={"year": "19x8"}).status_code == 400