bggpipe export: the library as static pages for any site
Eric wanted the library on his blog; the spec always called games.json "the seed for a future web frontend." The new export stage renders it as self-contained static pages — an index with search, one page per game with facts, chips, the owner's edition and the description — that drop into any static host (Hugo's static/ folder included). No server, no build step, no external requests from the published pages. Public pages carry obligations a localhost app doesn't. Cover art is downloaded once from BGG's CDN instead of hotlinked (0.3s between fetches — a guest, not a crawler; part-file writes so a failure never leaves a truncated image; re-runs skip what exists, so the export is idempotent and resumable like every stage). The footer shows a Powered-by-BGG badge per BGG's public-app policy — text by default, upgraded to the official logo when the owner saves it from their registered-application page as data/powered-by-bgg.png — plus the trademark attribution. And one privacy rule, tested: shelf photos are never exported; they picture the inside of the owner's home. Covers and hand-added local art only, per Eric's explicit choice. Slugs are deterministic and collision-stable (two editions of one game get -2 suffixes in sorted-key order) so re-exports keep every URL. Descriptions un-double-encode BGG's entities. First real run: 136 pages, 254 covers, 64MB, live on the blog's static directory. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016jXZFSTZQKzAC8fqpWSz9g
This commit is contained in:
Eric Wagoner
co-authored by
Claude Fable 5
Claude Fable 5
parent
e778452dd1
commit
f677c7ce65
@@ -14,6 +14,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
|
|||||||
| 4 | `bggpipe diff` | diff approved matches against the existing BGG collection | working |
|
| 4 | `bggpipe diff` | diff approved matches against the existing BGG collection | working |
|
||||||
| 5 | `bggpipe upload` | add games via a logged-in Playwright session | working — all browser flows verified live 2026-08-06 (62 adds + 36 version updates landed) |
|
| 5 | `bggpipe upload` | add games via a logged-in Playwright session | working — all browser flows verified live 2026-08-06 (62 adds + 36 version updates landed) |
|
||||||
| 6 | `bggpipe enrich` | fetch full game/version metadata into `games.json` | working |
|
| 6 | `bggpipe enrich` | fetch full game/version metadata into `games.json` | working |
|
||||||
|
| + | `bggpipe export` | render the library as self-contained static pages (covers downloaded, never hotlinked; shelf photos never included; Powered-by-BGG badge slot) | working |
|
||||||
|
|
||||||
Full design lives in `docs/spec.md` (read it before changing pipeline semantics); the upload-stage walkthrough is in `docs/bgg-upload-flow.md`.
|
Full design lives in `docs/spec.md` (read it before changing pipeline semantics); the upload-stage walkthrough is in `docs/bgg-upload-flow.md`.
|
||||||
|
|
||||||
|
|||||||
@@ -27,7 +27,7 @@ BGG has no bulk import and no write API. Cataloging a few hundred games by hand
|
|||||||
3. **review** — A local review step for ambiguous matches: pick the right game/version, or leave the version blank. Wrong guesses never reach your collection.
|
3. **review** — A local review step for ambiguous matches: pick the right game/version, or leave the version blank. Wrong guesses never reach your collection.
|
||||||
4. **diff** — Your existing BGG collection is fetched and compared, per copy (owning one edition of a game doesn't hide a second edition you also own).
|
4. **diff** — Your existing BGG collection is fetched and compared, per copy (owning one edition of a game doesn't hide a second edition you also own).
|
||||||
5. **upload** — A Playwright browser session logs into your BGG account and adds each game (with its version, when known) politely and slowly. Dry-run mode, per-game logging, and resumability included.
|
5. **upload** — A Playwright browser session logs into your BGG account and adds each game (with its version, when known) politely and slowly. Dry-run mode, per-game logging, and resumability included.
|
||||||
6. **enrich** — Full metadata for every game (designers, player counts, weight, rank, mechanics, artwork, version details) lands in `data/games.json`, feeding a browsable library of your shelves.
|
6. **enrich** — Full metadata for every game (designers, player counts, weight, rank, mechanics, artwork, version details) lands in `data/games.json`, feeding a browsable library of your shelves — which `bggpipe export` can publish as [static pages on your own site](docs/guide.md#publishing-your-library-on-your-own-site).
|
||||||
|
|
||||||
Everything runs locally, every stage survives being killed mid-run, and all artifacts are flat CSV/JSON files you can inspect and edit. You can drive it from the terminal or from a local web app:
|
Everything runs locally, every stage survives being killed mid-run, and all artifacts are flat CSV/JSON files you can inspect and edit. You can drive it from the terminal or from a local web app:
|
||||||
|
|
||||||
|
|||||||
@@ -109,6 +109,29 @@ BGG application approval can take a week or more. Until then: `extract` works im
|
|||||||
- `https://boardgamegeek.com/xmlapi2/collection?username=YOU&own=1&version=1`
|
- `https://boardgamegeek.com/xmlapi2/collection?username=YOU&own=1&version=1`
|
||||||
- `https://boardgamegeek.com/xmlapi2/collection?username=YOU&own=1&version=1&subtype=boardgameexpansion`
|
- `https://boardgamegeek.com/xmlapi2/collection?username=YOU&own=1&version=1&subtype=boardgameexpansion`
|
||||||
|
|
||||||
|
## Publishing your library on your own site
|
||||||
|
|
||||||
|
```sh
|
||||||
|
bggpipe export --out ~/my-site/static/library --title "My Game Shelves"
|
||||||
|
```
|
||||||
|
|
||||||
|
Writes the library as self-contained static pages: an index with search, one
|
||||||
|
page per game, and a local copy of every cover (public pages must not hotlink
|
||||||
|
BGG's image CDN). No server, no build step — drop the directory into any
|
||||||
|
static host, Hugo/Jekyll `static/` folder included. Re-runs are idempotent:
|
||||||
|
already-downloaded covers are kept, failed downloads are retried, and page
|
||||||
|
URLs stay stable.
|
||||||
|
|
||||||
|
What's included follows a privacy rule: cover art, stats, your editions, and
|
||||||
|
your hand-added local games — **never your shelf photos** (they picture the
|
||||||
|
inside of your home; they stay in the private app).
|
||||||
|
|
||||||
|
BGG's API policy asks public-facing apps to display the "Powered by BGG"
|
||||||
|
badge. The footer carries a text badge automatically; to show the official
|
||||||
|
logo, save it from your registered application page at
|
||||||
|
[boardgamegeek.com/applications](https://boardgamegeek.com/applications) as
|
||||||
|
`data/powered-by-bgg.png` and re-export.
|
||||||
|
|
||||||
## Keeping your data safe from git
|
## Keeping your data safe from git
|
||||||
|
|
||||||
The [README's quick start](../README.md#quick-start) — installed as a tool, run in a directory of your own — is the only supported way to use bggpipe on your collection. Everything the pipeline produces lives where you run it, and `uv tool upgrade bggpipe` picks up fixes without going anywhere near your data.
|
The [README's quick start](../README.md#quick-start) — installed as a tool, run in a directory of your own — is the only supported way to use bggpipe on your collection. Everything the pipeline produces lives where you run it, and `uv tool upgrade bggpipe` picks up fixes without going anywhere near your data.
|
||||||
|
|||||||
@@ -178,6 +178,25 @@ def upload(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@app.command()
|
||||||
|
def export(
|
||||||
|
out: Annotated[
|
||||||
|
Path,
|
||||||
|
typer.Option("--out", help="Directory to write the static site into"),
|
||||||
|
],
|
||||||
|
title: Annotated[
|
||||||
|
str,
|
||||||
|
typer.Option("--title", help="Site title shown on every page"),
|
||||||
|
] = "My Game Library",
|
||||||
|
config: ConfigOpt = None,
|
||||||
|
) -> None:
|
||||||
|
"""Export the library as self-contained static pages (for a blog etc.)."""
|
||||||
|
from bggpipe.export import run_export
|
||||||
|
|
||||||
|
cfg = load_config(config)
|
||||||
|
run_export(cfg, out, title=title)
|
||||||
|
|
||||||
|
|
||||||
@app.command()
|
@app.command()
|
||||||
def enrich(
|
def enrich(
|
||||||
refresh: Annotated[
|
refresh: Annotated[
|
||||||
|
|||||||
@@ -0,0 +1,431 @@
|
|||||||
|
"""Export the library as self-contained static pages for any web host.
|
||||||
|
|
||||||
|
Reads games.json (the enrich output) and writes an index page, one page
|
||||||
|
per game, and a local copy of every cover image — no server, no build
|
||||||
|
step, no external requests from the published pages. Cover art is
|
||||||
|
downloaded once from BGG's CDN (public pages must not hotlink it) and
|
||||||
|
re-runs skip images already on disk, so the export is idempotent and
|
||||||
|
resumable like every other stage.
|
||||||
|
|
||||||
|
Shelf photos are deliberately NOT exported: they picture the inside of
|
||||||
|
the owner's home. Covers and hand-added local art only.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import html
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import shutil
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
from urllib.parse import urlsplit
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
import typer
|
||||||
|
|
||||||
|
from bggpipe.config import Config
|
||||||
|
|
||||||
|
_SLUG_RE = re.compile(r"[^a-z0-9]+")
|
||||||
|
|
||||||
|
# BGG's API policy: public-facing apps must display the "Powered by BGG"
|
||||||
|
# logo linking back. The official asset comes from the owner's registered
|
||||||
|
# application page — save it as data/powered-by-bgg.png and the export
|
||||||
|
# ships the image badge; until then, a text badge keeps the link.
|
||||||
|
BADGE_NAME = "powered-by-bgg.png"
|
||||||
|
|
||||||
|
|
||||||
|
def _powered_by(rel: str, badge_present: bool) -> str:
|
||||||
|
inner = (
|
||||||
|
f'<img src="{rel}art/{BADGE_NAME}" alt="Powered by BGG" height="40">'
|
||||||
|
if badge_present
|
||||||
|
else "Powered by BGG"
|
||||||
|
)
|
||||||
|
return f'<a class="pbb" href="https://boardgamegeek.com" rel="noopener">{inner}</a>'
|
||||||
|
|
||||||
|
|
||||||
|
def _slug(name: str) -> str:
|
||||||
|
return _SLUG_RE.sub("-", (name or "game").casefold()).strip("-") or "game"
|
||||||
|
|
||||||
|
|
||||||
|
def _slugs(games: dict[str, dict]) -> dict[str, str]:
|
||||||
|
"""Deterministic, unique, human-readable page names. Collisions (two
|
||||||
|
editions of one game share a name) get -2, -3... in sorted-key order
|
||||||
|
so re-exports keep every URL stable."""
|
||||||
|
out: dict[str, str] = {}
|
||||||
|
taken: set[str] = set()
|
||||||
|
for key in sorted(games):
|
||||||
|
base = _slug(games[key].get("name", ""))
|
||||||
|
slug, n = base, 2
|
||||||
|
while slug in taken:
|
||||||
|
slug = f"{base}-{n}"
|
||||||
|
n += 1
|
||||||
|
taken.add(slug)
|
||||||
|
out[key] = slug
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _art_name(key: str, url: str, *, thumb: bool) -> str:
|
||||||
|
ext = Path(urlsplit(url).path).suffix.lower() or ".jpg"
|
||||||
|
safe = _SLUG_RE.sub("-", key.casefold()).strip("-")
|
||||||
|
return f"{safe}-thumb{ext}" if thumb else f"{safe}{ext}"
|
||||||
|
|
||||||
|
|
||||||
|
def _fetch_art(
|
||||||
|
games: dict[str, dict],
|
||||||
|
art_dir: Path,
|
||||||
|
cfg: Config,
|
||||||
|
fetch: httpx.Client,
|
||||||
|
sleep,
|
||||||
|
) -> tuple[dict[str, dict[str, str]], int, list[str]]:
|
||||||
|
"""Local filename per game for thumb/full art. Downloads only what is
|
||||||
|
missing; a failed download is reported and the page ships without
|
||||||
|
that image — never a broken half-written file."""
|
||||||
|
art: dict[str, dict[str, str]] = {}
|
||||||
|
downloaded = 0
|
||||||
|
failures: list[str] = []
|
||||||
|
art_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
for key, game in games.items():
|
||||||
|
entry: dict[str, str] = {}
|
||||||
|
image = game.get("image") or ""
|
||||||
|
thumb = game.get("thumbnail") or ""
|
||||||
|
if image.startswith("/local-art/"):
|
||||||
|
# hand-added cover: already on disk, just carried across
|
||||||
|
src = cfg.local_art_dir / Path(image).name
|
||||||
|
if src.exists():
|
||||||
|
shutil.copyfile(src, art_dir / src.name)
|
||||||
|
entry["full"] = entry["thumb"] = src.name
|
||||||
|
else:
|
||||||
|
for url, is_thumb in ((image, False), (thumb, True)):
|
||||||
|
if not url.startswith("http"):
|
||||||
|
continue
|
||||||
|
name = _art_name(key, url, thumb=is_thumb)
|
||||||
|
target = art_dir / name
|
||||||
|
if not target.exists():
|
||||||
|
try:
|
||||||
|
resp = fetch.get(url)
|
||||||
|
resp.raise_for_status()
|
||||||
|
except httpx.HTTPError as err:
|
||||||
|
failures.append(f"{game.get('name', key)}: {err}")
|
||||||
|
continue
|
||||||
|
tmp = target.with_suffix(target.suffix + ".part")
|
||||||
|
tmp.write_bytes(resp.content)
|
||||||
|
tmp.replace(target)
|
||||||
|
downloaded += 1
|
||||||
|
sleep(0.3) # a guest on BGG's CDN, not a crawler
|
||||||
|
entry["thumb" if is_thumb else "full"] = name
|
||||||
|
if "thumb" not in entry and "full" in entry:
|
||||||
|
entry["thumb"] = entry["full"]
|
||||||
|
art[key] = entry
|
||||||
|
return art, downloaded, failures
|
||||||
|
|
||||||
|
|
||||||
|
_CSS = """
|
||||||
|
:root { --board:#faf3e3; --ink:#2a2438; --ink-soft:#6b6478; --line:#d8cdb4;
|
||||||
|
--sky:#cfe4f5; --navy:#2f3d5c; --accent:#7a4a9e; }
|
||||||
|
* { box-sizing:border-box; }
|
||||||
|
body { margin:0; font:16px/1.55 system-ui, sans-serif; color:var(--ink);
|
||||||
|
background:var(--sky); }
|
||||||
|
main { max-width:72rem; margin:0 auto; padding:1.2rem; }
|
||||||
|
header.site { background:var(--navy); color:#fff; padding:.9rem 1.2rem; }
|
||||||
|
header.site a { color:#fff; text-decoration:none; font-weight:700; }
|
||||||
|
header.site .sub { color:#cfe4f5; font-size:.85rem; }
|
||||||
|
.controls { display:flex; gap:.5rem; flex-wrap:wrap; margin:1rem 0; }
|
||||||
|
.controls input[type=search] { flex:1 1 14rem; padding:.45rem .6rem;
|
||||||
|
border:2px solid var(--line); border-radius:.5rem; font:inherit; }
|
||||||
|
.controls select, .controls button { font:inherit; padding:.4rem .7rem;
|
||||||
|
border:2px solid var(--line); border-radius:.5rem; background:var(--board);
|
||||||
|
cursor:pointer; }
|
||||||
|
.controls button[aria-pressed=true] { background:var(--accent); color:#fff; }
|
||||||
|
.grid { display:grid; grid-template-columns:repeat(auto-fill,minmax(11.5rem,1fr));
|
||||||
|
gap:1rem; }
|
||||||
|
a.game { background:var(--board); border:1px solid var(--line);
|
||||||
|
border-radius:.8rem; overflow:hidden; text-decoration:none; color:inherit;
|
||||||
|
display:flex; flex-direction:column; box-shadow:0 2px 5px rgba(42,36,56,.15); }
|
||||||
|
a.game:hover { outline:3px solid var(--accent); }
|
||||||
|
a.game img { width:100%; aspect-ratio:1; object-fit:cover; display:block;
|
||||||
|
background:#e8ddc5; }
|
||||||
|
a.game .noart { width:100%; aspect-ratio:1; display:flex; align-items:center;
|
||||||
|
justify-content:center; font-size:3rem; font-weight:700; color:var(--navy);
|
||||||
|
background:#e8ddc5; }
|
||||||
|
a.game .info { padding:.55rem .7rem .7rem; }
|
||||||
|
.gname { font-weight:700; }
|
||||||
|
.meta { color:var(--ink-soft); font-size:.82rem; }
|
||||||
|
.chip { display:inline-block; font-size:.7rem; border:1px solid var(--accent);
|
||||||
|
color:var(--accent); border-radius:1rem; padding:0 .5rem; }
|
||||||
|
.detail { display:flex; gap:1.4rem; flex-wrap:wrap; align-items:flex-start;
|
||||||
|
margin-top:1rem; }
|
||||||
|
.detail .art { flex:0 0 clamp(11rem,26vw,17rem); }
|
||||||
|
.detail .art img { width:100%; border-radius:.8rem; border:1px solid var(--line); }
|
||||||
|
.facts { flex:1 1 18rem; background:var(--board); border:1px solid var(--line);
|
||||||
|
border-radius:.8rem; padding:.9rem 1.1rem; }
|
||||||
|
.factrow { margin:.45rem 0; }
|
||||||
|
.factrow b { font-size:.72rem; text-transform:uppercase; letter-spacing:.06em;
|
||||||
|
color:var(--ink-soft); margin-right:.4rem; }
|
||||||
|
.card { background:var(--board); border:1px solid var(--line);
|
||||||
|
border-radius:.8rem; padding:.9rem 1.1rem; margin-top:1rem; }
|
||||||
|
.desc { white-space:pre-wrap; }
|
||||||
|
footer { margin:2rem 0 1rem; color:var(--ink-soft); font-size:.8rem;
|
||||||
|
text-align:center; }
|
||||||
|
footer a { color:var(--accent); }
|
||||||
|
a.pbb { font-weight:700; }
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def _chips(label: str, values: list | None) -> str:
|
||||||
|
if not values:
|
||||||
|
return ""
|
||||||
|
spans = " ".join(f'<span class="chip">{html.escape(str(v))}</span>' for v in values)
|
||||||
|
return f'<div class="factrow"><b>{html.escape(label)}</b> {spans}</div>'
|
||||||
|
|
||||||
|
|
||||||
|
def _fact(label: str, value) -> str:
|
||||||
|
if value in (None, "", []):
|
||||||
|
return ""
|
||||||
|
return (
|
||||||
|
f'<div class="factrow"><b>{html.escape(label)}</b> '
|
||||||
|
f"{html.escape(str(value))}</div>"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _players(g: dict) -> str:
|
||||||
|
lo, hi = g.get("min_players"), g.get("max_players")
|
||||||
|
lo, hi = lo or hi, hi or lo
|
||||||
|
if not lo:
|
||||||
|
return ""
|
||||||
|
span = str(lo) if lo == hi else f"{lo}–{hi}"
|
||||||
|
noun = "player" if lo == hi == 1 else "players"
|
||||||
|
best = g.get("best_player_counts") or []
|
||||||
|
return f"{span} {noun}" + (f" (best at {', '.join(best)})" if best else "")
|
||||||
|
|
||||||
|
|
||||||
|
def _playtime(g: dict) -> str:
|
||||||
|
lo, hi = g.get("min_playtime"), g.get("max_playtime")
|
||||||
|
if lo and hi and lo != hi:
|
||||||
|
return f"{lo}–{hi} min"
|
||||||
|
minutes = g.get("playtime") or lo
|
||||||
|
return f"{minutes} min" if minutes else ""
|
||||||
|
|
||||||
|
|
||||||
|
def _footer(title: str, rel: str, badge_present: bool) -> str:
|
||||||
|
return f"""<footer>
|
||||||
|
<p>{_powered_by(rel, badge_present)}</p>
|
||||||
|
<p>{html.escape(title)} · cataloged from shelf photos by
|
||||||
|
<a href="https://git.kestrelsnest.social/eric/bggpipe"
|
||||||
|
rel="noopener">bggpipe</a></p>
|
||||||
|
<p>Game data and cover images from <a href="https://boardgamegeek.com"
|
||||||
|
rel="noopener">BoardGameGeek</a>. BoardGameGeek and BGG are trademarks of
|
||||||
|
BoardGameGeek, LLC; this page is an independent project, not affiliated
|
||||||
|
with or endorsed by BoardGameGeek.</p>
|
||||||
|
</footer>"""
|
||||||
|
|
||||||
|
|
||||||
|
def _page(
|
||||||
|
title: str,
|
||||||
|
body: str,
|
||||||
|
*,
|
||||||
|
rel: str,
|
||||||
|
site_title: str,
|
||||||
|
badge_present: bool,
|
||||||
|
) -> str:
|
||||||
|
return f"""<!doctype html>
|
||||||
|
<html lang="en">
|
||||||
|
<meta charset="utf-8">
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||||
|
<title>{html.escape(title)}</title>
|
||||||
|
<link rel="stylesheet" href="{rel}style.css">
|
||||||
|
<header class="site"><a href="{rel}./">{html.escape(site_title)}</a>
|
||||||
|
<span class="sub">a board game library</span></header>
|
||||||
|
<main>
|
||||||
|
{body}
|
||||||
|
{_footer(site_title, rel, badge_present)}
|
||||||
|
</main>
|
||||||
|
</html>"""
|
||||||
|
|
||||||
|
|
||||||
|
def _card(key: str, g: dict, slug: str, art: dict[str, str]) -> str:
|
||||||
|
img = (
|
||||||
|
f'<img src="art/{html.escape(art["thumb"])}" alt="" loading="lazy">'
|
||||||
|
if art.get("thumb")
|
||||||
|
else '<div class="noart">'
|
||||||
|
+ html.escape((g.get("name") or "?")[0].upper())
|
||||||
|
+ "</div>"
|
||||||
|
)
|
||||||
|
year = f' <span class="meta">({g["year"]})</span>' if g.get("year") else ""
|
||||||
|
bits = " · ".join(
|
||||||
|
b
|
||||||
|
for b in (
|
||||||
|
_players(g),
|
||||||
|
_playtime(g),
|
||||||
|
f"rank {g['rank']}" if g.get("rank") else "",
|
||||||
|
)
|
||||||
|
if b
|
||||||
|
)
|
||||||
|
kind = ""
|
||||||
|
if g.get("type") == "rpgitem":
|
||||||
|
kind = ' <span class="chip">RPG</span>'
|
||||||
|
elif g.get("type") == "localgame":
|
||||||
|
kind = ' <span class="chip">not on BGG</span>'
|
||||||
|
name = html.escape(g.get("name") or "?")
|
||||||
|
return f"""<a class="game" href="{slug}/" data-name="{name}">
|
||||||
|
{img}
|
||||||
|
<div class="info"><div class="gname">{name}{year}{kind}</div>
|
||||||
|
<div class="meta">{html.escape(bits)}</div></div>
|
||||||
|
</a>"""
|
||||||
|
|
||||||
|
|
||||||
|
def _detail_body(g: dict, art: dict[str, str]) -> str:
|
||||||
|
art_html = (
|
||||||
|
f'<img src="../art/{html.escape(art["full"])}" '
|
||||||
|
f'alt="box art for {html.escape(g.get("name") or "")}">'
|
||||||
|
if art.get("full")
|
||||||
|
else ""
|
||||||
|
)
|
||||||
|
bgg_link = ""
|
||||||
|
if g.get("bgg_id") and g.get("type") != "localgame":
|
||||||
|
site = (
|
||||||
|
f"https://rpggeek.com/rpgitem/{g['bgg_id']}"
|
||||||
|
if g.get("type") == "rpgitem"
|
||||||
|
else f"https://boardgamegeek.com/boardgame/{g['bgg_id']}"
|
||||||
|
)
|
||||||
|
label = "RPGGeek" if g.get("type") == "rpgitem" else "BGG"
|
||||||
|
bgg_link = f'<a href="{site}" rel="noopener">view on {label} ↗</a>'
|
||||||
|
facts = "".join(
|
||||||
|
(
|
||||||
|
_fact("players", _players(g)),
|
||||||
|
_fact("playing time", _playtime(g)),
|
||||||
|
_fact("ages", f"{g['min_age']}+" if g.get("min_age") else ""),
|
||||||
|
_fact("weight", f"{g['weight']:.2f} / 5" if g.get("weight") else ""),
|
||||||
|
_fact("BGG rank", g.get("rank")),
|
||||||
|
_fact("BGG rating", f"{g['rating']:.2f}" if g.get("rating") else ""),
|
||||||
|
_chips("designers", g.get("designers")),
|
||||||
|
_chips("artists", (g.get("artists") or [])[:8]),
|
||||||
|
_chips("publishers", (g.get("publishers") or [])[:6]),
|
||||||
|
_chips("categories", g.get("categories")),
|
||||||
|
_chips("mechanics", g.get("mechanics")),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
version = ""
|
||||||
|
if g.get("version"):
|
||||||
|
v = g["version"]
|
||||||
|
vyear = f" · {v['year']}" if v.get("year") else ""
|
||||||
|
version = (
|
||||||
|
'<div class="card"><div class="factrow">'
|
||||||
|
f"<b>my copy</b> {html.escape(v.get('name') or '—')}{vyear}</div>"
|
||||||
|
+ _chips("publishers", v.get("publishers"))
|
||||||
|
+ _chips("languages", v.get("languages"))
|
||||||
|
+ "</div>"
|
||||||
|
)
|
||||||
|
desc = (
|
||||||
|
# BGG descriptions arrive with entities still encoded (’ etc);
|
||||||
|
# unescape their layer, then escape once for this page
|
||||||
|
f'<div class="card desc">{html.escape(html.unescape(g["description"]))}</div>'
|
||||||
|
if g.get("description")
|
||||||
|
else ""
|
||||||
|
)
|
||||||
|
year = f' <span class="meta">({g["year"]})</span>' if g.get("year") else ""
|
||||||
|
return f"""<p><a href="../">← the whole library</a></p>
|
||||||
|
<h1>{html.escape(g.get("name") or "?")}{year} {bgg_link}</h1>
|
||||||
|
<div class="detail">
|
||||||
|
<div class="art">{art_html}</div>
|
||||||
|
<div class="facts">{facts}</div>
|
||||||
|
</div>
|
||||||
|
{version}
|
||||||
|
{desc}"""
|
||||||
|
|
||||||
|
|
||||||
|
_INDEX_JS = """
|
||||||
|
<script>
|
||||||
|
"use strict";
|
||||||
|
const cards = Array.from(document.querySelectorAll("a.game"));
|
||||||
|
const rows = cards.map(c => ({el: c, text: (c.dataset.name + " " +
|
||||||
|
c.textContent).toLowerCase()}));
|
||||||
|
const q = document.getElementById("q");
|
||||||
|
q.addEventListener("input", () => {
|
||||||
|
const needle = q.value.trim().toLowerCase();
|
||||||
|
rows.forEach(r => r.el.style.display =
|
||||||
|
!needle || r.text.includes(needle) ? "" : "none");
|
||||||
|
});
|
||||||
|
</script>
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def run_export(
|
||||||
|
cfg: Config,
|
||||||
|
out_dir: Path,
|
||||||
|
*,
|
||||||
|
title: str = "My Game Library",
|
||||||
|
fetch: httpx.Client | None = None,
|
||||||
|
sleep=time.sleep,
|
||||||
|
) -> dict:
|
||||||
|
if not cfg.games_path.exists():
|
||||||
|
raise typer.Exit(
|
||||||
|
typer.secho(
|
||||||
|
f"{cfg.games_path} not found — run `bggpipe enrich` first.", err=True
|
||||||
|
)
|
||||||
|
or 1
|
||||||
|
)
|
||||||
|
games: dict[str, dict] = json.loads(cfg.games_path.read_text())
|
||||||
|
slugs = _slugs(games)
|
||||||
|
out_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
own_client = fetch is None
|
||||||
|
fetch = fetch or httpx.Client(
|
||||||
|
timeout=30, follow_redirects=True, headers={"User-Agent": "bggpipe-export"}
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
art, downloaded, failures = _fetch_art(
|
||||||
|
games, out_dir / "art", cfg, fetch, sleep
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
if own_client:
|
||||||
|
fetch.close()
|
||||||
|
|
||||||
|
(out_dir / "style.css").write_text(_CSS)
|
||||||
|
# the official badge, when the owner has saved it from their BGG
|
||||||
|
# application page; the footer falls back to a text badge without it
|
||||||
|
badge_src = cfg.data_dir / BADGE_NAME
|
||||||
|
badge_present = badge_src.exists()
|
||||||
|
if badge_present:
|
||||||
|
shutil.copyfile(badge_src, out_dir / "art" / BADGE_NAME)
|
||||||
|
else:
|
||||||
|
typer.echo(
|
||||||
|
" note: no data/powered-by-bgg.png — footer uses a text "
|
||||||
|
'"Powered by BGG" link; save the official badge from '
|
||||||
|
"boardgamegeek.com/applications to upgrade it"
|
||||||
|
)
|
||||||
|
ordered = sorted(games, key=lambda k: (games[k].get("name") or "").casefold())
|
||||||
|
cards = "\n".join(_card(k, games[k], slugs[k], art.get(k, {})) for k in ordered)
|
||||||
|
index_body = (
|
||||||
|
f"<h1>{html.escape(title)}</h1>"
|
||||||
|
f'<p class="meta">{len(games)} games, cataloged from shelf photos.</p>'
|
||||||
|
'<div class="controls"><input type="search" id="q" '
|
||||||
|
'placeholder="search the shelves…" aria-label="search"></div>'
|
||||||
|
f'<div class="grid">{cards}</div>{_INDEX_JS}'
|
||||||
|
)
|
||||||
|
(out_dir / "index.html").write_text(
|
||||||
|
_page(title, index_body, rel="", site_title=title, badge_present=badge_present)
|
||||||
|
)
|
||||||
|
for key, game in games.items():
|
||||||
|
page_dir = out_dir / slugs[key]
|
||||||
|
page_dir.mkdir(exist_ok=True)
|
||||||
|
(page_dir / "index.html").write_text(
|
||||||
|
_page(
|
||||||
|
f"{game.get('name') or 'game'} · {title}",
|
||||||
|
_detail_body(game, art.get(key, {})),
|
||||||
|
rel="../",
|
||||||
|
site_title=title,
|
||||||
|
badge_present=badge_present,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
for line in failures:
|
||||||
|
typer.echo(f" warning: cover not fetched — {line}")
|
||||||
|
typer.echo(
|
||||||
|
f"exported {len(games)} game page(s) to {out_dir} "
|
||||||
|
f"({downloaded} cover(s) downloaded, "
|
||||||
|
f"{len(failures)} failed — re-run to retry)"
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"games": len(games),
|
||||||
|
"downloaded": downloaded,
|
||||||
|
"failures": len(failures),
|
||||||
|
}
|
||||||
@@ -0,0 +1,166 @@
|
|||||||
|
"""Static export: pages, art downloads, badge slot, privacy. Offline —
|
||||||
|
cover downloads run against a mock CDN transport."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from bggpipe.config import Config
|
||||||
|
from bggpipe.export import run_export
|
||||||
|
|
||||||
|
GAMES = {
|
||||||
|
"240:24621": {
|
||||||
|
"bgg_id": 240,
|
||||||
|
"type": "boardgame",
|
||||||
|
"name": "Britannia",
|
||||||
|
"year": 1986,
|
||||||
|
"description": "A game of <invasions> & settlement.",
|
||||||
|
"image": "https://cf.geekdo-images.com/orig/pic1.jpg",
|
||||||
|
"thumbnail": "https://cf.geekdo-images.com/thumb/pic1.jpg",
|
||||||
|
"min_players": 3,
|
||||||
|
"max_players": 5,
|
||||||
|
"best_player_counts": ["4"],
|
||||||
|
"playtime": 240,
|
||||||
|
"designers": ["Lewis Pulsipher"],
|
||||||
|
"rank": 1210,
|
||||||
|
"weight": 3.16,
|
||||||
|
"version": {
|
||||||
|
"version_id": 24621,
|
||||||
|
"name": "Avalon Hill second edition",
|
||||||
|
"year": 1987,
|
||||||
|
"publishers": ["Avalon Hill"],
|
||||||
|
"languages": ["English"],
|
||||||
|
},
|
||||||
|
"source_photos": ["IMG_1.jpeg"],
|
||||||
|
},
|
||||||
|
"311654": {
|
||||||
|
"bgg_id": 311654,
|
||||||
|
"type": "rpgitem",
|
||||||
|
"name": "Alice is Missing",
|
||||||
|
"year": 2020,
|
||||||
|
"image": "https://cf.geekdo-images.com/orig/pic2.png",
|
||||||
|
"thumbnail": "",
|
||||||
|
"source_photos": ["IMG_2.jpeg"],
|
||||||
|
},
|
||||||
|
"local:homebrew:x.jpg": {
|
||||||
|
"type": "localgame",
|
||||||
|
"name": "Britannia", # deliberate name collision with 240:24621
|
||||||
|
"image": "/local-art/abc123.jpg",
|
||||||
|
"source_photos": ["x.jpg"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _cfg(tmp_path: Path) -> Config:
|
||||||
|
cfg = Config(data_dir=tmp_path / "data")
|
||||||
|
cfg.data_dir.mkdir(parents=True)
|
||||||
|
cfg.games_path.write_text(json.dumps(GAMES))
|
||||||
|
cfg.local_art_dir.mkdir()
|
||||||
|
(cfg.local_art_dir / "abc123.jpg").write_bytes(b"\xff\xd8local")
|
||||||
|
return cfg
|
||||||
|
|
||||||
|
|
||||||
|
def _cdn(requests_log: list[str]) -> httpx.Client:
|
||||||
|
def handler(request: httpx.Request) -> httpx.Response:
|
||||||
|
requests_log.append(str(request.url))
|
||||||
|
return httpx.Response(200, content=b"\xff\xd8fake-image")
|
||||||
|
|
||||||
|
return httpx.Client(transport=httpx.MockTransport(handler))
|
||||||
|
|
||||||
|
|
||||||
|
def test_export_writes_pages_art_and_no_shelf_photos(tmp_path):
|
||||||
|
cfg = _cfg(tmp_path)
|
||||||
|
out = tmp_path / "site"
|
||||||
|
log: list[str] = []
|
||||||
|
summary = run_export(
|
||||||
|
cfg, out, title="Test Shelves", fetch=_cdn(log), sleep=lambda s: None
|
||||||
|
)
|
||||||
|
assert summary == {"games": 3, "downloaded": 3, "failures": 0}
|
||||||
|
|
||||||
|
index = (out / "index.html").read_text()
|
||||||
|
assert "Test Shelves" in index and "Powered by BGG" in index
|
||||||
|
assert "trademarks of" in index
|
||||||
|
|
||||||
|
# name collision -> distinct stable slugs
|
||||||
|
assert (out / "britannia" / "index.html").exists()
|
||||||
|
assert (out / "britannia-2" / "index.html").exists()
|
||||||
|
assert (out / "alice-is-missing" / "index.html").exists()
|
||||||
|
|
||||||
|
detail = (out / "britannia" / "index.html").read_text()
|
||||||
|
assert "Avalon Hill second edition" in detail
|
||||||
|
assert "<invasions> & settlement" in detail # escaped, not raw
|
||||||
|
# BGG double-encodes entities; the page shows the character, not ’
|
||||||
|
assert "&rsquo" not in detail
|
||||||
|
assert "boardgamegeek.com/boardgame/240" in detail
|
||||||
|
rpg = (out / "alice-is-missing" / "index.html").read_text()
|
||||||
|
assert "rpggeek.com/rpgitem/311654" in rpg
|
||||||
|
|
||||||
|
# covers landed locally and pages never hotlink the CDN
|
||||||
|
assert len(list((out / "art").glob("*"))) == 4 # 2 full + 1 thumb + local art
|
||||||
|
all_html = index + detail + rpg
|
||||||
|
assert "geekdo-images.com" not in all_html
|
||||||
|
|
||||||
|
# the privacy rule: shelf photo names appear nowhere in the output
|
||||||
|
assert "IMG_1.jpeg" not in all_html and "IMG_2.jpeg" not in all_html
|
||||||
|
|
||||||
|
|
||||||
|
def test_export_is_idempotent_and_offline_on_rerun(tmp_path):
|
||||||
|
cfg = _cfg(tmp_path)
|
||||||
|
out = tmp_path / "site"
|
||||||
|
run_export(cfg, out, fetch=_cdn([]), sleep=lambda s: None)
|
||||||
|
|
||||||
|
def refuse(request: httpx.Request) -> httpx.Response:
|
||||||
|
raise AssertionError(f"re-run fetched {request.url}")
|
||||||
|
|
||||||
|
summary = run_export(
|
||||||
|
cfg,
|
||||||
|
out,
|
||||||
|
fetch=httpx.Client(transport=httpx.MockTransport(refuse)),
|
||||||
|
sleep=lambda s: None,
|
||||||
|
)
|
||||||
|
assert summary["downloaded"] == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_failed_cover_reports_and_ships_page_anyway(tmp_path):
|
||||||
|
cfg = _cfg(tmp_path)
|
||||||
|
|
||||||
|
def flaky(request: httpx.Request) -> httpx.Response:
|
||||||
|
if "pic2" in str(request.url):
|
||||||
|
return httpx.Response(503)
|
||||||
|
return httpx.Response(200, content=b"img")
|
||||||
|
|
||||||
|
summary = run_export(
|
||||||
|
cfg,
|
||||||
|
tmp_path / "site",
|
||||||
|
fetch=httpx.Client(transport=httpx.MockTransport(flaky)),
|
||||||
|
sleep=lambda s: None,
|
||||||
|
)
|
||||||
|
assert summary["failures"] == 1
|
||||||
|
page = (tmp_path / "site" / "alice-is-missing" / "index.html").read_text()
|
||||||
|
assert "Alice is Missing" in page # the page exists, just coverless
|
||||||
|
# nothing half-written left behind to poison the idempotent re-run
|
||||||
|
assert not list((tmp_path / "site" / "art").glob("*.part"))
|
||||||
|
|
||||||
|
|
||||||
|
def test_official_badge_upgrades_the_footer(tmp_path):
|
||||||
|
cfg = _cfg(tmp_path)
|
||||||
|
(cfg.data_dir / "powered-by-bgg.png").write_bytes(b"\x89PNG-badge")
|
||||||
|
out = tmp_path / "site"
|
||||||
|
run_export(cfg, out, fetch=_cdn([]), sleep=lambda s: None)
|
||||||
|
assert (out / "art" / "powered-by-bgg.png").exists()
|
||||||
|
index = (out / "index.html").read_text()
|
||||||
|
assert 'img src="art/powered-by-bgg.png" alt="Powered by BGG"' in index
|
||||||
|
detail = (out / "britannia" / "index.html").read_text()
|
||||||
|
assert 'img src="../art/powered-by-bgg.png"' in detail
|
||||||
|
|
||||||
|
|
||||||
|
def test_export_without_games_json_exits_with_guidance(tmp_path):
|
||||||
|
cfg = Config(data_dir=tmp_path / "data")
|
||||||
|
import typer
|
||||||
|
|
||||||
|
with pytest.raises(typer.Exit):
|
||||||
|
run_export(cfg, tmp_path / "site", fetch=_cdn([]), sleep=lambda s: None)
|
||||||
Reference in New Issue
Block a user