eric/bggpipe
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

bggpipe web: the whole pipeline as a local web app

Browse Source 
A dashboard at / joins the review page (now at /review): drag-and-drop
photo upload (re-uploading a photo drops its raw cache so extract
re-reads it), per-stage status cards fed by /api/pipeline (counts and
key NAMES only — never values), and run buttons that execute stages
one-at-a-time in a background JobRunner with captured output streamed
to the page. The real upload sits behind a confirmation, defaults to
dry-run at the API layer, and stays disabled while stub data is
present. The CLI is unchanged and shares all state with the web UI.
python-multipart joins the deps for the upload endpoint; RunBody lives
at module scope because postponed annotations keep FastAPI from
resolving function-local models.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Eric Wagoner
2026-08-02 17:08:00 -04:00
parent bf9795235a
commit 6ecdd43ed2
9 changed files with 776 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+2 -1
View File
@@ -20,13 +20,14 @@ Full design lives in `bgg-shelf-pipeline-spec.md` (read it before changing pipel
## Commands
- `uv sync` — install deps (Python 3.12+, managed by **uv**; use `uv add`, never pip). `uv run bggpipe init` handles first-run setup (folders, .env credentials, the one-time `playwright install chromium`).
- `uv run bggpipe web` — the full pipeline as a local web app (dashboard at `/`, review at `/review`); stage runs execute one-at-a-time in a background job.
- `uv run bggpipe <stage>` — run a pipeline stage. Non-secret settings come from `config.toml` (username, dirs, vision model, rate limit); `--config` overrides the path.
- `uv run pytest` — the suite runs fully offline against fixtures. Tests marked `live` hit the real BGG API (read-only) and are skipped unless you pass `--run-live`.
- `uv run ruff check` / `uv run ruff format` — lint (rules E, F, I, UP, B, SIM) and format.
## Layout
- `src/bggpipe/` — `cli.py` (typer app), one module per stage (`extract`, `resolve`, `review` + `webreview`, `diff`, `upload`, `enrich`), plus `bgg_client.py` (rate-limited XML API2 client that caches responses to `data/bgg_cache/`), `normalize.py` (title normalization), `models.py` (dataclasses), `config.py`.
- `src/bggpipe/` — `cli.py` (typer app), one module per stage (`extract`, `resolve`, `review` + `webreview`, `diff`, `upload`, `enrich`), plus `bgg_client.py` (rate-limited XML API2 client that caches responses to `data/bgg_cache/`), `jobs.py` (single-slot background stage runner for the web UI), `normalize.py` (title normalization), `models.py` (dataclasses), `config.py`, `fsio.py` (atomic writes), `init_wizard.py` (first-run setup).
- `scripts/` — `write_stub_fixtures.py` / `write_photo_fixtures.py` generate synthetic fixtures; `record_fixtures.py` re-records real API responses once a token exists.
- `tests/fixtures/bgg_cache/` — stub XML fixtures the offline tests run against.
- `data/` — pipeline state (CSV/JSON artifacts are committed; caches are not — see Git).