Defuse the clone-and-run trap: tracked pipeline data warns loudly

Eric spotted it: the README told people to clone this repo, rm the
committed data, and run — which writes THEIR pipeline artifacts at
git-TRACKED paths. The next `git pull` (this repo commits data every
session) refuses to merge, and the internet's standard remedies for
that error — reset --hard, checkout ., stash, clean -fdx — destroy
their review decisions, hand-written games, upload log, and photos.

Two layers. The README's "Bring your own shelves" now leads with
`uv tool install git+…` and running in a directory of your own: data
lands untracked by construction and a bug fix is `uv tool upgrade`,
which cannot touch it. And because nobody re-reads a README, Config
gains tracked_data_warning(): if artifacts under data_dir are
git-tracked, `bggpipe init` and the web dashboard both warn in plain
words. The owner's exemption is data/.own_repo — a GITIGNORED marker,
so the author's checkout is silent while a fresh clone of the same
repo still gets the warning (a committed marker or config key would
have shipped the exemption to exactly the people who need warning).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016jXZFSTZQKzAC8fqpWSz9g
This commit is contained in:
Eric Wagoner
2026-08-06 00:43:35 -04:00
co-authored by Claude Fable 5
parent 77d330748d
commit dec2bfc7b6
7 changed files with 94 additions and 5 deletions
+10 -4
View File
@@ -143,14 +143,20 @@ Straight-on, one shelf (or part of one) per shot, close enough that spine text i
## Bring your own shelves
This repo doubles as its author's live pipeline, so `data/` ships with his real artifacts — extracted titles, matches, and 2018 collection snapshots. Before running against *your* shelves, clear the data:
Install bggpipe as a tool and run it in its own directory — **not inside a clone of this repo**:
```sh
rm data/*.csv data/*.json data/*.xml data/STUB_DATA.marker
rm -rf data/bgg_cache data/extract_raw
uv tool install git+https://git.kestrelsnest.social/eric/bggpipe
mkdir ~/shelves && cd ~/shelves
bggpipe init # folders, credentials, one-time browser install
bggpipe web # or run stages from the terminal
```
Two of those files deserve a word:
Everything the pipeline produces — photos, matches, review decisions, your upload log — lives in the directory where you run it. A bug fix is `uv tool upgrade bggpipe`, which by construction cannot touch your data.
**Why not clone and run?** This repo doubles as its author's live pipeline: `data/` ships with their real artifacts, committed and updated often. Run inside a clone and *your* data lands at git-tracked paths — the next `git pull` will refuse to merge, and the usual remedies (`git reset --hard`, `git checkout .`, `git stash`, `git clean -fdx`) would destroy your review decisions, hand-written games, upload log, and photos. bggpipe detects this arrangement and warns at `init` and on the web dashboard; don't ignore it. Clone only to develop (see [Development](#development)), and keep your own pipeline runs elsewhere.
Two files of the author's data deserve a word:
- **`data/STUB_DATA.marker`** — normally absent. It appears only if the synthetic stub fixtures (from `scripts/write_stub_fixtures.py`) regenerate the CSVs, and the upload stage refuses to run while it exists — so placeholder data can never reach a real BGG account. The committed CSVs are real API data.
- **`data/collection_snapshot_*.xml`** — with `BGG_API_TOKEN` set, `diff` fetches your collection live and you don't need these. Without a token (still waiting on approval?), you can use the logged-in-browser exemption: while signed in to BGG, save these two URLs as `data/collection_snapshot_base.xml` and `data/collection_snapshot_expansions.xml` (if you get a "queued" message, refresh after a few seconds):