Review decisions outrank the queue; stop waiting on OUR name for a heading

Two bugs behind one failure. Dungeons & Dragons timed out waiting for a
dialog heading matching our stored name — but BGG 140509's primary name
is "Dragones Y Mazmorras"; we matched it through an ALTERNATE name, so
that heading never appears. The add flow now waits for the Own checkbox
(the form itself) instead: /boardgame/<id>/ already establishes which
game the page is.

And the job should not have run at all. to_add.csv is a snapshot from
the last diff, so any review decision taken afterwards — local,
rejected, wrong-match — was invisible to upload. run_upload now
cross-checks every queued job against the CURRENT matches.csv and skips
those it no longer endorses, naming each and pointing at diff. When
matches.csv is absent or empty it condemns nothing: absence is not a
verdict.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016jXZFSTZQKzAC8fqpWSz9g
This commit is contained in:
Eric Wagoner
2026-08-05 23:02:59 -04:00
co-authored by Claude Fable 5
parent 6e72ef22d1
commit 99c1fbf02d
3 changed files with 111 additions and 5 deletions
+45
View File
@@ -20,3 +20,48 @@ add,280794,,Etherfields,458300,,added,2026-08-06T02:49:53+00:00,
add,380837,,Botany,650545,,added,2026-08-06T02:50:00+00:00, add,380837,,Botany,650545,,added,2026-08-06T02:50:00+00:00,
add,393672,,Gloomhaven: Buttons & Bugs,669782,,failed,2026-08-06T02:50:07+00:00,RuntimeError: 'Gloomhaven: Buttons & Bugs (English edition) (2024)' was listed a moment ago but vanished on the second pass — retryable add,393672,,Gloomhaven: Buttons & Bugs,669782,,failed,2026-08-06T02:50:07+00:00,RuntimeError: 'Gloomhaven: Buttons & Bugs (English edition) (2024)' was listed a moment ago but vanished on the second pass — retryable
add,14535,,SPANC: Space Pirate Amazon Ninja Catgirls,29075,,added,2026-08-06T02:50:15+00:00, add,14535,,SPANC: Space Pirate Amazon Ninja Catgirls,29075,,added,2026-08-06T02:50:15+00:00,
add,255984,,Sleeping Gods,701034,,added,2026-08-06T02:54:05+00:00,
add,393672,,Gloomhaven: Buttons & Bugs,669782,,added,2026-08-06T02:54:12+00:00,
add,9666,,Greed Quest,215137,,added,2026-08-06T02:54:19+00:00,
add,40508,,Scrabble Slam!,644326,,added,2026-08-06T02:54:27+00:00,
add,3181,,Farkle,87931,,added,2026-08-06T02:54:36+00:00,
add,193485,,Dastardly Dirigibles,300446,,added,2026-08-06T02:54:54+00:00,
add,226065,,Sheriff of Nottingham: Merry Men,353389,,added,2026-08-06T02:55:01+00:00,
add,143884,,Machi Koro,453051,,added,2026-08-06T02:55:09+00:00,
add,358880,,Etherfields: Kittenburg Expansion,603694,,added,2026-08-06T02:55:15+00:00,
add,432858,,Artistry,734890,,added,2026-08-06T02:55:21+00:00,
add,417060,,Botany: Tantalizing Trees,706506,,added,2026-08-06T02:55:27+00:00,
add,417064,,Botany: Perilous Perfumes,706510,,added,2026-08-06T02:55:33+00:00,
add,161928,,Utter Nonsense,,,added,2026-08-06T02:55:40+00:00,
add,50381,,Cards Against Humanity,,,added,2026-08-06T02:55:44+00:00,
add,1269,,Skip-Bo,,,added,2026-08-06T02:55:50+00:00,
add,202982,,Scrawl,,,added,2026-08-06T02:55:56+00:00,
add,195314,,Herbaceous,,,added,2026-08-06T02:56:02+00:00,
add,320202,,Dragon Land,,,added,2026-08-06T02:56:07+00:00,
add,2452,,Jenga,,,added,2026-08-06T02:56:12+00:00,
add,42063,,Code Master,,,added,2026-08-06T02:56:18+00:00,
add,340420,,Throw Throw Avocado,,,added,2026-08-06T02:56:23+00:00,
add,1260,,Rook,,,added,2026-08-06T02:56:28+00:00,
add,116,,Guillotine,,,added,2026-08-06T02:56:34+00:00,
add,169124,,Flick 'em Up!,,,added,2026-08-06T02:56:40+00:00,
add,20866,,The Rocky Horror Trivia Game,,,added,2026-08-06T02:56:45+00:00,
add,269564,,Captain Marvel: Secret Skrulls,,,added,2026-08-06T02:57:37+00:00,
add,18755,,Pie Face!,,,added,2026-08-06T02:57:41+00:00,
add,140509,,Dungeons & Dragons,,,failed,2026-08-06T02:58:01+00:00,"TimeoutError: Locator.wait_for: Timeout 15000ms exceeded. Call log: - waiting for get_by_role(""dialog"").get_by_role(""heading"", name=re.compile(r""Dungeons\ \&\ Dragons"", re.IGNORECASE)) to be visible"
add,201248,,Evolution: The Beginning,,,added,2026-08-06T02:58:05+00:00,
add,362205,,Sleeping Gods: Primeval Peril,,,added,2026-08-06T02:58:11+00:00,
add,453845,,Murder at the Manor,,,added,2026-08-06T02:58:16+00:00,
add,329230,,Bluffaneer,,,added,2026-08-06T02:58:20+00:00,
add,358320,,Sleeping Gods: Distant Skies,,,added,2026-08-06T02:58:26+00:00,
add,389113,,Rivals,,,added,2026-08-06T02:58:31+00:00,
add,324937,,Wiz-War (9th Edition),,,added,2026-08-06T02:58:35+00:00,
add,295770,,Frosthaven,,,added,2026-08-06T02:58:40+00:00,
add,174430,,Gloomhaven,,,added,2026-08-06T02:58:46+00:00,
add,442312,,Deadly Doodles,,,added,2026-08-06T02:58:51+00:00,
add,332398,,Everdell: The Complete Collection,,,added,2026-08-06T02:58:56+00:00,
add,334065,,Verdant,,,added,2026-08-06T02:59:01+00:00,
add,283155,,Calico,,,added,2026-08-06T02:59:06+00:00,
add,291457,,Gloomhaven: Jaws of the Lion,,,added,2026-08-06T02:59:12+00:00,
add,150925,,Hold Your Breath!,,,added,2026-08-06T02:59:18+00:00,
add,2980,,The Simpsons Trivia Game,,,added,2026-08-06T02:59:23+00:00,
add,1197,,Blink,,,added,2026-08-06T02:59:29+00:00,
1 action bgg_id collid name version_id second_copy status timestamp error
20 add 380837 Botany 650545 added 2026-08-06T02:50:00+00:00
21 add 393672 Gloomhaven: Buttons & Bugs 669782 failed 2026-08-06T02:50:07+00:00 RuntimeError: 'Gloomhaven: Buttons & Bugs (English edition) (2024)' was listed a moment ago but vanished on the second pass — retryable
22 add 14535 SPANC: Space Pirate Amazon Ninja Catgirls 29075 added 2026-08-06T02:50:15+00:00
23 add 255984 Sleeping Gods 701034 added 2026-08-06T02:54:05+00:00
24 add 393672 Gloomhaven: Buttons & Bugs 669782 added 2026-08-06T02:54:12+00:00
25 add 9666 Greed Quest 215137 added 2026-08-06T02:54:19+00:00
26 add 40508 Scrabble Slam! 644326 added 2026-08-06T02:54:27+00:00
27 add 3181 Farkle 87931 added 2026-08-06T02:54:36+00:00
28 add 193485 Dastardly Dirigibles 300446 added 2026-08-06T02:54:54+00:00
29 add 226065 Sheriff of Nottingham: Merry Men 353389 added 2026-08-06T02:55:01+00:00
30 add 143884 Machi Koro 453051 added 2026-08-06T02:55:09+00:00
31 add 358880 Etherfields: Kittenburg Expansion 603694 added 2026-08-06T02:55:15+00:00
32 add 432858 Artistry 734890 added 2026-08-06T02:55:21+00:00
33 add 417060 Botany: Tantalizing Trees 706506 added 2026-08-06T02:55:27+00:00
34 add 417064 Botany: Perilous Perfumes 706510 added 2026-08-06T02:55:33+00:00
35 add 161928 Utter Nonsense added 2026-08-06T02:55:40+00:00
36 add 50381 Cards Against Humanity added 2026-08-06T02:55:44+00:00
37 add 1269 Skip-Bo added 2026-08-06T02:55:50+00:00
38 add 202982 Scrawl added 2026-08-06T02:55:56+00:00
39 add 195314 Herbaceous added 2026-08-06T02:56:02+00:00
40 add 320202 Dragon Land added 2026-08-06T02:56:07+00:00
41 add 2452 Jenga added 2026-08-06T02:56:12+00:00
42 add 42063 Code Master added 2026-08-06T02:56:18+00:00
43 add 340420 Throw Throw Avocado added 2026-08-06T02:56:23+00:00
44 add 1260 Rook added 2026-08-06T02:56:28+00:00
45 add 116 Guillotine added 2026-08-06T02:56:34+00:00
46 add 169124 Flick 'em Up! added 2026-08-06T02:56:40+00:00
47 add 20866 The Rocky Horror Trivia Game added 2026-08-06T02:56:45+00:00
48 add 269564 Captain Marvel: Secret Skrulls added 2026-08-06T02:57:37+00:00
49 add 18755 Pie Face! added 2026-08-06T02:57:41+00:00
50 add 140509 Dungeons & Dragons failed 2026-08-06T02:58:01+00:00 TimeoutError: Locator.wait_for: Timeout 15000ms exceeded. Call log: - waiting for get_by_role("dialog").get_by_role("heading", name=re.compile(r"Dungeons\ \&\ Dragons", re.IGNORECASE)) to be visible
51 add 201248 Evolution: The Beginning added 2026-08-06T02:58:05+00:00
52 add 362205 Sleeping Gods: Primeval Peril added 2026-08-06T02:58:11+00:00
53 add 453845 Murder at the Manor added 2026-08-06T02:58:16+00:00
54 add 329230 Bluffaneer added 2026-08-06T02:58:20+00:00
55 add 358320 Sleeping Gods: Distant Skies added 2026-08-06T02:58:26+00:00
56 add 389113 Rivals added 2026-08-06T02:58:31+00:00
57 add 324937 Wiz-War (9th Edition) added 2026-08-06T02:58:35+00:00
58 add 295770 Frosthaven added 2026-08-06T02:58:40+00:00
59 add 174430 Gloomhaven added 2026-08-06T02:58:46+00:00
60 add 442312 Deadly Doodles added 2026-08-06T02:58:51+00:00
61 add 332398 Everdell: The Complete Collection added 2026-08-06T02:58:56+00:00
62 add 334065 Verdant added 2026-08-06T02:59:01+00:00
63 add 283155 Calico added 2026-08-06T02:59:06+00:00
64 add 291457 Gloomhaven: Jaws of the Lion added 2026-08-06T02:59:12+00:00
65 add 150925 Hold Your Breath! added 2026-08-06T02:59:18+00:00
66 add 2980 The Simpsons Trivia Game added 2026-08-06T02:59:23+00:00
67 add 1197 Blink added 2026-08-06T02:59:29+00:00
+47 -5
View File
@@ -39,7 +39,8 @@ import typer
from bggpipe.bgg_client import BGGAuthError, BGGClient, client_for from bggpipe.bgg_client import BGGAuthError, BGGClient, client_for
from bggpipe.config import Config from bggpipe.config import Config
from bggpipe.fsio import atomic_write_text from bggpipe.fsio import atomic_write_text
from bggpipe.models import CollectionItem from bggpipe.models import CollectionItem, is_recognized
from bggpipe.resolve import read_matches
BGG = "https://boardgamegeek.com" BGG = "https://boardgamegeek.com"
UPLOAD_LOG_COLUMNS = [ UPLOAD_LOG_COLUMNS = [
@@ -89,6 +90,33 @@ def _job_key(row: dict) -> tuple[str, str, str]:
return _key(row["action"], row["bgg_id"], row["collid"], row["version_id"]) return _key(row["action"], row["bgg_id"], row["collid"], row["version_id"])
def stale_jobs(queue_rows: list[dict], match_rows: list[dict]) -> dict[str, str]:
"""bgg_id -> why, for queued games the CURRENT matches.csv no longer
endorses. to_add.csv is a snapshot from the last diff; a review
decision taken afterwards (marking a game local, rejecting it, calling
a match wrong) must not still upload."""
if not match_rows:
# nothing to compare against (no matches.csv, or a test harness
# driving the queue directly): absence is not a verdict
return {}
live: dict[str, list[dict]] = {}
for row in match_rows:
if row.get("bgg_id"):
live.setdefault(row["bgg_id"], []).append(row)
stale = {}
for row in queue_rows:
bgg_id = row.get("bgg_id") or ""
rows = live.get(bgg_id, [])
if any(is_recognized(r) for r in rows):
continue
if not rows:
stale[bgg_id] = "no longer matched to this game in matches.csv"
else:
statuses = sorted({r["match_status"] for r in rows})
stale[bgg_id] = f"now {', '.join(statuses)} in matches.csv"
return stale
def outstanding_failures(log_rows: list[dict]) -> int: def outstanding_failures(log_rows: list[dict]) -> int:
"""Jobs whose LATEST attempt failed. upload_log.csv is an append-only """Jobs whose LATEST attempt failed. upload_log.csv is an append-only
audit trail, so counting every 'failed' row ever written would keep audit trail, so counting every 'failed' row ever written would keep
@@ -515,10 +543,12 @@ class PlaywrightUploader:
self._goto(f"{BGG}/boardgame/{job.bgg_id}/") self._goto(f"{BGG}/boardgame/{job.bgg_id}/")
add_btn = page.get_by_role("button", name="Add To").first add_btn = page.get_by_role("button", name="Add To").first
dialog = self._open_dialog(add_btn) dialog = self._open_dialog(add_btn)
# Content settles when the game-name heading replaces "Loading...". # Wait for the form itself, NOT for a heading matching our stored
dialog.get_by_role( # name: a match made through an ALTERNATE name (BGG 140509 is
"heading", name=re.compile(re.escape(job.name), re.I) # "Dungeons & Dragons" to search, "Dragones Y Mazmorras" on the
).wait_for(timeout=15_000) # page) would never show it. The /boardgame/<id>/ URL already
# guarantees which game this is.
dialog.get_by_role("checkbox", name="Own", exact=True).wait_for(timeout=15_000)
# exact: "Own" is a substring of "Prev. Owned", and a loose label # exact: "Own" is a substring of "Prev. Owned", and a loose label
# match resolves to both checkboxes (strict-mode violation) # match resolves to both checkboxes (strict-mode violation)
dialog.get_by_role("checkbox", name="Own", exact=True).check() dialog.get_by_role("checkbox", name="Own", exact=True).check()
@@ -747,6 +777,18 @@ def run_upload(
to_update = _read_csv(cfg.to_update_path) to_update = _read_csv(cfg.to_update_path)
log_rows = _read_csv(log_path) log_rows = _read_csv(log_path)
# The queue is a snapshot; review decisions since the last diff win.
stale = stale_jobs(to_add + to_update, read_matches(cfg.matches_path))
if stale:
to_add = [r for r in to_add if r.get("bgg_id") not in stale]
to_update = [r for r in to_update if r.get("bgg_id") not in stale]
typer.echo(
f"Skipping {len(stale)} queued game(s) whose review decision "
"changed since the last diff — re-run diff to refresh the queue:"
)
for bgg_id, why in stale.items():
typer.echo(f" {bgg_id}: {why}")
jobs, skipped_done, skipped_failed, deferred = build_queue( jobs, skipped_done, skipped_failed, deferred = build_queue(
to_add, to_update, log_rows, retry_failed=retry_failed to_add, to_update, log_rows, retry_failed=retry_failed
) )
+19
View File
@@ -754,3 +754,22 @@ def test_paging_state_survives_reopen_so_page_one_is_clicked():
assert picked is True assert picked is True
assert why == "" assert why == ""
assert dialog.clicked.startswith("Sleeping Gods (English Gamefound") assert dialog.clicked.startswith("Sleeping Gods (English Gamefound")
def test_stale_queue_jobs_are_skipped(tmp_path):
"""to_add.csv is a snapshot from the last diff. A review decision made
afterwards — marking a game local, rejecting it, calling the match
wrong — must win over the stale queue."""
from bggpipe.upload import stale_jobs
queue = [{"bgg_id": "13"}, {"bgg_id": "140509"}, {"bgg_id": "999"}]
matches = [
{"bgg_id": "13", "match_status": "auto"},
{"bgg_id": "140509", "match_status": "local"},
]
stale = stale_jobs(queue, matches)
assert "13" not in stale # still endorsed
assert "now local" in stale["140509"]
assert "no longer matched" in stale["999"]
# unverifiable (no matches.csv at all) never condemns a job
assert stale_jobs(queue, []) == {}