Stage 5 upload: Playwright-driven adds and version updates

Queue from to_add/to_update minus upload_log.csv (append-per-attempt,
so runs resume); per-game failure isolation with 2-4s pacing;
--dry-run/--verify/--retry-failed/--limit; stub-fixture marker blocks
real runs, dry-run warns. Headed browser by default: live recon showed
Cloudflare Turnstile hard-blocks headless, and BGG never reaches
networkidle. Login selectors verified anonymously; version-picker
pagination and the collection-row update flow remain unverified until
real data exists. Client collection fetches gain a refresh passthrough
so --verify sees the live collection, not cache.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Eric Wagoner
2026-08-01 19:03:25 -04:00
parent 196862e243
commit 3ca7e7f650
8 changed files with 968 additions and 19 deletions
+3 -3
View File
@@ -12,21 +12,21 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
| 2 | `bggpipe resolve` | match titles to BGG IDs and versions via XML API2 | working (stub data — see hard rules) | | 2 | `bggpipe resolve` | match titles to BGG IDs and versions via XML API2 | working (stub data — see hard rules) |
| 3 | `bggpipe review` | human review of ambiguous/unmatched items; `--web` serves a FastAPI UI on port 8377 | working | | 3 | `bggpipe review` | human review of ambiguous/unmatched items; `--web` serves a FastAPI UI on port 8377 | working |
| 4 | `bggpipe diff` | diff approved matches against the existing BGG collection | working | | 4 | `bggpipe diff` | diff approved matches against the existing BGG collection | working |
| 5 | `bggpipe upload` | add games via a logged-in Playwright session | **not built** — CLI stub exits 1; Playwright not yet a dependency | | 5 | `bggpipe upload` | add games via a logged-in Playwright session | built; browser flows unverified until real data exists (`--dry-run` works now) |
| 6 | `bggpipe enrich` | fetch full game/version metadata into `games.json` | working | | 6 | `bggpipe enrich` | fetch full game/version metadata into `games.json` | working |
Full design lives in `bgg-shelf-pipeline-spec.md` (read it before changing pipeline semantics); the upload-stage walkthrough is in `docs/bgg-upload-flow.md`. Full design lives in `bgg-shelf-pipeline-spec.md` (read it before changing pipeline semantics); the upload-stage walkthrough is in `docs/bgg-upload-flow.md`.
## Commands ## Commands
- `uv sync` — install deps (Python 3.12+, managed by **uv**; use `uv add`, never pip). - `uv sync` — install deps (Python 3.12+, managed by **uv**; use `uv add`, never pip). Playwright needs a one-time `uv run playwright install chromium`.
- `uv run bggpipe <stage>` — run a pipeline stage. Non-secret settings come from `config.toml` (username, dirs, vision model, rate limit); `--config` overrides the path. - `uv run bggpipe <stage>` — run a pipeline stage. Non-secret settings come from `config.toml` (username, dirs, vision model, rate limit); `--config` overrides the path.
- `uv run pytest` — 105 tests, all offline against fixtures. Tests marked `live` hit the real BGG API (read-only) and are skipped unless you pass `--run-live`. - `uv run pytest` — 105 tests, all offline against fixtures. Tests marked `live` hit the real BGG API (read-only) and are skipped unless you pass `--run-live`.
- `uv run ruff check` / `uv run ruff format` — lint (rules E, F, I, UP, B, SIM) and format. - `uv run ruff check` / `uv run ruff format` — lint (rules E, F, I, UP, B, SIM) and format.
## Layout ## Layout
- `src/bggpipe/``cli.py` (typer app), one module per stage (`extract`, `resolve`, `review` + `webreview`, `diff`, `enrich`), plus `bgg_client.py` (rate-limited XML API2 client that caches responses to `data/bgg_cache/`), `normalize.py` (title normalization), `models.py` (dataclasses), `config.py`. - `src/bggpipe/``cli.py` (typer app), one module per stage (`extract`, `resolve`, `review` + `webreview`, `diff`, `upload`, `enrich`), plus `bgg_client.py` (rate-limited XML API2 client that caches responses to `data/bgg_cache/`), `normalize.py` (title normalization), `models.py` (dataclasses), `config.py`.
- `scripts/``write_stub_fixtures.py` / `write_photo_fixtures.py` generate synthetic fixtures; `record_fixtures.py` re-records real API responses once a token exists. - `scripts/``write_stub_fixtures.py` / `write_photo_fixtures.py` generate synthetic fixtures; `record_fixtures.py` re-records real API responses once a token exists.
- `tests/fixtures/bgg_cache/` — stub XML fixtures the offline tests run against. - `tests/fixtures/bgg_cache/` — stub XML fixtures the offline tests run against.
- `data/` — pipeline state (CSV/JSON artifacts are committed; caches are not — see Git). - `data/` — pipeline state (CSV/JSON artifacts are committed; caches are not — see Git).
+22
View File
@@ -78,3 +78,25 @@ runs): pagination controls in the version sub-view, exact post-save behavior
(toast? dialog close? redirect?), and how the dialog differs when the game is (toast? dialog close? redirect?), and how the dialog differs when the game is
ALREADY in the collection (second-copy flow must create a new entry, not edit ALREADY in the collection (second-copy flow must create a new entry, not edit
the existing one). the existing one).
## Login page (recon 2026-08-01, anonymous probe via Playwright)
- **Cloudflare Turnstile blocks headless browsers outright**: the headless
shell never gets past "Just a moment..." (`cf-turnstile-response` hidden
input, no form). A normal **headed** Chromium passed the check without
interaction. Hence `bggpipe upload` runs headed by default; `--headless`
exists but expect login to fail there. A first login in headed mode may
still need one human click on the challenge widget; the session then
persists via `storage_state.json` (gitignored).
- **BGG pages never reach Playwright's `networkidle`** — ad/analytics
requests poll forever. Navigate with `wait_until="domcontentloaded"` and
rely on element-level auto-waiting.
- Verified form selectors at `/login`: `#inputUsername` (name=`username`,
formcontrolname=`username`), `#inputPassword`, and a button with
accessible name **"Sign In"** (`type="button"` — Angular handles submit,
so click the button rather than pressing Enter and hoping for a form
submit). Labels "Username"/"Password" point at those ids. Cookie-consent
checkboxes (Essential, Performance Analytics, ...) render on the same
page but did not overlay the form in the probe.
- Logged-in detection heuristic (unverified): the header shows a "Sign In"
link only when logged out.
+1
View File
@@ -14,6 +14,7 @@ dependencies = [
"rich>=15.0.0", "rich>=15.0.0",
"fastapi>=0.141.1", "fastapi>=0.141.1",
"uvicorn>=0.52.1", "uvicorn>=0.52.1",
"playwright>=1.62.0",
] ]
[project.scripts] [project.scripts]
+13 -5
View File
@@ -157,17 +157,25 @@ class BGGClient:
username: str, username: str,
subtype: str | None = None, subtype: str | None = None,
version: bool = True, version: bool = True,
*,
refresh: bool = False,
) -> list[CollectionItem]: ) -> list[CollectionItem]:
params = {"username": username, "own": "1"} params = {"username": username, "own": "1"}
if subtype: if subtype:
params["subtype"] = subtype params["subtype"] = subtype
if version: if version:
params["version"] = "1" params["version"] = "1"
return parse_collection(self.get_xml("collection", params)) return parse_collection(self.get_xml("collection", params, refresh=refresh))
def collection_full(self, username: str) -> list[CollectionItem]: def collection_full(
"""Owned items incl. expansions (excluded from the default subtype).""" self, username: str, *, refresh: bool = False
base = self.collection(username) ) -> list[CollectionItem]:
expansions = self.collection(username, subtype="boardgameexpansion") """Owned items incl. expansions (excluded from the default subtype).
refresh=True bypasses the cache — upload --verify must see the live
collection, not the snapshot resolve ran against."""
base = self.collection(username, refresh=refresh)
expansions = self.collection(
username, subtype="boardgameexpansion", refresh=refresh
)
seen = {item.coll_id for item in base} seen = {item.coll_id for item in base}
return base + [e for e in expansions if e.coll_id not in seen] return base + [e for e in expansions if e.coll_id not in seen]
+30 -11
View File
@@ -20,13 +20,6 @@ ConfigOpt = Annotated[
] ]
def _not_implemented(stage: str, build_order: int) -> None:
typer.echo(
f"bggpipe {stage}: not implemented yet (build-order step {build_order})."
)
raise typer.Exit(code=1)
@app.command() @app.command()
def extract( def extract(
only: Annotated[ only: Annotated[
@@ -89,13 +82,39 @@ def diff(config: ConfigOpt = None) -> None:
@app.command() @app.command()
def upload( def upload(
dry_run: Annotated[bool, typer.Option("--dry-run")] = False, dry_run: Annotated[
verify: Annotated[bool, typer.Option("--verify")] = False, bool, typer.Option("--dry-run", help="Show the queue without a browser")
retry_failed: Annotated[bool, typer.Option("--retry-failed")] = False, ] = False,
verify: Annotated[
bool, typer.Option("--verify", help="Re-fetch the collection and cross-check")
] = False,
retry_failed: Annotated[
bool, typer.Option("--retry-failed", help="Re-attempt previously failed games")
] = False,
limit: Annotated[
int | None, typer.Option("--limit", help="Upload at most N games this run")
] = None,
headless: Annotated[
bool,
typer.Option(
"--headless",
help="Run the browser headless (Cloudflare may block login)",
),
] = False,
config: ConfigOpt = None, config: ConfigOpt = None,
) -> None: ) -> None:
"""Stage 5: add games to the BGG collection via Playwright.""" """Stage 5: add games to the BGG collection via Playwright."""
_not_implemented("upload", 5) from bggpipe.upload import run_upload
cfg = load_config(config)
run_upload(
cfg,
dry_run=dry_run,
verify=verify,
retry_failed=retry_failed,
limit=limit,
headless=headless,
)
@app.command() @app.command()
+507
View File
@@ -0,0 +1,507 @@
"""Stage 5 — upload: add games / set versions on boardgamegeek.com via Playwright.
BGG has no write API, so this stage drives the real website with a logged-in
browser session. Etiquette (spec + bgg-api skill):
- 2-4 s randomized delay between games;
- credentials only from BGG_USERNAME / BGG_PASSWORD env vars, never disk/logs;
- browser storage state persists locally (gitignored) so login is rare;
- every attempt is appended to data/upload_log.csv immediately, so a killed
run loses nothing and re-runs skip completed work;
- refuses to touch the site while data/bgg_cache/STUB_FIXTURES.marker exists
(stub-resolved version ids must never reach BGG); --dry-run still works,
loudly labeled as synthetic.
Cloudflare: BGG fronts the site with a Turnstile check that blocks headless
browsers outright (verified 2026-08-01 — headless shell never gets past
"Just a moment..."). The stage therefore runs HEADED by default; a first
login may need one human click on the challenge widget. BGG pages also never
reach Playwright's networkidle (ad/analytics polling), so navigation waits on
domcontentloaded plus explicit element waits.
"""
from __future__ import annotations
import contextlib
import csv
import os
import random
import re
import time
from collections.abc import Callable
from dataclasses import dataclass
from datetime import UTC, datetime
from pathlib import Path
from typing import Protocol
import typer
from bggpipe.bgg_client import BGGAuthError, BGGClient
from bggpipe.config import Config
from bggpipe.models import CollectionItem
BGG = "https://boardgamegeek.com"
STORAGE_STATE_PATH = Path("storage_state.json") # gitignored, credential-adjacent
UPLOAD_LOG_COLUMNS = [
"action",
"bgg_id",
"collid",
"name",
"version_id",
"status",
"timestamp",
"error",
]
DONE_STATUSES = {"added", "updated", "already_present"}
MAX_VERSION_PAGES = 40
@dataclass(frozen=True)
class UploadJob:
action: str # "add" | "update"
bgg_id: str
name: str
version_id: str = ""
version_name: str = ""
collid: str = ""
@property
def key(self) -> tuple[str, str, str]:
# A second copy of the same game (different version) is a distinct
# add; updates are keyed by the physical copy they amend.
if self.action == "update":
return ("update", self.collid, "")
return ("add", self.bgg_id, self.version_id)
def _row_key(row: dict) -> tuple[str, str, str]:
if row["action"] == "update":
return ("update", row["collid"], "")
return ("add", row["bgg_id"], row["version_id"])
def _read_csv(path: Path) -> list[dict]:
if not path.exists():
return []
with path.open(newline="") as f:
return list(csv.DictReader(f))
def append_log_row(path: Path, row: dict) -> None:
"""Append one attempt, creating the file with a header on first write.
One row per attempt, flushed immediately — the log is the resume point."""
new = not path.exists()
path.parent.mkdir(parents=True, exist_ok=True)
with path.open("a", newline="") as f:
writer = csv.DictWriter(f, fieldnames=UPLOAD_LOG_COLUMNS, extrasaction="ignore")
if new:
writer.writeheader()
writer.writerow(row)
def build_queue(
to_add: list[dict],
to_update: list[dict],
log_rows: list[dict],
*,
retry_failed: bool = False,
) -> tuple[list[UploadJob], int, int]:
"""Turn the diff outputs into pending jobs, minus work the log says is
done. Returns (jobs, skipped_done, skipped_failed)."""
latest: dict[tuple[str, str, str], str] = {}
for row in log_rows:
latest[_row_key(row)] = row["status"]
candidates = [
UploadJob(
action="add",
bgg_id=row["bgg_id"],
name=row["bgg_name"],
version_id=row.get("version_id", ""),
version_name=row.get("version_name", ""),
)
for row in to_add
] + [
UploadJob(
action="update",
bgg_id=row["bgg_id"],
name=row["bgg_name"],
version_id=row["version_id"],
version_name=row["version_name"],
collid=row["collid"],
)
for row in to_update
]
jobs: list[UploadJob] = []
skipped_done = skipped_failed = 0
for job in candidates:
status = latest.get(job.key)
if status in DONE_STATUSES:
skipped_done += 1
elif status == "failed" and not retry_failed:
skipped_failed += 1
else:
jobs.append(job)
return jobs, skipped_done, skipped_failed
def _scrub(text: str) -> str:
"""Credentials must never reach the log, even via a selector error that
echoes filled form values."""
for key in ("BGG_PASSWORD", "BGG_USERNAME"):
value = os.environ.get(key)
if value:
text = text.replace(value, "***")
return text
class Uploader(Protocol):
def add_game(self, job: UploadJob) -> tuple[str, str]: ...
def update_entry(self, job: UploadJob) -> tuple[str, str]: ...
class PlaywrightUploader:
"""Drives the real site. Selector documentation: docs/bgg-upload-flow.md.
Verified selectors (2026-08-01): the login form (#inputUsername /
#inputPassword / "Sign In") and the Add-to-Collection dialog structure.
UNVERIFIED: version-picker pagination, post-save behavior, and the whole
collection-row edit flow for updates — expect first-real-run adjustments.
"""
def __init__(
self,
username: str,
storage_state: Path = STORAGE_STATE_PATH,
headless: bool = False,
) -> None:
self._username = username
self._storage_state = storage_state
self._headless = headless
self._authed = False
def __enter__(self) -> PlaywrightUploader:
from playwright.sync_api import TimeoutError as PWTimeoutError
from playwright.sync_api import sync_playwright
self._timeout_error = PWTimeoutError
self._pw = sync_playwright().start()
self._browser = self._pw.chromium.launch(headless=self._headless)
state = str(self._storage_state) if self._storage_state.exists() else None
self._context = self._browser.new_context(storage_state=state)
self._page = self._context.new_page()
return self
def __exit__(self, *exc: object) -> None:
self._context.close()
self._browser.close()
self._pw.stop()
def _goto(self, url: str) -> None:
# networkidle never arrives on BGG (ad polling) — domcontentloaded
# plus per-element auto-waiting is the reliable pattern.
self._page.goto(url, wait_until="domcontentloaded")
def _signed_out(self) -> bool:
# Heuristic: the header shows a "Sign In" link only when logged out.
return (
self._page.get_by_role("link", name=re.compile(r"^sign in$", re.I)).count()
> 0
)
def _ensure_logged_in(self) -> None:
if self._authed:
return
page = self._page
self._goto(f"{BGG}/")
if self._signed_out():
user = os.environ.get("BGG_USERNAME", "")
password = os.environ.get("BGG_PASSWORD", "")
if not (user and password):
raise RuntimeError(
"BGG_USERNAME and BGG_PASSWORD env vars are required to log in"
)
self._goto(f"{BGG}/login")
# Generous timeout: in headed mode a human may need to click the
# Cloudflare Turnstile widget before the form renders.
page.locator("#inputUsername").wait_for(timeout=120_000)
page.locator("#inputUsername").fill(user)
page.locator("#inputPassword").fill(password)
page.get_by_role("button", name="Sign In").click()
page.wait_for_url(lambda url: "/login" not in url, timeout=120_000)
self._context.storage_state(path=str(self._storage_state))
self._authed = True
def _open_dialog(self, opener) -> object:
"""Click an opener that may no-op right after page load (hydration
race) and wait for the dialog to actually show."""
dialog = self._page.get_by_role("dialog")
for attempt in (1, 2):
opener.click()
try:
dialog.wait_for(state="visible", timeout=5_000)
return dialog
except self._timeout_error:
if attempt == 2:
raise
return dialog
def _select_version(self, dialog, version_name: str) -> bool:
"""Page through the version sub-view matching the canonical version
NAME (the list has no search box). Returns False — with the sub-view
cancelled — when the name never shows up."""
dialog.get_by_role("button", name="Set version/edition").click()
pattern = re.compile(re.escape(version_name), re.I)
with contextlib.suppress(self._timeout_error): # empty list is legal
dialog.get_by_role("listitem").first.wait_for(timeout=10_000)
for _ in range(MAX_VERSION_PAGES):
items = dialog.get_by_role("listitem").filter(has_text=pattern)
if items.count():
items.first.click()
return True
# Pagination controls are UNVERIFIED (docs/bgg-upload-flow.md);
# best guess is a next-page button, stopping when absent/disabled.
nxt = dialog.get_by_role("button", name=re.compile("next|", re.I)).first
if nxt.count() == 0 or nxt.is_disabled():
break
nxt.click()
# Two-level dismissal: the sub-view has its own Cancel distinct from
# the main dialog's.
dialog.get_by_role("button", name="Cancel").first.click()
return False
def add_game(self, job: UploadJob) -> tuple[str, str]:
self._ensure_logged_in()
page = self._page
# /boardgame/<id> redirects to the canonical slug for any subtype.
self._goto(f"{BGG}/boardgame/{job.bgg_id}/")
add_btn = page.get_by_role("button", name="Add To").first
dialog = self._open_dialog(add_btn)
# Content settles when the game-name heading replaces "Loading...".
dialog.get_by_role(
"heading", name=re.compile(re.escape(job.name), re.I)
).wait_for(timeout=15_000)
dialog.get_by_label("Own").check()
note = ""
if job.version_name and not self._select_version(dialog, job.version_name):
note = f"version {job.version_name!r} not in picker; added without version"
dialog.get_by_role("button", name="Save").click()
# The dialog is hidden after save, not removed from the DOM.
dialog.wait_for(state="hidden", timeout=15_000)
return "added", note
def update_entry(self, job: UploadJob) -> tuple[str, str]:
"""Set the version on an EXISTING entry — strictly additive.
UNVERIFIED FLOW: the collection table can't target a collid directly,
so this filters by game and opens the row's status link. With several
copies of one game the wrong row could open — acceptable only while
every 2018 entry is version-less; revisit after the first real run.
"""
self._ensure_logged_in()
page = self._page
self._goto(
f"{BGG}/collection/user/{self._username}?objectid={job.bgg_id}&own=1"
)
row = (
page.get_by_role("row")
.filter(has_text=re.compile(re.escape(job.name), re.I))
.first
)
opener = row.get_by_role("link", name=re.compile("own", re.I)).first
dialog = self._open_dialog(opener)
if not self._select_version(dialog, job.version_name):
# Never guess a version: close without touching the entry.
dialog.get_by_role("button", name="Cancel").first.click()
raise RuntimeError(
f"version {job.version_name!r} not in picker — entry left untouched"
)
dialog.get_by_role("button", name="Save").click()
dialog.wait_for(state="hidden", timeout=15_000)
return "updated", "row-edit flow is unverified; confirm with --verify"
def _process(
uploader: Uploader,
jobs: list[UploadJob],
log_path: Path,
*,
sleep: Callable[[float], None],
rng: random.Random,
now: Callable[[], str],
) -> list[dict]:
results = []
for i, job in enumerate(jobs):
if i:
sleep(rng.uniform(2.0, 4.0)) # polite pacing between games (spec)
try:
if job.action == "add":
status, note = uploader.add_game(job)
else:
status, note = uploader.update_entry(job)
except Exception as exc: # per-game isolation: log it, keep going
status, note = "failed", _scrub(f"{type(exc).__name__}: {exc}")
row = {
"action": job.action,
"bgg_id": job.bgg_id,
"collid": job.collid,
"name": job.name,
"version_id": job.version_id,
"status": status,
"timestamp": now(),
"error": note if status == "failed" else note,
}
append_log_row(log_path, row)
results.append(row)
suffix = f"{note}" if note else ""
typer.echo(f" {job.name}: {status}{suffix}")
return results
def verify_uploads(log_rows: list[dict], collection: list[CollectionItem]) -> list[str]:
"""Cross-check the log's successes against a fresh collection fetch."""
by_object: dict[int, list[CollectionItem]] = {}
by_collid: dict[int, CollectionItem] = {}
for item in collection:
by_object.setdefault(item.object_id, []).append(item)
by_collid[item.coll_id] = item
problems = []
latest: dict[tuple[str, str, str], dict] = {}
for row in log_rows:
latest[_row_key(row)] = row
for row in latest.values():
if row["status"] == "added":
copies = by_object.get(int(row["bgg_id"]), [])
if not copies:
problems.append(f"{row['name']}: logged added but not in collection")
elif row["version_id"] and not any(
str(c.version_id or "") == row["version_id"] for c in copies
):
problems.append(
f"{row['name']}: in collection but no copy has "
f"version {row['version_id']}"
)
elif row["status"] == "updated":
item = by_collid.get(int(row["collid"]))
if item is None:
problems.append(
f"{row['name']}: collid {row['collid']} not in collection"
)
elif str(item.version_id or "") != row["version_id"]:
problems.append(
f"{row['name']}: collid {row['collid']} does not carry "
f"version {row['version_id']}"
)
return problems
def run_upload(
cfg: Config,
*,
dry_run: bool = False,
verify: bool = False,
retry_failed: bool = False,
limit: int | None = None,
headless: bool = False,
uploader: Uploader | None = None,
client: BGGClient | None = None,
storage_state: Path = STORAGE_STATE_PATH,
sleep: Callable[[float], None] = time.sleep,
rng: random.Random | None = None,
now: Callable[[], str] | None = None,
) -> list[dict]:
rng = rng or random.Random()
now = now or (lambda: datetime.now(UTC).isoformat(timespec="seconds"))
marker = cfg.cache_dir / "STUB_FIXTURES.marker"
if marker.exists():
if not dry_run:
typer.echo(
f"Refusing to upload: {marker} exists — every resolved "
"version_id is a synthetic stub placeholder. Once "
"BGG_API_TOKEN arrives: delete both cache dirs, re-record "
"fixtures, `resolve --force`, re-review, re-diff."
)
raise typer.Exit(code=1)
typer.echo(
"WARNING: stub fixtures active — the ids below are SYNTHETIC "
"placeholders, not real BGG data.\n"
)
log_path = cfg.data_dir / "upload_log.csv"
to_add = _read_csv(cfg.data_dir / "to_add.csv")
to_update = _read_csv(cfg.data_dir / "to_update.csv")
log_rows = _read_csv(log_path)
jobs, skipped_done, skipped_failed = build_queue(
to_add, to_update, log_rows, retry_failed=retry_failed
)
if limit is not None:
jobs = jobs[:limit]
skip_note = ""
if skipped_done or skipped_failed:
skip_note = (
f" (skipping {skipped_done} already done, {skipped_failed} "
"previously failed — use --retry-failed)"
)
results: list[dict] = []
if dry_run:
typer.echo(f"Dry run: {len(jobs)} job(s) pending{skip_note}")
for job in jobs:
if job.action == "add":
version = f" [version {job.version_name}]" if job.version_name else ""
typer.echo(f" would add {job.name} ({job.bgg_id}){version}")
else:
typer.echo(
f" would set version {job.version_name!r} on existing "
f"entry collid {job.collid} ({job.name})"
)
elif jobs:
typer.echo(f"Uploading {len(jobs)} job(s){skip_note}")
if uploader is None:
if not (os.environ.get("BGG_USERNAME") and os.environ.get("BGG_PASSWORD")):
typer.echo(
"BGG_USERNAME and BGG_PASSWORD env vars are required "
"(direnv loads them from .env)."
)
raise typer.Exit(code=1)
with PlaywrightUploader(
cfg.bgg_username, storage_state=storage_state, headless=headless
) as real:
results = _process(real, jobs, log_path, sleep=sleep, rng=rng, now=now)
else:
results = _process(uploader, jobs, log_path, sleep=sleep, rng=rng, now=now)
counts: dict[str, int] = {}
for row in results:
counts[row["status"]] = counts.get(row["status"], 0) + 1
summary = " · ".join(f"{n} {status}" for status, n in sorted(counts.items()))
typer.echo(f"\n{summary or 'nothing to do'}")
else:
typer.echo(f"Nothing to upload{skip_note}.")
if verify and not dry_run:
_run_verify(cfg, client, log_path)
return results
def _run_verify(cfg: Config, client: BGGClient | None, log_path: Path) -> None:
if not cfg.bgg_username:
typer.echo("--verify needs bgg_username (config.toml or BGG_USERNAME).")
return
client = client or BGGClient(cfg.cache_dir, cfg.rate_limit_seconds)
try:
collection = client.collection_full(cfg.bgg_username, refresh=True)
except BGGAuthError as exc:
typer.echo(f"--verify skipped: {exc}")
return
problems = verify_uploads(_read_csv(log_path), collection)
if problems:
typer.echo("\nVerification problems:")
for line in problems:
typer.echo(f" - {line}")
else:
typer.echo("\nVerification OK: every logged success is in the collection.")
+292
View File
@@ -0,0 +1,292 @@
"""Upload-stage tests: queue building, idempotency via upload_log.csv, the
stub-fixture guard, per-game failure isolation, pacing, and verify — all
against a fake uploader. No browser, no network."""
from __future__ import annotations
import csv
import random
from pathlib import Path
import pytest
import typer
from bggpipe.config import Config
from bggpipe.models import CollectionItem
from bggpipe.upload import (
UPLOAD_LOG_COLUMNS,
UploadJob,
_scrub,
build_queue,
run_upload,
verify_uploads,
)
NOW = lambda: "2026-08-01T00:00:00+00:00" # noqa: E731
def _cfg(tmp_path: Path) -> Config:
return Config(bgg_username="tester", data_dir=tmp_path)
def _write_csv(path: Path, columns: list[str], rows: list[dict]) -> None:
with path.open("w", newline="") as f:
writer = csv.DictWriter(f, fieldnames=columns)
writer.writeheader()
writer.writerows(rows)
def _add_row(bgg_id="1", name="Wingspan", version_id="", version_name=""):
return {
"bgg_id": bgg_id,
"bgg_name": name,
"year": "2019",
"type": "boardgame",
"version_id": version_id,
"version_name": version_name,
"title_raw": name.upper(),
"source_photos": "x.jpg",
}
def _update_row(collid="9", bgg_id="2", name="Britannia", vid="25", vname="AH ed."):
return {
"collid": collid,
"bgg_id": bgg_id,
"bgg_name": name,
"version_id": vid,
"version_name": vname,
}
def _log_row(action="add", bgg_id="1", collid="", version_id="", status="added"):
return {
"action": action,
"bgg_id": bgg_id,
"collid": collid,
"name": "Game",
"version_id": version_id,
"status": status,
"timestamp": NOW(),
"error": "",
}
def _seed_data(tmp_path: Path, to_add=None, to_update=None, log=None) -> None:
_write_csv(
tmp_path / "to_add.csv",
list(_add_row().keys()),
to_add if to_add is not None else [],
)
_write_csv(
tmp_path / "to_update.csv",
list(_update_row().keys()),
to_update if to_update is not None else [],
)
if log is not None:
_write_csv(tmp_path / "upload_log.csv", UPLOAD_LOG_COLUMNS, log)
class FakeUploader:
"""Records jobs; raises for names listed in `failures`."""
def __init__(self, failures: set[str] | None = None):
self.calls: list[UploadJob] = []
self.failures = failures or set()
def _handle(self, job: UploadJob, status: str) -> tuple[str, str]:
self.calls.append(job)
if job.name in self.failures:
raise RuntimeError("dialog never appeared")
return status, ""
def add_game(self, job: UploadJob) -> tuple[str, str]:
return self._handle(job, "added")
def update_entry(self, job: UploadJob) -> tuple[str, str]:
return self._handle(job, "updated")
# -- build_queue --------------------------------------------------------
def test_build_queue_skips_logged_successes():
jobs, done, failed = build_queue(
[_add_row(bgg_id="1"), _add_row(bgg_id="2", name="Catan")],
[_update_row(collid="9")],
[
_log_row(action="add", bgg_id="1", status="added"),
_log_row(action="update", bgg_id="2", collid="9", status="updated"),
],
)
assert [j.bgg_id for j in jobs] == ["2"]
assert done == 2
assert failed == 0
def test_build_queue_second_copy_is_a_distinct_job():
# Same game, different version: a separate physical copy, so a
# logged add of one version must not swallow the other.
jobs, done, _ = build_queue(
[
_add_row(bgg_id="1", version_id="10", version_name="First ed."),
_add_row(bgg_id="1", version_id="11", version_name="Second ed."),
],
[],
[_log_row(action="add", bgg_id="1", version_id="10", status="added")],
)
assert [j.version_id for j in jobs] == ["11"]
assert done == 1
def test_build_queue_failures_need_retry_flag():
log = [_log_row(action="add", bgg_id="1", status="failed")]
jobs, _, skipped = build_queue([_add_row(bgg_id="1")], [], log)
assert jobs == [] and skipped == 1
jobs, _, skipped = build_queue([_add_row(bgg_id="1")], [], log, retry_failed=True)
assert len(jobs) == 1 and skipped == 0
def test_build_queue_latest_log_entry_wins():
# failed then added on retry -> done, not retriable
log = [
_log_row(action="add", bgg_id="1", status="failed"),
_log_row(action="add", bgg_id="1", status="added"),
]
jobs, done, _ = build_queue([_add_row(bgg_id="1")], [], log, retry_failed=True)
assert jobs == [] and done == 1
# -- stub-fixture guard -------------------------------------------------
def test_real_run_refuses_while_stub_marker_exists(tmp_path):
cfg = _cfg(tmp_path)
cfg.cache_dir.mkdir(parents=True)
(cfg.cache_dir / "STUB_FIXTURES.marker").write_text("stub")
_seed_data(tmp_path, to_add=[_add_row()])
with pytest.raises(typer.Exit):
run_upload(cfg, uploader=FakeUploader(), sleep=lambda s: None, now=NOW)
def test_dry_run_allowed_with_stub_marker_and_writes_nothing(tmp_path, capsys):
cfg = _cfg(tmp_path)
cfg.cache_dir.mkdir(parents=True)
(cfg.cache_dir / "STUB_FIXTURES.marker").write_text("stub")
_seed_data(tmp_path, to_add=[_add_row()], to_update=[_update_row()])
run_upload(cfg, dry_run=True, now=NOW)
out = capsys.readouterr().out
assert "WARNING" in out and "SYNTHETIC" in out
assert "would add Wingspan" in out
assert "collid 9" in out
assert not (tmp_path / "upload_log.csv").exists()
# -- run_upload with a fake browser -------------------------------------
def test_run_logs_every_attempt_and_continues_past_failures(tmp_path):
cfg = _cfg(tmp_path)
_seed_data(
tmp_path,
to_add=[_add_row(bgg_id="1"), _add_row(bgg_id="2", name="Catan")],
to_update=[_update_row(collid="9")],
)
fake = FakeUploader(failures={"Catan"})
results = run_upload(cfg, uploader=fake, sleep=lambda s: None, now=NOW)
assert [r["status"] for r in results] == ["added", "failed", "updated"]
logged = list(csv.DictReader((tmp_path / "upload_log.csv").open()))
assert len(logged) == 3
assert logged[1]["error"] == "RuntimeError: dialog never appeared"
assert logged[2]["collid"] == "9"
def test_rerun_skips_completed_work(tmp_path):
cfg = _cfg(tmp_path)
_seed_data(tmp_path, to_add=[_add_row(bgg_id="1"), _add_row(bgg_id="2", name="C")])
fake = FakeUploader()
run_upload(cfg, uploader=fake, sleep=lambda s: None, now=NOW)
assert len(fake.calls) == 2
again = FakeUploader()
results = run_upload(cfg, uploader=again, sleep=lambda s: None, now=NOW)
assert again.calls == [] and results == []
def test_pacing_sleeps_2_to_4s_between_games_only(tmp_path):
cfg = _cfg(tmp_path)
_seed_data(tmp_path, to_add=[_add_row(bgg_id=str(i)) for i in range(1, 5)])
sleeps: list[float] = []
run_upload(
cfg,
uploader=FakeUploader(),
sleep=sleeps.append,
rng=random.Random(42),
now=NOW,
)
assert len(sleeps) == 3 # between games, not before the first
assert all(2.0 <= s <= 4.0 for s in sleeps)
def test_limit_caps_the_queue(tmp_path):
cfg = _cfg(tmp_path)
_seed_data(tmp_path, to_add=[_add_row(bgg_id=str(i)) for i in range(1, 5)])
fake = FakeUploader()
run_upload(cfg, uploader=fake, limit=2, sleep=lambda s: None, now=NOW)
assert len(fake.calls) == 2
# -- credential hygiene -------------------------------------------------
def test_scrub_removes_credentials_from_error_text(monkeypatch):
monkeypatch.setenv("BGG_USERNAME", "erics-user")
monkeypatch.setenv("BGG_PASSWORD", "s3cret-pw")
text = 'fill("erics-user") then fill("s3cret-pw") timed out'
assert "s3cret-pw" not in _scrub(text)
assert "erics-user" not in _scrub(text)
# -- verify -------------------------------------------------------------
def _item(object_id, coll_id, name="Game", version_id=None):
return CollectionItem(
object_id=object_id,
coll_id=coll_id,
name=name,
subtype="boardgame",
own=True,
year=None,
version_id=version_id,
)
def test_verify_flags_missing_and_confirms_present():
log = [
_log_row(action="add", bgg_id="1", status="added"),
_log_row(action="add", bgg_id="2", status="added"),
_log_row(
action="update", bgg_id="3", collid="30", version_id="7", status="updated"
),
]
collection = [_item(1, 10), _item(3, 30, version_id=7)]
problems = verify_uploads(log, collection)
assert len(problems) == 1
assert "not in collection" in problems[0]
def test_verify_checks_version_on_adds_and_updates():
log = [
_log_row(action="add", bgg_id="1", version_id="99", status="added"),
_log_row(
action="update", bgg_id="3", collid="30", version_id="7", status="updated"
),
]
collection = [_item(1, 10, version_id=11), _item(3, 30, version_id=8)]
problems = verify_uploads(log, collection)
assert len(problems) == 2
def test_verify_ignores_failed_rows():
log = [_log_row(action="add", bgg_id="5", status="failed")]
assert verify_uploads(log, []) == []
Generated
+100
View File
@@ -63,6 +63,7 @@ dependencies = [
{ name = "httpx" }, { name = "httpx" },
{ name = "pillow" }, { name = "pillow" },
{ name = "pillow-heif" }, { name = "pillow-heif" },
{ name = "playwright" },
{ name = "rapidfuzz" }, { name = "rapidfuzz" },
{ name = "rich" }, { name = "rich" },
{ name = "typer" }, { name = "typer" },
@@ -83,6 +84,7 @@ requires-dist = [
{ name = "httpx", specifier = ">=0.27" }, { name = "httpx", specifier = ">=0.27" },
{ name = "pillow", specifier = ">=12.3.0" }, { name = "pillow", specifier = ">=12.3.0" },
{ name = "pillow-heif", specifier = ">=1.5.0" }, { name = "pillow-heif", specifier = ">=1.5.0" },
{ name = "playwright", specifier = ">=1.62.0" },
{ name = "rapidfuzz", specifier = ">=3.9" }, { name = "rapidfuzz", specifier = ">=3.9" },
{ name = "rich", specifier = ">=15.0.0" }, { name = "rich", specifier = ">=15.0.0" },
{ name = "typer", specifier = ">=0.12" }, { name = "typer", specifier = ">=0.12" },
@@ -168,6 +170,73 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/cb/03/10388a42375ee7e4ac9b94eb2c5c569c8b5795e377e701c9ac3ad63de890/fastapi-0.141.1-py3-none-any.whl", hash = "sha256:bfb91aa2d334c61cb35ba9a116fc123b3d3df31640b801cf57a7a78ec3f603b3", size = 131954, upload-time = "2026-07-29T17:18:04.364Z" }, { url = "https://files.pythonhosted.org/packages/cb/03/10388a42375ee7e4ac9b94eb2c5c569c8b5795e377e701c9ac3ad63de890/fastapi-0.141.1-py3-none-any.whl", hash = "sha256:bfb91aa2d334c61cb35ba9a116fc123b3d3df31640b801cf57a7a78ec3f603b3", size = 131954, upload-time = "2026-07-29T17:18:04.364Z" },
] ]
[[package]]
name = "greenlet"
version = "3.5.4"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/a3/74/b13368064b09053253555d3f2839cc2684d22d5aed0d2ccffbf7a6736558/greenlet-3.5.4.tar.gz", hash = "sha256:0232ae1de90a8e07867bb127d7a6ba2301e859145489f25cda8a6096dabe1d20", size = 206538, upload-time = "2026-07-22T12:47:14.468Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/f3/04/81bd731d6d1e3a469d9a4c36f5eb069bcf0cbb2d5d342c9fec22245b91fc/greenlet-3.5.4-cp312-cp312-macosx_11_0_universal2.whl", hash = "sha256:3d66250e8b09f182ede05490998c818b5961f7a3640332d44c4927caec7bbfe4", size = 295909, upload-time = "2026-07-22T11:38:09.261Z" },
{ url = "https://files.pythonhosted.org/packages/cc/dd/f5f22903a6ae70f5ea328ed0beaec92ad903f0e3b7d2845133b354abc4b8/greenlet-3.5.4-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c90e930c9c192e5b3ee9fb8bcd920ea3926155e2e3ded39fc697323addecee17", size = 612011, upload-time = "2026-07-22T12:26:40.69Z" },
{ url = "https://files.pythonhosted.org/packages/8e/10/92a4a88d12b915d74ea5b6d288e4afefda4771647caa34442c156f7a454f/greenlet-3.5.4-cp312-cp312-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:791fdfeeb9c6e0c7b10fa151bf110d2a6974866f13dcb5b1c7efae698245893a", size = 624299, upload-time = "2026-07-22T12:29:02.089Z" },
{ url = "https://files.pythonhosted.org/packages/6c/f9/03e26be3487c5238e81f2b84714959a86ea8515a869828cf41f4fc54b34e/greenlet-3.5.4-cp312-cp312-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:b7c895310363f310361e0fe2072af85269d2a2a285cd04c0c59e79a5e3670dcf", size = 629603, upload-time = "2026-07-22T12:43:43.456Z" },
{ url = "https://files.pythonhosted.org/packages/50/6d/0b14bb9db2989f32cd9fe7f76afedea01ee8bee3f87c07e69f24adfe7e63/greenlet-3.5.4-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f88193799d43dbf8c8a806d6405c9c52fe2af40bf75072a606357b33cc336c7f", size = 621541, upload-time = "2026-07-22T11:51:09.464Z" },
{ url = "https://files.pythonhosted.org/packages/57/6b/7c55ca72ef80d57c16c4a55210f82582622462dc4485799a30f4ec6f3372/greenlet-3.5.4-cp312-cp312-manylinux_2_39_riscv64.whl", hash = "sha256:13b980043cb1b3134e81ea469da1250ddcc6bfe6d245bbaa59168d9cdc8f228f", size = 432554, upload-time = "2026-07-22T12:39:51.379Z" },
{ url = "https://files.pythonhosted.org/packages/48/3d/25e9a2d9eb6b2e8b7ca4e80a3a26cb887cce6c8e0a87c921164f11bc5574/greenlet-3.5.4-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:b7a5f095767c4493afcd06067f2bb3b8716e3f3f9e92b99c88e7e99f885b3d4d", size = 1581444, upload-time = "2026-07-22T12:25:03.818Z" },
{ url = "https://files.pythonhosted.org/packages/b9/96/4c9bf2e2c408dcc0556edce69efa9f802e82223573c53240136a086821f1/greenlet-3.5.4-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:42afdc1ab5f66da8c586c32af9224a74a706b4f0ea0dc3a4188a0860a09c65c9", size = 1645842, upload-time = "2026-07-22T11:51:12.295Z" },
{ url = "https://files.pythonhosted.org/packages/b5/41/303ecb26a3a56122c0f4d4073ee078881847bd6b6f463ae0ec57ec20223b/greenlet-3.5.4-cp312-cp312-win_amd64.whl", hash = "sha256:60149df8f462d1b230038e6590c23c3b4768bb5d6c022b3b6e82532b34b0b8a3", size = 247169, upload-time = "2026-07-22T11:38:19.893Z" },
{ url = "https://files.pythonhosted.org/packages/a4/e3/ef56864b4c35fcb3eb3b41b869f6cc46f4cd3f5e2c68e74acde8ac433951/greenlet-3.5.4-cp312-cp312-win_arm64.whl", hash = "sha256:77d6ce04fed0d9aeed42e0f37923cc43eba9b027bdd9c34546bb4ccd143d0fe0", size = 245565, upload-time = "2026-07-22T11:38:27.061Z" },
{ url = "https://files.pythonhosted.org/packages/c0/9a/e51225dcd58713f16ccbdcc501a8da21098ea14515b7870f1f94459e5ff5/greenlet-3.5.4-cp313-cp313-macosx_11_0_universal2.whl", hash = "sha256:24e61b88cb7e1b1d794b32a10cc346ac779681d6d74ff137a3e0a444d2bf1f02", size = 294831, upload-time = "2026-07-22T11:38:53.389Z" },
{ url = "https://files.pythonhosted.org/packages/9f/ea/de50a50fadf979713ab18b46f22ad5ff5f2dcfc637a3ebdecf669801e1a5/greenlet-3.5.4-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:870d730fec833f5a06906a32596cc099b9161594642a92a520b7a88911c95356", size = 614619, upload-time = "2026-07-22T12:26:42.282Z" },
{ url = "https://files.pythonhosted.org/packages/db/c7/2aae27fea41205b8650294c301f042a2a4bb6155eea48c995b890a92f2c1/greenlet-3.5.4-cp313-cp313-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ec5ff0d1878df6af3bf9b638a5a92a7d5693291de77c91bff10fa48519c604ef", size = 627021, upload-time = "2026-07-22T12:29:03.445Z" },
{ url = "https://files.pythonhosted.org/packages/1b/80/fb4d4788bbc8e54761f1fc88533af9523a6e86299fa113d6e8a8503ed9fc/greenlet-3.5.4-cp313-cp313-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:07bd44616608d873d06735b63ef1a88191d6ca57c8d291d6559c71bc14c0893c", size = 632845, upload-time = "2026-07-22T12:43:45.19Z" },
{ url = "https://files.pythonhosted.org/packages/eb/56/79fd826f9ccaae0b84e1b4ef68dabba5e105bb044ffcd448a0b782fcba9a/greenlet-3.5.4-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d84d993f6e575c950d91a23c1345d18fe1a4310d447bf630849d7809196b52f0", size = 624002, upload-time = "2026-07-22T11:51:11.391Z" },
{ url = "https://files.pythonhosted.org/packages/42/e3/6086fa578ebb72772722cdc4bcd628459814b42e0c2db1e3cbd6552b3271/greenlet-3.5.4-cp313-cp313-manylinux_2_39_riscv64.whl", hash = "sha256:3529a8a933582ad19e224792cac7372489526576b75b4c124e8e4f29948f4861", size = 435053, upload-time = "2026-07-22T12:39:52.715Z" },
{ url = "https://files.pythonhosted.org/packages/0a/1a/27319f97e731298513dcba1a2e91b63e9d8811d9de22130f960b129b1bf1/greenlet-3.5.4-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:58023945f421093de5e6fa108c0985a8659d43f49e0216da25099369a121bcbd", size = 1581533, upload-time = "2026-07-22T12:25:05.322Z" },
{ url = "https://files.pythonhosted.org/packages/b1/6d/24240bf562e9786dd2799ee0a4a4dadb4ded22510f41b20245099159ac8c/greenlet-3.5.4-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:bae2728e1897aa8df8cb1af38cd48b3a743aefe29372de7b8b7a9f532501e69f", size = 1645781, upload-time = "2026-07-22T11:51:14.805Z" },
{ url = "https://files.pythonhosted.org/packages/c1/5a/442ab1a9ef7ca6bf7210e5397a95972206a91a31033a03c8900866a10039/greenlet-3.5.4-cp313-cp313-win_amd64.whl", hash = "sha256:ca5726c0b08ca35ae873557266a78b2c3f3b2b7d7401aa5ff886c2045dd0111c", size = 247133, upload-time = "2026-07-22T11:39:20.661Z" },
{ url = "https://files.pythonhosted.org/packages/3e/e6/9160210222386b1a378ff94db846b9508ca24a121cf684991561fdb69280/greenlet-3.5.4-cp313-cp313-win_arm64.whl", hash = "sha256:7c1303791d603080cac6fc3b34df51c3b75b723739c282c8029e48a0d241672f", size = 245500, upload-time = "2026-07-22T11:40:22.185Z" },
{ url = "https://files.pythonhosted.org/packages/a5/a7/6ab1d4f9cd548d15ab90da29947f2076100130bb179b0bde59f795a459e3/greenlet-3.5.4-cp314-cp314-macosx_11_0_universal2.whl", hash = "sha256:7e8afa5eac028f8140ceafe5ceec66e6aa127ddcb21452d2a564dcd2900b5f22", size = 295410, upload-time = "2026-07-22T11:40:35.747Z" },
{ url = "https://files.pythonhosted.org/packages/cd/7a/422f63b4715cbc0b24385305407adf38b48f6bb68b3e6b04090e994d0f5a/greenlet-3.5.4-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:73b37afe369021423ea53dd3123e04bffa7e93ac64429b9f50835b2e4fcae7cf", size = 661286, upload-time = "2026-07-22T12:26:43.8Z" },
{ url = "https://files.pythonhosted.org/packages/d0/31/5a1cac663bf5582190c5a714ef81364f03cde232227f39748f8ae4c11da5/greenlet-3.5.4-cp314-cp314-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:3ef964f56dfcb6f9bbef2a190d9126795eac408716aeae47b5e7c73c32aafca9", size = 673517, upload-time = "2026-07-22T12:29:04.815Z" },
{ url = "https://files.pythonhosted.org/packages/9c/bf/250c2921c7b585dde12f5239e313ca2dcbc464d161ecca36e4e6ef21762d/greenlet-3.5.4-cp314-cp314-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:cef589bc65fae02d10bca2ac341191c5b33acc2967892ebf4fcbd10eabb7a74c", size = 677968, upload-time = "2026-07-22T12:43:46.788Z" },
{ url = "https://files.pythonhosted.org/packages/15/4a/2a82a1e3f8aaca020853ac8d12211280ca2b231aa08ea39f636f1060c319/greenlet-3.5.4-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9c53ff01a5c53a40f2c16820ebc56d7c61a77f5fbe009dadd96292d5682f80f8", size = 670917, upload-time = "2026-07-22T11:51:13.589Z" },
{ url = "https://files.pythonhosted.org/packages/18/40/10bfcf6513558d82f7b95dd728001c63bd388259fe27d3e30ae01f103430/greenlet-3.5.4-cp314-cp314-manylinux_2_39_riscv64.whl", hash = "sha256:dfc41ae893d9ceaf22c824f2153a88b30651b20e8758c2cd9ac143f23640563c", size = 480643, upload-time = "2026-07-22T12:39:54.149Z" },
{ url = "https://files.pythonhosted.org/packages/68/b0/e379a152b17bfdfa95795af4049e37c0fd1b4d81f020d426db104ed07c77/greenlet-3.5.4-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:ecca4d80d55a01ad6b23b33262662956149fbb7b2c6be2910f1705921958cbf3", size = 1628478, upload-time = "2026-07-22T12:25:06.678Z" },
{ url = "https://files.pythonhosted.org/packages/5e/43/bffdfa64f7317f954c5c1230b5dd5922676ce198689a68c1ac1ed4b1b1a5/greenlet-3.5.4-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:2ffbc533e0eaf8e80d8471411646ab88fe58f641d508c0b02b24494479f4d9ec", size = 1692021, upload-time = "2026-07-22T11:51:17.008Z" },
{ url = "https://files.pythonhosted.org/packages/d0/11/f799f9637e2c6e9b0b716015e339040598b058cf7654dfc0d67468b177ed/greenlet-3.5.4-cp314-cp314-win_amd64.whl", hash = "sha256:305f69e6c4523d7f6979ed001cff4e5853c063e5da04880296603aa0227e544c", size = 248031, upload-time = "2026-07-22T11:40:11.007Z" },
{ url = "https://files.pythonhosted.org/packages/05/75/625bcdd74d5e6b2dca1ecba3c3ac77bcf8a026c21a649a46cef23e421f97/greenlet-3.5.4-cp314-cp314-win_arm64.whl", hash = "sha256:f260930bbbbcf9caee661211235a5111c86dfe5832fdf6ae4570da1e0995320f", size = 246892, upload-time = "2026-07-22T11:40:27.357Z" },
{ url = "https://files.pythonhosted.org/packages/ec/69/35c62ed49c320cb4d98e14698ccca5467d3bfe683984172be9cb564d9ce3/greenlet-3.5.4-cp314-cp314t-macosx_11_0_universal2.whl", hash = "sha256:41ddab54e4b238f4a6c323f39b4e59e176affd5a94d461a9fb7583dac74240a3", size = 305571, upload-time = "2026-07-22T11:40:31.659Z" },
{ url = "https://files.pythonhosted.org/packages/5b/6c/64d60216b3640dcb0b62d913dd9e0d80030c09115bb2e4ba70c95d10ca45/greenlet-3.5.4-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b3dabe3e2809013052c68bdf0b7fa5f5f2859c43a80803131ad61af9cabd7867", size = 672568, upload-time = "2026-07-22T12:26:45.298Z" },
{ url = "https://files.pythonhosted.org/packages/5c/de/ba3ab0a96292e53039530333b0d2ae18d9e508f3a325cd7bf15f8172944c/greenlet-3.5.4-cp314-cp314t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:27d3f00718634d4520a3a150154ac5da36f257869d41321953375b90bfbbc72c", size = 680076, upload-time = "2026-07-22T12:29:06.125Z" },
{ url = "https://files.pythonhosted.org/packages/ae/db/24a10af12bf8e639cec46c38b9ce1a282543ba42ff4fb0b31a970f1ab603/greenlet-3.5.4-cp314-cp314t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:39169a11d87a6a263afda3e9a27d1df16d0f919d40a4837cc73986c9884c0dd8", size = 681690, upload-time = "2026-07-22T12:43:48.109Z" },
{ url = "https://files.pythonhosted.org/packages/a5/be/aeada79083c6f1c15f45d77a332f9c441af263ee298e3eb17522cd337d22/greenlet-3.5.4-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:cbd60b5763c6543c1827e48faaf14ea9bfbad245f52b1a4d76a2a2d8884c6c66", size = 676733, upload-time = "2026-07-22T11:51:16.027Z" },
{ url = "https://files.pythonhosted.org/packages/f4/60/44a2eca7b9fd71ae0fae7ff184da1cd3169d176652b97aa1cffcbb0ef961/greenlet-3.5.4-cp314-cp314t-manylinux_2_39_riscv64.whl", hash = "sha256:bd3d1145f603b2db19feb9078c2e6855eb7c67e15580c010ed815cee519b86fd", size = 510263, upload-time = "2026-07-22T12:39:55.678Z" },
{ url = "https://files.pythonhosted.org/packages/e9/10/2392fc3a98948652ef5fd1e7275c04f861dd13f74b78a2b4309f4ee4d090/greenlet-3.5.4-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:f00f910f0e7b35416c63b23ad78b769aeccfc1775f712b43c4ee525624a2eef7", size = 1637327, upload-time = "2026-07-22T12:25:07.879Z" },
{ url = "https://files.pythonhosted.org/packages/55/c6/e7237a3dfa1f205ed0d9ea1e46d70bd2811b32d516266399fb59d28ab90a/greenlet-3.5.4-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:91c26423753b92caf41ab3f98fd547d7374d4d9fc2d85be041886c1579d9255e", size = 1697493, upload-time = "2026-07-22T11:51:19.214Z" },
{ url = "https://files.pythonhosted.org/packages/55/e3/4ba8154ba2a3d43729e499f72471b4b5c993f3826d3e24da81d5f06d6572/greenlet-3.5.4-cp314-cp314t-win_amd64.whl", hash = "sha256:ee032b91fd8ec29ec6c4cea2b8c561b178435134bd0752c7334b94e9c736c132", size = 251637, upload-time = "2026-07-22T11:40:37.44Z" },
{ url = "https://files.pythonhosted.org/packages/90/03/e3f96dfc100261a29545ddc8270cafe58f9195b6651466910e820910de77/greenlet-3.5.4-cp315-cp315-macosx_11_0_universal2.whl", hash = "sha256:178111881dd7a6c946471fda85485ec796e1043c2b939f694b096e2ecf986809", size = 296076, upload-time = "2026-07-22T11:39:38.364Z" },
{ url = "https://files.pythonhosted.org/packages/a4/3d/da52d208e5c977bce8667e784729e584e38b5785f4c1ec0f4c836e9a1c42/greenlet-3.5.4-cp315-cp315-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d92df08dd65fede97fc37aad36c2e9dcda3b31c467f8e0c2c096456cb818e927", size = 666870, upload-time = "2026-07-22T12:26:46.691Z" },
{ url = "https://files.pythonhosted.org/packages/2d/8a/7e6dee25cb8a8cf9b362c8e597cc269593378bd916f16c736c059a52e85a/greenlet-3.5.4-cp315-cp315-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:99e8f8c4ebc4fd80aa26c1280ae9ad43a0976e786349703a181cf0bae60413e5", size = 677678, upload-time = "2026-07-22T12:29:07.508Z" },
{ url = "https://files.pythonhosted.org/packages/51/a7/dafc7415d430b0a43a16396eb49ecb3b62fd720877fb259cc4dcfaf5f31e/greenlet-3.5.4-cp315-cp315-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:1f17e362d78e37559e0506c5a7d066bdd45073c36a0127a543e8a0df27242ff3", size = 681428, upload-time = "2026-07-22T12:43:49.623Z" },
{ url = "https://files.pythonhosted.org/packages/6c/21/5a38699fa45de749e3857d93b8f07e4c20489e77c2d35d915a2e1c456606/greenlet-3.5.4-cp315-cp315-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:394de08dad5ffcb1f50c2159d93e398d9d2da3ed437645eaa54771fa720db9f0", size = 676067, upload-time = "2026-07-22T11:51:18.163Z" },
{ url = "https://files.pythonhosted.org/packages/2e/d9/6298f3432de301d4718766cf934bd73c418c73f81fbb77247319364b0d96/greenlet-3.5.4-cp315-cp315-manylinux_2_39_riscv64.whl", hash = "sha256:cd320d998cbaa032932830448e39abf3c6a12901295e386e8114db926e10cffb", size = 487446, upload-time = "2026-07-22T12:39:57.044Z" },
{ url = "https://files.pythonhosted.org/packages/5e/ba/863116ab8ff1ca7a729e327800268939d182db47aa433db70e216e7d9194/greenlet-3.5.4-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:c883d61f2282d72c767a14936641b3efcbde9d82f1080712aaea0b1d3126cb88", size = 1633489, upload-time = "2026-07-22T12:25:09.605Z" },
{ url = "https://files.pythonhosted.org/packages/fa/06/7466ced82818d6132462d7f26b3f83c66ea15d2b193a6c0088d558ed7d95/greenlet-3.5.4-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:2a924f15d17957e252a810acefcb5942f5ca712298e8b6fcaed9a307d357522c", size = 1696584, upload-time = "2026-07-22T11:51:21.304Z" },
{ url = "https://files.pythonhosted.org/packages/f9/4d/55b638489260065de9ffce606c8b5d04507bef705de4b212a0c3d6a1a0df/greenlet-3.5.4-cp315-cp315-win_amd64.whl", hash = "sha256:ed17e5f3420360d5b459de8462efb52060399a5326a613d4cde31cef63ef95da", size = 248297, upload-time = "2026-07-22T11:42:04.055Z" },
{ url = "https://files.pythonhosted.org/packages/bb/08/9dd4ae635da93d41dc268bc34bd62a9d711ed8b8825c5d22ac910c7d6e6d/greenlet-3.5.4-cp315-cp315-win_arm64.whl", hash = "sha256:f908898d6fa484ce4b6f447ce70ea99b52c503fee419e53cf74d60a16bc9e667", size = 247423, upload-time = "2026-07-22T11:44:00.764Z" },
{ url = "https://files.pythonhosted.org/packages/19/66/7c87ed9cdbf1d49c2c6cd1c7b9dd4d16c33b24235ca03972293a1876b30c/greenlet-3.5.4-cp315-cp315t-macosx_11_0_universal2.whl", hash = "sha256:1833637f17d5e7472548a48575c394fe39f1b1890d676d162d86593610f44d8c", size = 306487, upload-time = "2026-07-22T11:41:25.118Z" },
{ url = "https://files.pythonhosted.org/packages/5b/05/0a4201e7c0054866eefc05da234f236dd4c950d0fbf9ca0517141f01b269/greenlet-3.5.4-cp315-cp315t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:12cda9122e03341f1cb6b8207a19d7a9d375e52f1b4e9243918375f40fd7b4b9", size = 676479, upload-time = "2026-07-22T12:26:48.129Z" },
{ url = "https://files.pythonhosted.org/packages/3d/c0/4b6b8c5a3aec70f0649cd89662d120fdd6421e2bdc8e3b15c3ab5ec568d8/greenlet-3.5.4-cp315-cp315t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:d83ae0e32d14957ab7170785a20f582635c8474deab1bfbb552b17e769a6ce25", size = 684321, upload-time = "2026-07-22T12:29:08.925Z" },
{ url = "https://files.pythonhosted.org/packages/88/15/0b167aeea95285b0e654ddce651922f666c089363c2ec528ca8b9a9ba74f/greenlet-3.5.4-cp315-cp315t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:123aa379c962ed5fe90a880327e0c3066124ac64ec99e12a238be9fd8eb3db3d", size = 685995, upload-time = "2026-07-22T12:43:50.993Z" },
{ url = "https://files.pythonhosted.org/packages/24/c9/b49c31c9a972eee91e260445770e922244a7efc542697f51a012ec046d0f/greenlet-3.5.4-cp315-cp315t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9f1467de1bb767f75db0aa34c195e3a496d8d1278c796e70c24ce205d3e99cde", size = 681293, upload-time = "2026-07-22T11:51:20.43Z" },
{ url = "https://files.pythonhosted.org/packages/de/90/c023ec337f32ff505be7db759c80d98f0532bb94d0c6fa13645efe9bee2e/greenlet-3.5.4-cp315-cp315t-manylinux_2_39_riscv64.whl", hash = "sha256:adf2244d7f69409925a8f22ed22cc5f93cdfe5c9dc87ff3476be2c2aaae61a05", size = 516928, upload-time = "2026-07-22T12:39:58.359Z" },
{ url = "https://files.pythonhosted.org/packages/95/6f/7f2d4653770500eee667866016d42d7a68e3d3462f80df6b8e3fcd48a0eb/greenlet-3.5.4-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:0fa53040b78b578120eecdc0265e3f1051487cc425d11a2b7c761daadf4feaa8", size = 1642474, upload-time = "2026-07-22T12:25:10.819Z" },
{ url = "https://files.pythonhosted.org/packages/5a/d8/8cba31036a4caae448087ba5d150660ab03b4a0f54d9150f6495a3be7262/greenlet-3.5.4-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:60e0bc961d367df506660e9ac0177a76bc6d81305300704b0977d1634f76efe2", size = 1701012, upload-time = "2026-07-22T11:51:23.17Z" },
{ url = "https://files.pythonhosted.org/packages/e3/cd/3f77a4cce3bae631b08eb52f53a82a976669600337e21dfdba811cb50267/greenlet-3.5.4-cp315-cp315t-win_amd64.whl", hash = "sha256:f680e549edb3eaf21eea4e7fe101e15ec180c74b7879ab46adc080f22d4015d2", size = 251977, upload-time = "2026-07-22T11:41:38.125Z" },
{ url = "https://files.pythonhosted.org/packages/93/e8/65e8707d00fe2a49bf12f609a9b2b39ba6dd23c2810eacad877c4fc94bfe/greenlet-3.5.4-cp315-cp315t-win_arm64.whl", hash = "sha256:08fc36de8442d5c3e95b044550dbea9bf144d31ec0cc58e36fb241cb6ef6a994", size = 250538, upload-time = "2026-07-22T11:40:17.985Z" },
]
[[package]] [[package]]
name = "h11" name = "h11"
version = "0.16.0" version = "0.16.0"
@@ -435,6 +504,25 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/17/53/2a52f64f399717adae560068e9b0ae12ae3e43be2c64991e246af5aedc3f/pillow_heif-1.5.0-cp314-cp314t-win_arm64.whl", hash = "sha256:508fc9dd8fb4df933b666c30b60b0930270d9e072fcd90b75990166050e44656", size = 4017728, upload-time = "2026-07-22T14:27:41.102Z" }, { url = "https://files.pythonhosted.org/packages/17/53/2a52f64f399717adae560068e9b0ae12ae3e43be2c64991e246af5aedc3f/pillow_heif-1.5.0-cp314-cp314t-win_arm64.whl", hash = "sha256:508fc9dd8fb4df933b666c30b60b0930270d9e072fcd90b75990166050e44656", size = 4017728, upload-time = "2026-07-22T14:27:41.102Z" },
] ]
[[package]]
name = "playwright"
version = "1.62.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "greenlet" },
{ name = "pyee" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/6c/5b/ca2abcf3aa69f9fb510215e3064f30b57fe57657c8d04ede45bb966d5606/playwright-1.62.0-py3-none-macosx_10_13_x86_64.whl", hash = "sha256:d8da938f3748841a8754f2e1f0216902c1c8f8ae3720de8b32ccf8e6913a7c4f", size = 43732091, upload-time = "2026-07-31T17:00:44.178Z" },
{ url = "https://files.pythonhosted.org/packages/af/1a/0bfbe9904350961f4dbb713f04342e40d548c5fc26c8157bd13617c81492/playwright-1.62.0-py3-none-macosx_11_0_arm64.whl", hash = "sha256:db755ab27db21a04186f1fe8169888e42356086e439b1059b923ef417f0b6034", size = 42510842, upload-time = "2026-07-31T17:00:48.596Z" },
{ url = "https://files.pythonhosted.org/packages/66/dc/c0486b407ad0699a250f6bbe3066fca95344009a99ca66e88ca175c69dc1/playwright-1.62.0-py3-none-macosx_11_0_universal2.whl", hash = "sha256:5108bd5b3e87169ddf269feee097da5893af7f8aea4634dfc840518d64c1f1da", size = 43732093, upload-time = "2026-07-31T17:00:52.218Z" },
{ url = "https://files.pythonhosted.org/packages/43/6b/b24aebc2b04bffcb342bccf96e287c78b363e1615bed5cea97500cc0393a/playwright-1.62.0-py3-none-manylinux1_x86_64.whl", hash = "sha256:ba33bae6a13b3d9d354c751cb618af357d20fe1d57767cbcce52079bbef17ad3", size = 47748926, upload-time = "2026-07-31T17:00:56.438Z" },
{ url = "https://files.pythonhosted.org/packages/36/43/b4b18bdc87e1949568fffdcde3ff9a0456266b2d0c6d4432cc34d89ea6eb/playwright-1.62.0-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:db2d76613a57ad844362ce42f7d0c2fa26b19a4f7a46d4f76b891c631e6e5aff", size = 47441423, upload-time = "2026-07-31T17:01:00.404Z" },
{ url = "https://files.pythonhosted.org/packages/81/22/af5d926fc2c32a339eec00a443644bc40ab9db1dd2dd9017873c59773c0c/playwright-1.62.0-py3-none-win32.whl", hash = "sha256:e5614fa89355d7081457680324bb219f79f69c423c5cb6fa250e30b0d8aebf1c", size = 38164450, upload-time = "2026-07-31T17:01:04.187Z" },
{ url = "https://files.pythonhosted.org/packages/2b/a9/4160c1033c07af98bf841ad079457dd78408a5ee0dd56cbfe50b8b6a1c22/playwright-1.62.0-py3-none-win_amd64.whl", hash = "sha256:92c0d98ed04eb35af557b709875edba415b1f548bdb22ddb5bb3e1e6c835c2f1", size = 38164458, upload-time = "2026-07-31T17:01:08.459Z" },
{ url = "https://files.pythonhosted.org/packages/6c/ec/06b55d619a7082a766aa04f2c6bb31435c87f02930087d8a0517119408fa/playwright-1.62.0-py3-none-win_arm64.whl", hash = "sha256:ea8d3055aa9d5a9f1832ac82517bd8b42c78fac7ebcbebb0107116735c8cb6a1", size = 34208868, upload-time = "2026-07-31T17:01:11.818Z" },
]
[[package]] [[package]]
name = "pluggy" name = "pluggy"
version = "1.6.0" version = "1.6.0"
@@ -534,6 +622,18 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/fa/c3/7c8b240552251faf6b3a957db200fcfbbcec36763c050428b601e0c9b83b/pydantic_core-2.46.4-graalpy312-graalpy250_312_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:00c603d540afdd6b80eb39f078f33ebd46211f02f33e34a32d9f053bba711de0", size = 2147590, upload-time = "2026-05-06T13:39:29.883Z" }, { url = "https://files.pythonhosted.org/packages/fa/c3/7c8b240552251faf6b3a957db200fcfbbcec36763c050428b601e0c9b83b/pydantic_core-2.46.4-graalpy312-graalpy250_312_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:00c603d540afdd6b80eb39f078f33ebd46211f02f33e34a32d9f053bba711de0", size = 2147590, upload-time = "2026-05-06T13:39:29.883Z" },
] ]
[[package]]
name = "pyee"
version = "13.0.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "typing-extensions" },
]
sdist = { url = "https://files.pythonhosted.org/packages/8b/04/e7c1fe4dc78a6fdbfd6c337b1c3732ff543b8a397683ab38378447baa331/pyee-13.0.1.tar.gz", hash = "sha256:0b931f7c14535667ed4c7e0d531716368715e860b988770fc7eb8578d1f67fc8", size = 31655, upload-time = "2026-02-14T21:12:28.044Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/a0/c4/b4d4827c93ef43c01f599ef31453ccc1c132b353284fc6c87d535c233129/pyee-13.0.1-py3-none-any.whl", hash = "sha256:af2f8fede4171ef667dfded53f96e2ed0d6e6bd7ee3bb46437f77e3b57689228", size = 15659, upload-time = "2026-02-14T21:12:26.263Z" },
]
[[package]] [[package]]
name = "pygments" name = "pygments"
version = "2.20.0" version = "2.20.0"