The Queue page reports what upload already did
All 36 version updates landed and all 36 still read as outstanding: to_add.csv and to_update.csv are diff-time snapshots that never shrink, and the page showed them without consulting the upload log. Rows now carry their last attempt's outcome — pending / done / failed, plus "retired" for jobs a later review decision withdrew — and each section heads with a tally instead of a raw row count. A note explains that finished rows persist until the next diff rebuilds the queue, and that the log is the permanent record. On Eric's data: to_update now reads 36 done, 0 pending. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016jXZFSTZQKzAC8fqpWSz9g
This commit is contained in:
co-authored by
Claude Fable 5
parent
0af9ae86c5
commit
e44c7e1b92
@@ -29,7 +29,7 @@
|
||||
<p><b><a href="/photos">Photos</a></b> — drag photos in, drop them in the <code>photos/</code> folder, or (on a paired phone) tap the drop zone and shoot straight from the camera; camera captures get unique <code>shelf-…</code> names so they can never overwrite each other. Each photo has its own page listing every title read from it and any reshoot tickets — boxes seen but not identified. Photograph those up close, drop the new shot in, and extract again. Re-uploading a photo under the same <i>file name</i> deliberately replaces it, and the next extract run re-reads it.</p>
|
||||
<p><b><a href="/titles">Titles</a></b> — every read off your shelves, alphabetized, with its status and photos. This is the proofread checkpoint: <a href="#curation">edit, split, remove</a>. Its badge counts <span class="chip shaky">shaky read</span> lines — the model wasn't sure and nothing has verified them; filter to them, then press <b>✓ looks right</b> or edit each one.</p>
|
||||
<p><b><a href="/review">Review</a></b> — the decisions only you can make: which game a title is, which edition a copy is, whether two same-game reads are really one box (merges show a veto), and whether an unmatched title is a real game BGG simply doesn't have (<b>keep locally</b>: it joins the Library, never uploads). Keyboard-first; see <a href="#keys">shortcuts</a>.</p>
|
||||
<p><b><a href="/queue">Queue</a></b> — exactly what upload will do (new entries and version upgrades) and the log of everything it has done. Nothing reaches BGG that isn't visible here first. A job that fails is skipped by later runs (so one broken game can't loop forever); when any exist, the Pipeline's upload card offers a <b>retry N failed</b> checkbox.</p>
|
||||
<p><b><a href="/queue">Queue</a></b> — exactly what upload will do (new entries and version upgrades) and the log of everything it has done. Nothing reaches BGG that isn't visible here first. A job that fails is skipped by later runs (so one broken game can't loop forever); when any exist, the Pipeline's upload card offers a <b>retry N failed</b> checkbox. Each queued row shows what upload did with it — <span class="chip open">pending</span>, <span class="chip ok">done</span>, <span class="chip no">failed</span>, or <span class="chip no">retired</span> (a review decision since the last diff withdrew it). Finished rows stay listed until the next <b>diff</b> rebuilds the queue; the log below them is the permanent record.</p>
|
||||
<p><b><a href="/library">Library</a></b> — your enriched collection: filter by board games or RPGs. RPG matches are identified and enriched but never uploaded — BGG collections can't hold them, so they stay local citizens.</p>
|
||||
</div>
|
||||
|
||||
|
||||
@@ -11,24 +11,48 @@ function table(headers, rows) {
|
||||
${rows.join("")}</table></div>`;
|
||||
}
|
||||
|
||||
// a queue row's state comes from the upload log: the CSVs are diff-time
|
||||
// snapshots and never shrink as work completes
|
||||
function stateChip(r) {
|
||||
if (r.stale) return `<span class="chip no" title="${esc(r.stale)}">retired</span>`;
|
||||
if (r.state === "done") return `<span class="chip ok">done</span>`;
|
||||
if (r.state === "failed") return `<span class="chip no">failed</span>`;
|
||||
return `<span class="chip open">pending</span>`;
|
||||
}
|
||||
|
||||
function tally(rows) {
|
||||
const n = s => rows.filter(r => !r.stale && r.state === s).length;
|
||||
const parts = [`<b>${n("")}</b> pending`];
|
||||
if (n("done")) parts.push(`<b>${n("done")}</b> done`);
|
||||
if (n("failed")) parts.push(`<b>${n("failed")}</b> failed`);
|
||||
const retired = rows.filter(r => r.stale).length;
|
||||
if (retired) parts.push(`<b>${retired}</b> retired by review`);
|
||||
return parts.join(" · ");
|
||||
}
|
||||
|
||||
function render(q) {
|
||||
let html = "";
|
||||
|
||||
html += `<h2>To add <span class="count">— ${q.to_add.length} new collection entr${q.to_add.length === 1 ? "y" : "ies"}</span></h2>`;
|
||||
html += `<h2>To add <span class="count">— ${tally(q.to_add)}</span></h2>`;
|
||||
html += q.to_add.length
|
||||
? table(["game", "version", "seen in"], q.to_add.map(r => `
|
||||
? table(["game", "version", "seen in", ""], q.to_add.map(r => `
|
||||
<tr><td class="t">${esc(r.bgg_name)} <span class="meta">· ${esc(r.bgg_id)}</span></td>
|
||||
<td>${r.version_name ? esc(r.version_name) : `<span class="meta">no version</span>`}</td>
|
||||
<td class="meta">${esc((r.source_photos ?? "").split(";").join(", "))}</td></tr>`))
|
||||
<td class="meta">${esc((r.source_photos ?? "").split(";").join(", "))}</td>
|
||||
<td>${stateChip(r)}</td></tr>`))
|
||||
: `<p class="empty">Nothing queued — run <b>diff</b> from the <a href="/">pipeline</a> first.</p>`;
|
||||
|
||||
html += `<h2>Version updates <span class="count">— ${q.to_update.length} existing entr${q.to_update.length === 1 ? "y" : "ies"} gaining a version</span></h2>`;
|
||||
html += `<h2>Version updates <span class="count">— ${tally(q.to_update)}</span></h2>`;
|
||||
html += q.to_update.length
|
||||
? table(["game", "version to set", "collection id"], q.to_update.map(r => `
|
||||
? table(["game", "version to set", "collection id", ""], q.to_update.map(r => `
|
||||
<tr><td class="t">${esc(r.bgg_name)} <span class="meta">· ${esc(r.bgg_id)}</span></td>
|
||||
<td>${esc(r.version_name)}</td>
|
||||
<td class="meta">${esc(r.collid)}</td></tr>`))
|
||||
<td class="meta">${esc(r.collid)}</td>
|
||||
<td>${stateChip(r)}</td></tr>`))
|
||||
: `<p class="empty">No version updates pending.</p>`;
|
||||
if (q.to_add.concat(q.to_update).some(r => r.state === "done"))
|
||||
html += `<p class="empty">Finished jobs stay listed until the next <b>diff</b>
|
||||
rebuilds the queue — the log below is the permanent record.</p>`;
|
||||
|
||||
html += `<h2>Upload log <span class="count">— every attempt ever made (${q.log.length})</span></h2>`;
|
||||
html += q.log.length
|
||||
|
||||
@@ -90,6 +90,40 @@ def _job_key(row: dict) -> tuple[str, str, str]:
|
||||
return _key(row["action"], row["bgg_id"], row["collid"], row["version_id"])
|
||||
|
||||
|
||||
def annotate_queue(
|
||||
queue_rows: list[dict], action: str, log_rows: list[dict]
|
||||
) -> list[dict]:
|
||||
"""Each queue row plus the outcome of its LAST upload attempt, in a
|
||||
`state` field: "" (pending), "done", or "failed". to_add.csv and
|
||||
to_update.csv are diff-time snapshots — nothing removes a row once its
|
||||
job succeeds — so a reader without the log sees finished work as
|
||||
outstanding forever."""
|
||||
last: dict[tuple[str, str, str], str] = {}
|
||||
for row in log_rows:
|
||||
last[_job_key(row)] = row["status"]
|
||||
out = []
|
||||
for row in queue_rows:
|
||||
key = _key(
|
||||
action,
|
||||
row.get("bgg_id", ""),
|
||||
row.get("collid", ""),
|
||||
row.get("version_id", ""),
|
||||
)
|
||||
status = last.get(key, "")
|
||||
out.append(
|
||||
{
|
||||
**row,
|
||||
"state": "done"
|
||||
if status in DONE_STATUSES
|
||||
else "failed"
|
||||
if status == "failed"
|
||||
else "",
|
||||
"last_status": status,
|
||||
}
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
def stale_jobs(queue_rows: list[dict], match_rows: list[dict]) -> dict[str, str]:
|
||||
"""bgg_id -> why, for queued games the CURRENT matches.csv no longer
|
||||
endorses. to_add.csv is a snapshot from the last diff; a review
|
||||
|
||||
@@ -762,10 +762,23 @@ def create_app(
|
||||
with path.open(newline="") as f:
|
||||
return list(csv.DictReader(f))
|
||||
|
||||
from bggpipe.upload import annotate_queue, stale_jobs
|
||||
|
||||
log = rows(cfg.upload_log_path)
|
||||
to_add = rows(cfg.to_add_path)
|
||||
to_update = rows(cfg.to_update_path)
|
||||
# a review decision taken after the last diff retires a queued job
|
||||
stale = stale_jobs(to_add + to_update, session.rows)
|
||||
return {
|
||||
"to_add": rows(cfg.to_add_path),
|
||||
"to_update": rows(cfg.to_update_path),
|
||||
"log": rows(cfg.upload_log_path),
|
||||
"to_add": [
|
||||
{**r, "stale": stale.get(r.get("bgg_id", ""), "")}
|
||||
for r in annotate_queue(to_add, "add", log)
|
||||
],
|
||||
"to_update": [
|
||||
{**r, "stale": stale.get(r.get("bgg_id", ""), "")}
|
||||
for r in annotate_queue(to_update, "update", log)
|
||||
],
|
||||
"log": log,
|
||||
}
|
||||
|
||||
@app.get("/api/library")
|
||||
|
||||
Reference in New Issue
Block a user