The rollup and the backup check in once, when they end, with the run's duration
The rollup finishes in two seconds on the shared host, and its "in progress" and "ok" check-ins reached Sentry in either order; when "ok" landed first, the late start was never closed and the monitor reported a timeout ten minutes later. One check-in at the end cannot race, and a run that hangs still shows: nothing arrives and the night is marked missed. Fixes TAFL-1 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GwFKMuQnPAEHJ5yA1q4orh
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
cc9d7330aa
commit
57ae338cdb
@@ -14,9 +14,14 @@ LOG="/var/log/hnefatafl/backup.log"
|
||||
STAMP=$(date +%Y-%m-%d)
|
||||
mkdir -p "$(dirname "$LOG")"
|
||||
CRON_URL=$(cat /root/.hnefatafl-sentry-cron 2>/dev/null || true)
|
||||
# One check-in, when the run ends, carrying its duration: a start and a
|
||||
# finish sent seconds apart can reach Sentry in either order, and a start
|
||||
# that lands second is never closed. A run that never ends sends nothing,
|
||||
# and Sentry marks the night missed.
|
||||
START=$SECONDS
|
||||
checkin() {
|
||||
[ -n "$CRON_URL" ] && curl -sf -o /dev/null -X POST -H "Content-Type: application/json" \
|
||||
-d "{\"status\":\"$1\",\"monitor_config\":{\"schedule\":{\"type\":\"crontab\",\"value\":\"$SCHEDULE\"},\"checkin_margin\":30,\"max_runtime\":10,\"timezone\":\"UTC\"}}" \
|
||||
-d "{\"status\":\"$1\",\"duration\":$((SECONDS - START)),\"monitor_config\":{\"schedule\":{\"type\":\"crontab\",\"value\":\"$SCHEDULE\"},\"checkin_margin\":30,\"max_runtime\":10,\"timezone\":\"UTC\"}}" \
|
||||
"$CRON_URL" || true
|
||||
}
|
||||
|
||||
@@ -27,7 +32,6 @@ fi
|
||||
|
||||
{
|
||||
echo "=== $(date -Is) backup start"
|
||||
checkin in_progress
|
||||
if rclone sync "$SRC" "spaces:$BUCKET/$PREFIX/current" --exclude 'rollup.jsonl' 2>&1 &&
|
||||
rclone copy "$SRC" "spaces:$BUCKET/$PREFIX/snapshots/$STAMP" 2>&1; then
|
||||
checkin ok
|
||||
|
||||
@@ -16,13 +16,17 @@ DAY="${1:-$(date -u -d 'yesterday' +%Y-%m-%d)}"
|
||||
SCHEDULE="12 0 * * *"
|
||||
CRON_URL=$(cat /root/.hnefatafl-sentry-cron-rollup 2>/dev/null || true)
|
||||
# The check-in carries the schedule: Sentry keeps a monitor only once told its shape.
|
||||
# One check-in, when the run ends, carrying its duration: a start and a
|
||||
# finish sent seconds apart can reach Sentry in either order, and a start
|
||||
# that lands second is never closed. A run that never ends sends nothing,
|
||||
# and Sentry marks the night missed.
|
||||
START=$SECONDS
|
||||
checkin() {
|
||||
[ -n "$CRON_URL" ] && curl -sf -o /dev/null -X POST -H "Content-Type: application/json" \
|
||||
-d "{\"status\":\"$1\",\"monitor_config\":{\"schedule\":{\"type\":\"crontab\",\"value\":\"$SCHEDULE\"},\"checkin_margin\":30,\"max_runtime\":10,\"timezone\":\"UTC\"}}" \
|
||||
-d "{\"status\":\"$1\",\"duration\":$((SECONDS - START)),\"monitor_config\":{\"schedule\":{\"type\":\"crontab\",\"value\":\"$SCHEDULE\"},\"checkin_margin\":30,\"max_runtime\":10,\"timezone\":\"UTC\"}}" \
|
||||
"$CRON_URL" || true
|
||||
}
|
||||
mkdir -p "$(dirname "$LOG")"
|
||||
checkin in_progress
|
||||
if python3 - "$DAY" "$OUT" <<'PY' >> "$LOG" 2>&1
|
||||
import datetime, glob, json, os, shutil, sys
|
||||
sys.path.insert(0, "/usr/local/lib/hnefatafl")
|
||||
|
||||
Reference in New Issue
Block a user