Eric WagonerandClaude Fable 5.1
60435fef84
Eight cron monitors, two per game, watched eight scripts that run on the same box at the same two minutes. hall-nightly.sh now runs the hall's own rollup and every game's rollup at 00:12, and every game's backup at 07:23, finding the games by their scripts in /usr/local/bin, and checks in once for each job to the gamehall project's hall-rollup and hall-backup monitors. The template's rollup and backup say by their exit status whether they managed, the skipped backup included, and its deploy leaves the game's own cron file off a host the hall schedules. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
53 lines
2.2 KiB
Bash
Executable File
53 lines
2.2 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# The hall's nightly work for every game on this host, with one Sentry Crons
|
|
# check-in for the lot: the rollups just after midnight, the backups at
|
|
# breakfast. Each game's own script does its work and says by its exit
|
|
# status whether it managed; the games are found by their scripts in
|
|
# /usr/local/bin, so a game deployed here is covered without being listed.
|
|
# hall-nightly.sh rollup|backup
|
|
# Cron: /etc/cron.d/gamehall, installed by gamehall/deploy.sh. SCHEDULE
|
|
# below must match it, because Sentry is told the same shape.
|
|
# Sentry Crons check-in: /root/.hall-sentry-cron-<job> holds the check-in
|
|
# URL (absent = no check-ins).
|
|
set -u
|
|
JOB="${1:-}"
|
|
case "$JOB" in
|
|
rollup) SCHEDULE="12 0 * * *" ;;
|
|
backup) SCHEDULE="23 7 * * *" ;;
|
|
*) echo "usage: hall-nightly.sh rollup|backup" >&2; exit 2 ;;
|
|
esac
|
|
LOG="/var/log/gamehall/nightly.log"
|
|
CRON_URL=$(cat "/root/.hall-sentry-cron-$JOB" 2>/dev/null || true)
|
|
# One check-in, when the run ends, carrying its duration and the schedule
|
|
# (Sentry keeps a monitor only once told its shape). A start and a finish
|
|
# sent seconds apart can reach Sentry in either order, and a start that lands
|
|
# second is never closed. A run that never ends sends nothing, and Sentry
|
|
# marks the night missed.
|
|
START=$SECONDS
|
|
checkin() {
|
|
[ -n "$CRON_URL" ] && curl -sf -o /dev/null -X POST -H "Content-Type: application/json" \
|
|
-d "{\"status\":\"$1\",\"duration\":$((SECONDS - START)),\"monitor_config\":{\"schedule\":{\"type\":\"crontab\",\"value\":\"$SCHEDULE\"},\"checkin_margin\":30,\"max_runtime\":30,\"timezone\":\"UTC\"}}" \
|
|
"$CRON_URL" || true
|
|
}
|
|
mkdir -p "$(dirname "$LOG")"
|
|
|
|
# The hall's own count first, then each game's script, in name order.
|
|
scripts=()
|
|
[ "$JOB" = rollup ] && scripts+=(/usr/local/bin/gamehall-rollup.sh)
|
|
for s in /usr/local/bin/*-"$JOB".sh; do
|
|
[ "$s" = /usr/local/bin/gamehall-rollup.sh ] || scripts+=("$s")
|
|
done
|
|
failed=()
|
|
for s in "${scripts[@]}"; do
|
|
name=$(basename "$s" "-$JOB.sh")
|
|
"$s" || failed+=("$name")
|
|
done
|
|
if [ ${#failed[@]} -eq 0 ]; then
|
|
echo "$(date -Is) $JOB: ${#scripts[@]} done" >> "$LOG"
|
|
checkin ok
|
|
else
|
|
echo "$(date -Is) $JOB: ${#scripts[@]} run, failed: ${failed[*]}" >> "$LOG"
|
|
checkin error
|
|
exit 1
|
|
fi
|