The hall keeps the night on the shared host: one job runs every game's rollup and backup and checks in with Sentry once for each

Eight cron monitors, two per game, watched eight scripts that run on the
same box at the same two minutes. hall-nightly.sh now runs the hall's own
rollup and every game's rollup at 00:12, and every game's backup at 07:23,
finding the games by their scripts in /usr/local/bin, and checks in once
for each job to the gamehall project's hall-rollup and hall-backup
monitors. The template's rollup and backup say by their exit status
whether they managed, the skipped backup included, and its deploy leaves
the game's own cron file off a host the hall schedules.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Eric Wagoner
2026-10-12 10:07:05 -04:00
co-authored by Claude Fable 5.1
parent 87c2215e5f
commit 60435fef84
11 changed files with 98 additions and 17 deletions
+12 -7
View File
@@ -46,9 +46,11 @@ A fresh host, when one is needed: `doctl compute droplet create hall --region ny
## Operations scripts on the droplet
`deploy.sh` installs these to /usr/local/bin, the `traffic.py` they share to
/usr/local/lib/__SLUG__, and the cron file to /etc/cron.d/__SLUG__ on every
deploy, so the live copies are the repo copies:
`deploy.sh` installs these to /usr/local/bin and the `traffic.py` they share
to /usr/local/lib/__SLUG__ on every deploy, so the live copies are the repo
copies. The cron file goes to /etc/cron.d/__SLUG__ only on a box of the game's
own: on the hall's shared host, `hall-nightly.sh` (in the kit's `gamehall/ops`)
runs every game's rollup and backup from /etc/cron.d/gamehall.
- `__SLUG__-visitors.sh [day]`: who is here now and who came that day.
- `__SLUG__-pulse.sh`: the weekly health check (service, errors, rollup
@@ -62,10 +64,13 @@ deploy, so the live copies are the repo copies:
wizwar droplet; until then it logs "skipped".
Errors from the game server go to Sentry, project `__SLUG__` in the
locallygrownnet organisation; the DSN is in the unit file. The nightly rollup
and backup check in with Sentry Crons when /root/.__SLUG__-sentry-cron-rollup
and /root/.__SLUG__-sentry-cron hold their check-in URLs (the ingest
URL with the project's cron path and public key), so a missed night is noticed.
locallygrownnet organisation; the DSN is in the unit file. On the shared host
the hall's nightly job checks in with Sentry Crons once for every game's
rollup and once for every backup (monitors `hall-rollup` and `hall-backup` in
the `gamehall` project), and each game's script reports by its exit status.
A game on its own box checks in for itself when /root/.__SLUG__-sentry-cron-rollup
and /root/.__SLUG__-sentry-cron hold check-in URLs (the ingest URL with the
project's cron path and public key); absent, it sends nothing.
To have every new issue, regression and reappearance posted to Slack the way
wizwar's are, run `deploy/sentry-slack-alert.sh [#channel]` once from your own