The hall keeps the night on the shared host: one job runs every game's rollup and backup and checks in with Sentry once for each
Eight cron monitors, two per game, watched eight scripts that run on the same box at the same two minutes. hall-nightly.sh now runs the hall's own rollup and every game's rollup at 00:12, and every game's backup at 07:23, finding the games by their scripts in /usr/local/bin, and checks in once for each job to the gamehall project's hall-rollup and hall-backup monitors. The template's rollup and backup say by their exit status whether they managed, the skipped backup included, and its deploy leaves the game's own cron file off a host the hall schedules. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5.1
parent
87c2215e5f
commit
60435fef84
@@ -46,9 +46,11 @@ A fresh host, when one is needed: `doctl compute droplet create hall --region ny
|
||||
|
||||
## Operations scripts on the droplet
|
||||
|
||||
`deploy.sh` installs these to /usr/local/bin, the `traffic.py` they share to
|
||||
/usr/local/lib/__SLUG__, and the cron file to /etc/cron.d/__SLUG__ on every
|
||||
deploy, so the live copies are the repo copies:
|
||||
`deploy.sh` installs these to /usr/local/bin and the `traffic.py` they share
|
||||
to /usr/local/lib/__SLUG__ on every deploy, so the live copies are the repo
|
||||
copies. The cron file goes to /etc/cron.d/__SLUG__ only on a box of the game's
|
||||
own: on the hall's shared host, `hall-nightly.sh` (in the kit's `gamehall/ops`)
|
||||
runs every game's rollup and backup from /etc/cron.d/gamehall.
|
||||
|
||||
- `__SLUG__-visitors.sh [day]`: who is here now and who came that day.
|
||||
- `__SLUG__-pulse.sh`: the weekly health check (service, errors, rollup
|
||||
@@ -62,10 +64,13 @@ deploy, so the live copies are the repo copies:
|
||||
wizwar droplet; until then it logs "skipped".
|
||||
|
||||
Errors from the game server go to Sentry, project `__SLUG__` in the
|
||||
locallygrownnet organisation; the DSN is in the unit file. The nightly rollup
|
||||
and backup check in with Sentry Crons when /root/.__SLUG__-sentry-cron-rollup
|
||||
and /root/.__SLUG__-sentry-cron hold their check-in URLs (the ingest
|
||||
URL with the project's cron path and public key), so a missed night is noticed.
|
||||
locallygrownnet organisation; the DSN is in the unit file. On the shared host
|
||||
the hall's nightly job checks in with Sentry Crons once for every game's
|
||||
rollup and once for every backup (monitors `hall-rollup` and `hall-backup` in
|
||||
the `gamehall` project), and each game's script reports by its exit status.
|
||||
A game on its own box checks in for itself when /root/.__SLUG__-sentry-cron-rollup
|
||||
and /root/.__SLUG__-sentry-cron hold check-in URLs (the ingest URL with the
|
||||
project's cron path and public key); absent, it sends nothing.
|
||||
|
||||
To have every new issue, regression and reappearance posted to Slack the way
|
||||
wizwar's are, run `deploy/sentry-slack-alert.sh [#channel]` once from your own
|
||||
|
||||
Reference in New Issue
Block a user