game-kit/template/deploy/README.md

Deploying NAME

Every game is a room on a small Node process that keeps it as an append-only ledger of moves in /var/lib/SLUG/rooms. Production is the shared Kestrel's Hall host: Caddy terminates TLS with automatic certificates, serves the static build from /opt/SLUG/build, and proxies /api and /ws to the game server on port PORT, which runs as the __SLUG__ user under systemd from /opt/SLUG/app.

The shared host

The games of Kestrel's Hall share one droplet, hall (nyc3, s-1vcpu-1gb, $6 a month, tag kestrels-hall, IP 159.203.115.252). Each game keeps its own Caddy site file in /etc/caddy/sites/.caddy, its own port, service user, /opt/ and /var/lib/ trees, access log (/var/lib/caddy/-access.log), cron file and ops scripts; /etc/caddy/Caddyfile only says import sites/*.caddy. A game server is one small Node process capped at 300 MB by its unit file, and a room is a few kilobytes of ledger, so the host carries several. Each game needs a port of its own on the host: wizwar 8787, waving-hands 8788, hnefatafl 8789, dip 8790.

Current production

• Host: IP
• URLs: https://DOMAIN (A record at Hover, where kestrelsnest.social's DNS lives) and https://SLUG.IP.sslip.io (always works, zero DNS).
• Everyday deploy: deploy/deploy.sh __IP__
• Players' reports: deploy/pull-reports.sh mirrors /var/lib/SLUG/feedback.jsonl and the screenshots to ~/Desktop/SLUG-reports with a digest; deploy/report-reply.sh __IP__ <id> <status> "text" answers one.

Adding this game to the host

1. scp deploy/setup-droplet.sh deploy/Caddyfile.tmpl root@<ip>:/root/ && ssh root@<ip> \ "bash /root/setup-droplet.sh '__DOMAIN__, __SLUG__.<ip>.sslip.io'" It installs Caddy and Node if the host lacks them, writes this game's site file, and touches nothing of another game's. Point the A record at the host first, or leave the real name out until it is; running it again with the new line is how a name is added later.
2. deploy/deploy.sh <ip>

A fresh host, when one is needed: doctl compute droplet create hall --region nyc3 \ --size s-1vcpu-1gb --image ubuntu-24-04-x64 --ssh-keys <your-key-ids> \ --tag-name kestrels-hall --wait, then step 1.

Operations scripts on the droplet

deploy.sh installs these to /usr/local/bin, the traffic.py they share to /usr/local/lib/SLUG, and the cron file to /etc/cron.d/SLUG on every deploy, so the live copies are the repo copies:

• __SLUG__-visitors.sh [day]: who is here now and who came that day.
• __SLUG__-pulse.sh: the weekly health check (service, errors, rollup trend, backup, box).
• __SLUG__-rollup.sh [day]: one JSON line per day of counts, run nightly at 00:12 UTC into /var/lib/SLUG/rollup.jsonl.
• __SLUG__-backup.sh: nightly at 07:23 UTC, mirrors /var/lib/SLUG to the kestrel-wizwar-backups Space under __SLUG__/ (a current copy and dated snapshots kept 90 days). It needs rclone with the Spaces credentials in /root/.config/rclone/rclone.conf, copied by hand from the wizwar droplet; until then it logs "skipped".

Errors from the game server go to Sentry, project __SLUG__ in the locallygrownnet organisation; the DSN is in the unit file. The nightly rollup and backup check in with Sentry Crons when /root/.SLUG-sentry-cron-rollup and /root/.SLUG-sentry-cron hold their check-in URLs (the ingest URL with the project's cron path and public key), so a missed night is noticed.

To have every new issue, regression and reappearance posted to Slack the way wizwar's are, run deploy/sentry-slack-alert.sh [#channel] once from your own shell with SENTRY_TOKEN (an org auth token with alerts:write) and SLACK_CHANNEL_ID (the channel's Slack ID) set; the token never leaves the shell.

Before every deploy, deploy/verify-ledgers.sh <ip> fetches every production ledger and replays it with the local engine, comparing each room with what the server shows. A ledger the new engine refuses or replays differently stops the deploy: the server would rewrite that game on restart.

Everyday deploys

deploy/deploy.sh <ip>

Runs the type-checks, the tests and the build locally, rsyncs build/ to the droplet keeping the previous week's hashed assets, rsyncs the server and engine sources, installs dependencies, and restarts the game server. A room is replayed from its ledger when the server comes back, which takes a few seconds; Caddy holds requests that land in the gap.

Operations

• Logs: ssh root@<ip> journalctl -u __SLUG__ -f for the game server, journalctl -u caddy -f and /var/lib/caddy/access.log for the web side.
• Restart: ssh root@<ip> systemctl restart __SLUG__
• Rooms: /var/lib/__SLUG__/rooms/<CODE>.jsonl, one ledger per game; the nightly backup keeps them in the Space.