waving-hands/deploy

Deploying Waving Hands

The single-player game runs entirely in the browser. Duels between people go through a small Node process that keeps each room as an append-only ledger of moves in /var/lib/waving-hands/rooms. Production is one DigitalOcean droplet: Caddy terminates TLS with automatic certificates, serves the static build from /opt/waving-hands/build, and proxies /api and /ws to the duel server on port 8788, which runs as the waving-hands user under systemd from /opt/waving-hands/app.

Sizing

The smallest droplet, s-1vcpu-512mb-10gb ($4 a month), carries both Caddy and the duel server comfortably: the server is one small Node process capped at 300 MB by its unit file, and a room is a few kilobytes of ledger.

The zero-cost alternative is a second site block in an existing Caddy server's configuration pointing at a second directory; the deploy script works unchanged against that host. A droplet of its own keeps this site's uptime and upgrades independent of anything else.

Current production

• Droplet: waving-hands (nyc3, s-1vcpu-512mb-10gb, tag waving-hands), IP 159.203.98.71
• URLs: https://hands.kestrelsnest.social (A record at Hover, where kestrelsnest.social's DNS lives) and https://waving-hands.159.203.98.71.sslip.io (always works, zero DNS). Caddy serves both from the first line of /etc/caddy/Caddyfile.
• Everyday deploy: deploy/deploy.sh 159.203.98.71
• Players' reports: deploy/pull-reports.sh mirrors /var/lib/waving-hands/feedback.jsonl and the screenshots to ~/Desktop/waving-hands-reports with a digest; deploy/report-reply.sh 159.203.98.71 <id> <status> "text" answers one.

New droplet from scratch

1. doctl compute droplet create waving-hands --region nyc3 \ --size s-1vcpu-512mb-10gb --image ubuntu-24-04-x64 \ --ssh-keys <your-key-ids> --tag-name waving-hands --wait
2. scp deploy/setup-droplet.sh deploy/Caddyfile.tmpl root@<ip>:/root/ && ssh root@<ip> \ "bash /root/setup-droplet.sh 'hands.kestrelsnest.social, waving-hands.<ip>.sslip.io'" (point the A record at the new IP first, or leave the real name out until it is).
3. scp deploy/setup-server.sh deploy/Caddyfile.tmpl root@<ip>:/root/ && ssh root@<ip> "bash /root/setup-server.sh"
4. deploy/deploy.sh <ip>

The sslip.io hostname works with no DNS at all. To add a real name, point an A record at the droplet and add the name to the first line of /etc/caddy/Caddyfile (space separated), then systemctl reload caddy; Caddy fetches the certificate on first request.

Operations scripts on the droplet

deploy.sh installs these to /usr/local/bin and the cron file to /etc/cron.d/waving-hands on every deploy, so the live copies are the repo copies:

• waving-hands-visitors.sh [day]: who is here now and who came that day.
• waving-hands-pulse.sh: the weekly health check (service, errors, rollup trend, backup, box).
• waving-hands-rollup.sh [day]: one JSON line per day of counts, run nightly at 00:12 UTC into /var/lib/waving-hands/rollup.jsonl.
• waving-hands-backup.sh: nightly at 07:23 UTC, mirrors /var/lib/waving-hands to the kestrel-wizwar-backups Space under waving-hands/ (a current copy and dated snapshots kept 90 days). It needs rclone with the Spaces credentials in /root/.config/rclone/rclone.conf, copied by hand from the wizwar droplet; until then it logs "skipped".

Errors from the duel server go to Sentry, project waving-hands in the locallygrownnet organisation; the DSN is in the unit file. The nightly rollup and backup check in with Sentry Crons when /root/.waving-hands-sentry-cron-rollup and /root/.waving-hands-sentry-cron hold their check-in URLs (the ingest URL with the project's cron path and public key), so a missed night is noticed.

To have every new issue, regression and reappearance posted to Slack the way wizwar's are, run deploy/sentry-slack-alert.sh [#channel] once from your own shell with SENTRY_TOKEN (an org auth token with alerts:write) and SLACK_CHANNEL_ID set; the token never leaves the shell.

Before every deploy, deploy/verify-ledgers.sh <ip> fetches every production ledger and replays it with the local engine, comparing each room with what the server shows. A ledger the new engine refuses or replays differently stops the deploy: the server would rewrite that duel on restart.

Everyday deploys

deploy/deploy.sh <ip>

Runs the type-checks, the tests and the build locally, rsyncs build/ to the droplet keeping the previous week's hashed assets, rsyncs the server and engine sources, installs dependencies, and restarts the duel server. A single-player game is in the player's own browser and loses nothing. A room is replayed from its ledger when the server comes back, which takes a few seconds; Caddy holds requests that land in the gap.

Operations

• Logs: ssh root@<ip> journalctl -u waving-hands -f for the duel server, journalctl -u caddy -f and /var/lib/caddy/access.log for the web side.
• Restart: ssh root@<ip> systemctl restart waving-hands
• Who has been playing: ssh root@<ip> waving-hands-visitors.sh [YYYY-MM-DD] prints today's visitors from Caddy's log (people and bots apart, by page), every room opened today with how far it got, and the names seated, marking those seen for the first time.
• Rooms: /var/lib/waving-hands/rooms/<CODE>.jsonl, one ledger per duel. Copy that directory to back them up; single-player games are in players' browsers.