Operations in wizwar's shape: the determinism gate, backups, the rollup and the pulse

Every deploy first replays every production ledger with the engine about to ship. The droplet gains a nightly rollup of counts and a nightly backup to Spaces, a pulse script the pulse skill reads, rate limits on opening rooms and taking seats, and eviction of idle rooms from memory with reload from their ledgers on the next visit. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0141G6xqLeNRYEtviLWSB5Up
2026-09-22 22:06:10 -04:00
parent 885dcf56e6
commit 9bd203ae60
13 changed files with 418 additions and 5 deletions
@@ -0,0 +1,28 @@
+---
+name: waving-hands-pulse
+description: The weekly Waving Hands operations pulse — duel server health, errors, the last week of traffic and rooms from the rollup, backup status, and the box's vitals. Use when Eric asks how the server is doing, wants an ops check, or says "run the pulse".
+---
+
+# Waving Hands pulse
+
+One command on the droplet prints everything; read it as a short report.
+
+    ssh root@159.203.98.71 waving-hands-pulse.sh
+
+- **duel server**: whether the service and Caddy are up, restarts and
+  journal errors in the last seven days with the last three error lines.
+- **rollup, last 7 days**: one line a day from /var/lib/waving-hands/rollup.jsonl,
+  written nightly at 00:12 UTC by waving-hands-rollup.sh: people (addresses
+  with a browser user agent), page requests, rooms opened, started and with
+  turns, turns played, bot addresses, and referrers. A missing day means the
+  rollup did not run; run it by hand with the date.
+- **ledgers**: rooms on disk, touched in the last day, and their size.
+- **backup**: the last start and done lines of the nightly backup to Spaces
+  (07:23 UTC), or "never run". "skipped" means rclone is not configured.
+- **box**: disk, memory, load, and the access log's size and rotation.
+
+Lead with anything wrong (service down, errors, backup skipped, disk over
+80%). Then the trend: are people coming, and are duels being played? For
+who exactly, use the waving-hands-visitors skill. Before an engine change
+ships, deploy/verify-ledgers.sh replays every production ledger locally;
+deploy.sh runs it, and a "DIFFERS" or "REFUSED" line stops the deploy.