Begin Hnefatafl from the game kit

This commit is contained in:
Eric Wagoner
2026-09-23 12:45:44 -04:00
commit ed1dcad259
59 changed files with 8048 additions and 0 deletions
+42
View File
@@ -0,0 +1,42 @@
__HOST__
root * /opt/hnefatafl/build
encode gzip zstd
header {
Strict-Transport-Security "max-age=31536000"
X-Content-Type-Options "nosniff"
X-Frame-Options "DENY"
Referrer-Policy "no-referrer"
}
log {
output file /var/lib/caddy/access.log {
roll_size 10MiB
roll_keep 30
}
}
# The game server answers the API and the websocket; a deploy restarts it for
# a few seconds, and the proxy holds requests that land in that gap.
@game path /api/* /ws
handle @game {
reverse_proxy localhost:8789 {
lb_try_duration 30s
lb_try_interval 250ms
}
}
# Everything else is the static site. SvelteKit writes each route as
# <route>.html, so /rules is tried as /rules.html before falling back to the
# app shell, which serves the rooms. Hashed assets under _app/immutable are
# cached for a year; every other response is revalidated so a deploy shows
# up on the next load. The two header matchers are disjoint.
handle {
@immutable path /_app/immutable/*
header @immutable Cache-Control "public, max-age=31536000, immutable"
@mutable not path /_app/immutable/*
header @mutable Cache-Control "no-cache"
try_files {path} {path}.html /index.html
file_server
}
+102
View File
@@ -0,0 +1,102 @@
# Deploying Hnefatafl
The single-player game runs entirely in the browser. Duels between people
go through a small Node process that keeps each room as an append-only
ledger of moves in /var/lib/hnefatafl/rooms. Production is one
DigitalOcean droplet: Caddy terminates TLS with automatic certificates,
serves the static build from /opt/hnefatafl/build, and proxies /api and
/ws to the game server on port 8789, which runs as the `hnefatafl` user
under systemd from /opt/hnefatafl/app.
## Sizing
The smallest droplet, `s-1vcpu-512mb-10gb` ($4 a month), carries both Caddy
and the game server comfortably: the server is one small Node process capped
at 300 MB by its unit file, and a room is a few kilobytes of ledger.
The zero-cost alternative is a second site block in an existing Caddy
server's configuration pointing at a second directory; the deploy script
works unchanged against that host. A droplet of its own keeps this site's
uptime and upgrades independent of anything else.
## Current production
- Droplet: `hnefatafl` (nyc3, s-1vcpu-512mb-10gb, tag `hnefatafl`), IP __IP__
- URLs: https://hnefatafl.kestrelsnest.social (A record at Hover, where kestrelsnest.social's
DNS lives) and https://hnefatafl.__IP__.sslip.io (always works, zero DNS).
- Everyday deploy: `deploy/deploy.sh __IP__`
- Players' reports: `deploy/pull-reports.sh` mirrors /var/lib/hnefatafl/feedback.jsonl
and the screenshots to ~/Desktop/hnefatafl-reports with a digest;
`deploy/report-reply.sh __IP__ <id> <status> "text"` answers one.
## New droplet from scratch
1. `doctl compute droplet create hnefatafl --region nyc3 \
--size s-1vcpu-512mb-10gb --image ubuntu-24-04-x64 \
--ssh-keys <your-key-ids> --tag-name hnefatafl --wait`
2. `scp deploy/setup-droplet.sh deploy/Caddyfile.tmpl root@<ip>:/root/ && ssh root@<ip> \
"bash /root/setup-droplet.sh 'hnefatafl.kestrelsnest.social, hnefatafl.<ip>.sslip.io'"`
(point the A record at the new IP first, or leave the real name out until it is).
3. `scp deploy/setup-server.sh deploy/Caddyfile.tmpl root@<ip>:/root/ && ssh root@<ip> "bash /root/setup-server.sh"`
4. `deploy/deploy.sh <ip>`
The sslip.io hostname works with no DNS at all. To add a real name, point an
A record at the droplet and add the name to the first line of
/etc/caddy/Caddyfile (space separated), then `systemctl reload caddy`; Caddy
fetches the certificate on first request.
## Operations scripts on the droplet
`deploy.sh` installs these to /usr/local/bin and the cron file to
/etc/cron.d/hnefatafl on every deploy, so the live copies are the repo copies:
- `hnefatafl-visitors.sh [day]`: who is here now and who came that day.
- `hnefatafl-pulse.sh`: the weekly health check (service, errors, rollup
trend, backup, box).
- `hnefatafl-rollup.sh [day]`: one JSON line per day of counts, run
nightly at 00:12 UTC into /var/lib/hnefatafl/rollup.jsonl.
- `hnefatafl-backup.sh`: nightly at 07:23 UTC, mirrors /var/lib/hnefatafl
to the `kestrel-wizwar-backups` Space under `hnefatafl/` (a current copy
and dated snapshots kept 90 days). It needs rclone with the Spaces
credentials in /root/.config/rclone/rclone.conf, copied by hand from the
wizwar droplet; until then it logs "skipped".
Errors from the game server go to Sentry, project `hnefatafl` in the
locallygrownnet organisation; the DSN is in the unit file. The nightly rollup
and backup check in with Sentry Crons when /root/.hnefatafl-sentry-cron-rollup
and /root/.hnefatafl-sentry-cron hold their check-in URLs (the ingest
URL with the project's cron path and public key), so a missed night is noticed.
To have every new issue, regression and reappearance posted to Slack the way
wizwar's are, run `deploy/sentry-slack-alert.sh [#channel]` once from your own
shell with `SENTRY_TOKEN` (an org auth token with alerts:write) and
`SLACK_CHANNEL_ID` set; the token never leaves the shell.
Before every deploy, `deploy/verify-ledgers.sh <ip>` fetches every production
ledger and replays it with the local engine, comparing each room with what the
server shows. A ledger the new engine refuses or replays differently stops the
deploy: the server would rewrite that game on restart.
## Everyday deploys
deploy/deploy.sh <ip>
Runs the type-checks, the tests and the build locally, rsyncs `build/` to
the droplet keeping the previous week's hashed assets, rsyncs the server and
engine sources, installs dependencies, and restarts the game server. A
single-player game is in the player's own browser and loses nothing. A room
is replayed from its ledger when the server comes back, which takes a few
seconds; Caddy holds requests that land in the gap.
## Operations
- Logs: `ssh root@<ip> journalctl -u hnefatafl -f` for the game server,
`journalctl -u caddy -f` and /var/lib/caddy/access.log for the web side.
- Restart: `ssh root@<ip> systemctl restart hnefatafl`
- Who has been playing: `ssh root@<ip> hnefatafl-visitors.sh [YYYY-MM-DD]`
prints today's visitors from Caddy's log (people and bots apart, by page),
every room opened today with how far it got, and the names seated, marking
those seen for the first time.
- Rooms: `/var/lib/hnefatafl/rooms/<CODE>.jsonl`, one ledger per game.
Copy that directory to back them up; single-player games are in players'
browsers.
+54
View File
@@ -0,0 +1,54 @@
#!/usr/bin/env bash
# Build locally, push the static site and the game server to the droplet,
# and restart the server. Usage: deploy/deploy.sh <droplet-ip-or-host>
set -euo pipefail
HOST="${1:?usage: deploy.sh <droplet-ip-or-host>}"
# The family list is the kit's; a game deploys with the kit's latest copy when the kit is beside it.
KIT_FAMILY="$(dirname "$0")/../../game-kit/family.json"
[ -f "$KIT_FAMILY" ] && cp "$KIT_FAMILY" static/family.json
npm run check
npm run check:server
npm test
npm run build
# Every production ledger must replay identically with the engine about to ship.
deploy/verify-ledgers.sh "$HOST"
# Old hashed chunks stay a week: a page loaded before this deploy can still
# fetch the module it was built against instead of failing mid-game.
rsync -az --delete --filter='P _app/immutable/*' \
build/ "root@$HOST:/opt/hnefatafl/build/"
# The server runs from source with tsx and carries its own small manifest;
# it needs only its code and the engine. Ledgers live outside this tree.
rsync -az --delete --exclude='/server/node_modules' \
--include='/server/***' --include='/src/' --include='/src/lib/' --include='/src/lib/game/***' --include='/src/lib/net/' --include='/src/lib/net/view.ts' \
--include='/deploy/' --include='/deploy/hnefatafl.service' --include='/deploy/hnefatafl.cron' --include='/deploy/*.sh' --include='/deploy/hash-tokens.ts' \
--exclude='*' \
./ "root@$HOST:/opt/hnefatafl/app/"
ssh "root@$HOST" '
find /opt/hnefatafl/build/_app/immutable -type f -mtime +7 -delete
chown -R root:caddy /opt/hnefatafl/build
chmod -R g+rX /opt/hnefatafl/build
cd /opt/hnefatafl/app/server && npm install --no-audit --no-fund
chown -R hnefatafl:hnefatafl /opt/hnefatafl/app
cp /opt/hnefatafl/app/deploy/hnefatafl.service /etc/systemd/system/hnefatafl.service
# Cron runs the rollup and the backup from /usr/local/bin: install them on
# every deploy so the live copies are always the repo copies.
install -m 755 /opt/hnefatafl/app/deploy/visitors.sh /usr/local/bin/hnefatafl-visitors.sh
install -m 755 /opt/hnefatafl/app/deploy/pulse.sh /usr/local/bin/hnefatafl-pulse.sh
install -m 755 /opt/hnefatafl/app/deploy/hnefatafl-rollup.sh /usr/local/bin/hnefatafl-rollup.sh
install -m 755 /opt/hnefatafl/app/deploy/hnefatafl-backup.sh /usr/local/bin/hnefatafl-backup.sh
install -m 644 /opt/hnefatafl/app/deploy/hnefatafl.cron /etc/cron.d/hnefatafl
mkdir -p /var/log/hnefatafl
systemctl daemon-reload
systemctl enable --now hnefatafl
systemctl restart hnefatafl
systemctl reload caddy
sleep 1
systemctl --no-pager -l status hnefatafl | head -3
'
echo "deployed."
+44
View File
@@ -0,0 +1,44 @@
// One-time migration: replace each seat line's raw token with its SHA-256,
// so no ledger on disk or in a backup holds a live seat key. Idempotent; a
// line already hashed is left alone. Run ON the droplet from the server
// directory (tsx is installed there):
// cd /opt/hnefatafl/app/server && npx tsx ../deploy/hash-tokens.ts /var/lib/hnefatafl/rooms
// Browsers keep their raw tokens; the server hashes what they send and
// compares, so nobody loses a seat.
import { createHash } from 'node:crypto';
import { readdirSync, readFileSync, renameSync, statSync, writeFileSync, chownSync } from 'node:fs';
import { join } from 'node:path';
const dir = process.argv[2];
if (!dir) {
console.error('usage: hash-tokens.ts <ledger-dir>');
process.exit(2);
}
let files = 0;
let lines = 0;
for (const file of readdirSync(dir).filter((f) => f.endsWith('.jsonl'))) {
const path = join(dir, file);
const raw = readFileSync(path, 'utf8');
let changed = 0;
const out = raw
.split('\n')
.map((line) => {
if (!line.trim()) return line;
const entry = JSON.parse(line) as Record<string, unknown>;
if (entry.t !== 'seat' || typeof entry.token !== 'string') return line;
const { token, ...rest } = entry;
changed += 1;
return JSON.stringify({ ...rest, tokenHash: token ? createHash('sha256').update(token).digest('hex') : '' });
})
.join('\n');
if (!changed) continue;
const { uid, gid } = statSync(path);
writeFileSync(path + '.tmp', out);
chownSync(path + '.tmp', uid, gid);
renameSync(path + '.tmp', path);
files += 1;
lines += changed;
}
console.log(`${files} ledger${files === 1 ? '' : 's'} rewritten, ${lines} seat line${lines === 1 ? '' : 's'} hashed`);
+47
View File
@@ -0,0 +1,47 @@
#!/bin/bash
# Nightly ledger backup to DigitalOcean Spaces, one prefix per game in the shared bucket.
# current/ - exact mirror of /var/lib/hnefatafl
# snapshots/ - one dated copy per day, pruned after 90 days
# Ledgers are append-only JSONL; the game server never needs stopping.
# Sentry Crons check-in: /root/.hnefatafl-sentry-cron holds the check-in
# URL (absent file = no check-ins). SCHEDULE must match /etc/cron.d/hnefatafl.
set -u
SCHEDULE="23 7 * * *"
BUCKET="kestrel-wizwar-backups"
PREFIX="hnefatafl"
SRC="/var/lib/hnefatafl"
LOG="/var/log/hnefatafl/backup.log"
STAMP=$(date +%Y-%m-%d)
mkdir -p "$(dirname "$LOG")"
CRON_URL=$(cat /root/.hnefatafl-sentry-cron 2>/dev/null || true)
checkin() {
[ -n "$CRON_URL" ] && curl -sf -o /dev/null -X POST -H "Content-Type: application/json" \
-d "{\"status\":\"$1\",\"monitor_config\":{\"schedule\":{\"type\":\"crontab\",\"value\":\"$SCHEDULE\"},\"checkin_margin\":30,\"max_runtime\":10,\"timezone\":\"UTC\"}}" \
"$CRON_URL" || true
}
if [ ! -s /root/.config/rclone/rclone.conf ]; then
echo "$(date -Is) skipped: rclone is not configured" >> "$LOG"
exit 0
fi
{
echo "=== $(date -Is) backup start"
checkin in_progress
if rclone sync "$SRC" "spaces:$BUCKET/$PREFIX/current" --exclude 'rollup.jsonl' 2>&1 &&
rclone copy "$SRC" "spaces:$BUCKET/$PREFIX/snapshots/$STAMP" 2>&1; then
checkin ok
else
checkin error
fi
CUTOFF=$(date -d "90 days ago" +%Y-%m-%d)
rclone lsf "spaces:$BUCKET/$PREFIX/snapshots/" --dirs-only 2>/dev/null | \
while read -r d; do
day="${d%/}"
if [[ "$day" < "$CUTOFF" ]]; then
echo "pruning snapshot $day"
rclone purge "spaces:$BUCKET/$PREFIX/snapshots/$day" 2>&1
fi
done
echo "=== $(date -Is) backup done"
} >> "$LOG" 2>&1
+110
View File
@@ -0,0 +1,110 @@
#!/bin/bash
# Nightly rollup: one JSON line per day in /var/lib/hnefatafl/rollup.jsonl:
# the day's traffic from Caddy's access log (people and bots apart, by page,
# with referrers and the campaign tags links carry), the rooms opened,
# started and finished, and the box's vitals. Counts only: no addresses are
# kept. The pulse reads it for trends the self-rotating access log loses.
# hnefatafl-rollup.sh [YYYY-MM-DD] (default: yesterday, UTC)
# Cron: /etc/cron.d/hnefatafl, installed by deploy.sh; SCHEDULE below
# must match it, because the Sentry monitor is told the same shape.
# Sentry Crons check-in: /root/.hnefatafl-sentry-cron-rollup holds the
# check-in URL (absent = no check-ins).
set -u
OUT="/var/lib/hnefatafl/rollup.jsonl"
LOG="/var/log/hnefatafl/rollup.log"
DAY="${1:-$(date -u -d 'yesterday' +%Y-%m-%d)}"
SCHEDULE="12 0 * * *"
CRON_URL=$(cat /root/.hnefatafl-sentry-cron-rollup 2>/dev/null || true)
# The check-in carries the schedule: Sentry keeps a monitor only once told its shape.
checkin() {
[ -n "$CRON_URL" ] && curl -sf -o /dev/null -X POST -H "Content-Type: application/json" \
-d "{\"status\":\"$1\",\"monitor_config\":{\"schedule\":{\"type\":\"crontab\",\"value\":\"$SCHEDULE\"},\"checkin_margin\":30,\"max_runtime\":10,\"timezone\":\"UTC\"}}" \
"$CRON_URL" || true
}
mkdir -p "$(dirname "$LOG")"
checkin in_progress
if python3 - "$DAY" "$OUT" <<'PY' >> "$LOG" 2>&1
import collections, datetime, glob, gzip, json, os, re, shutil, sys
day, out = sys.argv[1], sys.argv[2]
d0 = datetime.datetime.fromisoformat(day).replace(tzinfo=datetime.timezone.utc)
t0, t1 = d0.timestamp(), (d0 + datetime.timedelta(days=1)).timestamp()
BOT = re.compile(r"bot|crawl|spider|slurp|facebookexternalhit|slack|discord|twitter|preview|fetch|curl|python|go-http|wget|headless|scan|monitor|uptime", re.I)
req = human = bot = 0
hips, bips = set(), set()
pages, joins, refs, campaigns = collections.Counter(), collections.Counter(), collections.Counter(), collections.Counter()
seen_campaign = set()
for f in sorted(glob.glob("/var/lib/caddy/access*.log*")):
if os.path.getmtime(f) < t0: continue
opener = gzip.open if f.endswith(".gz") else open
with opener(f, "rt", errors="ignore") as fh:
for line in fh:
try: r = json.loads(line)
except ValueError: continue
ts = r.get("ts", 0)
if not (t0 <= ts < t1): continue
q = r.get("request", {}); h = q.get("headers", {})
ua = " ".join(h.get("User-Agent", [""]))
ip = q.get("client_ip") or q.get("remote_ip") or "?"
uri = q.get("uri", ""); path = uri.split("?")[0]
req += 1
if BOT.search(ua) or not ua:
bot += 1; bips.add(ip); continue
if path.startswith(("/_app/", "/api/")) or path == "/ws" or path.endswith((".png", ".ico", ".txt")): continue
human += 1; hips.add(ip)
if path == "/": pages["hall"] += 1
elif path == "/rules": pages["rules"] += 1
elif path == "/guide": pages["guide"] += 1
elif path.startswith("/join/"): joins[path.split("/")[2].upper()] += 1
ref = " ".join(h.get("Referer", [""]))
if ref and "hnefatafl.kestrelsnest.social" not in ref:
refs[re.sub(r"^https?://", "", ref).split("/")[0]] += 1
m = re.search(r"[?&](ref|utm_source|fbclid)=([^&]*)", uri)
if m and (ip, m.group(1)) not in seen_campaign:
seen_campaign.add((ip, m.group(1)))
campaigns[m.group(1) + ("=" + m.group(2)[:24] if m.group(1) != "fbclid" else "")] += 1
created = started = finished = turned = turns = 0
humans, bots = set(), 0
for f in glob.glob("/var/lib/hnefatafl/rooms/*.jsonl"):
try: lines = [json.loads(l) for l in open(f) if l.strip()]
except Exception: continue
if not lines or lines[0].get("t") != "room": continue
made = lines[0]["createdAt"] / 1000
if t0 <= made < t1:
created += 1
if any(x["t"] == "start" for x in lines): started += 1
if any(x["t"] == "turn" for x in lines): turned += 1
for x in lines:
if x["t"] == "seat":
if x["bot"]: bots += 1
else: humans.add(x["name"])
elif x["t"] == "unseat": bots -= 1
day_turns = [x for x in lines if x["t"] == "turn" and t0 <= x["at"] / 1000 < t1]
turns += len(day_turns)
# A game finished today: its last turn is today's and the engine would say so; approximate by replay-free means:
# the ledger's final turn falls today and no turn follows it within the day's end.
if day_turns and day_turns[-1] is lines[-1] and False: finished += 1
disk = shutil.disk_usage("/")
mem = {}
for line in open("/proc/meminfo"):
k, v = line.split(":", 1); mem[k] = int(v.split()[0])
load = os.getloadavg()[1]
row = dict(day=day, requests=req, human=human, bot=bot, humanAddresses=len(hips), botAddresses=len(bips),
paths=dict(pages), roomLinks=sum(joins.values()), referrers=dict(refs), campaigns=dict(campaigns),
rooms=dict(created=created, started=started, withTurns=turned, humanNames=len(humans), botSeats=bots),
turns=turns, ledgers=len(glob.glob("/var/lib/hnefatafl/rooms/*.jsonl")),
vitals=dict(diskUsedPct=round(disk.used * 100 / disk.total), memAvailableMb=mem.get("MemAvailable", 0) // 1024, load5=round(load, 2)),
at=datetime.datetime.now(datetime.timezone.utc).isoformat(timespec="seconds"))
rows = []
if os.path.exists(out):
rows = [json.loads(l) for l in open(out) if l.strip()]
rows = [r for r in rows if r.get("day") != day] + [row]
rows.sort(key=lambda r: r["day"])
with open(out + ".tmp", "w") as fh:
for r in rows: fh.write(json.dumps(r) + "\n")
os.replace(out + ".tmp", out)
print("%s rolled: %d people, %d requests, %d rooms" % (day, len(hips), human, created))
PY
then checkin ok; else checkin error; fi
+3
View File
@@ -0,0 +1,3 @@
# Hnefatafl: nightly rollup of yesterday's counts, then the ledger backup. UTC.
12 0 * * * root /usr/local/bin/hnefatafl-rollup.sh
23 7 * * * root /usr/local/bin/hnefatafl-backup.sh
+36
View File
@@ -0,0 +1,36 @@
[Unit]
Description=Hnefatafl game server
After=network.target
[Service]
Type=simple
User=hnefatafl
WorkingDirectory=/opt/hnefatafl/app/server
Environment=PORT=8789
Environment=KEEPER=Kestrel
Environment=KEEPER_TZ=America/New_York
# Caddy terminates TLS; the plaintext port must not face the internet.
Environment=HOST=127.0.0.1
Environment=DATA_DIR=/var/lib/hnefatafl/rooms
# Create the Sentry project (the Sentry MCP can) and paste its DSN here; empty means no error reporting.
Environment=SENTRY_DSN=__SENTRY_DSN__
ExecStart=/opt/hnefatafl/app/server/node_modules/.bin/tsx src/index.ts
Restart=always
RestartSec=3
# Sandbox: the process reads /opt/hnefatafl and writes only its data dir.
NoNewPrivileges=yes
PrivateTmp=yes
ProtectSystem=strict
ProtectHome=yes
ReadWritePaths=/var/lib/hnefatafl
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectControlGroups=yes
RestrictSUIDSGID=yes
# A runaway process gets killed and restarted before it can take the box down.
MemoryMax=300M
LimitNOFILE=4096
[Install]
WantedBy=multi-user.target
+41
View File
@@ -0,0 +1,41 @@
#!/bin/bash
# Mirror the player reports, the keeper's replies and the players' screenshots
# to a local folder, and write a digest beside them for reading.
# deploy/pull-reports.sh [host] [folder] (default ~/Desktop/hnefatafl-reports)
set -euo pipefail
HOST="${1:-__IP__}"
OUT="${2:-$HOME/Desktop/hnefatafl-reports}"
mkdir -p "$OUT/images"
scp -q "root@$HOST:/var/lib/hnefatafl/feedback.jsonl" "$OUT/feedback.jsonl" 2>/dev/null || : > "$OUT/feedback.jsonl"
rsync -aq "root@$HOST:/var/lib/hnefatafl/feedback-images/" "$OUT/images/" 2>/dev/null || true
python3 - "$OUT" <<'PY'
import json, sys, os
out = sys.argv[1]
reports, order = {}, []
for raw in open(os.path.join(out, "feedback.jsonl"), encoding="utf8"):
raw = raw.strip()
if not raw: continue
line = json.loads(raw)
if "reportId" in line:
r = reports.get(line["reportId"])
if not r: continue
if "image" in line: r["image"] = line["image"]
else: r["replies"].append(line) # the keeper's replies and the player's answers alike, in order
continue
reports[line["id"]] = dict(line, replies=[]); order.append(line["id"])
lines = ["# Hnefatafl reports", "", f"{len(order)} reports; newest first. Pictures are in `images/`.", ""]
for rid in reversed(order):
r = reports[rid]
lines.append(f"## {r.get('at','')[:16].replace('T',' ')} — {r.get('player','?')} in {r.get('roomId','?')} (turn {r.get('turn','?')}, seq {r.get('seq','?')}) — id {rid}")
lines.append("")
lines.append(f"**What happened:** {r.get('happened','').strip()}")
if r.get("expected","").strip(): lines.append(f"**What they expected:** {r['expected'].strip()}")
if r.get("image"): lines.append(f"**Picture:** ![{r['image']}](images/{r['image']})")
for rep in r["replies"]:
who = f"{r.get('player','the player')} answers" if rep.get("from") == "player" else f"**{rep.get('status','')}**"
lines.append(f"> {who} ({rep.get('at','')[:16].replace('T',' ')}): {rep.get('text','').strip()}")
if not r["replies"] or r["replies"][-1].get("from") == "player": lines.append("> _awaiting the keeper_")
lines.append("")
open(os.path.join(out, "reports.md"), "w", encoding="utf8").write("\n".join(lines))
print(f"{len(order)} reports, {sum(1 for r in reports.values() if r.get('image'))} with pictures -> {out}/reports.md")
PY
+36
View File
@@ -0,0 +1,36 @@
#!/bin/bash
# The operations pulse, printed ON the droplet: the game server's health,
# errors in its journal, the last week of the rollup, the backup's last word,
# and the box's vitals. Read by the hnefatafl-pulse skill.
set -u
echo "as of $(date -u '+%F %H:%M') UTC on $(hostname); up $(uptime -p | sed 's/up //')"
echo "--- game server"
systemctl is-active hnefatafl >/dev/null && echo "hnefatafl: active since $(systemctl show hnefatafl -p ActiveEnterTimestamp --value)" || echo "hnefatafl: NOT ACTIVE"
systemctl is-active caddy >/dev/null && echo "caddy: active" || echo "caddy: NOT ACTIVE"
echo "restarts in 7 days: $(journalctl -u hnefatafl --since '7 days ago' -o cat | grep -c 'Started hnefatafl')"
ERR=$(journalctl -u hnefatafl --since '7 days ago' -p err -o cat | grep -vc '^$')
echo "journal errors in 7 days: $ERR"
journalctl -u hnefatafl --since '7 days ago' -o cat | grep -iE "error|unhandled|exception" | grep -v "Rate" | tail -3 | sed 's/^/ /'
echo "--- rollup, last 7 days (people / requests / rooms opened / turns)"
if [ -s /var/lib/hnefatafl/rollup.jsonl ]; then
tail -7 /var/lib/hnefatafl/rollup.jsonl | python3 -c '
import json, sys
for l in sys.stdin:
r = json.loads(l); rm = r["rooms"]
print(" %s %3d people %4d requests %2d rooms (%d started, %d with turns) %3d turns bots %d addr refs %s" % (
r["day"], r["humanAddresses"], r["human"], rm["created"], rm["started"], rm["withTurns"], r["turns"], r["botAddresses"],
",".join(sorted(r["referrers"])) or "-"))'
else
echo " (no rollup yet)"
fi
echo "--- ledgers"
echo "rooms on disk: $(ls /var/lib/hnefatafl/rooms/*.jsonl 2>/dev/null | wc -l); touched in 24h: $(find /var/lib/hnefatafl/rooms -name '*.jsonl' -mmin -1440 2>/dev/null | wc -l); size $(du -sh /var/lib/hnefatafl/rooms 2>/dev/null | cut -f1)"
echo "--- backup"
if [ -f /var/log/hnefatafl/backup.log ]; then
grep -E "backup (start|done)|skipped|error|ERROR" /var/log/hnefatafl/backup.log | tail -2 | sed 's/^/ /'
else
echo " never run"
fi
echo "--- box"
echo "disk: $(df -h / | awk 'NR==2{print $5" used of "$2}'); memory: $(free -m | awk 'NR==2{print $7" MB available of "$2}'); load: $(cut -d' ' -f1-3 /proc/loadavg)"
echo "caddy access log: $(du -sh /var/lib/caddy/access.log 2>/dev/null | cut -f1) live, $(ls /var/lib/caddy/access*.log* 2>/dev/null | wc -l) files"
+74
View File
@@ -0,0 +1,74 @@
// Replays every ledger in a directory with THIS checkout's engine and, when
// a server address is given, compares the result with what that server is
// showing for the room: the round reached and the outcome. A ledger the
// engine cannot replay, or replays differently, is reported and fails the
// run. Used by deploy/verify-ledgers.sh.
// tsx deploy/replay-ledgers.ts <ledger-dir> [https://host]
import { readdirSync, readFileSync } from 'node:fs';
import { join } from 'node:path';
import { game } from '../src/lib/game';
import type { SeatId } from '../src/lib/game/spec';
interface Seat {
id: SeatId;
name: string;
bot: boolean;
}
const [dir, host] = process.argv.slice(2);
if (!dir) {
console.error('usage: replay-ledgers.ts <ledger-dir> [https://host]');
process.exit(2);
}
async function main(): Promise<void> {
const files = readdirSync(dir).filter((f) => f.endsWith('.jsonl'));
let failures = 0;
for (const file of files) {
const code = file.slice(0, -'.jsonl'.length);
const lines = readFileSync(join(dir, file), 'utf8')
.split('\n')
.filter((l) => l.trim())
.map((l) => JSON.parse(l));
const seats: Seat[] = [];
let state: ReturnType<typeof game.create> | null = null;
let turns = 0;
try {
for (const line of lines) {
if (line.t === 'seat') seats.push(line);
else if (line.t === 'unseat') seats.splice(seats.findIndex((s) => s.id === line.id), 1);
else if (line.t === 'start') state = game.create(Object.fromEntries(seats.map((s) => [s.id, s.name])), line.seed, line.rules ?? 1);
else if (line.t === 'turn' && state) {
state = game.resolve(state, line.inputs);
turns += 1;
}
}
} catch (e) {
console.log(`${code}: REFUSED at turn ${turns + 1}: ${e instanceof Error ? e.message : String(e)}`);
failures += 1;
continue;
}
const outcome = state ? game.over(state) : null;
let verdict = state ? `${turns} turns, ${outcome ? 'over' : 'in play'}` : 'not started';
if (host && state) {
// The gallery's view carries the round and the outcome, which is all the comparison needs.
const res = await fetch(`${host}/api/rooms/${code}`);
if (!res.ok) {
verdict += `, server ${res.status}`;
} else {
const theirs = (await res.json()) as { turn: number | null; over: { winner: SeatId | null } | null };
const same = theirs.turn === game.turn(state) && (theirs.over?.winner ?? null) === (outcome?.winner ?? null);
if (!same) {
failures += 1;
verdict += `, DIFFERS from the server (server round ${theirs.turn}, local round ${game.turn(state)})`;
} else verdict += ', matches the server';
}
}
console.log(`${code}: ${verdict}`);
}
console.log(`${files.length} ledger${files.length === 1 ? '' : 's'}, ${failures} problem${failures === 1 ? '' : 's'}`);
process.exit(failures ? 1 : 0);
}
void main();
+15
View File
@@ -0,0 +1,15 @@
#!/bin/bash
# Answer a player's report; the reply appears under it in their hall.
# deploy/report-reply.sh <host> <reportId> <resolved|by-design|open> <text...>
set -euo pipefail
HOST="${1:?usage: report-reply.sh <host> <reportId> <status> <text...>}"
REPORT_ID="${2:?usage: report-reply.sh <host> <reportId> <status> <text...>}"
STATUS="${3:?usage: report-reply.sh <host> <reportId> <status> <text...>}"
shift 3
TEXT="$*"
case "$STATUS" in resolved|by-design|open) ;; *) echo "status must be resolved, by-design, or open" >&2; exit 1;; esac
case "$REPORT_ID" in *[!0-9a-f]*|"") echo "a report id is eight hex characters" >&2; exit 1;; esac
LINE=$(REPORT_ID="$REPORT_ID" STATUS="$STATUS" TEXT="$TEXT" python3 -c '
import json, os, datetime
print(json.dumps({"reportId": os.environ["REPORT_ID"], "at": datetime.datetime.now(datetime.timezone.utc).isoformat().replace("+00:00", "Z"), "status": os.environ["STATUS"], "text": os.environ["TEXT"]}))')
printf '%s\n' "$LINE" | ssh "root@$HOST" 'cat >> /var/lib/hnefatafl/feedback.jsonl && chown hnefatafl:hnefatafl /var/lib/hnefatafl/feedback.jsonl && tail -1 /var/lib/hnefatafl/feedback.jsonl'
+75
View File
@@ -0,0 +1,75 @@
#!/bin/bash
# Route every Hnefatafl issue to a Slack channel: new issues (any level or
# priority), regressions, and reappearances. Creates one Sentry workflow
# via the alerts API. The token never leaves your shell:
# SENTRY_TOKEN=sntryu_... deploy/sentry-slack-alert.sh [#channel]
# The channel defaults to #hnefatafl-notifications (__SLACK_CHANNEL_ID__); pass
# SLACK_CHANNEL_ID with the channel name to route elsewhere.
# The token needs the alerts:write scope (an org auth token from
# https://locallygrownnet.sentry.io/settings/auth-tokens/), and Slack must
# already be connected to the org under Settings → Integrations.
set -euo pipefail
: "${SENTRY_TOKEN:?SENTRY_TOKEN=sntryu_... is required (alerts:write)}"
ORG="${SENTRY_ORG:-locallygrownnet}"
PROJECT="${SENTRY_PROJECT:-hnefatafl}"
CHANNEL="${1:-#hnefatafl-notifications}"
API="https://us.sentry.io/api/0/organizations/$ORG"
AUTH="Authorization: Bearer $SENTRY_TOKEN"
# The Slack action takes the channel's ID as well as its name; the
# project's error detector is looked up by project id; a workflow is bound
# to its detectors by PUT after creation.
SLACK_ID="${SENTRY_SLACK_INTEGRATION:-__SENTRY_SLACK_INTEGRATION__}"
CHANNEL_ID="${SLACK_CHANNEL_ID:-__SLACK_CHANNEL_ID__}"
PROJECT_ID="${SENTRY_PROJECT_ID:-__SENTRY_PROJECT_ID__}"
echo "Slack integration $SLACK_ID; project $PROJECT_ID; channel $CHANNEL ($CHANNEL_ID)"
DETECTOR_ID=$(curl -s "$API/detectors/?project=$PROJECT_ID" -H "$AUTH" | python3 -c '
import json, sys
rows = json.load(sys.stdin)
for d in (rows if isinstance(rows, list) else []):
if str(d.get("projectId")) == sys.argv[1] and d.get("type") == "error": print(d["id"]); break' "$PROJECT_ID")
[ -n "$DETECTOR_ID" ] || { echo "no error detector for project $PROJECT_ID" >&2; exit 1; }
PAYLOAD=$(CHANNEL="$CHANNEL" CHANNEL_ID="$CHANNEL_ID" SLACK_ID="$SLACK_ID" DETECTOR_ID="$DETECTOR_ID" python3 -c '
import json, os
print(json.dumps({
"name": "Hnefatafl → Slack",
"enabled": True,
"environment": None,
"config": {"frequency": 0},
"triggers": {
"logicType": "any-short",
"conditions": [
{"type": "first_seen_event", "comparison": True, "conditionResult": True},
{"type": "regression_event", "comparison": True, "conditionResult": True},
{"type": "reappeared_event", "comparison": True, "conditionResult": True},
],
"actions": [],
},
"actionFilters": [{
"logicType": "all",
"conditions": [],
"actions": [{
"type": "slack",
"integrationId": int(os.environ["SLACK_ID"]),
"data": {},
"config": {"targetType": "specific", "targetIdentifier": os.environ["CHANNEL_ID"], "targetDisplay": os.environ["CHANNEL"]},
"status": "active",
}],
}],
"detectorIds": [int(os.environ["DETECTOR_ID"])],
}))')
echo "creating the workflow…"
RESPONSE=$(curl -s -w "\n%{http_code}" -X POST "$API/workflows/" -H "$AUTH" -H "Content-Type: application/json" -d "$PAYLOAD")
CODE=$(printf '%s' "$RESPONSE" | tail -1)
BODY=$(printf '%s' "$RESPONSE" | sed '$d')
if [ "$CODE" != "201" ]; then
echo "Sentry answered $CODE:" >&2; echo "$BODY" >&2; exit 1
fi
WORKFLOW_ID=$(printf '%s' "$BODY" | python3 -c 'import json,sys; print(json.load(sys.stdin).get("id",""))')
echo "created workflow $WORKFLOW_ID"
# Binding to the detector is done by PUT, the whole workflow resent.
curl -s -o /dev/null -w "bound to detector $DETECTOR_ID: %{http_code}\n" -X PUT "$API/workflows/$WORKFLOW_ID/" -H "$AUTH" -H "Content-Type: application/json" -d "$PAYLOAD"
echo "done: https://$ORG.sentry.io/monitors/alerts/$WORKFLOW_ID/"
+39
View File
@@ -0,0 +1,39 @@
#!/usr/bin/env bash
# One-time droplet setup. Copy this script and Caddyfile.tmpl to the droplet
# and run ON the droplet as root:
# bash setup-droplet.sh 'hnefatafl.kestrelsnest.social, hnefatafl.<droplet-ip>.sslip.io'
# The argument is the Caddy site address line: one name, or several
# separated by commas. Every name must already resolve to this droplet.
# Hnefatafl is a static site: Caddy serves the built files and terminates
# TLS. There is no application process to install or supervise.
set -euo pipefail
HOST="${1:?usage: setup-droplet.sh <hostname>}"
apt-get update -q
apt-get install -qy curl rsync
# Caddy (auto-HTTPS)
apt-get install -qy debian-keyring debian-archive-keyring apt-transport-https
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' \
| gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' \
| tee /etc/apt/sources.list.d/caddy-stable.list
apt-get update -q && apt-get install -qy caddy
# Site directory, readable by Caddy.
mkdir -p /opt/hnefatafl/build
chown -R root:caddy /opt/hnefatafl
chmod -R g+rX /opt/hnefatafl
# Caddy: the site and the game server's paths, from the template beside this script.
sed "s|__HOST__|$HOST|" "$(dirname "$0")/Caddyfile.tmpl" > /etc/caddy/Caddyfile
caddy validate --config /etc/caddy/Caddyfile
systemctl reload caddy
# Firewall: ssh + web only.
ufw allow OpenSSH
ufw allow 80/tcp
ufw allow 443/tcp
ufw --force enable
echo "droplet ready: now run deploy/deploy.sh <ip> from your machine"
+25
View File
@@ -0,0 +1,25 @@
#!/usr/bin/env bash
# Adds the game server to a droplet that setup-droplet.sh already prepared.
# Copy this script and Caddyfile.tmpl to the droplet and run ON the droplet
# as root; safe to run again. Installs Node 22, creates the service user and
# data directory, and teaches Caddy to hand /api and /ws to the server while
# it keeps serving the static site itself.
set -euo pipefail
if ! command -v node >/dev/null || [[ "$(node -v)" != v22* ]]; then
curl -fsSL https://deb.nodesource.com/setup_22.x | bash -
apt-get install -qy nodejs
fi
id -u hnefatafl &>/dev/null || useradd -r -m -d /opt/hnefatafl-home hnefatafl
mkdir -p /opt/hnefatafl/app /var/lib/hnefatafl/rooms
chown -R hnefatafl:hnefatafl /var/lib/hnefatafl
# Caddy: rewrite the site's configuration from the template, keeping its
# address line, so the game server's paths are routed before the files.
HOST_LINE=$(head -1 /etc/caddy/Caddyfile)
sed "s|__HOST__|$HOST_LINE|" "$(dirname "$0")/Caddyfile.tmpl" > /etc/caddy/Caddyfile
caddy validate --config /etc/caddy/Caddyfile
systemctl reload caddy
echo "server prerequisites ready: now run deploy/deploy.sh <ip> from your machine"
+13
View File
@@ -0,0 +1,13 @@
#!/usr/bin/env bash
# The determinism gate: fetch every production ledger and replay it with the
# LOCAL engine, comparing each room with what the server shows. A ledger the
# new engine replays differently would rewrite a game in progress when the
# server restarts, so this runs before every deploy.
# deploy/verify-ledgers.sh <droplet-ip-or-host> [https://site]
set -euo pipefail
HOST="${1:?usage: verify-ledgers.sh <droplet-ip-or-host> [https://site]}"
SITE="${2:-https://hnefatafl.kestrelsnest.social}"
DIR="$(mktemp -d)"
trap 'rm -rf "$DIR"' EXIT
rsync -az "root@$HOST:/var/lib/hnefatafl/rooms/" "$DIR/"
npx tsx deploy/replay-ledgers.ts "$DIR" "$SITE"
+120
View File
@@ -0,0 +1,120 @@
#!/bin/bash
# The visitors digest: who is at the table right now and who came today,
# the rooms opened and how far each got, with today's traffic so far.
# Runs ON the droplet: visitors.sh [YYYY-MM-DD] (default: today, UTC)
# Every game, bot games included, is a room with a ledger, so all show in full.
set -u
DAY="${1:-$(date -u +%F)}"
python3 - "$DAY" <<'PY'
import datetime, glob, json, re, subprocess, sys
from collections import Counter
day = sys.argv[1]
now = datetime.datetime.now(datetime.timezone.utc)
print("as of %s UTC, day %s" % (now.strftime("%H:%M"), day))
def sh(cmd):
try:
return subprocess.run(cmd, shell=True, capture_output=True, text=True, timeout=20).stdout.strip()
except Exception:
return "?"
print("live sockets: %s | rooms touched in the last hour: %s" % (
sh("ss -tn state established '( sport = :8789 )' | tail -n +2 | wc -l"),
sh("find /var/lib/hnefatafl/rooms -name '*.jsonl' -mmin -60 2>/dev/null | wc -l")))
# --- traffic: Caddy's access log, people and bots apart ---
BOT = re.compile(r"bot|crawl|spider|slurp|facebookexternalhit|preview|fetch|curl|python|go-http|headless|scan|monitor|uptime", re.I)
PAGES = {"/": "hall", "/rules": "rules", "/guide": "guide"}
addresses, bots = set(), set()
pages, joins, referrers, campaigns, agents = Counter(), Counter(), Counter(), Counter(), Counter()
requests = 0
for path in sorted(glob.glob("/var/lib/caddy/access.log*")):
try:
with open(path) as f:
for line in f:
try:
x = json.loads(line)
except Exception:
continue
ts = datetime.datetime.fromtimestamp(x["ts"], datetime.timezone.utc)
if ts.strftime("%F") != day:
continue
req = x["request"]
ua = " ".join(req.get("headers", {}).get("User-Agent", [""]))
ip = req.get("remote_ip") or req.get("client_ip", "?")
uri = req.get("uri", "")
path_only = uri.split("?")[0]
if BOT.search(ua) or not ua:
bots.add(ip)
continue
if path_only.startswith("/_app/") or path_only.startswith("/api/") or path_only == "/ws" or path_only.endswith((".png", ".ico", ".txt")):
continue
requests += 1
addresses.add(ip)
if path_only in PAGES:
pages[PAGES[path_only]] += 1
elif path_only.startswith("/join/"):
joins[path_only.split("/")[2].upper()] += 1
ref = " ".join(req.get("headers", {}).get("Referer", [""]))
if ref and "hnefatafl.kestrelsnest.social" not in ref:
referrers[re.sub(r"^https?://", "", ref).split("/")[0]] += 1
m = re.search(r"[?&](ref|utm_source|fbclid)=([^&]*)", uri)
if m:
campaigns[m.group(1) + "=" + m.group(2)[:24]] += 1
agents[re.sub(r"\(.*?\)", "", ua)[:40].strip()] += 1
except OSError:
continue
print("traffic: %d addresses of people, %d page requests (%d bot addresses); hall %d, rules %d, guide %d, room links %s; referrers %s; campaigns %s" % (
len(addresses), requests, len(bots), pages["hall"], pages["rules"], pages["guide"],
dict(joins) or "none", dict(referrers) or "none", dict(campaigns) or "none"))
# --- rooms: every ledger, today's in detail ---
first_seen = {}
rooms = []
for f in glob.glob("/var/lib/hnefatafl/rooms/*.jsonl"):
try:
lines = [json.loads(l) for l in open(f) if l.strip()]
meta = lines[0]
assert meta["t"] == "room"
except Exception:
continue
created = datetime.datetime.fromtimestamp(meta["createdAt"] / 1000, datetime.timezone.utc)
humans, bots_seated, started, turns, last, over, talk = [], [], False, 0, created, None, 0
names = {}
for x in lines[1:]:
if x["t"] == "seat":
(bots_seated if x["bot"] else humans).append(x["name"])
names[x["id"]] = x["name"]
elif x["t"] == "unseat":
bots_seated.remove(names.pop(x["id"], None)) if x["id"] in names else None
elif x["t"] == "start":
started = True
elif x["t"] == "turn":
turns += 1
last = datetime.datetime.fromtimestamp(x["at"] / 1000, datetime.timezone.utc)
elif x["t"] == "chat":
talk += 1
elif x["t"] == "over":
over = "a draw" if x["winner"] is None else "%s won" % names.get(x["winner"], x["winner"])
for h in humans:
if h not in first_seen or created < first_seen[h]:
first_seen[h] = created
rooms.append(dict(id=meta["id"], size=meta["seats"], created=created, humans=humans, bots=bots_seated, started=started, turns=turns, last=last, over=over, talk=talk))
today = sorted([r for r in rooms if r["created"].strftime("%F") == day], key=lambda r: r["created"])
names = sorted({h for r in today for h in r["humans"]})
new = [n for n in names if first_seen[n].strftime("%F") == day]
print("players today: %s | NEW today: %s" % (", ".join(names) or "none", ", ".join(new) or "none"))
print("rooms today: %d (of %d ever)" % (len(today), len(rooms)))
for r in today:
idle_h = (now - r["last"]).total_seconds() / 3600
state = "lobby only" if not r["started"] else ("started, no turns" if not r["turns"] else (r["over"] or ("abandoned" if idle_h > 12 else "in play")))
span = ("%d min" % round((r["last"] - r["created"]).total_seconds() / 60)) if r["turns"] else ""
seated = "+".join(r["humans"]) + ((" vs %d bot%s" % (len(r["bots"]), "" if len(r["bots"]) == 1 else "s")) if r["bots"] else "")
empty = r["size"] - len(r["humans"]) - len(r["bots"])
print(" %s %s %s%s %s %d turns %s last turn %s%s" % (
r["created"].strftime("%H:%M"), r["id"], seated, (" (%d seat%s empty)" % (empty, "" if empty == 1 else "s")) if empty else "",
state, r["turns"], span, r["last"].strftime("%H:%M") if r["turns"] else "never",
(" %d line%s of talk" % (r["talk"], "" if r["talk"] == 1 else "s")) if r["talk"] else ""))
PY