diff --git a/assets/css/_extra.scss b/assets/css/_extra.scss index 5fc6091b..684a4f12 100644 --- a/assets/css/_extra.scss +++ b/assets/css/_extra.scss @@ -177,6 +177,29 @@ figure { } } +// Two (or more) figures side by side, wrapping to a stack on narrow screens. +// Use by wrapping images in a
via the rawhtml shortcode. +.figure-row { + display: flex; + flex-wrap: wrap; + justify-content: center; + align-items: flex-start; + gap: 1.5rem; + margin: 2rem 0; + + figure { + flex: 1 1 0; + min-width: 220px; + margin: 0; + } + + img { + max-height: 70vh; + width: auto; + max-width: 100%; + } +} + // Also style standalone images in posts .post-content img:not(figure img) { display: block; diff --git a/content/posts/2026-07-18-a-ghost-hand-and-an-audience-of-one/index.md b/content/posts/2026-07-18-a-ghost-hand-and-an-audience-of-one/index.md new file mode 100644 index 00000000..cf28fd3e --- /dev/null +++ b/content/posts/2026-07-18-a-ghost-hand-and-an-audience-of-one/index.md @@ -0,0 +1,111 @@ +--- +title: "A ghost hand and an audience of one" +date: 2026-07-18T09:00:00-04:00 +draft: false +description: "I taught my software test suite to record training videos, first for an app with exactly one user." +tags: + - projects + - ai + - video +--- + +{{< callout type="info" title="TL;DR" >}} +I taught my software test suite to record training videos. They star the real application, they narrate themselves with captions, and when the app changes, I rebuild them with one command instead of re-filming anything. The system was invented in an evening to teach one person her own custom software; the next night it made a forty-nine-video library for LocallyGrown.net. This is the story; a [companion post](/posts/2026-07-18-your-test-suite-is-a-video-studio/) has the technical details. +{{< /callout >}} + +Modern software developers write automated tests for their applications that use the app the way a person does. The tests open a browser, sign in, tap the buttons, fill in the forms, and check that the right things happened. Normally all of this runs invisibly, in what's called a headless browser, which is exactly what it sounds like: a browser with no window, doing its work where nobody can see. You just get a list of green checkmarks at the end. + +Earlier this week I gave the tests for one of my projects a visible mode, mostly out of curiosity, so I could watch them run instead of trusting the checkmarks. It was oddly satisfying. A ghost hand glided through the app, tapping and typing, doing in seconds what would take me a minute. And watching it, I sent a message to the AI coding assistant I work with that said, nearly verbatim, "I watched the tests run and they're nice! It made me wonder what it would take to generate a training video I could give to Carol." + +The answer turned out to be about an hour. + +--- + +## Built for Carol + +Carol is my partner, and Refreshing Spaces is her professional organizing and decluttering business. This week, I built her an app from scratch to run it, covering her clients and visits, invoices and payments, mileage, and the before-and-after photos that really show off all of her work. She works from her phone in clients' homes, so everything is designed for a phone screen. The app's entire user base is one person, and she is not a software developer. + +That makes for a strangely pure version of a problem every software company faces: how do you teach someone to use a thing? There is no manual for an app that exists nowhere else, and nobody has recorded a YouTube tutorial for software with one user. If the app was going to be learnable, I had to make it teach itself. And in my household, as in most, "watch this forty-second video" beats "read this document I wrote." + +--- + +## What a generated video looks like + +{{< rawhtml >}} + +{{< /rawhtml >}} + +Each video is under a minute. It opens on the app already signed in, with a caption bar across the bottom narrating each step in a sentence. A small blue ring plays the part of a finger, gliding to a button and pulsing when it taps, moving at a pace you can follow. Text gets typed one character at a time, the way a person types. Everything on screen is the real application, doing real work. + +Now, viewers of my YouTube cooking series will know I'm no stranger to the world of video editing, but still I found this part genuinely pleasing: no camera was involved, and no video editor either. The videos are recorded by the same kind of automation that runs the tests. A short script says, in effect, "show this caption, tap this button, type this name, wait for the save," and the browser does it while recording itself. The video file that comes out is less like a film and more like a compiled program. The script is the source; the video is just what it produces today. + +That one property makes maintaining these a dream, and I'll come back to it at the end. + +The videos are silent on purpose, though that wasn't the original plan. The plan was voiceover, and each recording still produces a little narration script with timestamps for a voice to read. Some day I might play around with automated voice generation (synthesized speech continues to improve by leaps and bounds) but while creating these I realized the silent, captioned versions were the better product anyway. Carol can watch one in a client's living room without headphones, in a spare minute, with nothing to unmute. + +All fifteen are public, so Carol can brag about her bespoke software to her friends: [refreshingspaces.life/help/videos](https://refreshingspaces.life/help/videos). They're grouped by the shape of her workday, with sections like "Start here," "Getting paid," and "Out and about," because when there's one user, the natural organization is her day, and I know exactly what her day looks like. + +--- + +## Feeling human + +Getting the recordings to feel like a person showing you something, rather than a machine executing steps, took several rounds of watching them and wincing. + +The first version jumped from place to place, the way automation naturally moves, because a computer has no reason to scroll gradually when it can arrive instantly. A human would scroll and swipe with smooth motions, so every movement became a smooth, eased glide. Jump-cuts read as a machine. Eased motion reads as someone showing you. + +The second problem only revealed itself on watching: the caption bar sits at the bottom of the phone screen, which is exactly where apps put their most important buttons, so several taps happened underneath the caption. Now, when a tap is headed for the bottom of the screen, the caption politely hops to the top for that moment. It's a small behavior, and it's the difference between a video you trust and one that visibly hides the thing it's teaching. + +The third appeared on camera inside an error message. A form arrived with an amount already filled in, the automation typed a new amount on top of it, and the video proudly displayed "36.5012.00." The typing now clears a field before it fills it, the way you would. + +All three of these were caught by a human (me!) watching the finished video, not by any automated check, and that's worth repeating for anyone leaning on automation for anything: reviewing your own generated material is the quality control, and it doesn't automate away. + +One more choice from this stage, which I think about more than I expected to. I sprinkled Carol's business tagline throughout her app, but I chose to leave it out of the training captions. It belongs to her voice, and somehow it felt wrong seeing her voice in the caption bar narrating to herself where to tap. Somebody has to decide where a brand's voice does and doesn't go, and it turns out that's true even when the brand is your partner's and the audience is her. + +--- + +## Porting it to LocallyGrown + +The prototype was built in an evening for one person. The next night I brought the system to LocallyGrown.net, the farmers market platform I've built and run since 2002, and everything became plural. Shoppers, growers, and market managers each needed their own walkthroughs. People often shop on phones while managers often run markets on laptops (but not always!), so most walkthroughs got recorded twice, at phone size and at computer size. Twenty-five walkthroughs now produce forty-nine videos at [locallygrown.net/docs/videos](https://locallygrown.net/docs/videos), and short "watch" links sit inside the written documentation, right next to the paragraphs they illustrate. + +{{< rawhtml >}} +
+
+ Phone screenshot of the Refreshing Spaces training videos page. Below the heading and a short introduction, a section labeled "Start here" lists four videos, each with a play button, a title, a one-sentence description, and a duration: Finding your way around, Adding a new client, Logging a visit, and Using the timer. +
Carol's gallery is organized around her workday, not the app's feature list.
+
+
+ Phone screenshot of LocallyGrown's video gallery in a cream and green color scheme. Under the heading "For shoppers," three cards titled Watch: Placing your first order, Watch: Searching your market, and Watch: Browsing by category, each with a play button, a description, and a duration. +
LocallyGrown's gallery has to ask who you are first, because strangers need to find their section.
+
+
+{{< /rawhtml >}} + +The two video libraries came out differently in a way I didn't plan, and the difference says something about audiences. LocallyGrown's is grouped by role, because strangers arriving at a documentation page need to find their section. Carol's is grouped by her workday, because there are no roles when there's one user. You can see it in the writing, too. LocallyGrown's descriptions say things like "read the story behind the farm before you buy," addressed to a stranger, while Carol's say "a receipt when she pays," because I know exactly who "she" is. + +Scaling up also meant the demo data had to grow up. The recordings run against a make-believe farmers market, seeded into the system the way test data always is, which just means the database gets filled with plausible fictional things before the cameras roll: a market called Sweetwater, farms with products, shoppers with baskets. One video introduces shoppers to their growers, and it's only as good as the fictional grower it features. The seeded farm turned out to have contradictory placeholder junk, including a "uses synthetic chemicals" badge on a market that bans them, a one-line bio, and an empty photo frame. So I wrote the farm properly: a real three-paragraph story, a coherent certification, a photo. Quality time writing fiction changed how real every frame of that video feels. Good demo data is a form of writing. And once I started I kept going, because this turned out to be my favorite part of the whole project. The fixtures grew into a small fictional world: growers with histories, neighbors who buy their food from each other, a recipe that travels from one kitchen to another, dinner made for family and friends. I spent nineteen years running a real market like that, so writing the fictional one drew on a deep well. Sweetwater became an aspirational story about what a market can be, and every video is a little window into it. + +And one walkthrough turned out to be impossible to record. The video for adding a photo to a product showed nothing happening at the moment the photo should appear, because nothing did happen. The automation and the app's user interface framework disagree, in a subtle and interesting way, about what counts as "the user picked a file," so the automated recording genuinely cannot do what a real person's finger does. Real growers are unaffected. Rather than fake it, I dropped the video and kept the written guide. The [companion post](/posts/2026-07-18-your-test-suite-is-a-video-studio/) will be full of technical dives into issues like this, if that's your bag. + +--- + +## A cautionary tale about junk drawers + +The whole pitch of this system is that it records the real application with realistic data, and that pitch has a failure mode worth knowing about before you point a camera at a development database. + +While getting Carol's videos ready for public sharing, I noticed that one table in the development copy of her database still held real information. A few days earlier, I had imported her actual mileage spreadsheet to test that feature, and the import was still sitting there. Nothing in it was sensitive, and none of it ever reached the public site, but it was real data in a database I treat as entirely fictional, and I only noticed because I happened to look. + +A development database is a junk drawer. Things get put into it for good reasons and never taken out, and after enough time nobody remembers what's in the back. So a rule came out of the moment, and it's now built into the recording process: before recording anything meant for sharing, verify that every screen the camera visits shows only fictional data. The rule is deliberately not "check the data you remember putting in," because the whole problem is the data you forgot. + +The fiction has to be constructed with some care in the other direction, too. Some features don't appear at all until they're configured, so the demo data has to be rich enough to unlock everything the videos teach. The one-tap mileage button needs a home-base address before it will show itself, so the demo data includes one. For the address I picked a brewery a few streets over that we like to hang out at, which satisfies both halves of the rule: rich enough to unlock the feature, and verifiably fake. + +--- + +## Living with it + +Here's why "the video is a compiled program" matters. Every product help video ever filmed starts rotting the day it's published. The interface changes, the video is now wrong, and re-filming is so tedious that mostly nobody does, which is why the internet is full of tutorials for versions of software that no longer exist. + +My videos still rot. Literally within two hours of shipping, I had redesigned one page, added a label to another, and added a whole new tab to the navigation, and three videos went stale almost immediately. What changed is what happens next. Each fix was one command and a couple of minutes, and the freshly rebuilt videos simply replaced the old ones. There was one wrinkle where the internet itself kept showing everyone the old copies for a while (networks keep cached copies of files to serve them faster, and mine didn't know the videos had changed) but that too had a small, boring fix, covered in the [companion post](/posts/2026-07-18-your-test-suite-is-a-video-studio/). + +The discipline that emerged is a single question, asked after every change I ship: which videos show the screens I just touched? I rebuild those and move on. Nothing reminds me to ask it yet, and the system only stays cheap as long as someone does, so the question is headed for a checklist. In the meantime, every rebuild means I get to watch the ghost hand glide through the apps again, and that hasn't gotten old yet. diff --git a/content/posts/2026-07-18-a-ghost-hand-and-an-audience-of-one/locallygrown-gallery.jpeg b/content/posts/2026-07-18-a-ghost-hand-and-an-audience-of-one/locallygrown-gallery.jpeg new file mode 100644 index 00000000..53f438c5 Binary files /dev/null and b/content/posts/2026-07-18-a-ghost-hand-and-an-audience-of-one/locallygrown-gallery.jpeg differ diff --git a/content/posts/2026-07-18-a-ghost-hand-and-an-audience-of-one/logging-a-visit.mp4 b/content/posts/2026-07-18-a-ghost-hand-and-an-audience-of-one/logging-a-visit.mp4 new file mode 100644 index 00000000..6f8d0a3d Binary files /dev/null and b/content/posts/2026-07-18-a-ghost-hand-and-an-audience-of-one/logging-a-visit.mp4 differ diff --git a/content/posts/2026-07-18-a-ghost-hand-and-an-audience-of-one/refreshing-spaces-gallery.jpeg b/content/posts/2026-07-18-a-ghost-hand-and-an-audience-of-one/refreshing-spaces-gallery.jpeg new file mode 100644 index 00000000..819bdf64 Binary files /dev/null and b/content/posts/2026-07-18-a-ghost-hand-and-an-audience-of-one/refreshing-spaces-gallery.jpeg differ diff --git a/content/posts/2026-07-18-your-test-suite-is-a-video-studio/index.md b/content/posts/2026-07-18-your-test-suite-is-a-video-studio/index.md new file mode 100644 index 00000000..06af4847 --- /dev/null +++ b/content/posts/2026-07-18-your-test-suite-is-a-video-studio/index.md @@ -0,0 +1,285 @@ +--- +title: "Your test suite is a video studio" +date: 2026-07-18T09:00:00-04:00 +draft: false +description: "How a Playwright end-to-end harness became a training-video studio: the helpers, the overlay, the fixtures, and the walkthrough that couldn't be recorded." +tags: + - projects + - programming + - playwright + - svelte +--- + +{{< callout type="info" title="TL;DR" >}} +The technical companion to [part one](/posts/2026-07-18-a-ghost-hand-and-an-audience-of-one/). If your web app has a Playwright end-to-end suite, you already own most of a training-video studio. This post walks through what it takes to turn one into the other: a small helper vocabulary, an injected overlay, honest fixtures, and a build pipeline, with the lessons I picked up at each step. +{{< /callout >}} + +Part one told the story; this post is the how. The pitch, one more time for the people who came straight here: I generate silent, captioned training videos for two web applications by driving them with browser automation, and when an interface changes I rebuild the affected videos with one command. If you want to see the output first, the two libraries live at [locallygrown.net/docs/videos](https://locallygrown.net/docs/videos) and [refreshingspaces.life/help/videos](https://refreshingspaces.life/help/videos). Everything below is what you'd need to know to do the same for your own project. + +--- + +## One insight starts it + +Playwright can record video of everything that happens in a browser context. Turn that option on, and every test you already have becomes a screen recording. That's the whole foundation, and it's why this took an evening rather than a month: an end-to-end test and a walkthrough video are the same sequence of navigations and interactions. A test asserts at each step. A walkthrough narrates at each step. Everything else is presentation. + +So the first thing to build isn't video infrastructure at all. It's a flow: a name, a viewport, a starting URL, and an async function that performs the steps. Flows are data, which means the generator can run one, several, or all of them, and your video library has a single source of truth you can read like a table of contents. + +```ts +export type Flow = { + title: string; + description: string; + /** Per-role storageState path (playwright/.auth/.json) — reused from + * the e2e global-setup so recordings start already signed in. Omit for an + * anonymous (signed-out) walkthrough. */ + storageState?: string; + /** Routes to pre-compile on an unrecorded page before recording, so the + * cold vite dev-server compile of the first page isn't recorded as dead + * air. List every route the flow navigates to. */ + warmup?: string[]; + /** Optional fixture work on an unrecorded page (full speed, no captions). */ + prepare?: (page: Page) => Promise; + /** Recording viewport. 'desktop' uses 1280×800 + a desktop UA + centered, + * larger captions; omit (or 'mobile') for the default phone size. */ + device?: 'mobile' | 'desktop'; + record: (page: Page) => Promise; +}; +``` + +A flow itself reads like stage directions: + +```ts +'01-place-an-order': { + title: 'Placing your first order', + description: 'Browse your market, add fresh items to your basket, and reserve them for pickup.', + username: 'e2e-customer', // Casey Customer + warmup: ['/market', '/cart/review'], + async record(page) { + await goto(page, '/market'); + await say(page, "Welcome to your market — let's place an order."); + + // Skim down to the fresh items, the way you would on your phone. + await scrollBy(page, 620); + await say(page, "Scroll through what's fresh from your local growers."); + await scrollBy(page, 380); + + // First item — tap() smooth-scrolls it to center before the finger + // lands. Anchor the cart write so the basket persists before we leave. + const tomatoes = productCard(page, 'Heirloom Tomatoes'); + await say(page, 'Found something you like? Tap Add to Order.'); + let flushed = page.waitForResponse( + (r) => r.url().includes('/api/cart/batch-update') && r.request().method() === 'POST' + ); + await tap(page, tomatoes.getByRole('button', { name: 'Add to Order' })); + await flushed; + await say(page, "It's reserved — add as much as you like, from any grower."); + + // … eggs from a second grower, the basket review, and Reserve Now + // follow the same shape … + }, +}, +``` + +One configuration rule I'll hand you up front, because I learned it by squinting at blurry output: the recording size must exactly equal the viewport size. The comment in the harness says it with the authority of experience: + +```ts +const context = await browser.newContext({ + ...contextOpts, + // recordVideo.size MUST equal the viewport 1:1 — a larger canvas + // letterboxes the page into a corner instead of scaling up. + recordVideo: { dir: OUT_DIR, size: viewport }, +}); +``` + +--- + +## A vocabulary of four helpers + +Nearly everything the flows do goes through four helpers, and building these well is most of the work. + +The navigation helper goes to a URL and waits for the app to hydrate rather than merely load. A SvelteKit page is visible before it's interactive, and early on I had recordings that clicked during that gap: the video shows a confident tap and then nothing, which reads as broken software. Waiting for hydration is the difference. Mine watches for an attribute the app sets when it's ready: + +```ts +/** Navigate and wait for client hydration (the app sets body[data-hydrated]). */ +export async function goto(page: Page, path: string) { + await page.goto(`${BASE}${path}`); + await page.locator('body[data-hydrated]').waitFor({ timeout: 30_000 }); +} +``` + +The caption helper puts a sentence on screen and holds it long enough to read. My first version used a flat delay, and watching those drafts taught me that a flat delay is wrong in both directions: short captions overstay and long captions flash past. Hold time now scales with word count, and each caption is also logged with a timestamp for the narration cue sheet: + +```ts +/** Show a caption and hold it on screen long enough to comfortably read + * before anything moves. Hold time scales with length (a flat delay left + * longer lines flashing past); pass readMs to override. */ +export async function say(page: Page, text: string, readMs?: number) { + const words = text.trim().split(/\s+/).length; + const hold = readMs ?? Math.max(2800, 1000 + words * 360); + narration.push({ atMs: Date.now() - recordingStart, text }); + await page.evaluate((t) => { + const el = document.getElementById('tv-caption'); + if (el) { + // Reset to the bottom each caption — a previous tap may have dodged + // it to the top; the next caption should read from its home spot. + el.classList.remove('top'); + el.textContent = t; + el.classList.add('show'); + } + }, text); + await pause(page, hold); +} +``` + +The tap helper is where the humanity lives. It smooth-scrolls the target toward the center of the viewport with an eased animation, then glides a synthetic finger to it over two dozen interpolated steps, pulses, and clicks. The glide exists because my first recordings jumped from place to place, the way automation naturally moves, and watching them I had the same reaction every time: it feels like a machine. Easing fixed it, and I never had that reaction again. Nobody else saw a single video until I was happy with them, which is its own small lesson — you are the first audience for your generated material, and if it makes you wince, that's a signal. Sending your material out into the world sight unseen guarantees someone else will be wincing on your behalf. + +```ts +export async function tap(page: Page, locator: Locator) { + // Bring the target into view within whatever scrolls it — a modal's own + // overflow container, not just the window (smoothCenter only scrolls the + // window). Without this, a button below the fold of a tall modal is never + // reachable and the tap lands nowhere. + await locator.scrollIntoViewIfNeeded().catch(() => {}); + // Smooth-scroll the target toward center (a human glide), then read its + // post-scroll box so the finger lands accurately. + await smoothCenter(page, locator); + const box = await locator.boundingBox(); + if (!box) throw new Error('tap target not visible'); + const viewportHeight = page.viewportSize()?.height ?? VIEWPORT.height; + const nearBottom = box.y + box.height / 2 > viewportHeight - 150; + await page.evaluate( + (toTop) => document.getElementById('tv-caption')?.classList.toggle('top', toTop), + nearBottom + ); + await page.mouse.move(box.x + box.width / 2, box.y + box.height / 2, { steps: 24 }); + await pause(page, 260); + await page.mouse.down(); + await pause(page, 90); + await page.mouse.up(); +} +``` + +The typing helper selects a field's contents before it types, character by character at human speed. The select-first rule arrived by way of a video that proudly displayed "36.5012.00" after typing a new amount into a prefilled field. Your users clear fields without thinking about it; your automation won't unless you tell it to. + +```ts +/** Tap a field, then type at a human pace — select-all first to overwrite any + * prefilled value. */ +export async function type(page: Page, locator: Locator, text: string) { + await tap(page, locator); + await page.keyboard.press('ControlOrMeta+a'); + await page.keyboard.type(text, { delay: 45 }); +} +``` + +--- + +## An overlay that isn't part of your app + +The caption bar and the finger are injected DOM: a hundred-odd lines of markup, CSS, and script added to every recorded page at load time, riding on top of the untouched application. This matters more than it might seem. Your app needs zero knowledge that it's being filmed, which means no test-mode conditionals bleeding into production code, and the overlay works identically across every page and, it turned out, across two entirely different applications. + +The finger is a ring that shadows the mouse and pulses on every press; the caption bar is a fixed element the `say()` helper talks to: + +```ts +#tv-cursor { position: fixed; z-index: 2147483646; width: 28px; height: 28px; + border-radius: 50%; background: rgba(58,107,53,0.30); + border: 2px solid rgba(58,107,53,0.85); pointer-events: none; + transform: translate(-50%,-50%); left: -100px; top: -100px; } +#tv-cursor.tap { animation: tv-tap 0.4s ease-out; } +@keyframes tv-tap { 0% { transform: translate(-50%,-50%) scale(1); } + 40% { transform: translate(-50%,-50%) scale(1.7); } + 100% { transform: translate(-50%,-50%) scale(1); } } +``` + +```ts +addEventListener('mousemove', (e) => { + cursor.style.left = e.clientX + 'px'; + cursor.style.top = e.clientY + 'px'; +}, true); +addEventListener('mousedown', () => { + cursor.classList.remove('tap'); + void cursor.offsetWidth; + cursor.classList.add('tap'); +}, true); +``` + +The overlay taught me a fun small lesson on adaptability. The caption bar sits at the bottom of the viewport, which is exactly where mobile interfaces put their primary buttons, and several early recordings tapped buttons hidden behind the caption. Now, when a tap target sits in the bottom portion of the screen, the caption hops to the top for that beat — you can see it in `tap()` above, in the `nearBottom` check. It's four lines of logic, and it's the difference between a video you trust and a video that covers the control it's teaching. + +--- + +## Fixtures are writing + +The flows run against the same seeded fixtures your test suite already uses, and sign-in happens through the same stored authentication state, which is why every video opens already signed in instead of spending ten seconds on a login form. + +But video asks more of fixtures than tests do, and this is the part I'd underline for anyone building one of these. A test only needs data that's valid. A video needs data that's believable, because the camera lingers on it. LocallyGrown's "meet your growers" walkthrough is only as good as the fictional farm it features, and my seeded farm turned out to have a one-line bio, an empty photo frame, and a "uses synthetic chemicals" badge on a market that bans them. I spent an evening hour writing great fixtures that show off the system and that farm properly, such as a three-paragraph story and a coherent certification and a photo, and it changed how real every frame feels. Budget time for fixture writing. It's writing. That farm was the first of many; the fixtures kept growing until the demo data told the story of a whole small community. + +Two disciplines go with it. Before recording anything for sharing, verify that every table your camera visits holds only fictional data, and make that a check the system runs rather than a thing you remember, because development databases accumulate real imports over time and my own had one. And seed richly enough to unlock every feature you're teaching, since plenty of interface doesn't render until settings exist: a home-base address for the one-tap mileage button, payment handles for the tap-to-pay buttons. I use a separate test database that gets torn down and rebuilt from scratch every run, but if you use a single database shared between development and testing, this can become a thorny problem. + +One more fixture-adjacent choice: each recording also emits a timecoded narration cue sheet, caption text mapped to timestamps. Mine were meant for a learning exercise in AI voiceover that I ended up not wanting to get into just yet, but they cost nothing to keep producing, and they'd make captions translatable or narratable later without touching a single flow. + +--- + +## From recording to serving + +Playwright records webm; the web wants mp4. The generator transcodes with ffmpeg as its last step, and the files are all gitignored, because they're build artifacts. The flow catalog is the source; the videos are what it compiles to today. + +```ts +// webm → mp4 (yuv420p for Safari/iOS, +faststart so playback starts before +// the file finishes downloading). Best-effort: keep the webm regardless. +execFileSync( + 'ffmpeg', + ['-y', '-i', webm, '-c:v', 'libx264', '-pix_fmt', 'yuv420p', '-movflags', '+faststart', mp4], + { stdio: 'pipe' } +); +``` + +Serving them had two lessons waiting, both one-line fixes. My ignore entry read `training/`, which also matched `src/routes/training/`, the route that serves the videos, so production returned 404 for every video while my laptop happily played them all. One day I'll have gitignore syntax correctly stored in my brain. The anchored pattern `/training/` fixed it, and the commit title is the honest summary: "Ship the /training/[file] route the gitignore swallowed." Then Cloudflare, which fronts the app, kept serving old copies after regeneration, because the route sends a one-day cache header and re-uploading a file changes nothing an edge cache cares about. Versioned URLs fixed that one: every source renders as `/training/{video.file}?v={VIDEO_VERSION}`, and the version bumps after each upload. If mutable files live behind an edge cache, the URL has to change when the bytes do. + +Regeneration reviews, by the way, don't require watching every video end to end. For a change confined to one screen, I can extract a single frame at the timestamp where that screen is on camera, and that answers the only question that matters. + +--- + +## Scaling from one user to three audiences + +The prototype was a single eleven-hundred-line file, phone-only, built in an evening for an app with one user. Porting it to LocallyGrown the next night is what forced real structure: the flow catalog, harness, overlay, and orchestration each became a module, and flows gained a device dimension, because shoppers use phones while market managers use laptops, so most flows record twice at different viewports. + +Desktop mode had a subtle surprise for me. The app needed to know it was rendering at desktop size before first paint, and the obvious approach, setting an attribute on the root element from an init script, silently did nothing. The comment in the harness explains why, better than I could from memory: + +```ts +// Let the overlay adapt captions (centered + larger) for desktop recordings. +// Set a window global rather than an attribute: init scripts run before the +// HTML is parsed, and the parser replaces the initial element, so an +// attribute set here is silently discarded. The overlay installer runs at +// DOMContentLoaded and stamps data-tv-device from this global. +if (desktop) { + await page.addInitScript(() => { + (window as unknown as { __tvDevice?: string }).__tvDevice = 'desktop'; + }); +} +``` + +The two-device pipeline also taught me the project's most transferable operational lesson. My first version recorded the phone pass, reseeded the database, and reused the still-running dev server for the desktop pass, and that server held stale module state and stale row IDs. Sometimes the run crashed; sometimes it completed and produced videos of stale data, which is quieter and therefore worse. Each device pass now gets its own full server lifecycle: start, seed, record, stop. And the fix got better on review — I always have AI coding assistants do several review passes over their own work, and it caught that the stop step returned silently when the port never freed, letting a wedged server survive into the next pass. It now escalates to a hard kill and throws if the port is still listening. A cleanup step that always reports success is a specific example of a surprisingly common gotcha: never trust anything that always reports success. + +--- + +## A walkthrough I couldn't record + +I'll close with the one that got away, because it's always nice to have a puzzle I can come back to. + +The LocallyGrown walkthrough for adding a photo to a product produced a video in which the photo never appears. The flow ran cleanly, the form saved, and the recording shows a save with no photo, because that's what actually happened. + +Probing it layer by layer: Playwright's `setInputFiles` does exactly what it documents. After the call, the input's `files` property contains the file, and a native change event fires. The application's handler simply never runs. The app is Svelte 5, which delegates common events to a single root listener rather than attaching handlers per element, and that delegation doesn't invoke the component's `onchange` for this synthetic selection. App state never updates, the preview never renders, and the form honestly submits without a photo. + +This is a known class of behavior, not a novel bug: [sveltejs/svelte issue 12147](https://github.com/sveltejs/svelte/issues/12147) documents the same shape, where programmatically dispatched events with native names reach handlers under the legacy `on:change` syntax and stop reaching them under the delegated `onchange` syntax. A real person operating a real file picker generates a trusted, browser-initiated event, which delegation handles normally, so no actual grower has ever hit this. The failure exists only for synthetic events. A recording rig is made of synthetic events. + +Every workaround I considered was a form of lying. Dispatch a hand-rolled event, call the handler directly, patch the state from outside: each produces a video of something the application doesn't do. So the flow is gone from the catalog and the written photo guide remains. The commit message stands as the summary of record: "A recorded demo can't show a real photo upload... Real growers using a real file picker are unaffected — there's just no way to film it truthfully." + +I find I like what this says about the whole approach. The system's entire value is that it refuses to fake anything: real app, real fixtures, real interactions, and a recording of whatever truly happens. The same property that makes the videos trustworthy is the one that made this flow unrecordable, for now. If you build one of these, you'll eventually meet your own version of this walkthrough, and I'd encourage you to let it win. + +--- + +## What two evenings bought + +The part I keep marveling at is how little of this system I actually had to build. Playwright already knew how to record video. The e2e suite already knew how to seed a believable market and sign in as anyone. ffmpeg already knew how to make Safari happy. The new code is a thin layer of glue and taste: four helpers, an overlay, and a catalog of flows that read like stage directions. That's why the first version took an evening and the port took another one, and it's a low-risk reason to try this yourself if you have a test suite sitting there: the expensive parts are already on your machine. + +What it bought is out of proportion to that effort. Both applications are better documented than anything I've shipped in twenty-plus years of building software, and more approachable too. A stranger landing on LocallyGrown can watch themselves place an order before they've read a word, and Carol can answer "how do I do that again?" in under a minute, standing in a client's kitchen. + +And my favorite part of the whole project was the least automated one. Writing the fixtures grew into writing a world: farmers bringing what they grow to a common market, neighbors filling baskets, recipes traveling from one kitchen to another, meals made for family and friends. What started as believable data in a database became an aspirational story about a community, and every video is a little window into it. The machine records what it sees, and it fell to me to make what it sees worth looking at. I could happily do more of that.