Prompt to Playable
The model does not return a code block. It emits a typed artifact protocol that a character-level streaming parser turns into live file writes and shell commands inside an in-browser Node runtime — so the dev server is booting while the model is still typing. A cheap pre-generation pass turns a one-line wish into a structured brief, follow-up edits travel as unified diffs and come back as patches, and every version is recoverable.
- Role
- AI Engineer — generation & runtime
- Domain
- Creative tooling for non-coders
- Output
- 2D · 3D · simulator · quiz · slides
- Runtime
- Remix · WebContainer · AI SDK
- Status
- Live in production
Outcomes
What was
broken
The problem as it actually presented itself, and the constraints that shaped every decision after it.
Waiting for a complete model response before doing anything with it wastes the entire generation window. On a multi-file game that is thirty to sixty seconds of a user staring at a spinner, followed by a large diff appearing all at once — which is the least trustworthy way to show someone their own project being built.
Regenerating on every follow-up is worse. 'Make the enemies faster' should not rewrite the render loop, and every full regeneration is a fresh opportunity to break something that already worked.
The users are not programmers. They cannot debug a broken bundle, read a stack trace, or tell the difference between a bad idea and a missing import — so the failure modes had to be designed out of the generation surface rather than explained afterwards.
What it
runs on
Models, retrieval, serving, and the operational layer that keeps it honest in production.
- Claude via AI SDK
- Artifact/action protocol
- Genre-tuned exemplars
- Idea + prompt enhancers
- WebContainer
- CodeMirror 6
- xterm.js
- Serialised action queue
- Unified diffs
- Patch/partial/full writes
- Version history
- Snapshot tests
- AI thumbnails
- Community publish
- Stripe
- Amplitude + session replay
How it
works
The pipeline end to end — each stage, and why it earns its place in the latency budget.
- 01
Idea in, brief out
Before the expensive call runs, a cheap pass does the work the user cannot: an idea generator turns 'something with dinosaurs' into concrete game concepts, and a prompt enhancer rewrites the chosen one into a structured brief — genre, mechanics, controls, win condition — surfaced back as an editable form. The generation model starts from a specification instead of a wish, which is the single cheapest quality win in the pipeline.
- 02
A typed artifact protocol
Instead of prose with fenced code, the model emits an artifact containing typed actions: file writes with a path, or shell commands. Because the path is an attribute on the opening tag, the runtime knows what a chunk of output is for before any of the content has arrived.
- 03
Character-level streaming parser
A state machine parses the response mid-tag, tracking position, nesting, and partial attributes per message. It fires callbacks on artifact open and close and on every action open, update, and close — so file contents stream into the editor character by character as they are generated.
- 04
Action runner over an in-browser runtime
Parsed actions queue onto a single execution promise against a WebContainer instance, each carrying an abort signal and a status — pending, running, complete, aborted, or failed. Files are written and dependencies install while the model is still producing the rest of the response.
- 05
Diffs in, patches out
When the user edits a file by hand, the change comes back to the model as a GNU unified diff — or the whole file, whichever is smaller. The model replies with one of three write modes: full replace, a targeted patch at a line, or a partial range rewrite. Iterations touch only what changed, and every generation is a restorable version rather than a destructive overwrite.
- 06
Genre-specialised prompting
2D, 3D, 3D simulator, quiz, math-survival, and slide-deck each carry their own worked exemplars in the system prompt rather than sharing one generic instruction. An asset-locked mode forbids inventing assets when a curated library is supplied — the single highest-leverage constraint for output that actually renders.
- 07
Publish, share, and teach
A finished game gets an AI-generated thumbnail and a share flow into the community gallery, so the creator ends with something they can send to someone. The same pipeline powers the classroom layer — classes, lessons, published worlds, per-student and per-lesson reports. The teacher never sees an artifact tag or a file tree; they see whether the class understood the material.
Decisions
and their cost
Every choice below bought something and gave something up. The second half is the part worth reading.
Stream into execution, don't wait for the message
Parsing mid-stream is meaningfully harder than parsing a finished string — you handle split tags, partial attributes, and per-message state. It buys the entire generation window back as useful time, and it is the difference between watching a project get built and watching a progress bar.
Diffs in both directions
Sending unified diffs up and accepting patch-mode writes down cut both the tokens per iteration and the blast radius of a bad generation. The cost is a real diff/patch implementation and a size heuristic for when to fall back to the whole file — worth it by the second follow-up prompt.
Exemplars beat instructions
Every attempt to describe a genre in prose underperformed a worked example of that genre in the prompt. Six exemplar sets are more prompt tokens and more to maintain than one generic instruction, and they are the reason output for a physics simulator does not look like output for a quiz.
Keep the model out of the build system
The runtime owns the toolchain, the dev server, and the shell. The model writes game logic against a known project shape. Every degree of freedom removed from generation is a class of failure that never needs a repair loop.
Spend a cheap call to save an expensive one
A small model rewriting a one-line prompt into a structured brief costs a fraction of a cent and measurably improves what the generation model produces. Skipping it and letting the big model interpret 'make a cool game' directly is faster to build, cheaper per request on paper, and produces more regenerations — which is where the money actually goes.
How it was
measured
Nothing shipped on intuition. Each number below is produced by a repeatable harness that gates deploys.
Files write as the model generates, not after the response completes.
A cheap model turns a one-line prompt into an editable structured brief.
Follow-ups send a unified diff and receive a patch, not a full rewrite.
Full replace, line-targeted patch, and partial range rewrite.
Each with its own worked examples and constraints in the system prompt.
Every generation is restorable — iteration is never destructive.
The streaming parser is pinned by snapshots — it is the load-bearing piece.
Games run in the browser runtime; no backend to provision per project.
Guardrails
and safety
What stands between a good demo and something you can leave running unattended.
Generated code executes inside a sandboxed in-browser runtime with no access to the host machine or to other users' projects.
Every queued action carries an abort signal and an explicit terminal status, so a runaway install or a failed write surfaces as state rather than a hung UI.
Actions run on a single serialised execution promise — no two file writes race, and ordering matches the model's intent.
The model cannot touch the toolchain or bundler configuration; the project shape is fixed by the runtime.
Asset-locked mode prevents the model from referencing assets that do not exist when a curated library is in play.
The streaming parser is covered by a snapshot suite, because a parser regression corrupts every project it touches.
What I'd
carry forward
The parts that generalise — earned the expensive way, on this build.
Streaming is an architecture, not a UI feature. Once the parser can act on partial output, latency stops being something you apologise for and becomes something you spend.
Constrain the output surface. A typed action protocol with a fixed project shape removed more failure modes than any amount of prompt tuning did.
Show examples of the thing you want. Six genre-specific exemplar sets beat one carefully worded universal instruction, every time.
Design the failure mode for the actual user. Non-programmers cannot recover from a broken bundle, so the system has to make broken bundles rare rather than legible.
Put a cheap model in front of the expensive one. Turning a vague prompt into a structured brief before generation was worth more than any change I made to the generation prompt itself.