PDF to Lesson
Pages are sent to the model as rendered images, not extracted text, so diagrams and worksheet layouts survive. Output is validated against a typed activity registry before it renders and calibrated across ten grade levels. Everything a student writes runs through a thirteen-category safety classifier first — and when it trips something serious, the incident is triaged, logged, and escalated to the teacher rather than silently refused. The classroom itself runs on a role-authenticated Socket.IO session.
- Role
- AI Engineer — generation, safety & realtime
- Domain
- K-12 / higher-ed courseware
- Audience
- Elementary through college
- Safety
- 13 categories · 4 severity tiers
- Runtime
- React · Vite · Socket.IO
- Status
- Live in production
Outcomes
What was
broken
The problem as it actually presented itself, and the constraints that shaped every decision after it.
Extracting text from a textbook page throws away the part that teaches. Diagrams, tables, labelled figures, and worksheet layout carry the meaning, and a text-only pipeline generates quiz questions about captions while ignoring the figure they caption.
Free-text generation also breaks interactive UI. An activity is a typed structure with options, a correct index, and a hint — one malformed field and the component renders nothing, in front of a class.
The realtime layer started on a hosted database's change-feed. Classroom control is not a data-sync problem — it is a command problem with roles, acknowledgements, and a resync story when a laptop lid closes — and modelling teacher commands as row changes meant no delivery guarantee, no host-vs-student authorisation, and no clean way to catch a student up mid-lesson.
And the room is hostile in a second sense. Students type into an AI in a classroom, and some of them will type something harmful — about themselves or about someone else. A generic content filter that silently rejects the message protects the vendor and abandons the child.
What it
runs on
Models, retrieval, serving, and the operational layer that keeps it honest in production.
- Multimodal page attachments
- Typed activity schemas
- Grade-level difficulty map
- React 18
- Zustand
- pdfjs-dist
- Fabric.js annotations
- Socket.IO gateway
- Role-authenticated sessions
- Typed event protocol
- Reconnect + state resync
- 13-category classifier
- 4 severity tiers
- Teacher escalation
- Compliance audit log
- IndexedDB via localforage
- Sync queue
- Offline cache
How it
works
The pipeline end to end — each stage, and why it earns its place in the latency budget.
- 01
Send the page, not the text
PDF pages render to images and travel to the model as multimodal attachments, so figures and layout are part of the input. Images are resized and compressed client-side before upload, and the detail level is chosen per request — low detail costs about 85 tokens per image against 765+ for high, and most pages do not need high.
- 02
Typed activity registry
Every activity type — single choice, question set, true/false, fill-in-the-blanks, questionnaire — has one canonical JSON shape shipped in the prompt and one validator in the client. Generation is checked against the registry before it renders, and item counts are capped so a runaway response cannot produce a forty-question worksheet.
- 03
Grade-calibrated prompting
A single difficulty map spans ten levels from first-grade elementary through college and general audience, and it changes vocabulary, tone, and question depth together. Hints are required to point at the reasoning and explicitly forbidden from containing the answer.
- 04
Progressive streaming with real stages
The client parses partial JSON as it arrives and reports an honest stage — analysing, generating, finishing — by counting completed question markers in the buffer rather than animating a fake progress bar. The user sees how much is actually done.
- 05
Student-intent moderation
Everything a student writes passes a classifier tuned for a classroom rather than a general audience: thirteen categories — sexual and sexual/minors, hate and threatening hate, harassment, self-harm intent and instructions, violence, graphic violence, illicit — every one of them at a 0.01 threshold. It cannot be turned off per request. This is the layer that reads what a student actually intends, not just what they typed.
- 06
Severity triage and teacher escalation
A flagged message is scored into four severity tiers, and the tiers are not symmetric: self-harm intent, self-harm instructions, and sexual content involving minors are CRITICAL regardless of confidence, while threatening hate and graphic violence sit at HIGH. Every event is written to a rotating, timestamped audit log with the user, the request id, the endpoint, the full flagged content, the category scores, and the action taken — and the incident surfaces to the teacher, so a child writing about hurting themselves reaches a human instead of a rejected request. The student sees a message pointing them at their teacher.
- 07
Live classroom over Socket.IO
A dedicated realtime gateway namespace over a secure WebSocket, replacing the hosted change-feed the first version used. Clients authenticate into a session by role — host or participant — with acknowledgement callbacks, then speak a typed command protocol: slide navigation, opening and closing an interaction, revealing answers, enabling and disabling focus, pushing annotations, toggling the leaderboard, sharing a submission, pausing, resuming, and ending. Students join by code or QR, identified by device fingerprint instead of an account.
- 08
Reconnect and resync
Classroom Wi-Fi drops. The client tracks its own connection state machine, retries with backoff up to ten attempts, and on rejoin requests a full session sync rather than assuming it can resume mid-stream — so a student who reconnects lands on the slide the class is actually on. Underneath, settings, annotations, drawings, and generated content persist to IndexedDB with a sync queue that drains when connectivity returns.
Decisions
and their cost
Every choice below bought something and gave something up. The second half is the part worth reading.
Images over extracted text
Multimodal input costs more per page than text extraction and is the only version that understands a diagram. Choosing detail level per request rather than globally recovered most of the cost difference — roughly a 9× swing in tokens per image between the two settings.
Validate at the type level
A shared registry defining each activity's shape, used by both the prompt and the client validator, means a malformed generation is caught before render instead of appearing as an empty component in front of a classroom. One definition, two consumers, no drift.
Honest progress over a spinner
Counting structural markers in the partial JSON buffer gives a progress signal that reflects reality. It is more code than a spinner and it is the difference between a thirty-second wait that feels broken and one that feels like work happening.
Escalate to a teacher, don't just block
The easy version rejects the message and moves on. That is the wrong outcome when a twelve-year-old writes about self-harm — the signal exists, and burying it in a generic refusal wastes the one moment an adult could have acted. So severity is triaged, the incident is logged with enough context to be reviewable, and it reaches the teacher. The cost is real: it means logging student-written content, which is a privacy obligation you take on deliberately, with retention and access as explicit decisions rather than defaults.
Thresholds at 0.01, and not configurable
Every category sits at a 0.01 score threshold and the moderator cannot be disabled per request. That is far stricter than a general-purpose product would tolerate and it produces false positives — a history lesson about a war trips violence. For a classroom that is the correct direction to be wrong in, and the escalation path means a false positive costs a teacher ten seconds rather than costing a student anything.
Socket.IO over a hosted change-feed
Rebuilding realtime on an explicit socket protocol meant writing role authentication, an event vocabulary, acknowledgements, reconnect logic, and a resync path by hand — all of which the managed change-feed appeared to give for free. It also meant classroom control stopped being modelled as database rows changing: a teacher command is a command, it is authorised as one, and a student who reconnects can ask for the current state instead of inferring it.
No student accounts
Device fingerprinting identifies students within a session without a signup flow. It is weaker identity than authentication and it is the right trade for a classroom, where the cost of friction is measured in minutes of a lesson.
How it was
measured
Nothing shipped on intuition. Each number below is produced by a repeatable harness that gates deploys.
Sexual, minors, hate, harassment, self-harm, violence, illicit — and threatening variants.
Every category, every age group. Not configurable per request.
Self-harm and sexual/minors are CRITICAL regardless of confidence.
User, request id, endpoint, content, category scores, and action — rotated daily.
~85 tokens at low detail vs 765+ at high — chosen per request, not globally.
Each with one schema in the prompt and one validator in the client.
Elementary 1-6, middle, high, college, general audience.
Backoff, then a full session resync on rejoin rather than a blind resume.
Guardrails
and safety
What stands between a good demo and something you can leave running unattended.
Every student message is classified across thirteen risk categories — sexual content, sexual content involving minors, hate, threatening hate, harassment, threatening harassment, self-harm intent, self-harm instructions, violence, graphic violence, and illicit activity — at a 0.01 threshold, before it reaches a model.
Moderation is always on. There is no per-request flag, no educational-context exemption, and no age group with looser thresholds.
Flagged content is triaged into four severity tiers, with self-harm intent, self-harm instructions, and sexual content involving minors treated as CRITICAL regardless of score.
Every incident is written to a rotating audit log with the user, request id, endpoint, flagged content, per-category scores, and the action taken — reviewable, and shared with the teacher rather than silently discarded.
The student-facing message points them at their teacher and the classroom's help resources instead of returning a bare refusal.
The moderator is fail-closed: an error in the safety layer blocks the request, because an unavailable check is not a passing check.
Every generated activity is validated against its registry schema before rendering; a malformed response is rejected, not partially drawn.
Realtime sessions are role-authenticated — a participant socket cannot emit host commands — and session identity is scoped to a device and a session, with no account required to join a class.
Local persistence means a network failure degrades to offline mode rather than to data loss.
What I'd
carry forward
The parts that generalise — earned the expensive way, on this build.
Blocking is not safeguarding. The moment you can detect that a child is in trouble, refusing the message and logging nothing is the one design choice that helps no one — the escalation path is the feature, the classifier is just the trigger.
Realtime control is a protocol, not a data feed. Modelling teacher commands as row changes cost authorisation, acknowledgements, and any sane answer to 'what happens when a laptop reconnects'.
Modality is a content decision. Whether you send text or pixels determines what the model can possibly know about the page, and no prompt recovers information the input never carried.
Share one schema between the prompt and the validator. Two definitions of the same structure will drift, and the drift shows up in a classroom.
Design for the worst network in the room. Offline-first turned out to be a feature teachers noticed and mentioned, not just insurance.