UAT Theory Paper Flow

How UAT reads a theory paper today

The exact deployed flow (Anant's v3 engine, live since 15 Sept evening) — from the teacher's upload to an approved answer key. Verified against the running code on the box, 16 Sept 2026.

The one-paragraph version: the teacher's PDF is turned into photos of its pages. A vision AI (Gemini 3.1 Pro) looks at all the pages twice — once to map the paper's skeleton (sections, question numbers, marks), once to copy out every question's words. The paper is then checked against itself: the total printed on the cover, the sum of the section totals, and the sum of the per-question marks must agree. Then a second AI (Luna, a GPT-class model with "thinking" on) writes a model answer + step-by-step marking scheme for each question, with real textbook passages pasted into its prompt when we have that subject's books. A teacher approves the key before any student sheet is marked. No OCR text engine runs in this flow — Mathpix is dormant.

1 Ā· The full journey

flowchart TD
    T["šŸ‘©ā€šŸ« Teacher uploads paper PDF
types name, subject, TOTAL MARKS"] --> G{"Gate checks
(cp-backend)"} G -->|"AI plan active Ā· quota left
≤ 30 pages Ā· budget ok"| S3["PDF saved to S3
test row created (Draft)"] G -->|any gate fails| RAW["Paper still uploads —
just no AI answer key
(teacher told why)"] S3 --> Q["Job queued
(Redis Ā· one paper at a time)"] Q --> W["Worker picks the job"] W --> CPAI["cp-ai Ā· /v1/extract
engine = v3"] CPAI --> IMG["šŸ“ø Pages → JPEG photos
150 DPI · no OCR text step"] IMG --> C1["🧠 CALL 1 · STRUCTURE
Gemini 3.1 Pro (vision + thinking)
sees ALL pages at once"] C1 --> SK["Skeleton: sections, question
numbers, marks, cover total"] SK --> C2["🧠 CALL 2 · CONTENT
Gemini 3.1 Pro Ā· pages re-sent,
15 questions per call
copies every question's words"] C2 --> REC{"āš–ļø RECONCILE
cover total vs section sum
vs question sum"} REC -->|two of three agree| OK["confidence OK"] REC -->|they disagree| DIS["confidence DISAGREE
šŸ”“ never auto-approved"] OK --> DB["Questions stored in Postgres
teacher sees them appear"] DIS --> DB DB --> AK["🧠 ANSWER KEY · one call per question
Luna (GPT-class) Ā· thinking ON
prompt = question + options + figures
+ marks + šŸ“š textbook passages (RAG)"] AK --> J["Judge re-checks each scheme:
steps must add up to the marks"] J --> REV{"clean?"} REV -->|yes| AUTO["AUTO-APPROVED"] REV -->|anything off| NEED["NEEDS REVIEW
with a plain-English reason"] AUTO --> TEACH["šŸ‘©ā€šŸ« Teacher reviews & hits
APPROVE ANSWER KEY"] NEED --> TEACH TEACH --> GRADE["Only now do student sheets
get marked against the key"]

2 Ā· Who talks to whom, with what context

sequenceDiagram
    participant T as Teacher (browser)
    participant BE as cp-backend
    participant S3 as S3
    participant AI as cp-ai (v3 engine)
    participant G as Gemini 3.1 Pro (vision)
    participant L as Luna (answers)
    participant B as šŸ“š Textbook store (RAG)

    T->>BE: PDF + subject + class + total marks
    BE->>S3: store the PDF
    BE->>AI: PDF bytes + hints (subject, class, teacher's total)
    AI->>AI: render pages → JPEG photos (150 DPI)
    AI->>G: ALL page photos + "map this paper" prompt
    G-->>AI: sections Ā· numbering Ā· marks Ā· cover total
    AI->>G: same photos + skeleton + "copy out Qs 1–15"
    G-->>AI: full question text (repeat per 15-question chunk)
    AI->>AI: reconcile the paper's 3 totals → confidence
    AI-->>BE: questions + sections + marks + verdict
    BE-->>T: questions appear on the Answer Key tab
    BE->>AI: write the key (per question)
    AI->>B: find matching textbook passages
    B-->>AI: relevant paragraphs (when the subject has books)
    AI->>L: question + options + figure + marks + passages
    L-->>AI: model answer + marking scheme (steps sum to marks)
    AI->>AI: judge: does the scheme add up?
    AI-->>BE: key + AUTO-APPROVED / NEEDS-REVIEW + reason
    BE-->>T: "38 of 38 answers written" → teacher approves

3 Ā· The models, in plain words

JobModelWhat it's givenWhat it returns
Read the paper's shape (structure call)gemini-3.1-pro-preview — a vision LLM, thinking onPhotos of every page + a prompt asking for the paper's mapSections as printed, each question's number & marks, the cover total
Copy the questions out (content calls)same Gemini, fresh callsThe same page photos again + the map + "transcribe questions N–M"Every question's exact words, options, passages
Split section totalssmall text call (its own slot)"Section Q.2 is 16 marks over 8 questions"2 marks each — flagged as worked out, not printed
Write each answerLuna (GPT-class, via OpenRouter), thinking on, prompt theory_vision_answer@v4One question + its options/figure + its marks + textbook passagesModel answer + step marking scheme whose steps add up
Check the answerjudge passThe scheme it just wrote"adds up" → auto-approve Ā· anything else → NEEDS REVIEW + reason
Is OCR used? Not in this flow. There is no text-extraction engine in the middle — the vision LLM reads the page photos directly, the way a person reads a printed page. Mathpix (the math-OCR service) is still installed but dormant: it only ever ran on older paths, and the v3 engine takes every paper before Mathpix is reached.

4 Ā· The self-check that guards the total

flowchart LR
    A["Cover says
Max. Marks: 80"] --- X{"must
corroborate"} B["Section totals
16+4+16+18+8+15+3"] --- X C["Per-question marks
added up"] --- X X -->|"two of three agree"| OK2["āœ… read is trusted"] X -->|"they disagree"| BAD["šŸ”“ DISAGREE —
key can never auto-approve,
teacher is pointed at it"]
Why this matters (yesterday's lesson): the older pipeline read an 80-mark paper as 60 and nothing shouted, because no two independent totals were compared. v3's whole design bet is that a paper carries its own answer key for "did you read me right?" — three numbers printed in different places that must agree. Measured on the eval corpus: 91.3% exact totals.

5 Ā· What each piece stores / owns

PieceOwnsLayman's role
cp-backendtest row, question rows, S3 files, the queue, all gatesThe clerk: checks entitlements, files everything, never reads the paper itself
cp-aithe reading + answering brains, model choices, retriesThe examiner: reads the paper, writes the key, says how sure it is
Gemini 3.1 Pro—The eyes: sees page photos, maps and transcribes
Luna—The subject teacher: writes each answer and its step marks
Textbook storebook passages per subject/classThe library: real pages quoted into the answer prompt so the key isn't from memory alone
Teacherthe final sayNothing is graded until they press Approve