How a document becomes a cited answer

§1Problem

§2Architecture

• Strict layering: routes → controller → service → repository → db. A service never imports hono, which is what lets a BullMQ job call the same service the HTTP route does.
• Frontend: Next.js 16 App Router, React 19, TanStack Query v5, ky + zod. The Hono proxy means the backend URL never reaches the bundle.
• No JWT layer. The backend authenticates the Better Auth session cookie directly; the proxy forwards it and adds nothing.
• The workspace cookie is a preference, not a credential. It is validated against the real membership list, and the backend answers 404 for a workspace you are not in.
• Roles are UX-only on the client (canContribute, canAdminister). The backend re-checks membership and role at the resource boundary on every request.
• Chunk text lives in Postgres; its vector lives in Qdrant, keyed by the chunk row id.

§3The pipeline, stage by stage

1. 01pending

   Ingest

   The payload is thin on purpose. A job queued before a deploy acts on whatever is true in the database after it, not on a snapshot from before.

   Two guards agree on the same id: a duplicate enqueue of the same attempt is a no-op in BullMQ, and the ledger charge for that attempt lands on the same key, so it cannot post twice.

   The three-attempt cap is a cost decision: every retry is a paid provider call, and a broken file does not become a working one on the fourth try.

2. 02parsing

   Parse

   The extractors are split by document shape, not just by extension: general, structured, tabular, email and hierarchy, on top of unpdf, mammoth and exceljs.

   Each section carries its page range, and chunks inherit it as fromPage and toPage.

   A file with no text layer stops here with a readable reason rather than becoming an empty document marked ready.

3. 03chunking

   Chunk

   The merge works in two passes: anything over budget is split at sentence delimiters, then the pieces are merged back up to the token budget (512 by default) with an overlap percentage carried from the previous chunk's tail.

   RAPTOR clusters passages by cosine similarity with a greedy nearest-neighbour pass, deliberately not UMAP+GMM. It is simpler to reason about, and grouping passages before a model summarises each cluster does not need more.

   Summary chunks exist so a whole-document question has something to match. They are marked as model-written, so a reader can tell a generated summary from a quoted passage.

4. 04embedding → ready

   Embed

   The batch size is a provider limit, not a tuning knob, so it is not a setting anyone can change.

   Truncating an over-long input at 8 000 tokens loses the tail of one chunk; failing the document would lose all of it.

   Freezing the model per knowledge base is why Qdrant has one collection per embedding model, and why two bases embedded with different models cannot be searched together later.

   Resumability matters most on large PDFs: a failure late in the run re-embeds only the ranges that were never paid for. A run that gives up ends the row as failed, not ready.

5. 05

   Retrieve

   The two scores live on different scales, which is why the lexical score is normalised against the best hit in the result set before fusion. Both searches run over the same corpus.

   Bases embedded with different models are refused with a message rather than searched together, because their vectors are not comparable.

   The document screen shows the chunks themselves, because retrieval can only ever return one of those passages. The documents list polls only while a row is in flight and stops when every row has settled.

6. 06

   Cite

   The passages are stored with the message, not looked up again at render time, so what a reader opens is exactly what the model was shown.

   An agent run can search several times across several tool calls. Numbering once per run instead of once per search keeps [[1]] stable for the whole reply.

§4The agent flow editor

• A graph DSL owned by the product, not by the canvas library. @xyflow/react is a view of it; the backend stores and runs it, so a canvas-library upgrade is never a data-format change.
• 16 node types: begin, llm, knowledge_search, agent, categorize, switch, user_input, message, http, ocr, vision, stt, tts, excel, browser, loop.
• Branching nodes (categorize, switch) route through their own params. switch.otherwise is capped at 4 targets, loop at 25 iterations, labels at 80 characters — all mirrored from the backend schema.
• Client-side validateFlow mirrors the backend's publish-time rules and adds the ones the backend only discovers mid-run: an empty prompt, a branch wired to nothing. A problem carries blocksPublish separately from level, so work in progress can be saved while certain failures still show as errors.
• Params reach the client as Record<string, unknown>. categoriesOf and casesOf check for arrays instead of casting, because a string where a list was expected once took the whole agent screen down.
• Runs are checkpointed on agent_run.state at node boundaries, values only. A run waiting on a person for days survives a deploy, and a crashed run is resumed by a different process.
• usage_ledger.reference = agent-run:{runId}:step:{seq} has a unique index. Preserving seq across attempts is the entire double-charge guard.
• user_input pauses a run for as long as it takes. http refuses private address ranges.

§5Hard decisions

1. 01

   Citations are frozen server-side.

   Rendering is a lookup by index, never a similarity match that can drift to the wrong paragraph. Why not re-match at render time: less code, but a similarity match can land on a neighbouring paragraph, and a wrong citation is worse than none.

2. 02

   Streaming state is local, not in the query cache.

   A setQueryData per token re-renders every subscriber. The stream writes local state; the cache is invalidated once at the end and the server's rows — with citations, model and cost — win.

3. 03

   Poll only while something is in flight.

   The documents list stops the moment every row is ready or failed. Why not a fixed interval: an idle page would keep hitting the API for nothing.

4. 04

   Refusals happen before the stream opens.

   No credits, a model outside the plan, a missing base: a normal 4xx with a message, never an error frame after the UI has already switched to "answering".

5. 05

   Credits are granted by the Stripe webhook, never by the checkout call.

   The grant is keyed on the Stripe object id, so a redelivered event lands on the ledger's unique (kind, reference) and does nothing. Why not grant on checkout: the browser can close before the payment settles, and a webhook can arrive twice.

6. 06

   Auto-reload's single-flight lock is a conditional UPDATE.

   Not read-then-write, so two worker replicas cannot both charge. A declined card switches auto-reload off instead of retrying every five minutes.

7. 07

   One credit = one input token of the baseline model.

   Every model's rate is derived from provider cost and frozen on the usage row with a pricingVersion. A price change never restates history; an old row still says what it cost the day it ran.

§6Result

§7What I'd change

1. Put OCR in the ingestion path. Today a scanned PDF fails with a clear reason, which is honest, but the right answer is to index it.
2. Keep the backend README in step with the code. It still says knowledge bases, chat and agents are not built yet while the modules sit in src/modules.
3. One HTTP client, not two. Chat streaming and upload bypass ky with raw fetch because ky buffers the response, which leaves two code paths that have to agree on auth and errors.