The Writing Bed: answering from my own words

Sprout, the digital twin from the last post, knows Usama’s CV because the CV is handed to it on every request.

But the writing is thousands of words, and it grows with every new post. You can’t paste all of it into every prompt — that’s slow, expensive, and mostly irrelevant to whatever you asked.

So the Greenhouse has a second resident: the Writing Bed 📚. You ask a question; it answers only from these blog posts, and shows you which post each answer came from.

Under the hood it’s the most useful pattern in applied AI right now — retrieval-augmented generation, or RAG. And it’s far simpler than the acronym suggests.

In one breath: Retrieval-augmented generation (RAG) means finding the few passages most relevant to a question, by comparing meaning as vectors. You hand only those to the model. So it answers from your documents, with sources, instead of from its blurry memory.

Let’s build it from nothing.

Pause & recall. You have 50 pages of notes and a question, but you can only show the model a paragraph or two. What’s the hard part? Notice it’s a problem you’d have even without any AI: how do you find the right paragraph?

The intuition: open book, not memorised

Think of two kinds of exam. In a closed-book exam you answer from memory — fast, but you misremember, and you can’t cite anything.

In an open-book exam you first find the relevant page, then answer from what’s in front of you.

A plain LLM is a closed-book student with an enormous, blurry memory. RAG turns it into an open-book one.

The whole pattern is two questions:

1. Retrieval — given a question, which pieces of the text are relevant?
2. Generation — given those pieces, write the answer.

The generation half is just a grounded prompt, exactly like Sprout’s: “answer only from these excerpts.”

The interesting half — the half that makes it work — is retrieval. And retrieval hinges on one idea: turning meaning into geometry.

From first principles: meaning as coordinates

Here’s the move that makes everything click. An embedding model takes a piece of text and returns a long list of numbers — a vector, a point in high-dimensional space.

It’s trained so that texts with similar meaning land near each other, even when they share no words.

“Why did you rebuild your site?” and “the reasons behind the redesign” have almost no words in common — but they point at nearly the same spot.

“How do you bake sourdough?” points somewhere else entirely.

So meaning has become distance. And distance is something a computer can measure trivially.

Once every paragraph of the writing is a point, and your question is a point, “find the relevant text” becomes “find the nearest points.”

That’s it. That’s the engine.

Pause & recall. Why can this match “redesign reasons” to “why did you rebuild” when a plain keyword search (Ctrl-F) can’t? Say it in one sentence before reading on.

Reveal the answer

Because keyword search matches strings; embeddings match meaning. The words differ; the coordinates don’t.

The two phases, concretely

RAG has an indexing phase that happens once (well — once per restart here, more on that shortly). And a query phase that happens on every question.

Indexing: turn the library into a map

None of these steps is bespoke — the splitter, the embedding model and the store are all stock LangChain components, the same toolbox Sprout’s server already pulls in.

So the work isn’t building them. It’s two setup choices, and they’re where beginners lose the most quality:

• Chunking. A whole post isn’t embedded as one point. A 1,500-word post covers ten different things, so its single vector would be a mushy average of all of them. Instead posts are split into ~1,500-character chunks, with a little overlap so an idea straddling a boundary isn’t sliced in half. Each chunk becomes its own point — so retrieval returns the relevant paragraph, not the whole essay. It’s a bigger decision than it looks: where you cut the text turns out to affect retrieval about as much as which embedding model you pick.
• The vector store. A vector store just holds those vectors and answers one question: “give me the k nearest to this one.” Mine lives in memory — a list of vectors it scans with cosine similarity. For a personal blog that isn’t cutting corners, it’s right-sizing: with a few dozen chunks, a “real” vector database costs more to set up than it saves. A dedicated one — pgvector, Pinecone, Chroma — earns its keep at thousands of vectors, or when they must survive restarts, be filtered by metadata, or take concurrent writes. None of that is true here, so a plain in-memory scan wins. Scale the tool to the problem, not to the tutorial.

Query: find, then answer

The question is embedded with the same model as the chunks. They have to share a coordinate system, or “nearness” means nothing.

The top four nearest chunks go into a prompt that says “answer only from these,” and the model writes from those.

Because each chunk carries the post it came from, the answer arrives with its sources.

That’s the “with receipts” bit, and it’s not a nicety. Citations let you verify the model didn’t drift — check any answer against the post it came from, in one click.

”Cosine similarity” without the fear

Here’s the one piece of maths, demystified. To measure how close two vectors are, ignore their length and look only at their direction.

Two paragraphs on the same topic point the same way, even if one is longer. Cosine similarity is just “the angle between these two arrows, as a number from -1 to 1” — 1 means identical direction, 0 means unrelated.

Nearest neighbours are simply the smallest angles. Picture two arrows on a page; the dimensions go from 2 to a few thousand, but the intuition doesn’t change.

(A small war story: the first deploy of the Writing Bed crashed because that cosine-similarity step needs the library numpy, and it wasn’t installed. The lesson isn’t Python packaging — it’s that the “one line of maths” is a real dependency doing real work. Nothing here is magic; it’s all arrows and angles somewhere down the stack.)

Where it stops — and why that’s the point

Ask the Writing Bed about something that was never written about, and it won’t improvise. It says the bed’s freshly planted on that topic and points you at /learning/.

That refusal is the whole value proposition. A tool that answers everything is a tool that answers confidently when wrong.

By fencing it to the actual posts and showing sources, “I don’t know” becomes a first-class, trustworthy answer.

This is also the clean line between the Writing Bed and Scout, the third resident. The Writing Bed only ever reads these posts; Scout goes out to the wider world.

Same retrieval instinct, very different blast radius — which is the whole subject of the next post.

Why retrieval — and when it’s the wrong tool

There’s another way to teach a model a body of writing: fine-tuning — baking the text into the model’s own weights. It wasn’t used here, and the reasons are the case for RAG.

Fine-tuning has to be redone every time a post goes up. It can’t tell you which post an answer came from. And it still hallucinates, because the knowledge is now a blur in the weights rather than text on the desk.

Retrieval updates the instant a file is added, answers with citations, and costs nothing to “re-train.” For knowledge that keeps changing, looking it up beats memorising it.

“But context windows are huge now — why not paste every post into the prompt?” Because you pay per token on every request, long prompts are slower, and models genuinely lose track in the middle of a giant dump.

Retrieval sends only the few paragraphs that matter. Lower cost, lower latency, sharper answers — and it works the same at ten posts or ten thousand.

The flip side: for a tiny, fixed set of facts, RAG is over-engineering — just put them in the prompt. Retrieval earns its keep exactly when the knowledge is too big to paste and changes too often to bake in.

That choice — retrieve, fine-tune, or paste everything into a long context — comes up on every project. There’s a full decision framework for it: RAG vs fine-tuning vs long context, and how to choose.

Rebuild it in your head

Cut your documents into paragraph-sized chunks. Turn each into a vector with an embedding model, so meaning becomes position. Store the vectors. When a question arrives, turn it into a vector too, grab the few nearest chunks, and ask the model to answer using only those — then show which documents they came from.

That is a working RAG system. Everything else is tuning.

Check yourself.

1. What does an embedding convert text into, and what’s the useful property of the result?
2. Why chunk a post instead of embedding it whole?
3. Why must the question and the chunks be embedded by the same model?
4. What’s the purpose of returning sources — beyond looking tidy?

Answers you can’t quite reach are a map of what to reread. Retrieve first; confirm second.

Make it yours

The useful exercise here is never “explain RAG.”

It’s this: point at something in your own world where the right answer already exists in writing, but nobody can find it fast enough. A policy handbook, a sprawling wiki, years of support tickets, a research corpus. Then design the retrieval for that.

Chunking and embedding that corpus is the part a tutorial can teach you. What to index, what to fence out, and when “I don’t know” is the right answer are the parts only you can decide.

Try the Writing Bed on this very post over in the Greenhouse, then meet Scout next. 📚