The Writing Bed: answering from my own words

Sprout, my digital twin from the last post, knows my CV because I hand it my CV on every request.

But my writing is thousands of words, and it grows every time I publish. I can’t paste all of it into every prompt — that’s slow, expensive, and mostly irrelevant to whatever you asked.

So the Greenhouse has a second resident: the Writing Bed 📚. You ask a question; it answers only from these blog posts, and shows you which post each answer came from.

Under the hood it’s the most useful pattern in applied AI right now — retrieval-augmented generation, or RAG. And it’s far simpler than the acronym suggests.

Let’s build it from nothing.

Pause & recall. You have 50 pages of notes and a question, but you can only show the model a paragraph or two. What’s the hard part? Notice it’s a problem you’d have even without any AI: how do you find the right paragraph?

The intuition: open book, not memorised

Think of two kinds of exam. In a closed-book exam you answer from memory — fast, but you misremember, and you can’t cite anything.

In an open-book exam you first find the relevant page, then answer from what’s in front of you.

A plain LLM is a closed-book student with an enormous, blurry memory. RAG turns it into an open-book one.

The whole pattern is two questions:

1. Retrieval — given a question, which pieces of my text are relevant?
2. Generation — given those pieces, write the answer.

The generation half is just a grounded prompt, exactly like Sprout’s: “answer only from these excerpts.”

The interesting half — the half that makes it work — is retrieval. And retrieval hinges on one idea: turning meaning into geometry.

From first principles: meaning as coordinates

Here’s the move that makes everything click. An embedding model takes a piece of text and returns a long list of numbers — a vector, a point in high-dimensional space.

It’s trained so that texts with similar meaning land near each other, even when they share no words.

“Why did you rebuild your site?” and “the reasons behind the redesign” have almost no words in common — but they point at nearly the same spot.

“How do I bake sourdough?” points somewhere else entirely.

So meaning has become distance. And distance is something a computer can measure trivially.

Once every paragraph of my writing is a point, and your question is a point, “find the relevant text” becomes “find the nearest points.”

That’s it. That’s the engine.

Pause & recall. Why can this match “redesign reasons” to “why did you rebuild” when a plain keyword search (Ctrl-F) can’t? Say it in one sentence before reading on.

(Because keyword search matches strings; embeddings match meaning. The words differ; the coordinates don’t.)

The two phases, concretely

RAG has an indexing phase that happens once (well — once per restart here, more on that shortly). And a query phase that happens on every question.

Indexing: turn the library into a map

None of these steps is bespoke — the splitter, the embedding model and the store are all stock LangChain components, the same toolbox Sprout’s server already pulls in.

So the work isn’t building them. It’s two setup choices, and they’re where beginners lose the most quality:

• Chunking. I don’t embed a whole post as one point. A 1,500-word post covers ten different things, so its single vector would be a mushy average of all of them. Instead I split posts into ~1,500-character chunks, with a little overlap so an idea straddling a boundary isn’t sliced in half. Each chunk becomes its own point — so retrieval returns the relevant paragraph, not the whole essay.
• The vector store. A vector store just holds those vectors and answers one question: “give me the k nearest to this one.” Mine lives in memory — a list of vectors it scans with cosine similarity. For a personal blog that isn’t laziness, it’s right-sizing: with a few dozen chunks, a “real” vector database costs more to set up than it saves. A dedicated one — pgvector, Pinecone, Chroma — earns its keep at thousands of vectors, or when they must survive restarts, be filtered by metadata, or take concurrent writes. None of that is true here, so a plain in-memory scan wins. Scale the tool to the problem, not to the tutorial.

Query: find, then answer

The question is embedded with the same model as the chunks. They have to share a coordinate system, or “nearness” means nothing.

I pull the top four nearest chunks, drop them into a prompt that says “answer only from these,” and let the model write.

Because I tracked which post each chunk came from, I can hand you the sources alongside the answer.

That’s the “with receipts” bit, and it’s not a nicety. Citations let you verify the model didn’t drift — the honest way to ship something that can still occasionally be wrong.

”Cosine similarity” without the fear

Here’s the one piece of maths, demystified. To measure how close two vectors are, I ignore their length and look only at their direction.

Two paragraphs on the same topic point the same way, even if one is longer. Cosine similarity is just “the angle between these two arrows, as a number from -1 to 1” — 1 means identical direction, 0 means unrelated.

Nearest neighbours are simply the smallest angles. Picture two arrows on a page; the dimensions go from 2 to a few thousand, but the intuition doesn’t change.

(A small war story: the first deploy of the Writing Bed crashed because that cosine-similarity step needs the library numpy, and it wasn’t installed. The lesson isn’t Python packaging — it’s that the “one line of maths” is a real dependency doing real work. Nothing here is magic; it’s all arrows and angles somewhere down the stack.)

Where it stops — and why that’s the point

Ask the Writing Bed something I’ve never written about, and it won’t improvise. It says the bed’s freshly planted on that topic and points you at /writing/.

That refusal is the whole value proposition. A tool that answers everything is a tool that answers confidently when wrong.

By fencing it to my actual words and showing sources, “I don’t know” becomes a first-class, trustworthy answer.

This is also the clean line between the Writing Bed and Scout, the third resident. The Writing Bed only ever reads my posts; Scout goes out to the wider world.

Same retrieval instinct, very different blast radius — which is the whole subject of the next post.

Why retrieval — and when it’s the wrong tool

There’s another way to teach a model my writing: fine-tuning — baking the text into the model’s own weights. I didn’t, and the reasons are the case for RAG.

Fine-tuning has to be redone every time I publish. It can’t tell you which post an answer came from. And it still hallucinates, because the knowledge is now a blur in the weights rather than text on the desk.

Retrieval updates the instant I add a file, answers with citations, and costs nothing to “re-train.” For knowledge that keeps changing, looking it up beats memorising it.

“But context windows are huge now — why not paste every post into the prompt?” Because you pay per token on every request, long prompts are slower, and models genuinely lose track in the middle of a giant dump.

Retrieval sends only the few paragraphs that matter. Cheaper, faster, sharper — and it works the same whether I have ten posts or ten thousand.

The honest flip side: for a tiny, fixed set of facts, RAG is over-engineering — just put them in the prompt. Retrieval earns its keep exactly when the knowledge is too big to paste and changes too often to bake in.

Rebuild it in your head

Cut your documents into paragraph-sized chunks. Turn each into a vector with an embedding model, so meaning becomes position. Store the vectors. When a question arrives, turn it into a vector too, grab the few nearest chunks, and ask the model to answer using only those — then show which documents they came from.

That is a working RAG system. Everything else is tuning.

Check yourself.

1. What does an embedding convert text into, and what’s the useful property of the result?
2. Why chunk a post instead of embedding it whole?
3. Why must the question and the chunks be embedded by the same model?
4. What’s the purpose of returning sources — beyond looking tidy?

Answers you can’t quite reach are a map of what to reread. Retrieve first; confirm second.

Make it yours

I teach this pattern constantly, and the exercise I set is never “explain RAG.”

It’s this: point at something in your own world where the right answer already exists in writing, but nobody can find it fast enough. A policy handbook, a sprawling wiki, years of support tickets, a research corpus. Then design the retrieval for that.

The technique is the easy, portable part. The judgement is the valuable part — what to index, what to fence out, when “I don’t know” is the honest answer.

That’s yours to bring, and it’s where the real impact lives.

Try the Writing Bed on this very post over in the Greenhouse, then meet Scout next. 📚