RAG interview questions

A model's knowledge is frozen at its training cutoff and never included your private data — so it can't answer about recent events or your own docs, and when pushed it tends to hallucinate a confident guess. RAG (retrieval-augmented generation) retrieves the most relevant real passages at question time and pastes them into the prompt, so the model answers from the page in front of it instead of from memory. Grounding it on real text (and telling it to answer only from that, or say "I don't know") sharply cuts made-up answers and lets it cite sources.

Offline: split each document into chunks, embed each chunk into a vector, and store them in a vector store — done once, re-run when docs change. Online: embed the incoming question the same way; the retriever compares that vector to the stored ones (usually by cosine similarity) and returns the top-k nearest chunks. Those chunks are pasted into the prompt as context, and the generative model writes the final answer grounded in them. The retriever's job is the make-or-break middle step: it decides which passages the model gets to read.

Because RAG never asks the model to remember that information — it supplies it at question time. The model's weights stay fixed; the private or fresh knowledge lives in your document index, which you can update the moment a file changes. Re-index the new doc and the same day the assistant is current — no re-training, no waiting for a new model. That's why RAG is the go-to for "know my internal docs" and "answer about things that happened after the cutoff".