Understand the retrieval, chunking, and embedding processes, and how to evaluate a well-founded query system.
RAG is a method of finding relevant data segments and putting them into context so that the model can generate a response. This method allows the system to use its own knowledge without having to retrain the model.
Basic RAG process
The document is cleaned and divided into segments. Each segment is represented so that it can be searched by meaning. When the user asks a question, the system retrieves the most recent segments, matches them to the prompt, and asks the model to respond based on them.
Chunking determines quality.
Paragraphs that are too short lose context; paragraphs that are too long contain too much noise. Divide the text into sections using natural structures like headings, subheadings, and paragraphs, and include metadata about the source, update date, and access permissions.
Don't just evaluate the answers.
Break the evaluation down into two parts: whether retrieval finds the correct segment and whether generation closely follows the found segment. If retrieval is incorrect, changing the prompt usually won't solve the root of the problem.
Implementation checklist
- Only access documents that users are permitted to view. - Display sources or excerpts to support answers. - Provide alternative solutions when insufficient data is found. - Track failed questions to improve the document repository.
