Start with the question and the evidence
Keep the original question beside the retrieved passages. Before judging the generated answer, ask whether those passages actually contain the information needed to respond. If they do not, changing the wording of the prompt is unlikely to fix the underlying problem.
Inspect the boundaries
A document is rarely retrieved as one complete object. Its boundaries matter: a definition separated from its example, or a table separated from its heading, can be difficult to interpret. Looking at the actual chunks is more informative than treating a chunk-size setting as a quality score.
Change one thing at a time
Compare a baseline with a single change: a different split, a different retrieval strategy, or a reranking step. Keep the questions fixed. Record which relevant passages move into or out of the results. The purpose is to understand the tradeoff, including the cases that become worse.
Make the intermediate state a learning tool
A visual interface can put those decisions within reach. RAG Playground is a place to explore embeddings, retrieval, reranking, and evaluation in the browser. The interesting outcome is not just a better answer; it is a clearer explanation of how the context arrived there.