<!-- Generated from the same source as https://ntinno.net/notes/making-retrieval-visible/ -->

[Notes](/notes/)/AI & systems

AI & systems / WORKING NOTE

# Make retrieval visible before making it clever

A convincing answer is not a complete explanation of a retrieval system. The useful questions start one step earlier: what did the system retrieve, and why?

3 min read · Open to revision

IN THIS NOTE

[Start with the question and the evidence](#section-1)[Inspect the boundaries](#section-2)[Change one thing at a time](#section-3)[Make the intermediate state a learning tool](#section-4)

## Start with the question and the evidence

Keep the original question beside the retrieved passages. Before judging the generated answer, ask whether those passages actually contain the information needed to respond. If they do not, changing the wording of the prompt is unlikely to fix the underlying problem.

## Inspect the boundaries

A document is rarely retrieved as one complete object. Its boundaries matter: a definition separated from its example, or a table separated from its heading, can be difficult to interpret. Looking at the actual chunks is more informative than treating a chunk-size setting as a quality score.

## Change one thing at a time

Compare a baseline with a single change: a different split, a different retrieval strategy, or a reranking step. Keep the questions fixed. Record which relevant passages move into or out of the results. The purpose is to understand the tradeoff, including the cases that become worse.

## Make the intermediate state a learning tool

A visual interface can put those decisions within reach. RAG Playground is a place to explore embeddings, retrieval, reranking, and evaluation in the browser. The interesting outcome is not just a better answer; it is a clearer explanation of how the context arrived there.

CONNECTED PROJECT

[RAG Playground ](/work/rag-playground/)

FOLLOW THE CONNECTIONS

## A little further down the path.

Community & learning · Working note

### [Learn in small loops. Share what changes.](/notes/learning-through-small-experiments/)

A simple practice for turning an unfamiliar tool into an experiment someone else can learn from.

3 min read

App development · Working note

### [Privacy starts at the architecture sketch](/notes/privacy-as-a-design-constraint/)

Work out what needs to leave the device before deciding what the interface should promise.

3 min read
