Relationship-based vs semantic retrieval for AI project context

Relationship-based vs semantic retrieval for AI project context is a choice about how an agent's context gets picked: by following links someone declared (this task depends on that one, this decision applies to this module), or by searching for text that looks similar to the task. Relationship-based retrieval is predictable and explainable: you can say exactly why each item was included, and the same task gets the same context every time. Its weakness is that it only finds what was linked. Semantic retrieval, usually embeddings in a vector index, finds material nobody thought to link, but it can return plausible and wrong items, ranks stale documents as readily as current ones, and makes omissions hard to explain. Keyword search sits in between: exact and cheap, blind to synonyms. For agent work on a real project, the strongest setup uses declared relationships as the backbone and search as a way to suggest links a person confirms. Sizing a context bundle is covered in bounded task context; this page is about how the items get chosen.

How relationship-based retrieval works

In relationship-based retrieval, the project keeps explicit links: task ENG-12 depends on ENG-9, decision D-4 applies to the payments module, billing.py is the subject of ENG-12. When ENG-12 is launched, the context is built by walking those links: the task itself, its dependencies and their outcomes, the decisions attached to it, the files it names and any notes linked to them. A size limit decides how far to walk and what gets cut.

Its strengths are the ones you want when an agent is about to change code. It is deterministic: the same task produces the same bundle, so a bad result can be reproduced. It is explainable: every item has a reason you can read. It respects permissions naturally, because links can be filtered by what the task is allowed to see. And omissions can be reported precisely: three linked notes were dropped for size.

Its weakness is equally clear. It finds nothing that was not linked. If a module imports a helper and nobody recorded that relationship, the module is invisible. The page on declared vs inferred dependencies covers how to decide what must be declared.

How semantic retrieval works

Semantic retrieval turns documents and code into vectors and returns the items closest to the query. Ask about checkout rounding and it may find a note titled cents handling in exports, even though nobody linked the two. That is its real strength: recall across material whose connection was never written down, and tolerance for different wording.

Its weaknesses show up in agent work. Similar is not relevant: an old design note that discusses the same function in a way the team rejected scores as well as the current decision. Similarity has no sense of time, so stale project memory gets retrieved with full confidence. Results move when the index, embedding model or chunking changes, so the same task can get different context next week, which makes failures hard to reproduce. And when something important is missing, there is no clean explanation of why; it simply ranked lower.

Plain keyword search behaves differently again. It is exact and explainable (the term matched), which makes it good for identifiers like function names and error strings. It misses synonyms and paraphrase, and it can return every file that mentions a common word. Many teams already get most of their value from it.

A concrete miss on each side

Consider a shared helper cents() used by a checkout module and a CSV export. A change switches it from rounding to truncation. Checkout was supposed to change; export was not.

With relationship-based retrieval, the export appears only if its use of the helper was declared. In a regression probe on a development build of BRAIN FLUID, the export's import was deliberately left undeclared, and the declared-relationship impact query nominated the checkout task and its parent, not the export, while reporting its coverage as incomplete [D6]. The export check then failed, returning 123 where 124 was expected. The method did what it promised and missed what nobody recorded.

With semantic retrieval, a search for the helper might well surface the export file, since it mentions cents. It might also surface four older notes about currency formatting, one of them describing a rule the team dropped, and rank one of those above the export. The agent cannot tell which is current.

Neither method catches everything. The difference is how the miss looks: a stated gap in a declared graph, or a silent ranking decision. For changes with a real blast radius, a stated gap is easier to act on.

Combining both without losing predictability

The practical design uses each method for the job it is good at.

Make declared relationships the source of what the agent is given. Dependencies, decisions and the files a task is about are linked on purpose, and the bundle is built from those links within a fixed budget, with omissions listed.

Use search to propose, not to inject. When a task is created or changed, run keyword or semantic search for related material and show the hits to a person. Anything useful becomes a declared link, so it will be included next time for a reason anyone can read. Anything irrelevant is discarded without polluting the context.

Use code analysis for code relationships. Imports and call sites can be found by static tools, which answer what uses this function more reliably than either similarity or memory. Record what they find as links.

Keep freshness visible. Whatever the selection method, mark when an item was last confirmed and whether its source files changed since.

The result keeps the predictability of relationships, recovers much of the recall of search, and leaves a person deciding what counts as relevant, which is where mistakes in context selection are easiest to fix.

QUESTIONS

Is RAG a good way to give a coding agent project context?

It helps with recall, especially for large document sets where nobody has linked related material. On its own it is risky for agent context, because similar text is not always current or relevant and results shift when the index changes. Use it to suggest material a person confirms, and build the agent's actual context from declared links.

What is the difference between keyword search and semantic search for code?

Keyword search matches exact terms, so it is precise for function names, error messages and identifiers, and easy to explain. Semantic search matches meaning through vector similarity, so it finds paraphrases and related concepts but can return plausible items that are wrong or outdated. For code, keyword search plus static analysis of imports often beats either alone.

How do I find code relationships nobody declared?

Use static analysis: search for imports and call sites of the function you are changing, or use your editor's find-references tooling. Run it before an agent changes shared code, record what it finds as explicit links, and treat every module it reports as part of the change's review. Similarity search is a weaker substitute for this job.

RELATED GUIDES

• How to find and organize work across projects

RELATED HARNESS PAGES

• Bounded task context Bounded task context: give an agent the slice of a project one task needs, chosen by relationship, capped in size, with omissions stated.
• Declared vs inferred dependencies in change-impact analysis Declared vs inferred dependencies in change-impact analysis: what task links and import graphs each catch, where each is blind, how to combine them.
• Stale AI project memory Stale AI project memory: how notes and evidence an agent reads fall out of date, how to detect it, and what a freshness check cannot tell you.
• The blast radius of AI code changes The blast radius of AI code changes: what one agent edit can break elsewhere, a worked rounding example, and how a reviewer finds affected features.