Local-first RAG for coding agents

Problem

The problem explored here is how to give a coding agent relevant project context without treating every file or document as part of every request.

A retrieval index offers a separate place to search that material, then return selected passages for the agent to inspect.

Implemented approach

Indexing requires an explicit confirmation argument before the tool calls the indexer; that check isn't an audit of the selected content.

Files, documentation and configured web sources can be selected independently, with chunk size and overlap passed into the collection process.

The documented chunking approach uses tokens and document structure, including headings and paragraphs, rather than treating each document as one indivisible search result.

Collected chunks are embedded with a local model and passed to a Postgres vector index alongside their source type and source reference.

Search uses cosine distance and accepts source-type filters, allowing a request to distinguish files, documentation and web material. Returned passages include source references, giving the reader a way to identify where the selected context came from.

Selected sources → confirmation → chunk collection → local embeddings → vector index → filtered search → referenced passages.

Trade-offs

Freshness and reindexing

The prototype exposes separate indexing and search tools, so adding source material and asking a question are distinct operations.

Local embeddings keep that processing step local, but they also require a model and database to be available in the configured environment. They don't establish that the whole workflow is private, particularly when configured web sources or a separate coding-agent runtime are involved.

Chunking and retrieval

Chunking makes passages the unit of retrieval, while overlap carries some neighboring context at the cost of repeating material in the index. Those choices affect what a search can return; they aren't evidence that a particular chunk size produces better answers.

Limits and current status

This is a developer-tooling prototype, not a claim of a released or audited retrieval service.

The available evidence describes implementation and configuration, not measured retrieval quality, latency or production adoption.

Retrieved material still needs to be assessed as source content, not treated as instructions that override the agent's task.