James BruntonandGitHub
672e81d286
# Description of Changes Adds storage in the database for full document content alongside the RAG content (and changes the service to `DocumentService` instead of `RagService`). Then adds a generic capability that should be usable by any agent (currently just used by the Question Agent) which allows the agent to pull out the full contents of the doc, chunks it into various sections that will fit in the context window, and then processes them in parallel to create an intermediate result, and then processes the intermediate result into a final answer. It will re-chunk as many times as necessary to get the content small enough for the actual answer to be analysed (I've tested on PDFs ~3500 pages long, which is well above the context limit and requires maybe 3 rounds of compression to get an answer). The new full doc analysis stuff is heavier than the RAG lookup so both remain. The agents should use RAG for targeted info and the chunked reasoner for info that requires reading the full doc.
Document Storage
The documents package owns all stored content for a document under a single
collection (file id):
- Vector chunks — small, embedded chunks for RAG-style retrieval.
- Ordered pages — the original page text retained in document order, used for whole-document reading.
Both representations are populated by a single ingest() call and removed
together by delete_collection().
Adding RAG to an Agent
from pydantic_ai import Agent
from stirling.services import AppRuntime
class MyAgent:
def __init__(self, runtime: AppRuntime) -> None:
rag = runtime.rag_capability
self.agent = Agent(
model=runtime.smart_model,
system_prompt="Your prompt here...",
instructions=rag.instructions,
toolsets=[rag.toolset],
)
That's it. The agent gets a search_knowledge tool it can call autonomously.
Scoping to Specific Collections
Collections are named buckets of indexed documents - think folders. By default
an agent searches everything in the store. Pass collections= to restrict it
to only the docs indexed under those names.
from stirling.documents import RagCapability
# Only searches docs indexed under "company-docs"
scoped = RagCapability(runtime.documents, collections=["company-docs"], top_k=3)
# Searches multiple collections
multi = RagCapability(runtime.documents, collections=["company-docs", "product-specs"])
# No collections arg = searches all collections in the store
everything = RagCapability(runtime.documents)
Config
Non-secret defaults live in the committed engine/.env:
STIRLING_RAG_BACKEND=sqlite # or "pgvector"
STIRLING_RAG_EMBEDDING_MODEL=voyageai:voyage-4
STIRLING_RAG_STORE_PATH=data/rag.db # used when backend=sqlite
STIRLING_RAG_PGVECTOR_DSN= # used when backend=pgvector
STIRLING_RAG_CHUNK_SIZE=512
STIRLING_RAG_CHUNK_OVERLAP=64
STIRLING_RAG_TOP_K=5
Provider credentials (and any local overrides) go in the uncommitted
engine/.env.local:
VOYAGE_API_KEY=your-key
Backends
sqlite - Embedded sqlite-vec. Single .db file, zero ops. Ideal for dev
and self-hosted deployments.
pgvector - External PostgreSQL with the vector extension. Point
STIRLING_RAG_PGVECTOR_DSN at your Postgres instance.
Both backends implement the same DocumentStore interface, so agents and the
service work identically regardless of which you pick.
For a self-hosted embedding server (e.g. Ollama, TEI, vLLM) set the model string accordingly and point at the server via its native env var:
# Ollama running on another machine
STIRLING_RAG_EMBEDDING_MODEL=ollama:nomic-embed-text
OLLAMA_HOST=http://192.168.1.50:11434
# Any OpenAI-compatible embedding server
STIRLING_RAG_EMBEDDING_MODEL=openai:my-model
OPENAI_BASE_URL=http://192.168.1.50:8080/v1
API Endpoints
| Method | Endpoint | Purpose |
|---|---|---|
| POST | /api/v1/documents |
Replace-ingest a document's pages |
| DELETE | /api/v1/documents/{document_id} |
Delete a document's stored content |