From query to passage
Retrieval is a commodity. The work worth doing is turning twenty noisy pages into five high-signal, citable, injection-safe passages that fit a token budget.
Plan
Classify intent without an LLM — keyword scoring on word boundaries. Intent picks the freshness contract: a realtime question will not be answered from a week-old page, while stable documentation will.
Ask our own index first
If the local index already covers every content term of the query within one document, the external providers are skipped entirely. The bar is strict because skipping discovery on thin evidence trades correctness for latency.
Fan out, race, and give up on time
Providers run in parallel against a soft budget. Whatever has answered when the budget expires is what gets ranked.
Fetch in tiers
Plain HTTP/2 first (~85% of pages). Escalate to TLS-fingerprint impersonation, then to a remote browser — each tier rate-limited and deadline-gated, because sending every URL to a browser is what kills these systems.
Extract and chunk on structure
Restricted markdown with heading breadcrumbs and byte offsets. Code fences and tables are atomic — half a code block is not an answer.
Rank, diversify, pack
BM25 into a rerank cascade, then maximal marginal relevance so the k passages between them cover the question. Near-identical passages are dropped outright rather than penalised.
Fence and return
Content is wrapped in nonce-bearing untrusted delimiters with a trust-boundary note, and offsets survive every transformation.
Caching, in layers
| Layer | Holds | Effect |
|---|---|---|
| L0 | Local full-text index of everything crawled | Answers with no network at all |
| L1 | Whole responses, keyed by query and request shape | ~15 ms |
| L2 | Extracted markdown per URL | Skips fetch and extraction |
| L3 | Provider result sets | Skips the fan-out |
| L4 | Known-bad URLs | Stops re-fetching what will fail again |
Chunk boundaries and content hashes are computed once at index time, not per request — re-chunking a 200 KB reference page cost 260–710 ms and produced the same answer every time.
Measured
| Cold | Hot | |
|---|---|---|
| Answer containment | 1.000 | 1.000 |
| p50 latency | ~2.0 s | 15 ms |
| Tokens per answer | 1,182 | 1,182 |
| Facts per 1k tokens | 2.06 | 2.06 |
Honest limits
Open-web breadth
The benchmark numbers are 24 questions on four documentation domains deliberately crawled into the index. The index has since grown to roughly 4,000 pages across 295 domains, but that is still a corpus, not the web — on a general question a general index still wins on discovery. That gap is about which URLs you can find, not about ranking.
Discovery depends on a provider
For queries outside the crawled corpus, Oort needs somewhere to get candidate URLs. Several lanes run in parallel and each credential falls back to a spare when a free tier runs out — but with every lane exhausted, Oort answers from its own index or not at all.
No change tracking
The index records when a document was fetched, but does not diff revisions.
Deterministic by default
Field extraction is pattern matching, so it misses things a model would catch. extract_mode: auto adds a model pass — but every value it returns must be quoted verbatim from the source or it is discarded.
Built with
| Layer | Choice |
|---|---|
| API | FastAPI · uvicorn · uvloop · httptools |
| Fetching | httpx (HTTP/2) · curl_cffi · remote browser pool |
| Extraction | selectolax (lexbor) · trafilatura fallback · pypdf |
| Index & state | SQLite FTS5 in WAL, per-thread reader connections |
| Caching | Redis, with an in-process fallback |
| Ranking | BM25 → bi-encoder → cross-encoder (ONNX INT8) |
| This site | Alpine.js · highlight.js · Inter & JetBrains Mono · hand-written CSS, no build step |