eval/*.baseline.json 1.00

Measured, not guessed

This page is generated from the evaluation harness's own committed reports — the same JSON oce eval writes and compares baselines against. If a number here is stale, so is the repo. Re-run any configuration yourself; the gold sets and baselines ship in the eval/ directory.

Open Context Engine's own gold set (44 cases)

configuration	mode	cases	recall@10	MRR	nDCG@10	hit-rate	ctx-recall	latency	measured
keyword-only (BM25)	keyword-only	44	0.886	0.676	0.716	0.909	0.864	8 ms	2026-10-26
hybrid — voyage-code-3, equal-weight fusion	hybrid	44	0.943	0.759	0.792	0.955	0.932	268 ms	2026-10-26
hybrid — local MiniLM, adaptive fusion	hybrid	44	0.932	0.690	0.738	0.955	0.909	38 ms	2026-10-27

Flask 3.1.2 — a codebase OCE had never seen (27 cases)

configuration	mode	cases	recall@10	MRR	nDCG@10	hit-rate	ctx-recall	latency	measured
keyword-only (BM25)	keyword-only	27	0.926	0.605	0.685	0.926	0.926	6 ms	2026-10-25
hybrid — voyage-code-3, equal-weight fusion	hybrid	27	0.889	0.604	0.676	0.889	0.926	288 ms	2026-10-26
hybrid — local MiniLM, adaptive fusion	hybrid	27	0.963	0.757	0.808	0.963	0.963	55 ms	2026-10-27

How to read this

keyword-only runs use BM25 alone (no embedding key, no vectors). hybrid fuses BM25 with vector similarity; since Oct 2026 the fusion weights adapt to query shape (definition-shaped queries lean BM25, paraphrases lean semantic) — the voyage rows above predate that change and are kept as the equal-weight reference. ctx-recall checks whether the gold file made it into the packed context an LLM actually receives. Gold sets are file-granular: a case passes when the gold file ranks in the top-10 unique files. Flask is an external, never-seen codebase — the honest test of a ranking change.