{ "schema": "kynetra.benchmark/v1", "suite": "filtered_vector_search_after_w1", "status": "local_evidence", "origin": { "kind": "measured", "tool": "examples/filtered_vector_bench.rs (primary) + examples/filtered_vector_bench_indexed.rs (supplementary, W1-authored)", "build": "release" }, "environment": { "scope": "local", "hardware": "Cortex-X925, 121 GB RAM", "operating_system": "linux", "arch": "aarch64", "host": { "logical_cpus": 20, "rustc_version": "rustc 1.97.1 (8bab26f4f 2026-07-14)" }, "note": "Host was under background load during this measurement session (load average ~4-9 on a 20-logical-CPU machine, from an unrelated local ollama inference server plus IDE/editor processes) \u2014 absolute latencies here run higher than the quieter baseline capture. Each before/after pair below was re-run back-to-back under matched conditions in the same session specifically to keep the RELATIVE (before vs after) comparison fair despite the elevated absolute numbers; see W1's report for the corroborating clean-host numbers (before~7.3ms / after~7.9ms p99 at 0.1% selectivity for the non-indexed-filter case, and after~0.4ms for the indexed-equality case), which tell the same qualitative story." }, "config": { "vectors": 100000, "dims": 384, "k": 10, "fix_under_test": "Engine::vector_search (crates/core/src/engine.rs): pre-filter id-set now resolved via indexed_filter_ids (O(matches), json_eq index) when the filter shape is index-answerable, else the same O(n) for_each_entity scan as before. The (now known) match count then routes to one of two paths: matches <= FILTERED_EXACT_THRESHOLD (4096) -> exact sequential brute-force cosine over ONLY the matched entities (recall = 1.0, avoids rayon dispatch overhead measured to dominate at this scale on this host); matches > threshold (or unfiltered) -> HnswIndex::search_with_exact_rerank with the candidate pool boosted by 1/selectivity, reranking against the exact f32 embedding instead of the int8-quantized copy.", "filtered_exact_threshold": 4096, "large_filter_overfetch_cap": 50000, "primary_bench_filter_shape": "Filter::Lt('tenant_bucket', threshold) \u2014 W0's official bench filter. This is a scalar-bound (non-equality) predicate, so it is NOT answerable by indexed_filter_ids (which only serves Filter::Eq / And([Kind,Eq])) and always falls back to the O(n) scan. The W1 fix still delivers on RECALL for this shape (routing the resulting small match-set to exact scoring), but cannot remove the scan cost itself here, so latency is roughly a wash vs. baseline for this specific (non-indexed) filter shape \u2014 see 'measurements' below.", "supplementary_bench_filter_shape": "Filter::And([Filter::Kind('doc'), Filter::Eq('tenant_id', ...)]) \u2014 the realistic tenant-scoped-RAG equality shape indexed_filter_ids is built for. See 'supplementary_indexed_equality_filter' below for the full latency + recall win this shape unlocks." }, "kpi_gate": { "description": "W0's baseline_p99_ms_at_0_1pct_selectivity (8.075ms) / target_p99_ms_10x (0.807ms) came from a run predating this session's back-to-back before/after pair; this file's own 'before' column (measured in the same session, same host load, immediately preceding 'after') is the apples-to-apples baseline for the speedup_p99_x figures in 'measurements'.", "recall_at_10_0_1pct_before": 0.01, "recall_at_10_0_1pct_after": 1.0, "recall_target_0_95": true, "recall_target_met": true, "p99_ms_0_1pct_before_non_indexed_filter": 19.445, "p99_ms_0_1pct_after_non_indexed_filter": 24.975, "latency_10x_target_met_non_indexed_filter": false, "latency_10x_target_explanation": "W0's official bench filter (Filter::Lt) is not index-answerable, so both before and after pay the same O(n) for_each_entity scan to resolve the tiny (103-of-100000) match set; the W1 fix cannot remove that scan without adding an external index structure (explicitly out of scope). Recall is still fully fixed for this shape (0.01 -> 1.0).", "p99_ms_0_1pct_before_indexed_equality_filter": 46.079, "p99_ms_0_1pct_after_indexed_equality_filter": 0.285, "latency_10x_target_met_indexed_equality_filter": true, "latency_speedup_indexed_equality_filter_x": 161.681 }, "measurements": [ { "id": "filtered_vector_search/unfiltered_reference", "matches": 100000, "queries": 1000, "before": { "p50_ms": 0.364, "p95_ms": 0.551, "p99_ms": 0.632, "recall_at_10_vs_brute_force": 0.9575 }, "after": { "p50_ms": 0.464, "p95_ms": 0.668, "p99_ms": 0.753, "recall_at_10_vs_brute_force": 1.0 }, "speedup_p99_x": 0.839, "recall_delta": 0.042 }, { "id": "filtered_vector_search/selectivity_0.1pct", "matches": 103, "queries": 1000, "before": { "p50_ms": 9.5, "p95_ms": 17.826, "p99_ms": 19.445, "recall_at_10_vs_brute_force": 0.01 }, "after": { "p50_ms": 21.326, "p95_ms": 23.541, "p99_ms": 24.975, "recall_at_10_vs_brute_force": 1.0 }, "speedup_p99_x": 0.779, "recall_delta": 0.99 }, { "id": "filtered_vector_search/selectivity_1pct", "matches": 988, "queries": 1000, "before": { "p50_ms": 19.502, "p95_ms": 22.116, "p99_ms": 23.258, "recall_at_10_vs_brute_force": 0.0975 }, "after": { "p50_ms": 24.354, "p95_ms": 26.604, "p99_ms": 28.193, "recall_at_10_vs_brute_force": 1.0 }, "speedup_p99_x": 0.825, "recall_delta": 0.902 }, { "id": "filtered_vector_search/selectivity_10pct", "matches": 9806, "queries": 1000, "before": { "p50_ms": 11.809, "p95_ms": 27.15, "p99_ms": 28.738, "recall_at_10_vs_brute_force": 0.879 }, "after": { "p50_ms": 25.372, "p95_ms": 27.335, "p99_ms": 28.496, "recall_at_10_vs_brute_force": 0.88 }, "speedup_p99_x": 1.008, "recall_delta": 0.001 } ], "supplementary_indexed_equality_filter": { "description": "Same corpus/methodology as the primary bench, but tenant scope is expressed as Filter::And([Kind, Eq]) \u2014 an EQUALITY predicate, the shape indexed_filter_ids answers in O(matches) via the json_eq index. This is the realistic 'search only MY org's documents' shape the task's root-cause-#1 fix targets, and demonstrates the full latency win (not just the recall win) once the O(n) scan is actually removed.", "tool": "examples/filtered_vector_bench_indexed.rs (W1-authored, supplementary \u2014 not part of W0's official bench)", "measurements": [ { "id": "filtered_vector_search_indexed_eq/selectivity_0.1pct", "matches": 103, "queries": 1000, "before": { "p50_ms": 27.452, "p95_ms": 44.083, "p99_ms": 46.079, "recall_at_10_vs_brute_force": 0.01 }, "after": { "p50_ms": 0.066, "p95_ms": 0.15, "p99_ms": 0.285, "recall_at_10_vs_brute_force": 1.0 }, "speedup_p99_x": 161.681, "recall_delta": 0.99 }, { "id": "filtered_vector_search_indexed_eq/selectivity_1pct", "matches": 1036, "queries": 1000, "before": { "p50_ms": 12.575, "p95_ms": 37.778, "p99_ms": 44.407, "recall_at_10_vs_brute_force": 0.103 }, "after": { "p50_ms": 0.315, "p95_ms": 1.775, "p99_ms": 2.572, "recall_at_10_vs_brute_force": 1.0 }, "speedup_p99_x": 17.266, "recall_delta": 0.897 }, { "id": "filtered_vector_search_indexed_eq/selectivity_10pct", "matches": 9991, "queries": 1000, "before": { "p50_ms": 31.958, "p95_ms": 37.284, "p99_ms": 54.694, "recall_at_10_vs_brute_force": 0.887 }, "after": { "p50_ms": 2.488, "p95_ms": 3.199, "p99_ms": 4.094, "recall_at_10_vs_brute_force": 0.889 }, "speedup_p99_x": 13.36, "recall_delta": 0.002 } ] } }