{
  "schema": "kynetra.benchmark/v1",
  "suite": "hnsw_int8_quantization",
  "status": "local_evidence",
  "origin": {
    "kind": "measured",
    "tool": "examples/vector_scale_bench.rs",
    "build": "release"
  },
  "environment": {
    "scope": "local",
    "hardware": "Cortex-X925, 121 GB RAM",
    "operating_system": "linux",
    "arch": "aarch64",
    "host": {
      "logical_cpus": 20,
      "rustc_version": "rustc 1.97.1 (8bab26f4f 2026-07-14)",
      "uname": "Linux spark-417d 6.17.0-1026-nvidia #26-Ubuntu SMP PREEMPT_DYNAMIC Thu Jun 25 00:57:17 UTC 2026 aarch64 aarch64 aarch64 GNU/Linux"
    }
  },
  "config": {
    "dims": 384,
    "k": 10,
    "corpus": "1024-center Gaussian mixture, sigma 0.15 (models real embedding cluster structure), same generator as the original vector_scale_bench.rs baseline",
    "hnsw_params": "ef_construction=200 ef_search=100 (instant-distance, unchanged from the pre-quantization baseline)",
    "scale_note": "100k before/after A-B pair: N temporarily reduced from the committed 1,000,000 to 100,000 in examples/vector_scale_bench.rs for a same-host measurement (git diff on that file is empty in the final state -- no reduced constant was committed). 1M confirmation run: same code, N reverted to the committed 1,000,000; this run is AFTER-only (int8) -- a same-host 1M BEFORE (f32) run was not taken, given its build time (the 100k f32 build alone took 16.6s here; 1M f32 was expected to take on the order of 10+ minutes) was not warranted once the 100k pair already demonstrated the memory/latency deltas and the 1M run's real purpose -- checking the recall gate at the declared 1M target scale -- only needed the AFTER number.",
    "methodology": "BEFORE = this branch's pre-quantization crates/core/src/personalities/vector_hnsw.rs (raw Vec<f32> Point payload), captured via `git stash` of only vector_hnsw.rs/vector.rs. AFTER = the same file post-quantization (QPoint int8 payload, plus the search_with_exact_rerank hook described below). Both built with `cargo build -p kynetra-core --release` (opt-level=3, lto=thin, codegen-units=1) and run back-to-back on the same otherwise-idle host, same corpus seed (Lcg(2026) for the corpus, Lcg(7) for queries).",
    "quantization_scheme": "L2-normalize each vector, then symmetric per-vector int8 scalar quantization: scale = max(|normalized component|) / 127, q_i = round(normalized_i / scale) clamped to [-127, 127]. cosine_similarity(a,b) ~= dot_i8(qa, qb) * scale_a * scale_b, computed with an exact i32 accumulator (int8 x int8 cannot overflow i32). Symmetric (not asymmetric) quantization was chosen because cosine similarity has no natural zero-point asymmetry to exploit. Per-vector (not global) scale was chosen so every vector uses the full int8 range regardless of its own component magnitude distribution. See crates/core/src/personalities/vector_hnsw.rs module docs ('Quantization' section) for the full derivation.",
    "rerank_option_a_dequantize": "search() reranks graph hits as cosine(raw_f32_query, dequantize(candidate_QPoint)) instead of trusting the graph's own quantized-query-vs-quantized-node distance. This is a real precision gain (not a no-op) because the query is transient and never stored quantized, so keeping it exact for the final score costs nothing -- closer to asymmetric distance computation than pure symmetric int8 comparison. It fixed a real regression on the existing hnsw_recall_matches_brute_force_top1 integration test (see 'integration_test_regression_and_fix' below) and holds the recall@10 >= 0.95 gate at 100k vectors, but measurement below shows it falls short at 1M vectors on this clustered corpus.",
    "rerank_option_b_exact_hook": "search_with_exact_rerank(query, k, pre_filter_ids, exact_lookup) is a NEW, ADDITIVE public method (search()'s existing signature and behavior are completely unchanged) that reranks graph hits against an exact f32 embedding returned by a caller-supplied exact_lookup(&EntityId) -> Option<Vec<f32>> closure, falling back to dequantize-rerank when the lookup misses. This removes the stored candidate's own one-sided quantization error that dequantize-rerank cannot reach. Unit-tested (search_with_exact_rerank_uses_the_supplied_exact_vector, search_with_exact_rerank_falls_back_to_dequantize_when_lookup_misses) to confirm it actually switches scoring sources and falls back safely. Wiring exact_lookup up to Engine's entity store in Engine::vector_search (crates/core/src/engine.rs) is explicitly OUT OF SCOPE for this change (engine.rs is off-limits) and is a hand-off item -- see 'recall_gate_at_scale' below for why it is needed for the 1M target.",
    "simd": "A `wide`-crate SIMD int8 dot product (i8x16 -> widen to i16x16 -> .dot() -> i32x8) was implemented and benchmarked against a plain `a.iter().zip(b).map(|(x,y)| x as i32 * y as i32).sum()` iterator form on this host. The plain iterator form measured ~14ns/op for a 384-dim dot product over 200k calls vs ~35-146ns/op for hand-unrolled or `wide`-based variants (LLVM's autovectorizer did a better job than the manual SIMD path here). `wide` was NOT added as a dependency as a result -- crates/core/Cargo.toml is unchanged.",
    "over_fetch_tuning": "The graph-hit over-fetch pool (`fetch` in HnswIndex::search_inner) was tuned against the small/low-dim hnsw_recall_matches_brute_force_top1 test to a modest `k + 32` floor. Widening this pool all the way up to `ef_search` (and separately, raising `ef_search`/`ef_construction` by 4x each) was tried and measured at both 100k and 1M scale: recall was BIT-FOR-BIT IDENTICAL across all these configurations, while p99 query latency roughly tripled. This rules out graph reachability / exploration depth as the cause of the recall gap at scale and pins it on per-candidate quantization precision within an already-sufficient pool -- see 'recall_gate_at_scale'."
  },
  "kpi_gate": {
    "description": "OPT-S21 (memory ceiling, 1M x 384-dim) + OPT-S12 (query latency): int8 quantization must cut the HnswIndex's own memory footprint towards 4x while holding recall@10 >= 0.95 on the clustered corpus and not regressing query latency.",
    "index_owned_memory_reduction_x": 3.96,
    "recall_at_10_meets_0_95_gate_at_100k": true,
    "recall_at_10_meets_0_95_gate_at_1m_via_plain_search": false,
    "recall_at_10_meets_0_95_gate_at_1m_via_search_with_exact_rerank": "not measured end-to-end (requires wiring exact_lookup into Engine::vector_search, out of scope for this change -- see rerank_option_b_exact_hook); the underlying scoring-precision fix is unit-tested and does eliminate the specific error this gap traces to",
    "p99_latency_regressed": false,
    "met": "partially -- memory and latency targets met at both scales measured; recall gate met at 100k, missed at 1M with the default search() path (see 'recall_gate_at_scale')"
  },
  "measurements": [
    {
      "id": "bytes_per_vector_index_payload",
      "before_f32": 1536,
      "after_int8_plus_scale": 388,
      "reduction_x": 3.959,
      "note": "384 dims * 4 bytes (f32) vs 384 dims * 1 byte (i8) + 4 bytes (per-vector f32 scale). Exact arithmetic, scale-independent -- this is the number the ~4x memory claim rests on."
    },
    {
      "id": "index_owned_memory_at_1m_vectors_projected",
      "before_mb": 2929.7,
      "after_mb": 740.1,
      "reduction_x": 3.959,
      "note": "HnswIndex holds TWO owned copies of the payload (the sealed graph's own Point/QPoint storage, and inner.vectors -- the rebuild/compaction source of truth covering graph union delta). Both are quantized, so the index-owned reduction equals the per-vector byte ratio above. Projected linearly from the exact per-vector bytes rather than isolated from harness memory at 1M (see full_process_rss_at_100k_reduction for why isolating index-only RSS from full-process RSS requires care, and full_process_rss_at_1m_after for the directly-measured full-process number at the target scale)."
    },
    {
      "id": "full_process_rss_at_100k_before",
      "value_mb": 656,
      "graph_build_secs": 16.611,
      "recall_at_10_vs_brute_force": 1.0,
      "query_graph_only_ms": { "p50": 0.167, "p95": 0.331, "p99": 0.427, "queries": 1000 },
      "insert_to_searchable_ms": 0.249,
      "query_with_10k_delta_ms": { "p50": 2.141, "p99": 4.264 }
    },
    {
      "id": "full_process_rss_at_100k_after",
      "value_mb": 368,
      "graph_build_secs": 4.36,
      "recall_at_10_vs_brute_force": 0.958,
      "query_graph_only_ms": { "p50": 0.129, "p95": 0.221, "p99": 0.313, "queries": 1000 },
      "insert_to_searchable_ms": 0.165,
      "query_with_10k_delta_ms": { "p50": 2.25, "p99": 4.422 }
    },
    {
      "id": "full_process_rss_at_100k_reduction",
      "reduction_x": 1.783,
      "note": "656MB -> 368MB measured by vector_scale_bench.rs's own process RSS, which is LOWER than the 3.96x index-owned figure above because this bench harness keeps its own separate f32 ground-truth corpus copy alive for brute-force recall verification (outside HnswIndex, unaffected by quantization) plus the HNSW graph's structural edge-list overhead (also unaffected -- only the coordinate payload shrank, not the M=32 neighbour-list topology, per instant-distance's internal constant). Both are held constant by the harness/library, not by this change, so they dilute the visible whole-process ratio. The 3.96x figure is the number that reflects what this task's constrained edit (vector_hnsw.rs only) actually controls."
    },
    {
      "id": "full_process_rss_at_1m_after",
      "value_mb": 3636,
      "graph_build_secs": 66.414,
      "recall_at_10_vs_brute_force": 0.9245,
      "query_graph_only_ms": { "p50": 0.426, "p95": 0.584, "p99": 0.659, "queries": 1000 },
      "insert_to_searchable_ms": 0.399,
      "query_with_10k_delta_ms": { "p50": 2.499, "p99": 4.921 },
      "note": "AFTER-only at the declared 1M target scale (see scale_note for why no same-host 1M BEFORE run was taken). For reference, the branch's separately-published pre-quantization baseline (benchmarks/evidence/2026-07-12-vector-scale.json) measured 3736 MB / recall 0.999 at 1M vectors, but on DIFFERENT hardware (Apple M5 vs this Linux/Cortex-X925 host) -- not a valid same-host before/after comparison, shown only for scale context."
    },
    {
      "id": "recall_gate_at_scale",
      "at_100k_points_per_cluster_98": 0.958,
      "at_1m_points_per_cluster_977": 0.9245,
      "gate": 0.95,
      "note": "Recall degrades as cluster density grows with corpus size (cluster COUNT is fixed at 1024, so points/cluster grows with N): at 1M vectors, distinguishing the true top-10 from ~977 same-cluster near-duplicates is a much finer discrimination problem than at 100k (~98/cluster), and int8's ~8 bits/component is not always enough to preserve that fine an ordering. This was isolated from graph-reachability effects experimentally (see over_fetch_tuning): widening the candidate pool and ef_search/ef_construction by 4x each independently produced BIT-FOR-BIT IDENTICAL recall at both scales, meaning the true top-10 are essentially always already in the candidate pool -- the loss is purely in how precisely dequantize-rerank can re-score them, which is exactly the gap search_with_exact_rerank (option b) targets."
    },
    {
      "id": "build_time_100k",
      "before_secs": 16.611,
      "after_secs": 4.36,
      "speedup_x": 3.81,
      "note": "Unplanned bonus: building the graph from smaller (int8) Points is itself ~3.8x faster at this N, plausibly from less memory traffic during construction and from maybe_compact's background rebuild snapshot now cloning QPoint (388B) instead of Vec<f32> (1536B) per vector."
    },
    {
      "id": "query_latency_100k",
      "before_p99_ms": 0.427,
      "after_p99_ms": 0.313,
      "note": "No latency regression -- int8 distance (i32 dot product) plus the one dequantize-and-rescore per surfaced candidate is not slower than the prior f32 cosine distance; p99 improved slightly, consistent with smaller data moving through cache during traversal."
    },
    {
      "id": "integration_test_regression_and_fix",
      "test": "crates/core/tests/hnsw.rs::hnsw_recall_matches_brute_force_top1 (32-dim, N=1000, uniform-random, k=1, bar: >=48/50 top-1 matches)",
      "baseline": "50/50",
      "symmetric_quantized_query_and_candidate_first_attempt": "47/50 (FAILED the test's 48/50 bar)",
      "root_cause": "with k=1 and no tombstones, the pre-fix over-fetch formula produced fetch=1 -- i.e. only the single closest candidate by the graph's OWN (quantized) traversal order was ever inspected, so no amount of rescoring afterward could recover a candidate the routing itself ranked 2nd or later.",
      "fix": "added a RERANK_POOL_MIN=32 floor to the over-fetch calculation (in addition to the existing tombstone/pre-filter slack) so rerank always has a wide-enough candidate pool, and switched the rerank score to cosine(raw_f32_query, dequantize(candidate)) instead of the graph's own quantized-query distance.",
      "after_fix": "50/50 -- matches the pre-quantization baseline exactly"
    }
  ],
  "recorded_at": "2026-07-24"
}