{
  "schema": "kynetra.benchmark/v1",
  "suite": "hnsw_binary_quantization",
  "status": "local_evidence",
  "origin": {
    "kind": "measured",
    "tool": "examples/binary_quant_bench.rs",
    "build": "release"
  },
  "environment": {
    "scope": "local",
    "hardware": "Cortex-X925, 121 GB RAM",
    "operating_system": "linux",
    "arch": "aarch64",
    "host": {
      "logical_cpus": 20,
      "rustc_version": "rustc 1.97.1 (8bab26f4f 2026-07-14)",
      "uname": "Linux spark-417d 6.17.0-1026-nvidia #26-Ubuntu SMP PREEMPT_DYNAMIC Thu Jun 25 00:57:17 UTC 2026 aarch64 aarch64 aarch64 GNU/Linux"
    }
  },
  "config": {
    "dims": 384,
    "k": 10,
    "n": 100000,
    "corpus": "1024-center Gaussian mixture, sigma 0.15 -- IDENTICAL seed/generator to the int8 evidence (2026-07-24-linux-int8-quantization.json) and to vector_scale_bench.rs, so this A/B isolates the quantization scheme, not the data.",
    "hnsw_params": "ef_construction=200 ef_search=100 (instant-distance, unchanged from the int8 baseline)",
    "methodology": "BEFORE = QuantMode::Int8 (this branch's current default, unchanged). AFTER = QuantMode::Binary, same HnswIndex code path, same corpus/query seeds (Lcg(2026) corpus, Lcg(7) queries), same host, back-to-back `cargo run -p kynetra-core --release --example binary_quant_bench` invocations selected via QUANT_MODE=int8|binary env var (kept the corpus generation IDENTICAL between runs by construction, not by re-seeding per-run). N=100,000 (not 1,000,000) per the task's 'fast scale' instruction. instant-distance's own HNSW level-assignment uses an internal (unseeded) random draw, so byte-for-byte identical builds are not guaranteed across runs -- recall figures below have measured run-to-run noise of roughly +-0.002 at this N, well within the conclusions drawn.",
    "binary_quantization_scheme": "Sign-based 1-bit-per-dimension: bit_i = 1 if v_i >= 0 else 0, packed 64 bits per u64 word (ceil(dim/64) words -- 384 dims -> 6 words -> 48 bytes of packed bits). Does NOT L2-normalize first (unlike int8): cosine similarity is scale-invariant, so sign(v_i) == sign(v_i / ||v||) for any ||v|| > 0 -- normalizing would not change a single bit, so this implementation skips that pass entirely. See crates/core/src/personalities/vector_hnsw.rs module docs ('Binary quantization' section) for the full derivation.",
    "distance": "Hamming distance = popcount(a XOR b) summed word-by-word via u64::count_ones() (a hardware POPCNT/CNT instruction on this aarch64 host -- no SIMD crate needed, same autovectorization argument as the int8 dot product). Smaller-is-closer already holds with no negation needed. Hamming distance is a monotonic (order-preserving) function of the angle between sign-quantized vectors -- the classical SimHash/random-hyperplane-LSH result, applied here with the embedding's own axes as the hyperplanes -- so it is adequate for HNSW graph ROUTING even though it is not numerically equal to cosine similarity; rerank (below) closes that numerical gap.",
    "rerank": "search_with_exact_rerank (the SAME additive hook W2/this task built for int8's own precision gap) is used UNCHANGED for Binary graphs -- it is effectively MANDATORY here, not optional, because Hamming distance's routing is much coarser than int8's quantized-cosine. Plain search() still works (falls back to BQPoint::dequantize, a uniform-magnitude +-1/sqrt(dim) reconstruction of the stored signs) but is markedly less accurate -- see 'recall' measurements below for exactly how much.",
    "over_fetch_tuning": "HnswParams.binary_overfetch_factor multiplies the existing tombstone/pre-filter/rerank-floor over-fetch math, ONLY in QuantMode::Binary (Int8 unaffected). Measured at 1x, 2x, 4x, 8x (same corpus/queries, BINARY_OVERFETCH env var) -- see 'over_fetch_sweep' below. recall@10 (exact rerank) climbs from 0.58 (1x) to 0.93 (2x, still short of the 0.95 gate) to 0.987 (4x, clears the gate with margin) to 0.985 (8x, no further gain -- within run-to-run noise of the 4x figure -- while p99 latency roughly doubles vs 4x). Default is 4: the point where recall clears the gate and additional over-fetch stops paying for itself.",
    "memory_scheme_bytes_per_vector": {
      "f32_raw": 1536,
      "int8_qpoint": 388,
      "binary_bqpoint_packed_bits_only": 48,
      "binary_bqpoint_struct_total_incl_dim_bookkeeping_field": 56,
      "note": "int8_qpoint = 384 dims * 1 byte + 4-byte per-vector f32 scale (see int8 evidence). binary_bqpoint_packed_bits_only = ceil(384/64) * 8 = 48 bytes -- the actual quantization payload, directly analogous to how int8_qpoint's 'dims * 1 byte' component is counted (i.e. NOT counting the surrounding Vec's own (ptr,len,cap) stack metadata, exactly as QPoint's own 'dims' figure doesn't count Vec<i8>'s metadata either). struct_total additionally counts BQPoint's own `dim: usize` field (8 bytes) -- redundant per-vector bookkeeping (analogous to a Vec's own length, which int8's figure also excludes) kept for defensive dimension-mismatch detection; shown for full honesty even though it is not the headline figure."
    },
    "memory_ratio": {
      "vs_f32_packed_bits_only": 32.0,
      "vs_f32_struct_total": 27.4286,
      "vs_int8_packed_bits_only": 8.0833,
      "vs_int8_struct_total": 6.9286,
      "note": "1536/48 = 32.0 EXACTLY -- this is the number that matches the task's ~32x-vs-f32 target and Qdrant's published figure. 388/48 = 8.08x vs int8, matching the ~8x-vs-int8 target. These are exact per-vector-byte ratios (N-independent, copy-count-independent): HnswIndex holds two owned copies of the payload (the sealed graph's own point storage, plus inner.vectors, the rebuild/compaction source of truth covering graph union delta) and BOTH are quantized in QuantMode::Binary as they are in QuantMode::Int8 (see module docs) -- so this per-vector ratio equals the whole index's owned-memory ratio, not just the graph's."
    }
  },
  "kpi_gate": {
    "description": "Binary quantization tier (opt-in via HnswParams.mode = QuantMode::Binary): target ~32x memory reduction vs f32 (~8x vs int8) while search_with_exact_rerank holds recall@10 >= 0.95 on the clustered corpus, at an honestly-reported over-fetch/latency cost.",
    "memory_reduction_vs_f32_x": 32.0,
    "memory_reduction_vs_int8_x": 8.0833,
    "recall_at_10_raw_no_rerank": 0.2015,
    "recall_at_10_plain_search_dequantize_fallback": 0.2015,
    "recall_at_10_meets_0_95_gate_via_search_with_exact_rerank_at_default_overfetch_4x": true,
    "recall_at_10_exact_rerank_at_default_overfetch_4x": 0.9855,
    "met": "yes, with an honest caveat: raw/plain Binary-mode search recall is poor (~0.20, as expected for 1-bit Hamming routing on 384-dim clustered near-duplicates) -- search_with_exact_rerank (mandatory, not optional, for this mode) recovers recall@10 to 0.9855 at the default 4x over-fetch, clearing the 0.95 gate. End-to-end query latency is NOT lower than int8's at this over-fetch (see 'latency_honesty' below) -- the memory and build-time wins are real and large; the 'speed' side of Qdrant's 32x-memory/40x-speed claim does not carry over end-to-end once recall-protecting rerank + wider over-fetch are accounted for, only for the raw distance primitive (Hamming/popcount vs int8 dot product)."
  },
  "measurements": [
    {
      "id": "full_process_rss_at_100k_int8_before",
      "value_mb": 525,
      "graph_build_secs": 6.4,
      "recall_at_10_plain": 0.958,
      "recall_at_10_exact_rerank": 1.0,
      "query_plain_ms": { "p50": 0.162, "p95": 0.414, "p99": 0.64, "queries": 1000 },
      "query_exact_rerank_ms": { "p50": 0.219, "p95": 0.506, "p99": 0.785, "queries": 1000 },
      "insert_to_searchable_ms": 0.205
    },
    {
      "id": "full_process_rss_at_100k_binary_after_default_overfetch_4x",
      "value_mb": 524,
      "graph_build_secs": 4.8,
      "recall_at_10_plain_dequantize_fallback": 0.2015,
      "recall_at_10_exact_rerank": 0.9855,
      "query_plain_ms": { "p50": 0.326, "p95": 0.544, "p99": 0.686, "queries": 1000 },
      "query_exact_rerank_ms": { "p50": 0.454, "p95": 0.702, "p99": 0.814, "queries": 1000 },
      "insert_to_searchable_ms": 0.537,
      "note": "RSS is BARELY lower than int8's (525 -> 524 MB), NOT a ~8x drop, despite the index's own payload shrinking ~8x (see memory_ratio) -- this benchmark harness keeps TWO separate full-precision f32 copies alive outside HnswIndex: the ground-truth corpus (for brute-force recall verification) AND a HashMap standing in for the engine's entity store (feeding search_with_exact_rerank's exact_lookup closure, which a real deployment already holds separately in its entity store regardless of quantization scheme). At 100k x 384-dim, those two f32 copies alone are ~2 x 147 MB =~ 294 MB, which swamps the actual index-owned difference (int8 ~74 MB vs binary ~9-11MB, i.e. ~63-65MB of real savings) in the whole-process number. This is the same dilution effect the int8 evidence already documented for its own full_process_rss_at_100k_reduction (1.78x measured vs 3.96x index-owned) -- worse here because search_with_exact_rerank's realistic usage requires a second f32 copy the int8-only benchmark didn't need. The per-vector byte-count ratio (memory_ratio, exact arithmetic) is the number that reflects what this change actually controls."
    },
    {
      "id": "index_owned_memory_at_100k_vectors",
      "int8_mb": 74.02,
      "binary_mb_bits_only": 9.15,
      "binary_mb_struct_total": 10.68,
      "reduction_x_vs_int8_bits_only": 8.083,
      "note": "100,000 vectors x 2 owned copies (sealed graph + inner.vectors) x bytes-per-vector from memory_scheme_bytes_per_vector, in MiB. Matches the exact per-vector ratio (N and copy-count cancel out)."
    },
    {
      "id": "build_time_100k",
      "int8_secs": 6.4,
      "binary_secs": 4.8,
      "speedup_x": 1.33,
      "note": "Binary graph construction is faster than int8's, consistent with the int8-vs-f32 build speedup already documented (less memory traffic moving through the builder per vector: 48-56 bytes vs 388)."
    },
    {
      "id": "recall_raw_vs_reranked",
      "plain_dequantize_fallback": 0.2015,
      "exact_rerank_default_overfetch_4x": 0.9855,
      "gate": 0.95,
      "note": "Binary quantization's raw routing recall (no exact rerank) is poor -- ~0.20, i.e. roughly 1/5 of the true top-10 survive Hamming-distance-only ranking on this 384-dim, 1024-cluster, 100k-vector corpus. This is EXPECTED and is exactly why the module docs describe exact rerank as effectively mandatory for QuantMode::Binary, not optional the way it is for Int8 (whose plain-search dequantize-rerank already clears the gate at this same N -- recall 0.958, see int8 evidence). With search_with_exact_rerank at the default 4x over-fetch, recall recovers to 0.9855 -- comfortably above the 0.95 gate."
    },
    {
      "id": "over_fetch_sweep",
      "corpus": "same 100k corpus/queries as the main A/B, QuantMode::Binary, recall@10 via search_with_exact_rerank",
      "factor_1x": { "recall": 0.5825, "query_exact_rerank_p99_ms": 0.781 },
      "factor_2x": { "recall": 0.9265, "query_exact_rerank_p99_ms": 0.908 },
      "factor_4x": { "recall": 0.987, "query_exact_rerank_p99_ms": 0.414 },
      "factor_8x": { "recall": 0.985, "query_exact_rerank_p99_ms": 0.892 },
      "note": "1x and 2x both MISS the 0.95 gate (0.58, 0.93). 4x clears it with margin. 8x is statistically indistinguishable from 4x (0.985 vs 0.987, within the +-0.002-ish run-to-run noise HNSW's unseeded level assignment introduces at this N) while costing roughly double the p99 latency -- i.e. over-fetching further than 4x buys no additional recall on this corpus, only latency. This is the basis for binary_overfetch_factor's default of 4. (The p99 figures bounce around within a run's own noise floor at this query count/N and should be read as 'same ballpark', not a precise curve -- the RECALL trend across factors is the reliable signal here.)"
    },
    {
      "id": "latency_honesty",
      "int8_plain_p50_ms": 0.162,
      "binary_plain_p50_ms": 0.326,
      "int8_exact_rerank_p50_ms": 0.219,
      "binary_exact_rerank_p50_ms": 0.454,
      "note": "Binary-mode end-to-end query latency at its RECALL-PROTECTING configuration (4x over-fetch + mandatory exact rerank) is HIGHER than int8's at this N, not lower -- the wider candidate pool (4x more graph-traversal results processed and reranked per query) costs more than the cheaper per-candidate Hamming/popcount distance saves. Qdrant's widely-cited '32x memory / 40x speed' claim for binary quantization is a claim about the raw distance PRIMITIVE (popcount vs float dot product) and/or about un-reranked or lightly-reranked configurations; it does not carry over end-to-end once recall is protected to int8's level via a 4x-wider rerank pool on this corpus/index. The memory reduction is the reliably real win here; 'speed' should not be claimed without this caveat."
    }
  ],
  "recorded_at": "2026-07-24"
}