Durability & recovery

Semi-synchronous quorum commit

Lets a primary acknowledge a write once enough replicas have persisted it, so a single node failure does not lose acknowledged writes.

BetaEngine edition 5f8ea3af

What it does

When KYNETRA_QUORUM_PEERS lists replicas, the primary pushes each locally durable batch to them and answers its callers once the required number have confirmed. A write that misses quorum returns an error even though the primary keeps it, so callers must treat it as indeterminate.

How it works

The hook runs on the engine's single writer thread, after a batch is locally durable and before any caller is answered. It posts the batch as newline-delimited records to every peer's /v1/replica/ingest in parallel and counts a peer when it answers success and reports an applied position at or beyond the batch's last position. KYNETRA_QUORUM_W sets how many must confirm; it defaults to one and cannot exceed the peer count. KYNETRA_QUORUM_TIMEOUT_MS bounds the wait, with a two-second default, and KYNETRA_QUORUM_API_KEY supplies the peer credential, defaulting to the instance key.

A fresh cluster is circular, because followers need the primary to sync and the primary needs followers for quorum. So enforcement engages on the first successful quorum, and until then a failure is logged and replication stays asynchronous. A node that boots with recovered history is rejoining, not bootstrapping, so it enforces from its first write. The cost is deliberate: a primary restarted while its followers are unreachable stops acknowledging writes until one returns. The healthz endpoint shows whether enforcement is engaged.

When quorum is missed after enforcement has engaged, every staged request in the batch is answered with a quorum-not-reached error. The batch still applies locally: the write is durable and visible on the primary, including after a restart. Callers should treat the result as indeterminate and retry with an idempotency key.

Measured: semi-synchronous quorum commit measured across three processes on one host over loopback. This is local scope, one host (Apple M5, 32 GB RAM). Three processes, one host, loopback networking. This does not measure quorum commit across a real network, which is where the interesting failures live.

Where you reach it

  • ConfigKYNETRA_QUORUM_PEERS (comma-separated peer base URLs, on the primary)
  • ConfigKYNETRA_QUORUM_W, KYNETRA_QUORUM_TIMEOUT_MS, KYNETRA_QUORUM_API_KEY

Limits

  • A write that misses quorum returns an error but remains on the primary. Treat it as indeterminate, not as absent.
  • Until the first quorum succeeds, a new primary replicates asynchronously.
  • Setting this on a replica is refused, and it cannot be combined with Raft consensus writes.
  • The measurement covers three processes on one host over loopback. Behaviour across a real network is not measured.
  • There is no automatic failover. Quorum commit protects acknowledged writes; it does not elect a new primary.

Evidence

Test coverage is listed as present in the engine repository at this edition. This page does not report test results.

KynetraDB is in private preview

Request a preview to talk through the limits above against your own workload.