Data model & realtime

Connector fabric

Pulls data from other databases and services into a KynetraDB project through one governed contract, with credentials sealed on the server and repeat pulls that converge instead of duplicating.

BetaEngine edition 5f8ea3af

What it does

The connector fabric, called Estuary in the engine, runs inlets that read from external sources and write into a project's namespace through a tenant-scoped sink. Credentials are sealed server-side and referenced by handle. Fourteen built-in inlets are registered; two declare change-data-capture and the rest do full pulls.

How it works

An inlet is an adapter with a name, a set of declared capabilities (backfill, incremental, change-data-capture), a soundings call that reports the source's schema before anything is read, and a pull that takes a resume cursor, called a tidemark, and returns a new one. An inlet writes to a sink alone, never to the engine, and the sink prefixes every kind and id with the tenant. Re-pulling unchanged data converges to a no-op through content hashing. A conformance harness runs any inlet against six invariants, including tidemark monotonicity.

Credentials are sealed by the server under a root key, KYNETRA_ROOT_KEY, and just a handle crosses the HTTP boundary. Responses to credential, pull and status calls never contain secret material or record bytes, and error text from providers is scrubbed. With no root key the routes report 503 rather than serving, and a malformed key stops the server at boot.

The built-in inlets are Algolia, Appwrite, Convex, Cloudflare D1, Firebase, Meilisearch, MongoDB, NDJSON files, Pinecone, PocketBase, PostgreSQL, SQLite, Supabase and Typesense. PostgreSQL drains a logical replication slot from a given position and MongoDB resumes a change stream from its resume token; those two declare backfill, incremental and change-data-capture. Every other inlet declares backfill alone and returns no cursor, so re-pull cost comes from the content-hash no-op, not from incremental reads.

HTTP routes live under /v1/estuary: health, the inlet catalogue, sealed credentials, soundings, pulls with status and cancel, and a changes endpoint for inlets that accept pushed changes. The kynetra command line wraps them for remote servers with ingest sources, connection add and revoke, inspect, run, status and cancel; credential material enters through standard input alone. A separate conversion planner returns plans that change nothing and does not execute them.

The fabric stores its own state through the server's HTTP API on the loopback address, and a pull runs inside the server process. See the migration page for per-source detail on what each source moves.

Where you reach it

  • HTTPGET /v1/estuary/inlets; POST /v1/estuary/credentials; POST /v1/estuary/soundings
  • HTTPPOST /v1/estuary/pulls; GET /v1/estuary/pulls/{id}; POST /v1/estuary/pulls/{id}/cancel
  • CLIkynetra --remote <url> ingest sources | connection add --tenant --id --source --credentials-stdin | inspect | run | status | cancel
  • ConfigKYNETRA_ROOT_KEY (enables credential sealing and the routes)

Limits

  • The PostgreSQL source connection does not use TLS yet. Run the server inside a private network with the source, or reach the source through an SSH tunnel.
  • The conformance tests for the PostgreSQL and MongoDB change-data-capture paths need live servers and are opt-in, so they do not run by default.
  • A pull runs inside the server process. A large pull competes with the engine for CPU, with no isolation between them.
  • The destination is always the project that owns the connection. There is no arbitrary source-to-destination ELT, and the conversion routes plan without executing.
  • Project identities that cannot write are refused entirely.

Evidence

  • On this siteMigration sources and limits
  • TestsCovered in the engine repository by more than 150 unit tests in the connector crate, one conformance test per inlet, and 23 HTTP tests (authentication, sealed credentials, no secret or record bytes in responses, rate-limit surfacing, root-key absence)

Test coverage is listed as present in the engine repository at this edition. This page does not report test results.

KynetraDB is in private preview

Request a preview to talk through the limits above against your own workload.