kynetradb documentation

Introduction

kynetradb is a data engine designed to collapse the typical "modern" search stack into a single process. Instead of Algolia for search, Pinecone for vectors, Redis for cache, and Postgres for analytics — each fed by its own CDC pipeline — you run one binary that accepts JSON writes and answers all four query shapes against the same on-disk log.

The architecture is intentionally boring: an append-only write-ahead log feeds a set of in-memory indexes ("personalities"). Restart replays the log. Replication ships the log. Backup is the log. There is no separate index that can drift from the source of truth.

When this is the right tool

• You need search + vector + KV + document over the same dataset.
• You want one binary you can self-host, not four SaaS bills.
• Your working set fits in RAM on a single node (typically <100M entities).
• You're comfortable with last-writer-wins semantics and no SQL.

When it isn't

• You need ACID multi-row transactions across arbitrary keys.
• Your dataset is hundreds of millions of rows and grows unboundedly.
• You need Postgres-wire-compatible query access (use the Postgres sink instead).

Quickstart

Run locally

docker run -p 8080:8080 -v kynetra-data:/data ghcr.io/kynetra/kynetradb:latest
# or with the source
cargo run -p kynetra-server --release

First user (becomes Admin automatically)

curl -X POST http://localhost:8080/v1/auth/signup \
  -H 'Content-Type: application/json' \
  -d '{"email":"you@example.com","password":"strong-pw","display_name":"You"}'

Login & write a row

TOKEN=$(curl -s -X POST http://localhost:8080/v1/auth/login \
  -H 'Content-Type: application/json' \
  -d '{"email":"you@example.com","password":"strong-pw"}' | jq -r .token)

curl -X POST http://localhost:8080/v1/entities \
  -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' \
  -d '{"kind":"product","attrs":{"title":"Aurora Espresso","price":2200}}'

Search

curl -X POST http://localhost:8080/v1/search \
  -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' \
  -d '{"q":"espresso","top_k":10,"kind":"product"}'

That's a complete write → search loop in four HTTP calls. Open http://localhost:8080/admin for the embedded admin UI.

Core concepts

The universal log

Every write — entity put, collection record, user signup, file upload metadata, agent run — is appended to a single on-disk log as a typed event. The log is the source of truth. Indexes are derived state and can be rebuilt by replay.

The log uses length-prefixed CBOR records with CRC32 checksums. Group fsync is configurable for write throughput vs. durability trade-off.

Entities

An entity is the atomic unit of writeable data:

{
  "id": "01K3X...",          // ULID, auto-assigned if omitted
  "kind": "product",          // free-form tag — your "table name"
  "attrs": { ... any JSON ... }
}

kind is a free-form string. attrs is arbitrary JSON. Entities are schema-free at this layer — use Collections (below) when you want field validation.

Personalities

A personality is an in-memory index built from the log. Four ship by default:

• KV — exact lookup by id.
• Document — JSON predicate filtering over attrs.
• Search — BM25 full-text over indexed text fields.
• Vector — ANN search over embeddings stored in attrs.vector.

All four read the same entity state. There is no separate write path per personality and no CDC between them.

Collections

A Collection is a typed schema layered over entities. Fields, validation, uniqueness, and per-role CRUD rules are enforced server-side. Records live in the same log as raw entities.

Last-writer-wins

Same-id writes overwrite. There is no row-level locking, no transactions across keys, no MVCC. If you need strict serializability across multiple entities, this isn't the right engine.

Auth & roles

Passwords are hashed with bcrypt (cost 12). Sessions are stateless JWTs signed with HS256, 7-day TTL. The signing secret is generated on first run and persisted to the universal log so tokens survive restarts.

Roles

• Admin — full access. First signup is auto-Admin.
• Editor — write to entities and collections (per rule).
• Viewer — read-only.

Sending the JWT

Authorization: Bearer <token>

Every /v1/* route except /v1/auth/signup and /v1/auth/login requires a valid token.

Entities

The lowest-level write API. Use it directly when you don't need a typed schema.

See API reference → Entities for exact payloads.

Search (BM25)

Indexed text fields in attrs are tokenized and added to a BM25 inverted index on every write. Queries against POST /v1/search return ranked hits with scores.

POST /v1/search
{
  "q": "espresso colombian",
  "top_k": 10,
  "kind": "product"   // optional filter
}

Tokenization is Unicode-aware, lowercase, and strips punctuation. BM25 parameters (k1, b) use sensible defaults; tunable per-deployment.

Vector search

Store an embedding in attrs.vector (an array of floats). Query with POST /v1/vector using a query vector and top_k.

POST /v1/vector
{
  "vector": [0.012, -0.34, ...],
  "top_k": 10,
  "kind": "product"
}

Index uses HNSW with cosine similarity. Dimensionality is inferred from the first vector inserted; subsequent writes with a different dim are rejected.

Filters

JSON predicate syntax over attrs:

POST /v1/filter
{
  "kind": "product",
  "where": {
    "and": [
      { "field": "price", "op": "lt", "value": 5000 },
      { "field": "vendor", "op": "eq", "value": "Aurora" }
    ]
  },
  "limit": 50
}

Operators: eq, neq, gt, gte, lt, lte, in, nin, contains, starts_with. Logical: and, or, not.

Collections

PocketBase-style typed schemas. Defines a name, field types, validation, and per-operation role rules.

Field types

text, long_text, number, integer, bool, email, url, date, json, select, relation, file.

Per-field validation

required, unique, min, max, options (for select), regex.

Per-operation rules

Each collection has a rules object mapping list / view / create / update / delete to an array of role names: ["admin", "editor"] or ["*"] for public.

POST /v1/collections
{
  "name": "posts",
  "fields": [
    { "name": "title", "type": "text", "required": true },
    { "name": "body",  "type": "long_text" }
  ],
  "rules": {
    "list":   ["*"],
    "view":   ["*"],
    "create": ["admin", "editor"],
    "update": ["admin", "editor"],
    "delete": ["admin"]
  }
}

File storage

Two backends, both content-addressable:

• Local — files in a configurable data directory.
• S3 — any S3-compatible endpoint (AWS S3, R2, MinIO, Wasabi, B2, DO Spaces). AWS SigV4 signed in-tree.

Pick via KYNETRA_FILE_BACKEND=local|s3.

# Multipart upload
curl -X POST http://localhost:8080/v1/files \
  -H "Authorization: Bearer $TOKEN" \
  -F file=@image.png

Realtime (SSE)

Open GET /v1/realtime with optional topics and kinds query params. The server pushes named events (change) with a JSON payload describing each write.

const es = new EventSource(
  "http://localhost:8080/v1/realtime?topics=entities&kinds=product",
  { headers: { Authorization: `Bearer ${token}` } }
);
es.addEventListener("change", (evt) => {
  const data = JSON.parse(evt.data);
  console.log("write:", data);
});

Backed by a tokio::broadcast channel. Slow consumers see dropped messages signaled via SSE comments rather than blocking the writer.

LLM runtime

A unified completion API in front of Ollama (local), Anthropic Claude, and OpenAI. Configure providers via /admin/providers; the same POST /v1/llm/complete call works against any of them.

POST /v1/llm/complete
{
  "provider": "anthropic",       // or "openai", "ollama"
  "model": "claude-opus-4-7",
  "messages": [{ "role": "user", "content": "Hello" }],
  "max_tokens": 256
}

Cost (input + output tokens × per-million rate) is tracked per call.

Agents

An agent is a stored LLM tool-use loop. You define a system prompt, pick which of the 10 typed actions it can call (e.g. put_entity, search_entities, summarise_recent_writes), and a model.

Run it ad-hoc via POST /admin/agents/:id/run or schedule it via POST /admin/agents/schedules. Each run is appended to the log with full message history, tool calls, results, and token cost — fully auditable.

Actions can require approval: when an agent attempts a guarded action, the run pauses and an entry appears in /admin/agents/approvals for a human to approve or reject.

DB sinks (outbound mirroring)

Every entity write can be mirrored asynchronously to one or more external databases. The log is the source of truth; sinks are derived consumers.

12 sinks supported:

• BigQuery, DynamoDB, Firestore
• Cloudflare D1, KV, R2, Vectorize
• MongoDB Atlas, Postgres / Supabase, Redis
• Pinecone

Configure via /admin/sinks. Ping every sink with one call to verify credentials before enabling.

Cloud providers (control plane)

A 10-cloud control plane that can provision and destroy kynetradb instances on Fly, Render, Railway, GCP, AWS, Azure, DO, Hetzner, Cloudflare, and OCI. Credentials are stored AES-256-GCM-encrypted in the universal log.

Routes live under /admin/providers/*. Real deploy + destroy — not stubs.

GDPR endpoints

Admin UI

Embedded single-page admin available at /admin/. No build step, no separate process — shipped as a static HTML/JS bundle inside the server binary.

Tabs:

• Database — Cloudflare D1-style: stat cards, query REPL, time-travel restore.
• Collections — visual schema editor.
• Files — browse/upload/delete.
• Agents — author, run, schedule, approve.
• Sinks — configure outbound DB mirrors.
• Providers — multi-cloud deploy.
• Connect — copy-paste snippets for 12 frameworks.

Deploy

18 hosting targets supported, each with a one-command deploy. All run the same Dockerfile. Pick by what you already use.

See the deploy page for the full matrix.

Environment variables

SDKs

Official client SDKs at sdks/:

• TypeScript / JavaScript — @kynetra/sdk (fetch-based, isomorphic).
• React — @kynetra/react: useEntity, useSearch, useCollection, useRealtime, useFiles.
• Vue 3 — composables mirroring the React hooks.
• Svelte — stores for entities, search, realtime.
• Python — sync + async clients (httpx-based).
• Go — single-import package using net/http.
• Rust — typed client using reqwest.

All SDKs share the same wire shape — pick by language, behavior is identical. See examples/06-clients/ for runnable single-file samples.

Data-access integrations

KynetraDB publishes twelve versioned ORM and query-tool starters. Prisma and Drizzle use contract-tested, familiar JavaScript APIs over /rest/v1; SQLAlchemy, Hibernate/JPA, EF Core, Django ORM, Active Record, Sequelize, TypeORM, Diesel, GORM, and Ecto map the PostgreSQL bridge table.

These are previews, not full upstream ORM drivers. Native HTTP mutations are individual requests. Direct ORM transactions cover only the bridge PostgreSQL database, and migration tools must not modify kynetra_entities or Kynetra helper functions.

See the integration matrix for capability states, examples, and transaction boundaries. Foundry publishes the same catalog with template fingerprints for agents and retrieval.

Operations

Health endpoints

• GET /healthz — liveness, returns 200 if process is up.
• GET /readyz — readiness, returns 200 once WAL replay finishes.

Metrics

GET /metrics exposes Prometheus-format counters: per-route p50/p95/p99, write throughput, entity count, sink lag, SLO breach counters.

Backups

The data directory is the backup. cp -a data/ backup-$(date +%F)/ while the server is running is safe (the log is append-only). For continuous backup, enable the S3 mirror sink with kind: "wal".

Restore

Drop the backup data directory in place and restart. The admin time-travel UI also exposes point-in-time restore from snapshot ranges via POST /admin/db/restore.

Replication

Raft-based log replication (in active development). One leader, N async followers with bounded staleness; quorum-write configurable.

Security

• Bcrypt cost 12 for passwords.
• JWT HS256, 7-day TTL, secret rotated on demand via admin endpoint.
• AES-256-GCM for at-rest credential storage (provider keys, sink keys).
• Per-route admin authorization; /admin/* requires Admin role.
• Constant-time password comparison.
• CORS allowlist configurable; no wildcard by default.
• Audit log: every admin action appended to the WAL with actor + payload.

Security disclosures: security@kynetra.dev.

FAQ

Does it speak Postgres wire protocol?

No. Postgres clients (psql, JDBC, psycopg2, sqlx) won't connect. Use the HTTP API or the SDKs. If you want a Postgres-readable mirror, enable the Postgres sink.

How big can it scale?

Single-node working set is bounded by RAM (indexes are in-memory). Typical deployment: 16 GB RAM holds ~30M entities comfortably. Beyond that, shard at the application layer or use the outbound sinks to offload.

How is it different from PocketBase / Supabase?

PocketBase is SQLite + custom server; Supabase is Postgres + extensions. kynetradb is a purpose-built engine where search, vector, KV, and document share one log — no extension or extra service per query shape. The trade-off: no SQL.

Is it production ready?

v0.5. Real users in production for ecommerce search workloads. Read the Roadmap section of the README before betting anything serious on it.