Gateway01
One connection for statements and live queries
HTTP serves session exchange, one-shot SQL and bulk writes. A WebSocket session multiplexes statements and continuous queries by request id, with demand-based delivery.
HTTP · WebSocket
ProductArchitecture
Every write lands once, then feeds everything you derive
A NYXDB node orders each table’s writes through a single owner per tablet, makes them durable, and publishes each committed write once. Everything derived from it, from transformations to live queries, subscribes to that one publication.
Three planes
Three planes inside one process
Where your statements arrive, where your writes are ordered and made durable, and how everything derived from them is fed.
Tablets02
A write is acknowledged only once it is durable
Each table is split into tablets by its shard key, and each tablet has one owner that orders its commits. Records go to a segmented, checksummed local WAL with group commit, and a write is acknowledged only after it is durable.
one owner each
PSI and derived state03
Producers never wait for readers
Each committed batch is published once. Transformations, materialized views, indexes and streaming queries subscribe by stream:key pair, each with its own durable cursor, so a slow subscriber never slows a write.
publish once
Data path
What happens to a statement inside the node
Four stops between your statement and everything that depends on it.
- 01
Your statement reaches the node
HTTP serves sessions and one-shot SQL; a WebSocket session multiplexes statements and streams.
Gateway · HTTP · WebSocket
- 02
It is bound to exact identities
The statement is parsed and bound against the catalog to exact table and function identities.
SQL · catalog
- 03
The write is ordered and made durable
Each tablet has one owner that orders its commits and acknowledges a write only after its WAL record is durable.
Tablets · WAL · group commit
- 04
Everything derived from it is fed
The committed batch is published once; transformations, views, indexes and streams read it by stream:key pair.
PSI · stream:key pair
Consistency
Durable before it is acknowledged, visible to the next read
Three guarantees your application builds on, in the order a write meets them.
- Receipt01
Acknowledged only once it is durable
The tablet’s owner orders your write and assigns it a commit sequence. Its records join a group, the group is written to the write-ahead log and passes one persistence barrier, and only then do you receive the receipt and does anything derived from it hear about it.
commit sequence → WAL group → barrier → receipt
- Read-after-write02
The next read sees your write, no retry
When a write returns its receipt, the read that follows sees it. A read that runs while a write is in flight returns the state before it or the state after it, never a partly applied row.
receipt → next read
- AS OF COMMIT03
Read any versioned table at a past commit
Every accepted version of a row stays beside the current one, so you can read a table exactly as it stood at an earlier commit sequence: the state behind a decision, an audit or a replay, under the schema you have now.
SELECT address, token, amount FROM balances AS OF COMMIT 42 WHERE chain_id = 1;
The protocol
Finite reads and live queries, one socket
HTTP carries sign-in, one-shot statements and bulk rows. A WebSocket session on one route multiplexes everything else by request id, and the server sends results only as fast as your client asks for them.
- GET /client
- nyxdb.public.v1
- binary frames, network byte order
| What you need | Transport | Route or frame |
|---|---|---|
| Sign in and open a session | HTTP | POST /auth/passwordPOST /auth/api-keyPOST /auth/session |
| One statement, the whole result | HTTP or WebSocket | POST /sqlExecuteSql |
| A query that stays open | WebSocket | ExecuteSqlContinuousOpenedContinuousResult |
| Flow control and cancellation | WebSocket | DemandContinuousDemandCancel |
| Session settings and federation statements | WebSocket | SETRESET |
| Bulk rows | HTTP or WebSocket | POST /write/batchPOST /bulk-load/framePOST /insert/logical-block |
| Progress and liveness | WebSocket | QueryProgressPingPong |
Every frame carries a request id; many requests run at once on one connection.
Protocol referenceStorage profile
Shape each table around how you read it
Every table declares how it is written and kept: its mode, its key, how it is sharded across tablets, and a storage profile that fits how you read it.
Storage profile · ROW · COLUMNAR · HYBRID · DEFAULT
See CREATE TABLE for the full grammar and the storage model for the profiles.
a versioned, sharded table with a row profile
CREATE TABLE balances ( chain_id UINT16 NOT NULL, address STRING NOT NULL, token STRING NOT NULL, amount UINT256 NULL, updated_at DATETIME64(3, 'UTC') NULL, balance_index UINT128 NULL, balance_version UINT64 NULL, transfer_count UINT64 NULL, last_transfer_at DATETIME64(3, 'UTC') NULL, activity_index UINT128 NULL, PRIMARY KEY (chain_id, address, token), ATTRIBUTE GROUP holding (amount, updated_at, balance_index, balance_version) VALID BY (balance_index) REVISION BY (balance_version), ATTRIBUTE GROUP activity (transfer_count, last_transfer_at, activity_index) VALID BY (activity_index))MODE VERSIONEDSHARD BY (address) SHARDS AUTOORDER BY (chain_id, address, token)STORAGE PROFILE ROW;Storage
From the log to the layout, declared per table
Every table declares its own layout, partitions, indexes and codecs, and every one of them sits on the same durable log.
Every commit lands in one log first
Commits join a group behind one persistence barrier in a segmented, checksummed write-ahead log. Your write is acknowledged only after its group is durable, and published to everything derived from it only after that.
A restart picks up where the log left off
Committed data is kept in immutable parts. When the node opens, each tablet loads its last checkpoint, replays the log written after it, and repairs a torn final record if the process stopped mid-write.
Row, columnar or both, per table
Choose a row-oriented projection, a columnar one, or both at once, to fit how you read the table. The profile applies to a table’s current rows and to the history it retains.
Split by key, partition by time
Each table is split into tablets by its shard key, with a count you set or one the engine picks, and can be partitioned by a time column. Tablets are replicated across nodes, and move, split and merge as data grows.
Indexes that skip data, never change an answer
Range, membership and exact-lookup indexes, each scoped to current rows, retained history or both. An index decides how much data a read touches, and every candidate row still passes the full predicate.
Codecs per column, policies per table
Repeated values go through a dictionary, and each column can declare a chain of codec stages checked against its type. Retention policies expire rows by age, and storage policies tier data between local devices and object storage.
Operations
Run it like a database, because it is one
Six pages take a node from first start to upgrade. Each one names the route, relation or key you read.
- Deployment01
Several nodes, one process each
Each node runs with its own data directory and configuration file. Tablets are replicated across the nodes, and a replica takes over when a node is lost.
- Configuration02
One file, read at start
Keys grouped by area: the issuer key, request deadlines, stream lifetimes, the drain time and the recovery workers. Every value in force is visible with its origin.
- Health and readiness03
One probe, one relation
The readiness route answers 200 while the node serves and 503 while it drains, and a system relation reports the health of each component.
- Metrics and events04
Telemetry you query
Metrics, events, query statistics and health signals are system relations you read like any table. Alert on write pressure, log growth, transformation lag and stream lag.
- Backup and recovery05
A consistent copy, restored at its commit
A backup captures a node’s tables, their versions and the definitions that maintain them. A restore brings them back at the backup’s commit, and transformations resume from their checkpoints.
- Upgrade compatibility06
A newer build opens the same directory
Tables, versions, definitions and checkpoints carry over to a newer build. Upgrade one node at a time while the replicas keep serving.
Read the concepts behind the node
Tables, attribute groups, versioned tables, transformations and PSI, each with the statements the engine tests run.