Skip to main content
DB
ProductStreaming

Queries that stay open

Put one word in front of a query and it keeps running: the current answer first, then every exact change as writes commit. Many clients following the same question share one arrangement, and no client is ever flooded.

Streaming reads

The answer now, then every change

When the stream opens it delivers one snapshot of the result. After that, each commit on the source produces only the change to the result, so your client never re-reads the table and never misses a change.

one snapshot
One row per group over everything already committed, sent once when the stream opens.
exact changes
Inserts, deletes and updates of result rows, cell updates, retractions and replacement-snapshot boundaries.
anything you select
Relations, views and joins, with filters, grouped aggregates, ordering and a limit that is kept ordered as values change.
live volume per token
STREAM SELECT chain_id, token, SUM(amount) AS volume, COUNT(*) AS transfers
FROM transfers
GROUP BY chain_id, token;

Windows

Tumble, hop or session, released on your terms

Window a relation by event time or by processing time, and choose when a window’s result is released and in what form. Every windowed stream outputs the window bounds beside its aggregates.

Fixed windows

Non-overlapping windows of one size over event time, each released when it closes.

transfers per token, every minute
STREAM SELECT window_start, window_end, token, COUNT() AS transfers
FROM TUMBLE(transfers, transferred_at, INTERVAL 1 MINUTE)
GROUP BY window_start, window_end, token
EMIT AFTER WINDOW CLOSE;

TUMBLE · event time

Overlapping windows

Windows of one size that advance by a smaller step, so each event lands in several of them.

an hour of volume, advancing every minute
STREAM SELECT window_start, window_end, SUM(amount) AS volume
FROM HOP(transfers, transferred_at, INTERVAL 1 MINUTE, INTERVAL 1 HOUR)
EMIT AFTER WINDOW CLOSE;

HOP · step and size

Windows closed by a gap

A session stays open while events keep arriving and closes after a gap, optionally bounded by a maximum duration.

sessions on processing time
STREAM SELECT window_start, window_end, COUNT() AS transfers
FROM SESSION(transfers, NOW64(), INTERVAL 30 SECONDS, INTERVAL 5 MINUTES);

SESSION · NOW64() · processing time

EMIT · when results are released
The emit triggers of a streaming query and when each releases results.
AFTER WINDOW CLOSE [WITH DELAY …] [AND TIMEOUT …]when a window closes
PERIODIC INTERVAL … [REPEAT]on a fixed interval
ON UPDATE [WITH BATCH INTERVAL …]when the result changes, optionally batched
PER EVENTfor every event

EMIT STREAM releases append-style output; EMIT CHANGELOG carries retractions. A trailing SETTINGS clause bounds the stream’s lifetime and how long its arrangement lingers.

Shared arrangements

Computed once, shared by every subscriber

Open the same streaming query from many clients and the engine does not run it many times. Each client subscribes to one shared continuous arrangement, the shared continuous channel that already holds the result, and every subscriber receives the same exact changes.

One arrangement, as many subscribers as you open

Every dashboard, service and console that opens the query follows one arrangement. A new subscriber joins what is already there, and the arrangement emits one set of output deltas for all of them.

When the last subscriber leaves, the arrangement stays ready for a time you set, so a returning subscriber joins without reseeding. Set that time per node, per definition or per statement.

Each arrangement is a row you can read: its state in rows and bytes, its subscribers by consumer kind, the input changes it applied and the output deltas it emitted.

derived_arrangement
the consumer kind of shared continuous channels and materialized views
public_session
the consumer kind of a STREAM SELECT session
subscriber_idle_ttl
how long a shared continuous arrangement stays ready after its last subscriber leaves
system.psi_arrangements
one row per shared arrangement: state rows and bytes, subscribers by consumer kind
system.psi_streams
one row per subscription, with its demand and lag
public_session × 3
Dashboardthe same STREAM SELECT
Servicethe same STREAM SELECT
Consolethe same STREAM SELECT
derived_arrangement

One shared continuous arrangement

  • a shared continuous channel
  • state rows · state bytes
  • subscribers by consumer kind
  • input changes applied · output deltas emitted
PSI · each committed write, published once
Three clients, one arrangement, one publication. A fourth client joins the same arrangement.

Delivery

One socket, many requests, at your pace

A stream travels over the WebSocket session your client already holds. Every frame carries a request id, so finite reads and live queries share one connection, and the server sends only what your client has asked for.

  1. ExecuteSql → ContinuousOpened01

    Open it like any statement

    Send the statement with its row and byte bounds and an initial demand. The server answers that the query is open, with a resumable cursor, then sends the snapshot and every change after it.

    ContinuousResult: snapshot first, then changes

  2. Demand · ContinuousDemand02

    Results only as fast as you ask

    Demand is credit. The server sends results while the request holds credit; your client acknowledges the delivered prefix and grants more, so a slow consumer is never flooded and a long stream never blocks a quick read on the same connection.

    flow control, per request id

  3. Cancel · ResumeContinuous03

    Stop it, or pick it up again

    Cancel ends one request and leaves the others on the connection running. Re-open a continuous subscription after a delivery cursor, and bound a stream’s lifetime with a setting when you want one.

    max_stream_duration · request_timeout

GET /client · subprotocol nyxdb.public.v1 · the session token travels only in the first frame · wss:// for any host other than loopback

The other subscribers

Your derived tables follow the same publication

Every committed write is published once. Streaming queries subscribe to it, and so do transformations, materialized views and indexes, each by the relation, component and key it reads, each with its own durable cursor.

published once

One feed per tablet

Each committed write is published once, as an immutable batch on its tablet’s committed feed. Only committed changes are published, and a write to one attribute group reaches continuous consumers as the key’s complete current row.

stream:key pair

Subscribers choose exactly what they read

A subscriber selects changes by relation, component and key, never by a stream name alone. A transformation that writes a group of its own source has that group filtered from its subscription, so it never receives its own writes back.

durable cursor

Producers never wait for readers

Each consumer keeps its own cursor and acknowledges only what it applied. A slow transformation or index does not hold back a write, and after a restart a consumer resumes from its cursor. Nothing is skipped silently.

Consumer kinds: public_session for streams · derived_arrangement for shared channels and materialized views · transformation · index · projection · cache · changefeed

PSI in the docs

Frequently asked questions

The questions a team asks before it replaces polling with a query that stays open. Anything else is in the docs.

One snapshot of the current result, then exact changes as commits land: inserts, deletes and updates of result rows, cell updates, retractions and replacement-snapshot boundaries. The client never re-reads the table.

Open a query and leave it open

Create a table, write a row and watch a streaming query follow it over one WebSocket session. The first-queries walkthrough takes you there.