Query identity, cancellation, and retries

Correlate wire requests with engine queries, cancel queued or executing work, handle disconnect-driven cancellation, and retry without duplicating side effects.

NYXDB has separate identities and terminal signals at the wire, engine, and streaming layers. A production client must keep them distinct: confusing an opaque request ID with a numeric engine query ID can make a terminal result impossible to reconcile.

The raw TCP/WebSocket protocol and current clients are Experimental. Pin client and server revisions, test every downgrade and cancellation path, and do not treat a cancellation acknowledgement as transactional rollback. A lost DDL/DML response remains ambiguous.

Three identities

Identity	Representation	Lifetime and use
Wire request ID	Exactly 16 opaque bytes in every request envelope	Correlates response frames. For supported one-shots, it also targets `OP_CANCEL_REQUEST`.
Engine query ID	Nonzero unsigned 64-bit integer	Identifies admitted work in `system.queries` and `system.query_log`; SQL `CANCEL QUERY` uses it.
Subscription ID	Engine-assigned subscription identity	Identifies one subscriber attached to a stream/channel; it is not a one-shot request-cancel key.

Generate a fresh nonzero request ID for every operation that may be outstanding and verify the byte-for-byte echo on every ordinary response. The request ID is not a numeric value and must never be parsed as query_id.

Metadata-bearing v1 responses expose engine identity additively:

#meta\telapsed_us=<n>\tserver=<id>[\tcache=hit|miss]\tquery_id=<n>

Handshake schema 2 plus typed-metadata capability 0x0004 carries the same identity as query_id:u64be inside TypedResultEnvelope. JavaScript adapters expose the exact one-shot value as bigint; streaming metadata callbacks use decimal text so the full uint64 range is preserved. Do not narrow either to a JavaScript number.

One-shot lifecycle

The normal cancellable SQL/endpoint one-shot path is:

client creates request ID
        |
        v
frame decoded; connection/auth-class and raw-operator access gates pass
        |
        v
pending request/query identity registered
        |
        v
bounded admission queue
        |
        v
pending identity atomically promoted to live query
        |
        v
remaining endpoint/provider/grant/governance checks when applicable
        |
        v
planning and execution checkpoints
        |
        v
one captured result encoded as typed data or TSV fallback
        |
        v
terminal response
        |
        v
live request/profile identity removed
        |
        v
query-log completion appended

The pending registration happens before admission. A cancellation request can therefore find work that is waiting for an execution slot, not only work already visible as an executing operator tree.

One execution, two possible encodings

When binary results are engaged, the server executes the statement once and captures its rows. A supported shape becomes a typed ColumnarFrame (or a schema-2 metadata envelope around one). An unsupported or no-row shape becomes escaped TSV from that same capture. The fallback does not replay SQL. Metadata-bearing OP_QUERY_META/OP_QUERY_PARAMS fallbacks keep the original timing, query identity, server identity, and parameter-cache state in #meta; bare OP_QUERY remains plain TSV with no metadata carrier.

This encoding rule is separate from transport retry. A client that loses the socket may choose to repeat a conservatively classified read on a fresh connection; that is a new execution and must follow the retry policy below.

Cancel by request ID

OP_CANCEL_REQUEST (0x0A) targets a pending or executing one-shot without waiting for its eventual metadata:

cancel envelope request_id := fresh 16-byte correlation ID
cancel payload             := target request_id:bytes[16]

The payload must be exactly 16 bytes and the target must be nonzero. The cancel connection passes the same auth and live-revocation gates as other operations, but dispatch occurs before query admission.

The stable acknowledgement is TSV:

query_id\tcancel_requested
<matched_query_id>\t1

or, when no unique pending/live match exists:

query_id\tcancel_requested
0\t0

1 means the shared cancellation flag was set for one request. 0 means the target was missing, stale, zero, or not unique; nothing was cancelled. Neither row proves that the original request has reached a terminal response.

Supported targets are:

OP_QUERY (0x02);
OP_QUERY_META (0x04);
OP_QUERY_PARAMS (0x06); and
OP_ENDPOINT_EXEC (0x0B).

Streams, endpoint subscriptions, auth, capability negotiation, ping, and asynchronous block ingest are deliberately not request-cancel targets.

Use a separate control connection

The server processes ordinary requests sequentially on each connection. A cancel frame queued behind the active one-shot cannot overtake it. Open a separate control socket, negotiate/authenticate it as required, and send the cancel operation there.

After a successful acknowledgement, keep the original request slot occupied until its terminal frame arrives. If no terminal arrives within a bounded grace period, close that exact target connection. Do not release the member to another queued statement while an unread terminal frame can still arrive.

Cancel by engine query ID

Operators can find the numeric identity and use SQL:

SELECT query_id, state, elapsed_us, statement
FROM system.queries
ORDER BY elapsed_us DESC;

CANCEL QUERY 42;

This uses the same cooperative execution flag but a different lookup key. Wait for the live row to disappear, then confirm the archived outcome in system.query_log. Live removal happens just before log publication, so use a bounded poll rather than treating one empty read as missing evidence.

Disconnect-driven cancellation

Production one-shots attach a bounded, non-blocking peer-liveness probe to their execution budget. The raw transport probe does not allocate or consume queued bytes; the WebSocket path may retain bounded decoded/application-frame state:

raw TCP FIN/reset is detected without consuming pipelined application bytes;
WebSocket Close is processed as a terminal peer signal;
WebSocket Ping receives Pong while the one-shot is executing; and
bounded pipelined application frames are retained for the normal connection loop instead of being discarded.

Operator and parallel storage checkpoints observe the same sticky peer state. The terminal reason is:

NYXDB_EXEC_CANCELLED: client disconnected

and is retained in system.query_log. Cancellation is cooperative: the configured checkpoint interval bounds work units between observations, not wall-clock time inside an arbitrary kernel or device operation.

Closing a stream or endpoint-subscription transport remains terminal for that subscription. There is no durable cross-connection cursor; reconnect, authenticate again, subscribe again, and treat the new snapshot as a replacement boundary.

Shipped client behavior

Client/path	One-shot connection strategy	Metadata and cancellation behavior
Web console	Six-member WebSocket pool; one request on wire per member, excess calls queued locally	Production pool uses `OP_QUERY_META` for ordinary SQL and `OP_QUERY_PARAMS` for parameterized calls; schema 2 yields strict typed metadata. A local timeout sends `0x0A` on a separate socket, authenticating it when credentials are configured, then bounds terminal grace and closes the exact member if necessary.
`nyxsql` CLI	One connection per one-shot	Negotiates schema 2, then schema 1, then plain TSV on fresh sockets. A timeout closes the owning connection and relies on disconnect cancellation. The UI has no one-shot cancel handle; `Ctrl+C` stops only a focused live stream.
JDBC	Lazily opens and then reuses one persistent one-shot query connection; stream result sets own dedicated sockets	Schema 2 uses `OP_QUERY_META` and retains transport-level `ResultMetadata`; standard JDBC `Statement`/`ResultSet` does not expose the engine query ID or wire-cancel operation. Conservatively classified read-only statements may be retried once after transport failure.
Custom endpoint client	Reuse only under a strictly sequential policy; dedicate sockets to subscriptions	Cancel `OP_ENDPOINT_EXEC` by request ID on a separately authenticated socket. Cancel `OP_ENDPOINT_SUB` by closing its dedicated transport.

The protocol defines LZ4 capability 0x0002, but the current in-tree TypeScript and JDBC one-shot clients request binary results plus schema-2 typed metadata (0x0001 | 0x0004) and do not enable LZ4.

Retry decision table

Outcome	Safe default
Server SQL/parameter/budget error	Surface it. Do not reconnect-loop; change the statement, input, policy, or capacity condition.
Cancel acknowledgement `1`	Await or bound the original terminal result. Do not immediately reuse its connection.
Cancel acknowledgement `0`	Reconcile whether the request already completed, never existed, or used the wrong ID. Do not cancel by guessed query ID.
Read-only transport loss	Retry only when the client classifies one complete read conservatively and applies bounded backoff/deadline. Treat it as a new execution.
DDL/DML transport loss	Outcome is ambiguous. Inspect catalog/table state or apply an application idempotency policy before repeating.
Stream transport loss	Open a new subscription and replace local state with its new snapshot; do not append it as cursor continuation.
Client disconnect cancellation	Confirm the distinct reason and cleanup in `system.query_log` before diagnosing an orphan.

Release validation checklist

Negotiate schema 2, reject malformed/mismatched replies, and prove downgrade uses fresh connections.
Validate typed results and same-execution TSV fallback with decimal, temporal, nullable, vector, unsupported, and no-row statements.
Keep request ID, engine query ID, and subscription ID type-distinct in the client API.
Cancel a one-shot while queued and while scanning persisted/committed data; confirm one terminal result and one query-log record.
Exercise raw TCP FIN/reset and WebSocket Close, plus Ping/Pong and a pipelined frame during execution.
Prove a cancelled or disconnected statement releases admission, operator memory, and exact-native global reservations before the next request.
Test ambiguous mutation loss without automatic replay.

For byte layouts, see Experimental wire protocol. For operator budgets, see Query admission and limits. For application-user one-shots, see Browser and platform endpoint clients.

Query identity, cancellation, and retries

On this page