Skip to main content
DB

Concepts

PSI

The engine's one internal contract for publishing committed changes to every consumer that derives state from them.

PSI is the engine's single internal change-propagation contract. Every feature that derives state from committed writes — streaming reads, transformations, materialized views, indexes, projections, caches and the change feed — receives those writes through PSI. There is no second delivery path with its own semantics.

PSI has no SQL of its own. You use it through STREAM SELECT, transformations and materialized views, and you observe it through system relations.

Publication

Each committed write to a table tablet is published once, as an immutable batch on that tablet's committed feed. A batch is identified by:

  • relation;
  • tablet;
  • owner epoch;
  • commit sequence;
  • batch ordinal.

Only committed changes are published. For keyed tables (UPSERT and VERSIONED), the published row is the authoritative post-image: the current winner for the key, or a tombstone for a deletion. A write to one attribute group reaches continuous consumers as the key's complete current row.

Subscription

Warning

PSI is a core engine primitive, not a memory-service implementation detail. Subscribers select changes by exact stream:key pairs (relation, component and key), never by a stream name alone. Indexes, materialized views, schema and cache updates, and future subscribers route through this shared contract.

A component is a part of a relation's stored state, such as one attribute group. A transformation that writes one group of its own source has that group's writes filtered from its subscription, so it does not receive its own writes back.

Cursors and acknowledgement

Each consumer keeps its own durable cursor and acknowledges only what it has applied:

  • producers never wait on consumers; a slow transformation or index does not hold back a write;
  • after a restart, a consumer resumes from its cursor;
  • a consumer whose cursor is no longer covered by the retained feed reseeds explicitly from a snapshot. It never skips changes silently.

Delivery is admitted per owner against the feed's retention capacity, so one consumer cannot grow the retained feed without bound.

Consumers

Consumer kindUsed by
public_sessionSTREAM SELECT sessions
derived_arrangementShared continuous channels and materialized views
transformationTransformations
indexSecondary indexes
projectionPhysical columnar projections
cacheCaches
changefeedThe change feed
systemSystem relations
federatedA continuous read forwarded from another instance

Observing PSI

RelationShows
system.psi_streamsOne row per subscription: consumer kind, demand, queued frames and bytes, delivered and acknowledged sequence, resnapshot count and reason, source-to-delivery lag, lifecycle state.
system.psi_arrangementsOne row per shared arrangement: generation, latest and checkpoint sequence, state rows and bytes, subscribers by consumer kind, input changes applied and output deltas emitted.

What PSI is not

PSI is an internal contract of one node. It is not an external message broker, it exposes no client offsets, and it does not replicate changes between nodes.

On this page