perf: Send the changes a peer lacks as they are read, on the server

Collecting pending changes ran in one repeatable-read transaction and
handed nothing out until every change was read. A device's first sync of
a large space thus heard nothing from the server until the end: on a home
server, a library of 56,621 rows took about 100 s, so the client's idle
timeout (40 s, its pings unanswered meanwhile) closed the stream first,
and every attempt started over. Reading the rows in batches made that
shorter, not bounded.

An engine may now hand each change out as soon as it is read
(streamCollectedChanges), the collection's transaction staying open until
the last is out or the listener cancels; cancelling ends the collection
and its transaction, and an error, an integrity violation among them,
reaches the listener once the transaction rolled back, recorded as before.
The server's initializeOfflineSync switches it on: PostgreSQL's snapshot
only reads. A device's engine keeps it off, since Serverpod's SQLite
transactions are write transactions and would hold off the device's own
writes for as long as the peer takes to receive. A receiver merges a batch
only once its end arrives, so a stream broken halfway still merges
nothing.

The new tests collect 600 inserts and 300 renames for a peer behind: as
read, the first change arrives before the renamed rows are read; whole, once
all are; both merge into the author's rows. On embedded PostgreSQL, a row
written by another session during the streamed collection stays out of it,
the changes are a whole collection's, in order, and a listener stopping
after the first lets the next collection run. The orphan-insert violation
test runs both ways. Every PostgreSQL test file passes alone; run together,
the files bind the same port, the two already there as well.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
Tizian 「ティツィアーン」 2026-11-09 15:47:48 +01:00
commit 6fdd926cf0
Signed by: raisondetredev
GPG key ID: 07F6FCF405488B64

View file

@ -37,16 +37,23 @@ extension OfflineSyncInitialize on Serverpod {
///
/// [continuousSyncInterval] controls how long a continuous sync session waits
/// after completing one sync round before checking for local changes again.
///
/// [streamCollectedChanges] sends the changes a peer lacks as they are read
/// rather than once all are, so a peer's first sync of a large space hears
/// from the server before its idle timeout; the server's PostgreSQL holds
/// the collection's read-only snapshot meanwhile.
void initializeOfflineSync({
required List<Table> syncTables,
int syncBatchSize = OfflineSyncEngine.defaultSyncBatchSize,
Duration continuousSyncInterval = OfflineSyncEngine.defaultContinuousSyncInterval,
bool streamCollectedChanges = true,
}) {
_offlineSyncByServerpod[this] = OfflineSyncEngine(
syncTables: syncTables,
serializationManager: serializationManager,
syncBatchSize: syncBatchSize,
continuousSyncInterval: continuousSyncInterval,
streamCollectedChanges: streamCollectedChanges,
);
}
}