A merge wrote each inserted row one by one: a savepoint for the domain
row, another for its tracker, two statements each, and then read every
tracker back to record attempted values. A device's first push of a
library of 8,000 cards (24,650 changes) took the server 34 to 37 s, nearly
all of it in those round trips.
A batch the projection pass leaves out (nothing in it touches a foreign
key or a unique index, or its inserts are proven to change no decision)
now writes its new rows many to a statement, up to 1,000 at a time, in
one savepoint: each table's domain rows in one insert, parents' tables
first, then their trackers. A row qualifies when the batch inserts it once
and touches it no other way, no tracker exists for it, nothing earlier in
the batch names it, and every row of the batch it names was written
before it. Its values are planned in the batch's order, as the per-row
path plans them. A chunk whose writes fail (a row there already, perhaps
another space's) is rolled back, the presence it claimed for its rows is
forgotten, and it and every row after it are left to the per-row path,
which tells those cases apart. Attempted values read the trackers the
merge holds instead of querying them.
The proof for merged inserts also accepts a parent of the row's own
table inserted earlier in the batch, so a library's nested studysets no
longer send its first push through the pass; a chunk ends before a row
whose parent of its own table it holds. A row naming itself, or a parent
inserted after it, still takes the pass.
Measured on PostgreSQL, the 24,650 changes merge in about 5 s instead of
34 to 37 s, and with the studysets nested instead of 41 s. A device on
SQLite, whose adapter still inserts row by row, receives the library in
44 s instead of 60 s. The new tests cover a batch spanning chunks, a chunk
failing on a row whose tracker was lost and falling back, and a tree of
one table's rows inserted parents first (the shortcut, a chunk per level)
and children first (the pass). The core, server, integration and
simulation suites pass, but for two PostgreSQL tests that can't start
their database while another embedded PostgreSQL runs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A merge that deleted anything ran the full projection pass, whose closure
follows foreign keys both ways and so loads every row connected to the
deleted ones. A device deleting a card with its revisions, tags and
reviews cost each device receiving it about 4 s, and the server merging
the push about 5.4 s, in a library of 8,000 cards whose tags join it.
A device deleting a row locally tombstones everything visible below it,
so its batch carries those deletes along. What the pass could still
decide is a row below that the deleting device never saw, inserted here
meanwhile, which it hides by cascade. A batch that only deletes (none
restoring) now leaves the pass out when the deleted tables cascade only,
by the same rules as a local delete (_deletesOnlyCascade, shared with
canSkipProjectionAfterLocalDelete now), no attempted value is pending in
the connected tables, and every visible row below a deleted one, found by
walking down the cascades alone, is deleted by the same batch
(canLeaveProjectionUnchangedByMergedDeletes). A batch both inserting and
deleting, or restoring, takes the pass as before.
Measured on PostgreSQL: such a card delete merges in about 35 ms instead
of 5.4 s; received on a device, 99 ms instead of 4 s. The new tests
cover a leaf deleted through the shortcut and a middle row's delete,
which isn't eligible, hiding its children by the pass; in the app's sync
tests a card deleted while another device rated it declines the shortcut,
and dropping the walk's check fails that test. The core, server,
integration and simulation suites pass, but for two PostgreSQL tests
that can't start their database while another embedded PostgreSQL runs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every ephemeral OfflineSyncDatabase shares one context but initializes a
recorder of its own, and each rebuilt the projections of every space
when the context's first initialize had changed the registry: on every
session and every merge, until the process stopped (upstream #150). A
new device, or one whose app changed its synced tables, spent about 3 s
of every inbound merge on it with a library of 8,000 cards, a server
after such a migration as much on every request.
The context now runs the rebuild once (Once): wrappers initializing
while it runs await the same run, later ones skip it, and a failed run
is tried again by the next.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A merge that touched any foreign key ran the planning pass for its new
rows and then the full projection pass, whose closure follows foreign
keys both ways and so loads every row connected to the merged ones. In a
library whose tags join its parts, one merged rating, edit, tag or new
card loaded all of it: about 9.5 s on a device receiving it and 12 s on
the server merging a push, for 8,000 cards.
A batch that only inserts new rows (each once) and updates columns no
projection reads now leaves both passes out when its inserts are proven,
as local inserts are (canLeaveProjectionUnchangedByMergedInserts, on
localProjectionDependenciesAreStable): each row references its parents by
id, each parent visible or inserted earlier in the same batch, nothing
refers to the row yet (no row, no attempted value), and no unique claim
is contested. A parent inserted after its child in the batch would find
the child's reference held back, which only the pass restores, so it
counts only when earlier. Deletes, inserts onto rows there already and
anything unproven take the passes as before. The batch's parents are
still resolved together for the writes.
The new tests cover a child of a visible parent, a parent and child in
one batch (both through the shortcut), and a child of a merged delete
and a parent arriving after its held-back child (both through the
passes); dropping the parent or the reference check fails the matching
test. The core, server, integration and simulation suites pass, but for
two PostgreSQL tests that cannot start their database while another
embedded PostgreSQL runs, with or without this change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
insteadOfDelete ran the full foreign key projection after every local
delete. Its closure follows edges both ways, so it loads every row
connected to the deleted one, which in a schema whose tables all hang off
one another is nearly the whole database: one deleted row of a person's
8,000 flashcards loaded 56,000 of 56,350 CRDT rows and took about 8 s on
the calling isolate, most of it deserialising rows. Inserts and updates
have had a proof that the pass is unnecessary
(canLeaveLocalProjectionUnchanged); deletes had none.
canSkipProjectionAfterLocalDelete proves it for a delete: every table
the delete can cascade into is reached by cascade edges on id only, none
of them restricts its parent (a deleted child can release a parent
delete held back by it), none has a unique index (a released claim can
bring a contestant back) or lies in a component with a SET DEFAULT edge,
and no attempted value is pending in the connected tables. The delete's
own cascade (applyDeleteActions) then is the whole projection; otherwise
the full pass runs as before. applyDeleteActions also asks for the
visible children of a whole batch of parents at once for cascades by id,
instead of one query per parent.
On that database a single delete now takes 6-63 ms, a card with its
revisions, tags and reviews 46-145 ms instead of 28-39 s. The new tests
cover a restrict child deleted locally releasing a merged parent delete
(on the person tables, and on the chain tables, where the shortcut is
asked and declines: the test fails with the restrict condition removed)
and a cascade child deleted locally through the shortcut, its parent
then deleted by a merged remote delete. The core, server, integration
and simulation suites pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
onMergeSuccess is called for every space a sync touched, also for one
this peer only sent changes of, so a client re-read its data after every
sync that pushed. onInboundMerge is called only for a space whose
changes from the remote peer were merged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
projectionValuesEqual took a Uint8List (how a bytea column arrives) down
its List branch and compared it byte by byte, and tryUuidValue parsed
every value as a UUID by throwing and catching an error when it was none.
Each byte of a payload thus cost a thrown TypeError, about 18 µs: two
payloads of 2 KB took 36 ms to compare. A sync whose projection loaded a
few thousand rows with payloads kept the server's isolate busy for
minutes, during which it answered no other request and the client's
method stream timed out, so the client sent the same changes again.
Byte values are now compared as bytes, and tryUuidValue only parses what
can hold a UUID (a UuidValue, 16 bytes, a 36-character string), returning
null for anything else without throwing. A UUID compares with its string
and its bytes as before. Longer byte values are no longer read as the
UUID of their first 16 bytes, which made canonicalProjectionValue group
payloads with the same beginning together.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMm7usAo6q9ZZRoPkWjoxi
