refactor!: Rename sync APIs and ownership spaces (#128)

This commit is contained in:
Marcelo Mendonça Soares 2026-09-14 13:39:07 -03:00 • committed by GitHub
commit b89725f6ea
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
9 changed files with 43650 additions and 9417 deletions

View file

@ -54,7 +54,7 @@ soft-deleted instead of removed:
- `ON DELETE SET DEFAULT`: visible child FK columns are updated to the column
default when the default is legal, and those child FK fields receive ordinary
CRDT field updates. A nullable null default is legal; a non-null default must
remain visible in the same scope after the delete. A default in the same
remain visible in the same space after the delete. A default in the same
delete batch is unavailable. If a visible child needs this repair, an invalid
default rejects the transaction, including any earlier repairs or tombstones
in that transaction. An unused invalid default does not itself block a delete.
@ -100,7 +100,7 @@ restrict edges are evaluated first internally.
Hidden children do not block parent deletions. They still participate in FK
projection: when no legal repair exists and the authored parent remains
physically present in the same scope, the FK candidate retains the authored
physically present in the same space, the FK candidate retains the authored
reference, even if that parent is hidden. This recomputes from authored values;
it does not preserve an earlier projected fallback. Unique projection can then
release that candidate, as described in the combined projection model below.
@ -134,7 +134,7 @@ common case skips the dependency walk.
When a default is changed directly, or affected through a possible deletion,
cascade, or missing parent, the loader queries sparse dependency seeds in the
current scope: authored tombstones and hidden rows in the original-parent and
current space: authored tombstones and hidden rows in the original-parent and
cascade-ancestor tables, hidden children, and children with FK overrides. The
ordinary reference walk then loads their children, parents, and unique claimants.
Blocked deletions are included even when the original parent is still visible
@ -145,7 +145,7 @@ starting from authored tombstones.
This keeps ordinary merges on the existing row-closure path; there is no
component-wide domain-row load. Default-specific invalidation can still visit
many rows when many deletes, hidden rows, or repair claims depend on that default.
The sparse metadata lookup itself may scan rows in the relevant scoped metadata
The sparse metadata lookup itself may scan rows in the relevant space-scoped metadata
tables, but it runs only for affected defaults. Full rebuilds still load the
complete requested components.

View file

@ -38,10 +38,10 @@ The three steps below contain all required changes on a project to use the packa
3. Wire the sync call on the client.
```dart
// Push local changes and merge remote ones once (i.e. pull-to-refresh).
await client.crdt.syncOnce(session);
await client.offlineSync.syncOnce(session);
// Or stream changes both ways until cancelled for a near-real-time sync.
final syncSession = client.crdt.syncContinuously(session);
final syncSession = client.offlineSync.syncContinuously(session);
```
After this setup, the app can be developed using normal repository calls against the `session` instance.
@ -50,7 +50,7 @@ The sync engine is a **black box that "just works"**: changes are tracked atomic
Understanding the behavior is also straightforward: conflict resolution respects unique and relational constraints, mirroring what would be expected if the merged data existed in a single database at the time of each operation.
Finally, the data is isolated per scope: each user has their own implicit personal scope, and can be a member of any number of shared scopes - with read-only or read-write access. On the server, database operations need to be wrapped in a special `session.db.transactionForUser` method to target the correct scope. On the client, it is possible to set a `persistentUserId` to target the personal scope by default, or use the `transactionForUser` to target a shared scope.
Finally, the data is isolated per space: each user has their own implicit personal space, and can be a member of any number of shared spaces - with read-only or read-write access. On the server, database operations need to be wrapped in a special `session.db.transactionForUser` method to target the correct space. On the client, it is possible to set a `persistentUserId` to target the personal space by default, or use the `transactionForUser` to target a shared space.
#### Data modeling limitations
@ -58,12 +58,12 @@ Due to the nature of merge conflicts, the package imposes some limitations on th
- Every synced table must:
- Have a UUID primary key.
- Declare the field `scopeId: int?, relation(parent=crdt_scopes, onDelete=Cascade)`.
- Unique indexes must include `scopeId` together with the target columns.
- Declare the field `spaceId: int?, relation(parent=offline_sync_spaces, onDelete=Cascade)`.
- Unique indexes must include `spaceId` together with the target columns.
- Global unique indexes are unsupported, except for FK-only indexes.
- Unique indexes are only supported with at least one `String`/`UuidValue`/nullable column.
- All 1:1 relations must have the foreign-key column nullable (`optional` relation).
- The only allowed non-synced-to-synced relation is `scopeId -> crdt_scopes.id`.
- The only allowed non-synced-to-synced relation is `spaceId -> offline_sync_spaces.id`.
- Foreign-key relations that foreign-key projection cannot repair — non-nullable columns and composite relations — must be `deferred`.
- `onDelete=Restrict` is unsupported on synced tables; use `onDelete=NoAction`.
@ -103,9 +103,9 @@ Below is one of the last performance reports of the package (available at each p
Delay per delete: 408.75 μs
```
```
🔭 SELECT scope impact (100 extra users, 100,000 CRDT noise rows):
find all (1,000 rows): 9.91 ms --> 10.91 ms scoped (+10.07%) / 11.19 ms unscoped (+12.96%)
findById (per lookup): 295.04 μs --> 375.24 μs scoped (+27.19%) / 348.88 μs unscoped (+18.25%)
🔭 SELECT space impact (100 extra users, 100,000 CRDT noise rows):
find all (1,000 rows): 9.91 ms --> 10.91 ms space-scoped (+10.07%) / 11.19 ms unscoped (+12.96%)
findById (per lookup): 295.04 μs --> 375.24 μs space-scoped (+27.19%) / 348.88 μs unscoped (+18.25%)
```
```
💽 Storage impact (base footprint removed):
@ -139,9 +139,9 @@ A report similar to the one above is generated per pull request as a comment on
**Visibility as a pure function.** Replicas never coordinate. Each merge applies remote facts monotonically, then derives the visible database — tombstone state, unique-conflict winners, and foreign-key repair — as a deterministic projection computed to a fixed point. This is the architecture introduced by *Synql* (Ignat et al., DAIS 2024), extended here to the full set of `ON DELETE` actions and to unique constraints over deeply relational schemas.
**Ownership & scopes.** A *scope* is the unit of replication and ownership; a *user* is an authentication identity. Each row is globally identified by `(table, rowId)` and owned by exactly one scope (`scopeId`), so merging is always one chain per row. Sharing is a membership layer in front of scopes, not a change to row ownership. Scopes can be purged for GDPR / account erasure via a database-enforced cascade. Ownership collisions (the same `(table, rowId)` claimed by two scopes, which is mostly a malicious attack vector) fail the sync and are recorded in a durable `crdt_sync_integrity_violations` table.
**Ownership & spaces.** A *space* is the unit of replication and ownership; a *user* is an authentication identity. Each row is globally identified by `(table, rowId)` and owned by exactly one space (`spaceId`), so merging is always one chain per row. Sharing is a membership layer in front of spaces, not a change to row ownership. Spaces can be purged for GDPR / account erasure via a database-enforced cascade. Ownership collisions (the same `(table, rowId)` claimed by two spaces, which is mostly a malicious attack vector) fail the sync and are recorded in a durable `offline_sync_integrity_violations` table.
**Sync protocol.** A session handshakes once to validate both ends have the same table schema and ensure that a merge will reach the same result. Then, per cycle the server sends the authoritative scope set and all scopes that the user has access to are synced (data is transferred all at once, but merged one at a time). The data is transferred in chunks to avoid pressuring the bandwidth with too many small messages. This same stream-based implementation is used for both "one-shot" and "continuous" sync modes, so a one-shot sync with large amounts of data will be as performant as a continuous sync. It also means that all tests hold for both sync modes with minimal duplication.
**Sync protocol.** A session handshakes once to validate both ends have the same table schema and ensure that a merge will reach the same result. Then, per cycle the server sends the authoritative space set and all spaces that the user has access to are synced (data is transferred all at once, but merged one at a time). The data is transferred in chunks to avoid pressuring the bandwidth with too many small messages. This same stream-based implementation is used for both "one-shot" and "continuous" sync modes, so a one-shot sync with large amounts of data will be as performant as a continuous sync. It also means that all tests hold for both sync modes with minimal duplication.
## Testing
@ -153,13 +153,13 @@ The package is functionally in a production-ready state, but some APIs need to i
### Technical debts
- **Rename the `scope` name for data isolation to `space`**. This is a small refactor to avoid confusion with the `scope` keyword used in the Serverpod Auth modules. Using `space` is also clearer for usage on an application level.
- **Rename the `space` name for data isolation to `space`**. This is a small refactor to avoid confusion with the `space` keyword used in the Serverpod Auth modules. Using `space` is also clearer for usage on an application level.
### Post-integration UX improvements
The package already requires minimal configuration from the developer, but it can be even smoother if the `serverpod_cli` is made aware of the offline sync capabilities.
- **Add a fourth `sync` option for the `database` keyword to be used on models**. Models that use it will automatically receive the `scopeId` relation and be included in a generated list of synced tables on the server and client. This feature will enable all improvements below.
- **Add a fourth `sync` option for the `database` keyword to be used on models**. Models that use it will automatically receive the `spaceId` relation and be included in a generated list of synced tables on the server and client. This feature will enable all improvements below.
- **Simplify the `Serverpod` initialization by generating the class**. This is mapped on Serverpod's issue #4544 and will allow the CRDT database interceptor to be injected on the `Serverpod` constructor without the user having to call it manually whenever a `sync` model exists.
- **Move the restrictions that are runtime-enforced to a static analysis step**. The package enforces the limitations of the CRDT layer at runtime (during the initialization), but a better UX would be to apply the restrictions to `sync` models during static analysis of `generate`.
- **Inject the `serverpod_offline_sync` packages on the `pubspec.yaml` file**. This would remove the last manual step from the developer to make the solution work. Otherwise, the user will have to manually add the dependencies to their `pubspec.yaml` to generate the project.

View file

@ -4,7 +4,7 @@
Outbound change collection reads CRDT metadata and the domain data it describes
in **separate, non-atomic queries**. Under concurrent domain writes — real on
the server, where co-members of a shared scope write while a sync runs — this
the server, where co-members of a shared space write while a sync runs — this
race can make an otherwise-healthy stream sync **fail and record a durable
integrity violation** for a condition that the very next round would resolve on
its own. Failing the session for a transient, self-correcting race is the design
@ -12,12 +12,12 @@ weakness to fix; no data is ever applied incorrectly, so this is a
robustness/liveness problem, not a corruption one.
This is **not yet implemented** — it is tracked design debt. The single-pass
multi-scope collection (`collectPendingChanges`) did not introduce the race
multi-space collection (`collectPendingChanges`) did not introduce the race
but widened its window, which is what surfaced it.
## The mechanism
Collection is *snapshot-then-fetch*, and lock-free by design (the per-scope
Collection is *snapshot-then-fetch*, and lock-free by design (the per-space
`transactionForUser` lock guards the inbound merge, not outbound collection):
1. `_streamInserts` / `_streamUpdates` / `_streamDeletes` read the CRDT metadata
@ -62,7 +62,7 @@ up the tombstone, the late insert, or the higher HLC, and replicas converge.
Tearing down the stream — and persisting an integrity-violation record that reads
like corruption — for a race that resolves itself next round converts an
ordering skew into an outage plus a misleading audit trail. A durable
`crdt_sync_integrity_violations` entry should mean a *genuine* invariant breach
`offline_sync_integrity_violations` entry should mean a *genuine* invariant breach
(an ownership collision, real corruption), not "two reads happened to straddle a
concurrent commit."
@ -71,9 +71,9 @@ concurrent commit."
- **Read a consistent snapshot (primary).** Run the whole collection — the three
metadata queries *and* the domain reads — inside one read-only,
snapshot-consistent transaction (repeatable-read). On Postgres this is MVCC:
**a read snapshot, not a write lock**, so it adds no cross-scope write
**a read snapshot, not a write lock**, so it adds no cross-space write
contention (and is therefore the right answer to "do we have to lock all
scopes?" — no, we take a snapshot, not a lock). A single snapshot closes the
spaces?" — no, we take a snapshot, not a lock). A single snapshot closes the
race outright:
- The delete in (1) is invisible if it commits after the snapshot, and fully
visible (metadata + domain) if before — never half-seen.
@ -98,7 +98,7 @@ concurrent commit."
Correctness is preserved today (convergence + fail-safe: no wrong data is ever
applied), so this does not block current use. But because it can fail a healthy
long-lived continuous sync under concurrent shared-scope writes — the exact
workload shared scopes enable — it should be addressed before such deployments
long-lived continuous sync under concurrent shared-space writes — the exact
workload shared spaces enable — it should be addressed before such deployments
are relied upon. The consistent-snapshot approach is the smallest change that
removes all three failure modes at once.

View file

@ -160,7 +160,7 @@ Outbound collection substitutes `CrdtDataAttemptedValue.value` whenever it
exists; otherwise it reads the domain value. It never exports an FK candidate,
unique loser value, or temporary parking value as an authored update.
An empty replica merging a complete scope export must reproduce normalized
An empty replica merging a complete space export must reproduce normalized
domain rows, hidden state, authored values/HLCs, and derived projection. A
rebuild may rewrite `projectionReason`, but it must preserve
`CrdtDataAttemptedValue.value`: while the row exists, that is the only copy of
@ -210,7 +210,7 @@ store an otherwise supported dynamic value.
### Initialization algorithm
Call `await crdtDatabase.initialize()` after application migrations and before
Call `await offlineSyncDatabase.initialize()` after application migrations and before
serving requests or starting sync. Reconciliation must plan before mutating:
1. Load the persisted CRDT registry and target `TableDefinition`s.

View file

@ -1,4 +1,4 @@
# Row ownership and scope isolation
# Row ownership and space isolation
Status: implemented.
@ -14,26 +14,26 @@ different users silently overwrite each other across tenant boundaries.
This document makes ownership explicit. The core decisions:
> **Every synced domain row is globally identified by `(table, uuidRowId)` and
> owned by exactly one scope.** `scopeId` is not part of row identity; it is
> ownership and isolation metadata on a globally identified row. Two scopes
> never materialize or sync the same `(table, uuidRowId)`. Cross-scope UUID
> owned by exactly one space.** `spaceId` is not part of row identity; it is
> ownership and isolation metadata on a globally identified row. Two spaces
> never materialize or sync the same `(table, uuidRowId)`. Cross-space UUID
> reuse is a terminal ownership collision, not valid multi-tenant state. Sync
> fails on that collision, records a durable violation, and does not advance
> the sender past the rejected change. Sharing is modeled as shared scopes that
> multiple identities subscribe to — never as two scopes aliasing one row.
> the sender past the rejected change. Sharing is modeled as shared spaces that
> multiple identities subscribe to — never as two spaces aliasing one row.
> **Every synced table declares `scopeId`, identically on server and
> **Every synced table declares `spaceId`, identically on server and
> client.** The column is a hard requirement, never `scope=serverOnly`, so
> both databases run one schema shape and the package has exactly one code
> path. "Single user" is a usage pattern (a database that happens to contain
> one scope), not a mode: there are no mode flags, no schema branching, and
> one space), not a mode: there are no mode flags, no schema branching, and
> no per-mode test matrix.
> **The sync layer owns the column: stamp if null, assert if set — and the
> value leaves the way it came in.** On every insert, a null `scopeId` is
> stamped from the effective scope; a non-null one must equal it or the
> value leaves the way it came in.** On every insert, a null `spaceId` is
> stamped from the effective space; a non-null one must equal it or the
> operation throws. Stamped values are stripped back to null on returned
> rows and on scoped reads, so the database-local value never leaks into
> rows and on space-scoped reads, so the database-local value never leaks into
> endpoint payloads; explicitly provided values are kept on the way out.
> One rule serves both personalities: users who never want to think about
> tenancy never do, and users who pass it explicitly get verified
@ -46,8 +46,8 @@ validated at `initialize()` with paste-ready error messages:
```yaml
fields:
id: UuidValue?, defaultPersist=random_v7
### Owner scope of this row. Maintained by the CRDT sync layer.
scopeId: int?
### Owner space of this row. Maintained by the CRDT sync layer.
spaceId: int?
```
No model inheritance is needed: like the UUID primary key requirement, the
@ -56,7 +56,7 @@ by `CrdtSchemaRegistry`), not nominal (a base class), which is the only
option anyway since Serverpod models cannot extend models from an imported
module. The same constraint shapes the implementation: with no common
interface to call, the package stamps the field through dynamic dispatch —
`(row as dynamic).copyWith(scopeId: …)` (or the plain field setter, since
`(row as dynamic).copyWith(spaceId: …)` (or the plain field setter, since
generated fields are mutable). The field name is a compile-time constant, so
the serialization manager's `patchTableRow` would be the wrong tool here: its
serialize–patch–deserialize round-trip costs 50–200× a dynamic call and is
@ -79,13 +79,13 @@ justified only where field names are data-driven, as in the merge path.
`patchTableRow`.
3. **Roles are not hard-coded.** Nothing in the package assumes the server
is multi-user or the client single-user. Multi-account devices and
single-tenant servers are the same code path with different scope counts.
single-tenant servers are the same code path with different space counts.
4. **Application code cannot get tenancy wrong.** The package must verify
any user-supplied scope against the effective scope regardless (a tenant
any user-supplied space against the effective space regardless (a tenant
id from application code can never be trusted), so correctness never
rests on call-site discipline.
5. **Multi-device per user keeps working unchanged.** Devices of one user
share the user's scope and merge into one chain.
share the user's space and merge into one chain.
6. **Sharing must not be foreclosed.** Collaborative data is a core
local-first use case; the design must leave a clean path to it.
@ -93,8 +93,8 @@ justified only where field names are data-driven, as in the merge path.
The server holds one physical domain row per `(table, uuidRowId)`, while CRDT
metadata (`crdt_data_rows` and its children) is keyed per
`(scopeId, tblId, uuidRowId)`. Before this design was implemented, that
metadata shape let multiple scopes track the same row even though the domain
`(spaceId, tblId, uuidRowId)`. Before this design was implemented, that
metadata shape let multiple spaces track the same row even though the domain
table could only materialize one copy. Symptoms, in increasing severity:
1. **Tombstone masking.** A tombstone recorded by one user (or for another
@ -125,31 +125,31 @@ All four share one root cause: the metadata says who owns a row, but nothing
enforces that the answer is unique, and the domain storage assumes it is.
"Leave it up to users" is rejected explicitly: symptom 2 means the storage
model cannot represent what multi-scope tracking promises, so allowing it is
model cannot represent what multi-space tracking promises, so allowing it is
not granting flexibility — it is permitting silent cross-tenant data
corruption. Applications that need shared data use shared *scopes* (see
`shared-scopes.md`), which preserve the one-chain-per-row property that makes
corruption. Applications that need shared data use shared *spaces* (see
`shared-spaces.md`), which preserve the one-chain-per-row property that makes
LWW merging coherent.
## The `scopeId` column
## The `spaceId` column
### Contract
`scopeId` is a reserved column name, required on every synced table:
`spaceId` is a reserved column name, required on every synced table:
- **It is not part of row identity.** The domain primary key remains the
required `id: UuidValue`, and `(table, uuidRowId)` remains globally unique
in the physical domain table. A row with the same UUID in a different scope
in the physical domain table. A row with the same UUID in a different space
cannot be represented as a second domain row; it is treated as a corrupt or
hostile ownership collision, the merge fails, and the violation is recorded.
- The value references `crdt_scopes.id` — the scope that owns the row. Today
a scope is a user; the name is chosen deliberately so shared scopes can be
- The value references `offline_sync_spaces.id` — the space that owns the row. Today
a space is a user; the name is chosen deliberately so shared spaces can be
added later without renaming storage again.
- **The value is database-local.** It is a normalization of the scope UUID
into the local `crdt_scopes` table; the client's int for a scope is not
the server's int for the same scope. The sync layer re-maps it on merge
- **The value is database-local.** It is a normalization of the space UUID
into the local `offline_sync_spaces` table; the client's int for a space is not
the server's int for the same space. The sync layer re-maps it on merge
and never ships it as data, and application code cannot ship it by
accident either, because scoped reads return it stripped (see *Keeping
accident either, because space-scoped reads return it stripped (see *Keeping
the value off the wire*). This is also why the field is nullable: a model
outside the database honestly carries "not mine to say" instead of a
meaningless local id.
@ -159,77 +159,77 @@ LWW merging coherent.
- The column is **owned by the sync layer** and immutable after insert.
The value round-trips the way it came in: callers that never provide it
never receive it (stamped values are stripped from returned rows, and
scoped reads return null), explicit callers keep seeing what they passed,
space-scoped reads return null), explicit callers keep seeing what they passed,
and unscoped admin reads always return it populated (see *Keeping the
value off the wire*).
### Write semantics: stamp if null, assert if set
On every local insert (and batch insert), the database proxy resolves the
effective scope — transaction scope, else persistent scope, else the
effective space — transaction space, else persistent space, else the
operation throws, exactly as recording already requires — and applies one
rule per row:
- `scopeId == null` → the row is stamped with the effective scope before the
- `spaceId == null` → the row is stamped with the effective space before the
delegate insert (one write, not an UPDATE after the fact). Stamping uses a
dynamic call, since no base interface can declare the field; this is the
per-row hot path of every insert, which is why the heavier `patchTableRow`
stays out of it. In-place mutation is preferred (generated fields are
mutable, so no per-row allocation); dynamic `copyWith` is the fallback
only if rows were ever immutable.
- `scopeId != null` → it must equal the effective scope, or the operation
- `spaceId != null` → it must equal the effective space, or the operation
throws. Explicit callers get verification for free.
Consequences of the rule:
- **Cross-scope writes cannot succeed.** A model carrying another scope's id
- **Cross-space writes cannot succeed.** A model carrying another space's id
trips the assertion; a model carrying null is stamped with the acting
scope and then the scoped `WHERE` filter (which includes
`scopeId = <scope>`) matches no foreign row, so the update or delete fails
space and then the space-scoped `WHERE` filter (which includes
`spaceId = <space>`) matches no foreign row, so the update or delete fails
with "no rows" instead of touching another tenant's data.
- **The column never appears in update SET lists**, exactly like `id`
(`_updateRowWithoutRecording` already filters `columnName != 'id'`).
A tampered model cannot attempt to move a row across scopes.
- **Merges stamp from the merging scope** (each merge acts in exactly one
scope), and incoming payloads have the field stripped by
A tampered model cannot attempt to move a row across spaces.
- **Merges stamp from the merging space** (each merge acts in exactly one
space), and incoming payloads have the field stripped by
`_sanitizeMergeRowData` — it is not portable data.
The same effective-scope requirement applies to every CRUD entrypoint, not
The same effective-space requirement applies to every CRUD entrypoint, not
only inserts:
- `insert`/`insertRow`/batch insert: stamp-or-assert before the delegate write;
- reinsert of a tombstoned row: only the same owning scope can revive it;
a foreign-scope row id is a UUID ownership collision, not a reinsert target;
- `update`/`updateRow`: the scoped `WHERE` includes `scopeId = <scope>`, and
`scopeId` is removed from the SET list even if it appears in `columns`;
- `updateWhere`/`updateById`: any `ColumnValue` for `scopeId` is rejected (or
- reinsert of a tombstoned row: only the same owning space can revive it;
a foreign-space row id is a UUID ownership collision, not a reinsert target;
- `update`/`updateRow`: the space-scoped `WHERE` includes `spaceId = <space>`, and
`spaceId` is removed from the SET list even if it appears in `columns`;
- `updateWhere`/`updateById`: any `ColumnValue` for `spaceId` is rejected (or
dropped before SQL generation, but rejection gives better caller feedback),
and the scoped `WHERE` still includes `scopeId = <scope>`;
and the space-scoped `WHERE` still includes `spaceId = <space>`;
- `delete`/`deleteRow`/`deleteWhere`: deletes only see rows owned by the
effective scope, so a foreign row id returns "no rows" instead of recording a
effective space, so a foreign row id returns "no rows" instead of recording a
tombstone for another owner;
- future `upsert`: must first resolve ownership of the conflicting row and may
update only when the existing row's `scopeId` equals the effective scope.
update only when the existing row's `spaceId` equals the effective space.
### Keeping the value off the wire
The wire discipline is enforced by construction and respects the caller's
choice: **the field leaves the package the way it came in.**
- **Stamped values stay internal.** When a model comes in with `scopeId`
null, the proxy stamps the effective scope for the database write, and
- **Stamped values stay internal.** When a model comes in with `spaceId`
null, the proxy stamps the effective space for the database write, and
the rows handed back by that operation carry null again. The same applies
to scoped reads, which have no input model to respect. Stripping applies to
to space-scoped reads, which have no input model to respect. Stripping applies to
every synced model instance returned by the proxy, including nested
`IncludeObject` and `IncludeList` result graphs. Callers who never provide
the value never observe it, and cannot serialize the database-local int into
endpoint payloads even by accident.
- **Explicit values are respected.** When a model comes in with `scopeId`
- **Explicit values are respected.** When a model comes in with `spaceId`
set (and verified by the assert arm), the rows returned by that operation
keep it — the package does not hide a value the caller knowingly manages.
In batch operations the rule applies per row.
- **Unscoped (admin) reads keep it populated.** Within a scoped read the
value is redundant — it always equals the acting scope — but for admin
- **Unscoped (admin) reads keep it populated.** Within a space-scoped read the
value is redundant — it always equals the acting space — but for admin
reads it is the only per-row ownership information, and those results are
consumed server-side.
@ -244,14 +244,14 @@ This mirrors mature row-level-security practice, where the tenant column is
populated from session context (`DEFAULT current_setting(...)`) precisely so
application code cannot get it wrong, rather than passed through call sites.
A convenience accessor exposing the effective scope's normalized id (e.g. on
A convenience accessor exposing the effective space's normalized id (e.g. on
the transaction or the database) may be added for explicit callers and
custom filtering; it is not required by any flow, and its documentation
carries the database-local warning above.
### Sync-layer exclusions
`scopeId` is infrastructure that happens to live in the domain row. Keyed by
`spaceId` is infrastructure that happens to live in the domain row. Keyed by
the reserved name, it is excluded from:
- `crdt_schema_columns` registration and field-level CRDT metadata (it has no
@ -259,38 +259,38 @@ the reserved name, it is excluded from:
the synced-schema hash symmetrically on both sides;
- update recording (`afterUpdate` ignores it) and merge payload sanitization;
- outbound sync insert payloads and outbound sync update payloads. The sender
must not serialize `scopeId` from the domain row; the receiver stamps the
merge-local normalized scope instead.
must not serialize `spaceId` from the domain row; the receiver stamps the
merge-local normalized space instead.
### Validation
`CrdtSchemaRegistry` extends the existing UUID primary key check:
- every synced table must have a nullable `int` column named `scopeId`
- every synced table must have a nullable `int` column named `spaceId`
(missing or mistyped → error with the paste-ready yaml, including the
"remove `scope=serverOnly`" hint when the server schema has the column and
the client schema does not);
- every non-primary unique index on a synced table must include `scopeId`
- every non-primary unique index on a synced table must include `spaceId`
(missing → error), except a foreign-key-only unique index whose referenced
tables are also synced. Ordinary global unique indexes are forbidden for now
because the package does not yet define deterministic cross-scope conflict
because the package does not yet define deterministic cross-space conflict
resolution.
## Unique indexes
Per-scope uniqueness is declared by the application as an ordinary composite
Per-space uniqueness is declared by the application as an ordinary composite
index, which now exists identically on both databases:
```yaml
indexes:
person_scope_name_unique_idx:
fields: scopeId, name
person_space_name_unique_idx:
fields: spaceId, name
unique: true
```
Ordinary global unique indexes are rejected until the package has a
deterministic cross-scope arbitration policy and a way to record the
losing-side release in that scope's CRDT chain. "Whoever syncs first" is not
deterministic cross-space arbitration policy and a way to record the
losing-side release in that space's CRDT chain. "Whoever syncs first" is not
an acceptable CRDT rule. The only permitted global unique indexes are
foreign-key-only indexes: every index element must be a column, and the
indexed columns must be composed only of complete foreign-key column sets whose
@ -299,47 +299,47 @@ policy, every indexed FK column in that exception must be nullable. A required
unique FK has no safe loser value: `null` violates the column, a synthetic UUID
violates the FK constraint, and keeping the original value violates uniqueness.
This covers optional Serverpod relation indexes, including optional one-to-one
and composite FK relation indexes, without forcing duplicate per-scope indexes
and composite FK relation indexes, without forcing duplicate per-space indexes
for include-query performance. Those columns point at globally unique synced row
ids, and FK enforcement already repairs or rejects cross-scope references, so
ids, and FK enforcement already repairs or rejects cross-space references, so
they are not treated as global application-level arbitration. The model must
still declare scoped composite unique indexes for application uniqueness. On a
deployment that happens to contain one scope, a composite unique with a constant
`scopeId` accepts exactly the same rows as a plain unique; nothing degrades.
still declare space-scoped composite unique indexes for application uniqueness. On a
deployment that happens to contain one space, a composite unique with a constant
`spaceId` accepts exactly the same rows as a plain unique; nothing degrades.
The unique-conflict resolver discovers indexes from the table definitions,
uses `scopeId` as part of the lookup predicate for scoped indexes, and releases
only releasable non-`scopeId` columns. For allowed FK-only global unique
uses `spaceId` as part of the lookup predicate for space-scoped indexes, and releases
only releasable non-`spaceId` columns. For allowed FK-only global unique
indexes, nullable FK columns may be released to `null`; required FK columns are
rejected during schema initialization. Same-scope application conflicts are
handled by scoped composite indexes, and cross-scope references are handled by
FK repair. Rows in different scopes never form a conflict group under the
rejected during schema initialization. Same-space application conflicts are
handled by space-scoped composite indexes, and cross-space references are handled by
FK repair. Rows in different spaces never form a conflict group under the
required composite index. The flag policy (see `unique-constraint-invariants.md`)
is otherwise unchanged.
## Enforcement mechanics
1. **Local writes: stamp-or-assert** (see *Write semantics*), with the
effective scope required exactly as it already is for CRDT recording.
effective space required exactly as it already is for CRDT recording.
2. **Merge-insert ownership check.** `_applyMergeInsertForMissingRow` treats
an existing domain row with another `scopeId` as a UUID ownership collision.
an existing domain row with another `spaceId` as a UUID ownership collision.
This is a terminal sync error, not a recoverable conflict policy. The check
is atomic with the physical write:
1. Look up the domain row by `(table, uuidRowId)` before writing.
2. If it exists with the merging scope, follow the same-scope
2. If it exists with the merging space, follow the same-space
update/reinsert path.
3. If it exists with another scope, roll back the merge work for the
3. If it exists with another space, roll back the merge work for the
incoming change, record a durable integrity violation with denormalized
operation, table, row, scope, node UUID, and HLC fields, then fail the
operation, table, row, space, node UUID, and HLC fields, then fail the
sync session.
4. If it does not exist, insert with `scopeId` stamped from the merging
scope.
5. If the insert races another scope and hits the domain primary key,
4. If it does not exist, insert with `spaceId` stamped from the merging
space.
5. If the insert races another space and hits the domain primary key,
re-read the row and apply the same ownership decision before any CRDT
metadata survives the transaction.
Step 5 carries an implementation constraint: today the merge records the
scope's CRDT row (`_upsertMergeRow`) — and unique-conflict resolution and
space's CRDT row (`_upsertMergeRow`) — and unique-conflict resolution and
FK-safe handling, which may mutate neighboring rows — *before* the domain
write where the collision surfaces. The insert application must therefore
run inside a savepoint rolled back on collision (or be reordered
@ -349,9 +349,9 @@ is otherwise unchanged.
diagnostic record survives even though the bad change does not.
The durable record is sparse exception state, not a second ownership index.
`crdt_sync_integrity_violations` stores one deduplicated row per observed
`offline_sync_integrity_violations` stores one deduplicated row per observed
violation key: `(type, operation, domainTableName, uuidRowId,
ownerScopeUuid, incomingScopeUuid)`. All other stored fields are
ownerSpaceUuid, incomingSpaceUuid)`. All other stored fields are
denormalized scalars so the record remains useful after merge rollbacks and
on databases that do not share the same local CRDT metadata rows. There are
no foreign keys into CRDT metadata tables (`crdt_schema_tables`,
@ -366,9 +366,9 @@ is otherwise unchanged.
causal context; optional `crdtDataRowId` as a same-database hint to the
local metadata row id when one was known (for example outbound sync with a
stale tracker); `firstSeenAt`, `lastSeenAt`, and `occurrences`.
`ownerScopeUuid` is nullable because a corrupt physical row can exist with
`ownerSpaceUuid` is nullable because a corrupt physical row can exist with
a null owner;
`incomingScopeUuid` is required and globally locatable. `crdtDataRowId` is
`incomingSpaceUuid` is required and globally locatable. `crdtDataRowId` is
often null on merge-insert collisions and is not portable across devices.
The record intentionally stores neither the rejected payload nor a payload
hash. The sync response returns the violation identifier and checkpoints do
@ -382,23 +382,23 @@ is otherwise unchanged.
between honest clients are negligible; a collision is a bug or an attack,
and failing the sync is the only response that preserves both the existing
owner's data and the sender's still-unaccepted local change.
3. **Foreign keys stay within a scope.** A synced row may only reference
synced rows of its own scope. Locally,
`_assertVisibleForeignKeyTargets` gains a scope-equality check next to
its visibility check; in merges, a foreign-scope target is treated
3. **Foreign keys stay within a space.** A synced row may only reference
synced rows of its own space. Locally,
`_assertVisibleForeignKeyTargets` gains a space-equality check next to
its visibility check; in merges, a foreign-space target is treated
exactly like an invisible target, so the existing attempt/override
machinery (see `foreign-key-invariants.md`) repairs the reference instead
of linking across scopes. Cascade deletes and SET NULL projections
therefore never cross a scope boundary. References to non-synced tables
of linking across spaces. Cascade deletes and SET NULL projections
therefore never cross a space boundary. References to non-synced tables
are unaffected.
4. **No duplicated metadata lookup index.** An earlier draft added a unique
index on `crdt_data_rows (tblId, uuidRowId)` as defense-in-depth. It is
dropped because the hot CRDT paths need the existing scoped index
`(scopeId, tblId, uuidRowId)`, and adding the unscoped index duplicates
dropped because the hot CRDT paths need the existing space-scoped index
`(spaceId, tblId, uuidRowId)`, and adding the unscoped index duplicates
most of the key space on every row. Instead, every consumer of CRDT row
metadata is required to prove ownership against the physical row: reads key
visibility through `domain.scopeId`; outbound sync serializes a domain row
only when `domain.scopeId` equals the collecting scope; merge, FK, and
visibility through `domain.spaceId`; outbound sync serializes a domain row
only when `domain.spaceId` equals the collecting space; merge, FK, and
unique helpers reject metadata that attempts to materialize or mutate a
foreign-owned row. A stray foreign tracker is therefore inert by consumer
discipline, while an observed attempt to use it is a durable sync violation.
@ -413,24 +413,24 @@ index-covered probe; the aggregate (`GROUP BY`/`MIN`) form, its full-scan
cost (~95 ms per point lookup at 1M CRDT rows in the SQLite benchmark), and
the covering index it wanted never ship.
- **Scoped query** (a `transactionForUser` read, or the persistent default
scope): rows are filtered by `scopeId = <scope>` — the isolation promise,
- **Space-scoped query** (a `transactionForUser` read, or the persistent default
space): rows are filtered by `spaceId = <space>` — the isolation promise,
and the central point of the design — plus the existing correlated
`NOT EXISTS` visibility probe on `(scopeId, tblId, uuidRowId)`.
- **Unscoped query** (no transaction scope, no persistent scope — admin
`NOT EXISTS` visibility probe on `(spaceId, tblId, uuidRowId)`.
- **Unscoped query** (no transaction space, no persistent space — admin
reads): no isolation filter; the visibility probe is keyed by the row's
own column — `NOT EXISTS (… WHERE c.scopeId = t."scopeId" AND tblId = ?
own column — `NOT EXISTS (… WHERE c.spaceId = t."spaceId" AND tblId = ?
AND uuidRowId = t.id AND visibility > …)` — fully covered by the existing
unique index. A row is visible while its owner sees it.
On a database containing one scope, the two views return the same rows; no
code anywhere branches on scope count. Rows with a null `scopeId` — created
On a database containing one space, the two views return the same rows; no
code anywhere branches on space count. Rows with a null `spaceId` — created
behind the sync layer's back (raw SQL, seed fixtures) — are visible only to
unscoped reads; seeding flows must stamp scopes, which ties into the
unscoped reads; seeding flows must stamp spaces, which ties into the
reference-table open question.
Multi-scope deployments additionally gain Postgres row-level security as an
option, since policies can reference `scopeId` directly — the package's
Multi-space deployments additionally gain Postgres row-level security as an
option, since policies can reference `spaceId` directly — the package's
isolation filter and RLS become two enforcements of one predicate.
## Sync path
@ -438,71 +438,71 @@ isolation filter and RLS become two enforcements of one predicate.
Sync also consumes CRDT metadata, so it follows the same row-ownership rule
as reads and writes:
- **Outbound inserts** collect `crdt_data_rows` for one scope, then fetch the
physical domain row only if `domain.scopeId` equals that same scope. A
- **Outbound inserts** collect `crdt_data_rows` for one space, then fetch the
physical domain row only if `domain.spaceId` equals that same space. A
stale or hostile tracker for a foreign-owned row fails the sync and records
a durable violation before any payload is produced.
- **Outbound updates** fetch the column value only after the field's CRDT row
is proven to match the domain row owner. `scopeId` itself is never emitted
is proven to match the domain row owner. `spaceId` itself is never emitted
as an update because it has no field metadata.
- **Outbound deletes** are scoped by CRDT metadata; a foreign tracker can at
worst try to produce a tombstone for a row the collecting scope never owned,
- **Outbound deletes** are space-scoped by CRDT metadata; a foreign tracker can at
worst try to produce a tombstone for a row the collecting space never owned,
so delete collection also verifies the domain owner when the domain row still
exists and fails with a durable violation on mismatch.
- **Inbound inserts and updates** sanitize any incoming `scopeId` before merge
and stamp from the merge scope. Incoming row UUIDs that already belong to
another scope fail the sync session as ownership collisions, record durable
- **Inbound inserts and updates** sanitize any incoming `spaceId` before merge
and stamp from the merge space. Incoming row UUIDs that already belong to
another space fail the sync session as ownership collisions, record durable
violations, and do not advance checkpoints past the rejected change.
This is the audit rule for all future sync helpers: no path may serialize,
materialize, resolve conflicts for, or project a synced domain row through
CRDT metadata unless the domain row's `scopeId` equals the effective CRDT
scope, except explicit unscoped admin reads.
CRDT metadata unless the domain row's `spaceId` equals the effective CRDT
space, except explicit unscoped admin reads.
## API surface
The package is unreleased, so renames land directly, with no aliases and no
data migration. Public API stays user-first (that is the semantics
developers expect to write); internals and storage are named after what they
actually key — scopes:
actually key — spaces:
- **`transactionForUser(userId, fn)`** keeps its name and shape: it resolves
the user's UUID to the local scope and pins the transaction to it — writes
the user's UUID to the local space and pins the transaction to it — writes
stamp and assert against it, reads are isolated to it. The optional
`scopeId` parameter is reserved for shared scopes (see *Future*).
- **`persistentUserId`** remains the way to pin a default scope outside
`spaceId` parameter is reserved for shared spaces (see *Future*).
- **`persistentUserId`** remains the way to pin a default space outside
explicit transactions — the standard single-user client configuration.
- **`mergeChanges(scopeId: …)`** keys the chain being merged — the parameter
- **`mergeChanges(spaceId: …)`** keys the chain being merged — the parameter
is renamed from `userId` to what it always was.
- **Internals and storage** adopt scope naming in one sweep: `crdt_users` →
`crdt_scopes` (`CrdtUser` → `CrdtScope`, `uuidUserId` → `uuidScopeId`),
`crdt_data_rows.userId` → `scopeId` (matching the domain column),
- **Internals and storage** adopt space naming in one sweep: `crdt_users` →
`offline_sync_spaces` (`CrdtUser` → `OfflineSyncSpace`, `uuidUserId` → `uuidSpaceId`),
`crdt_data_rows.userId` → `spaceId` (matching the domain column),
`userForTransaction`, the effective-user helpers, and the HLC manager keys.
- **Unscoped reads** remain supported as admin reads.
Consequences for application code, stated as contract:
1. **Every synced table declares `id: UuidValue` and `scopeId: int?`.**
1. **Every synced table declares `id: UuidValue` and `spaceId: int?`.**
Startup fails with a paste-ready error otherwise.
2. **`id` remains globally unique per synced table.** The same
`(table, uuidRowId)` cannot exist in two scopes. If merge observes that
UUID already owned by another scope, it records a durable ownership
`(table, uuidRowId)` cannot exist in two spaces. If merge observes that
UUID already owned by another space, it records a durable ownership
violation and fails the sync.
3. **`scopeId` round-trips the way it came in** — left null, the sync layer
stamps it internally and returns it null (scoped reads return it null
too); provided explicitly, it must match the effective scope or the
3. **`spaceId` round-trips the way it came in** — left null, the sync layer
stamps it internally and returns it null (space-scoped reads return it null
too); provided explicitly, it must match the effective space or the
operation throws, and it stays visible on the way out. The column is
immutable after insert.
4. **Scoped reads are isolated to their scope.** Reading across scopes is an
4. **Space-scoped reads are isolated to their space.** Reading across spaces is an
explicit unscoped (admin) read.
5. **Merge-inserts colliding with another scope's row fail the sync**, never
5. **Merge-inserts colliding with another space's row fail the sync**, never
apply over the owner's data, and never advance the sender past the rejected
change.
6. **Foreign keys on synced tables target same-scope rows.** Cross-scope
6. **Foreign keys on synced tables target same-space rows.** Cross-space
references are repaired by the existing FK override machinery instead of
being linked.
7. **Application unique indexes on synced tables include `scopeId`.** Global
unique indexes are rejected at startup until a deterministic cross-scope
7. **Application unique indexes on synced tables include `spaceId`.** Global
unique indexes are rejected at startup until a deterministic cross-space
policy exists, except FK-only indexes whose referenced tables are synced.
## Implementation plan
@ -510,165 +510,165 @@ Consequences for application code, stated as contract:
Each phase lands independently green; later phases depend on earlier ones
only where stated. No generator or Serverpod fork changes are required.
- **Phase 1 — naming, contract, validation.** The user→scope renames in
- **Phase 1 — naming, contract, validation.** The user→space renames in
internals and storage (above), done first so later phases build on the
final names. `CrdtSchemaRegistry`: the `scopeId` presence/type check with
final names. `CrdtSchemaRegistry`: the `spaceId` presence/type check with
the `serverOnly` hint, and the global unique-index rejection. Reserved-name
exclusions: `crdt_schema_columns` registration, field-level merge, update
recording, inbound merge sanitization, and outbound sync serialization.
Test models gain `scopeId`.
Test models gain `spaceId`.
- **Phase 2 — write-path enforcement.** Stamp-or-assert in the proxy insert
paths (dynamic dispatch on the known field); `scopeId` rejected from
paths (dynamic dispatch on the known field); `spaceId` rejected from
`updateWhere`/`updateById` column values and excluded from model-update SET
lists; ownership assertion on update, delete, reinsert, and upsert; atomic
merge-insert ownership check with durable fail-fast collision handling,
including primary-key race handling; merge stamping from the merging scope;
FK same-scope assertion and merge repair routing; unique resolver limited
to scoped indexes, with ordinary global indexes rejected by validation.
- **Phase 3 — read paths.** Isolation filter for scoped reads; row-keyed
visibility probe for unscoped reads; stripping of `scopeId` from scoped
including primary-key race handling; merge stamping from the merging space;
FK same-space assertion and merge repair routing; unique resolver limited
to space-scoped indexes, with ordinary global indexes rejected by validation.
- **Phase 3 — read paths.** Isolation filter for space-scoped reads; row-keyed
visibility probe for unscoped reads; stripping of `spaceId` from space-scoped
root and include-graph read results; outbound sync owner checks before
serializing inserts, updates, or deletes; delete the `GROUP BY`/`MIN` form
and the server-only-index TODO from `tombstone.dart`.
- **Phase 4 — tests and benchmarks.** The plan below, plus updating
`TombstoneScopeBenchmark` to ownership-conformant seeding (one tracker per
`TombstoneSpaceBenchmark` to ownership-conformant seeding (one tracker per
row); its current second-tracker fixtures survive only as the enforcement
scenario.
### Test plan
Because no code branches on scope count or deployment role, mechanisms are
Because no code branches on space count or deployment role, mechanisms are
tested **once**, not per combination:
- **Mechanism suites (run once).** All existing CRUD, tombstone visibility,
unique-conflict, FK-invariant, merge, and sync suites continue on the
uniform model; test models declare `scopeId` and are otherwise unchanged
uniform model; test models declare `spaceId` and are otherwise unchanged
(the stamp rule means call sites do not change).
- **Enforcement tests.** Stamping on insert; assert-on-mismatch throwing for
insert and update; immutability of the column; round-trip symmetry (null
in → null out, explicit in → kept, per row in batches); stripping on
scoped root and included read results and retention on unscoped reads; the
space-scoped root and included read results and retention on unscoped reads; the
caller's input instance left unmutated; merge-insert collision fails sync,
records a durable violation, and covers the primary-key race regression;
stale foreign tracker fails before producing outbound insert/update/delete
payloads and records a durable violation (the multi-user fixtures in
`tombstone_scope_test.dart` become these tests); FK cross-scope repair;
`tombstone_space_test.dart` become these tests); FK cross-space repair;
schema validation rejects ordinary global unique indexes and accepts
FK-only unique indexes whose referenced tables are synced.
- **Combination smoke tests (one each, not per mechanism).** A two-scope
database proving read isolation and per-scope uniqueness on SQLite; one
- **Combination smoke tests (one each, not per mechanism).** A two-space
database proving read isolation and per-space uniqueness on SQLite; one
`withServerpod` sync roundtrip proving the wiring end to end. These exist
to prove deployment shapes compose, not to re-verify mechanisms.
## Shared scopes
## Shared spaces
`crdt_scopes` is the entity everything is keyed by: sync, locking, HLC
`offline_sync_spaces` is the entity everything is keyed by: sync, locking, HLC
chains, and visibility. Sharing becomes an indirection in front of it, not a
change to row ownership:
- an authenticated identity maps to one personal scope plus any number of
shared scopes;
- a shared list/document/workspace is a scope that several identities
- an authenticated identity maps to one personal space plus any number of
shared spaces;
- a shared list/document/workspace is a space that several identities
subscribe to and sync;
- every row still has exactly one owning scope (its `scopeId`), so merging
- every row still has exactly one owning space (its `spaceId`), so merging
stays one chain per row and nothing in this document's enforcement
changes.
This is the standard shape of sharing in local-first systems, and it is the
reason the column and concept are named `scope` rather than `user`.
Implementation details and current limits live in `shared-scopes.md`; the rest
reason the column and concept are named `space` rather than `user`.
Implementation details and current limits live in `shared-spaces.md`; the rest
of this section summarizes why the ownership rules remain compatible.
### Users authenticate, scopes act
### Users authenticate, spaces act
The two concepts separate cleanly:
- a **user** is an authentication identity — what endpoints and the sync
handshake verify;
- a **scope** is the unit of replication and ownership — what transactions
act in, what chains and tombstones are keyed by, what `scopeId`
- a **space** is the unit of replication and ownership — what transactions
act in, what chains and tombstones are keyed by, what `spaceId`
references.
Two conventions bridge them with no storage migration:
1. **A user's personal scope has the user's own UUID.** This is already true
(the scope UUID keys the chain today), so every deployment without
sharing is the degenerate case "each user has exactly one scope: their
personal one". Shared scopes are new `crdt_scopes` rows whose UUID is no
1. **A user's personal space has the user's own UUID.** This is already true
(the space UUID keys the chain today), so every deployment without
sharing is the degenerate case "each user has exactly one space: their
personal one". Shared spaces are new `offline_sync_spaces` rows whose UUID is no
user's id.
2. **Membership** is a server-side relation, `crdt_scope_members(scopeId,
2. **Membership** is a server-side relation, `offline_sync_space_members(spaceId,
userUuid, role?)`, managed by application code (invitations and roles are
app domain). Personal-scope membership is implicit; no row is stored for
app domain). Personal-space membership is implicit; no row is stored for
it.
### The transaction API
`transactionForUser` keeps its name and user-first semantics — that is what
developers expect to write — and gains an optional scope:
developers expect to write — and gains an optional space:
```dart
db.transactionForUser(userId, fn); // acts in the personal scope
db.transactionForUser(userId, fn, scopeId: listId); // acts in a shared scope
db.transactionForUser(userId, fn); // acts in the personal space
db.transactionForUser(userId, fn, spaceId: listId); // acts in a shared space
```
- Without `scopeId`, the scope resolves to the user's personal scope — by
- Without `spaceId`, the space resolves to the user's personal space — by
convention 1, simply `userId` itself, which is exactly today's behavior.
- With `scopeId`, the `userId` argument is **not dead weight**: the package
asserts `membership(userId, scopeId)` before acting. The parameter pair
reads as "authenticated as `userId`, acting in `scopeId`, verified
- With `spaceId`, the `userId` argument is **not dead weight**: the package
asserts `membership(userId, spaceId)` before acting. The parameter pair
reads as "authenticated as `userId`, acting in `spaceId`, verified
member" — the same predicate row-level security policies mirror. The
caller remains responsible for `userId` being authenticated, exactly as
today.
- A transaction acts in **exactly one scope**. Cross-scope writes need
- A transaction acts in **exactly one space**. Cross-space writes need
separate transactions; this is deliberate honesty about CRDT semantics,
since two scopes' chains replicate independently and remote replicas can
never observe a cross-scope write atomically anyway.
since two spaces' chains replicate independently and remote replicas can
never observe a cross-space write atomically anyway.
### Reads: membership-wide; writes: scope-pinned
### Reads: membership-wide; writes: space-pinned
A user-scoped read in a sharing world naturally means "rows in any scope I am
a member of": the isolation filter becomes `scopeId IN (<member scopes>)`
instead of `scopeId = <scope>`. The visibility probe is the row-keyed form —
A user-space-scoped read in a sharing world naturally means "rows in any space I am
a member of": the isolation filter becomes `spaceId IN (<member spaces>)`
instead of `spaceId = <space>`. The visibility probe is the row-keyed form —
the same probe as unscoped admin reads — so sharing simplifies the predicate
matrix rather than growing it: reads differ only in their membership filter
(one scope, a user's scopes, or none for admin).
(one space, a user's spaces, or none for admin).
This is also exactly the shape of a Postgres row-level security policy
(`scopeId IN (SELECT scopeId FROM crdt_scope_members WHERE userUuid =
(`spaceId IN (SELECT spaceId FROM offline_sync_space_members WHERE userUuid =
current_setting(...))`), which is the standard practice for multi-tenant
isolation and on the Serverpod backlog; the package's application-level
filter and RLS become two enforcements of one predicate.
### Sync and clients
- A device syncs one chain per scope it holds: each sync call enumerates the
authenticated user's member scopes, verified server-side against
`crdt_scope_members` at the start of every sync cycle; the membership table
itself is not synced. `CrdtScopeRole` is enforced as described in
`shared-scopes.md`: reads stay membership-wide, writes are role-gated, and the
- A device syncs one chain per space it holds: each sync call enumerates the
authenticated user's member spaces, verified server-side against
`offline_sync_space_members` at the start of every sync cycle; the membership table
itself is not synced. `OfflineSyncSpaceRole` is enforced as described in
`shared-spaces.md`: reads stay membership-wide, writes are role-gated, and the
server records `unauthorizedWrite` for rejected inbound changes.
- Every database is already sharing-capable schema-wise — the column is
universal — so a sharing-enabled client simply holds several scopes' rows.
One database file per scope remains a valid alternative layout.
- Checkpoints stay per `(scope, node)`: `crdt_nodes` stores the stable replica
identity, while `crdt_scope_nodes` stores each scope's checkpoint for that
universal — so a sharing-enabled client simply holds several spaces' rows.
One database file per space remains a valid alternative layout.
- Checkpoints stay per `(space, node)`: `crdt_nodes` stores the stable replica
identity, while `offline_sync_space_nodes` stores each space's checkpoint for that
node.
### What sharing does not change
Every enforcement rule in this document survives unchanged: `(table,
uuidRowId)` stays globally identified, rows have exactly one owning scope,
merge-insert ownership checks compare scopes, unique conflict groups form
within a scope, ordinary global unique indexes stay rejected, and
stamp-or-assert applies with the transaction's resolved scope. Sharing adds a
membership layer in front of scopes; it does not touch row ownership.
uuidRowId)` stays globally identified, rows have exactly one owning space,
merge-insert ownership checks compare spaces, unique conflict groups form
within a space, ordinary global unique indexes stay rejected, and
stamp-or-assert applies with the transaction's resolved space. Sharing adds a
membership layer in front of spaces; it does not touch row ownership.
## Resolved during review
Decisions that were open questions in earlier drafts, kept here so they are
not relitigated by accident:
- **Optional vs. required `scopeId`:** required, on every synced table and
- **Optional vs. required `spaceId`:** required, on every synced table and
in both schemas. The optional-column draft and the schema-presence-mode
draft were both rejected: each created a second ownership architecture (or
a second schema shape) that every mechanism would have to support and
@ -677,8 +677,8 @@ not relitigated by accident:
cost of ~1–2 bytes per row and a visible read-only field on domain models.
- **Global UUID row identity:** retained. Serverpod's generated domain model
and ORM expect one primary key column, so the package does not introduce
composite `(scopeId, id)` identity or a surrogate physical row id. A
duplicate `(table, uuidRowId)` in another scope is an ownership collision,
composite `(spaceId, id)` identity or a surrogate physical row id. A
duplicate `(table, uuidRowId)` in another space is an ownership collision,
not a valid second row.
- **`scope=serverOnly` on the column:** rejected (would reintroduce
asymmetric schemas); validation forbids it.
@ -694,7 +694,7 @@ not relitigated by accident:
sync layer guarantees the value on every synced row.
- **Wire transport prevented by construction, respecting the caller:** the
value leaves the way it came in — stamped values are stripped from
returned rows and scoped reads, including include graphs; explicitly
returned rows and space-scoped reads, including include graphs; explicitly
provided (and verified) values are kept on the way out; unscoped admin
reads keep the value as the only per-row ownership information server-side;
outbound sync never serializes it. Mechanism: in-place dynamic mutation on
@ -703,7 +703,7 @@ not relitigated by accident:
field names are data-driven and its 50–200× cost is justified), never
leaving the caller's input observably mutated.
- **Unscoped-read semantics:** an unscoped read is an admin view; a row is
visible while its owner sees it. Scoped reads are isolated to their scope
visible while its owner sees it. Space-scoped reads are isolated to their space
— confirmed as the central goal of the design.
- **Fail vs. skip vs. re-key on colliding inserts:** fail and record a
durable violation. Skipping can silently consume a client's rejected change
@ -711,9 +711,9 @@ not relitigated by accident:
would break the incoming chain's own FK references.
- **Metadata backstop index:** dropped only with the consumer audit rule:
every read, write, sync, merge, FK, and unique path that consumes CRDT row
metadata must prove `domain.scopeId` equals the effective metadata scope
metadata must prove `domain.spaceId` equals the effective metadata space
before materializing domain data.
- **Audit surface:** a durable `crdt_sync_integrity_violations` table is part
- **Audit surface:** a durable `offline_sync_integrity_violations` table is part
of the terminal sync-integrity contract. Session logs may mirror it, but
they are not the source of truth.
- **No compatibility aliases:** the package is unreleased, so renames
@ -724,23 +724,23 @@ not relitigated by accident:
## Open questions
1. **Globally shared reference tables.** Read-only data synced to every user
would violate "every row has one owning scope" and is out of scope; if
would violate "every row has one owning space" and is out of space; if
the use case materializes, it likely becomes a third table kind rather
than a relaxation of the contract. Related: rows seeded behind the sync
layer's back carry a null `scopeId` and are visible only to admin reads.
2. **`scopeId` naming collisions.** The name is reserved by convention; an
application with a pre-existing, semantically different `scopeId` column
layer's back carry a null `spaceId` and are visible only to admin reads.
2. **`spaceId` naming collisions.** The name is reserved by convention; an
application with a pre-existing, semantically different `spaceId` column
on a synced table has no escape hatch other than renaming its column.
Acceptable for now; revisit if it bites real users.
3. **Membership revocation semantics (sharing).** When a user loses membership
of a scope, the server stops streaming it at the next sync cycle. The device
may still store the scope's data and unsynced local changes, though
of a space, the server stops streaming it at the next sync cycle. The device
may still store the space's data and unsynced local changes, though
membership-wide reads no longer expose revoked rows. Purge-vs-keep policy,
and whether revoked unsynced changes are surfaced or dropped, need their own
design.
4. **Roles within a scope (sharing).** Resolved in `shared-scopes.md`.
`crdt_scope_members.role` is enforced for writes through the closed
`CrdtScopeRole` enum: the server rejects and records unauthorized inbound
4. **Roles within a space (sharing).** Resolved in `shared-spaces.md`.
`offline_sync_space_members.role` is enforced for writes through the closed
`OfflineSyncSpaceRole` enum: the server rejects and records unauthorized inbound
writes, the client rejects local writes from projected `readOnly` roles, and
reads remain membership-wide. The remaining cleanup question for demoted or
revoked unsynced local changes is part of question 3.

649
docs/shared-spaces.md Normal file
View file

@ -0,0 +1,649 @@
# Shared spaces
Status: implemented. This records the shared-space behavior built on top of
the row-ownership work in `row-ownership.md`. Revocation cleanup is still a
deferred follow-up; where the ownership summary and this document differ, this
one wins.
## Summary
A user is an authentication identity; a space is the unit of replication and
ownership. Each user has a personal space and may also belong to any number of
shared spaces. The load-bearing
requirement is:
> **A single `sync` call syncs every space the authenticated user has access
> to.** One `syncOnce` cycles through all accessible spaces exactly once; a
> continuous session keeps cycling forever. There is no separate stream or
> separate call per space.
The row-ownership work keys storage, locking, HLC chains, visibility, and
ownership by **space**, not user, so sharing is an indirection in front of
`offline_sync_spaces`, not a re-architecture. The implemented pieces are a server-side
membership relation and a sync protocol that iterates spaces.
> **Merges serialize one space at a time.** A merge runs in exactly one space's
> transaction and lock, so spaces always serialize on the merge step regardless
> of framing. A cycle's outbound changes are instead collected in a single pass
> over the active spaces and carried in one combined batch, each change tagged
> with its `uuidSpaceId`; the receiver regroups by space and merges each group
> in its own transaction. The wire stays a near-verbatim repetition of today's
> frames, now space-tagged.
> **The server is the authority on membership; the client follows.** The set of
> spaces a session syncs is computed server-side from `offline_sync_space_members` and
> re-resolved at the start of every sync cycle, so access changes take effect
> mid-session. The server dictates the set; the client adopts spaces it
> announces and stops cycling the ones it omits. The reconciliation is
> directional — not a set operation toggled by a flag — and membership itself is
> never synced. Local data cleanup for omitted spaces is a separate revocation
> policy, not part of the sync loop.
> **Row ownership is unchanged.** Every row still has exactly one owning
> `spaceId`, `(table, uuidRowId)` stays globally identified, and every
> enforcement rule in `row-ownership.md` survives untouched. Sharing adds a
> membership layer in front of spaces; it does not touch row ownership.
## Goals and constraints
1. **One call, every space.** The public client API (`syncOnce`,
`syncContinuously`) does not change shape: callers pass a session, never a
space. A device with only a personal space behaves exactly as today.
2. **Smallest protocol delta.** The exchanged frames are today's, plus a
per-cycle space-set exchange and a space tag on each frame. `collectNextBatch`
demultiplexes a cycle's combined batch, `collectPendingChanges` collects every
active space in one pass, and the per-space merge plus `recordSyncCheckpoint`
run once per inbound space group.
3. **Chunked streaming.** Outbound changes are collected in a single pass over
the active spaces and streamed in `syncBatchSize` chunks, so the wire payload
stays bounded regardless of space count. The collection step itself
materializes a cycle's pending rows across the active spaces (one
`spaceId IN (…)` query per change kind), so peak memory tracks a cycle's
pending rows rather than being held strictly one space at a time. See
`outbound-collection-consistency.md`.
4. **Membership cannot be forged.** The server computes its own space set from
`offline_sync_space_members` (plus the implicit personal space) and never widens it
from anything the client sends. A space the user is not a member of is never
synced, even if the client names it.
5. **One space per transaction.** Reads are membership-wide; writes stay pinned
to exactly one space. This is deliberate honesty about CRDT semantics: two
spaces' chains replicate independently and a remote replica can never
observe a cross-space write atomically.
6. **Roles are closed CRDT access roles.** The package stores and projects
non-null `OfflineSyncSpaceRole` values in space grants and shared memberships:
`readWrite` allows CRDT writes and `readOnly` blocks them. The implicit
personal space is announced as a `readWrite` grant, so single-tenant servers
and multi-account devices remain the same code path with different space
counts.
## Membership
### The `offline_sync_space_members` table
```yaml
class: OfflineSyncSpaceMember
table: offline_sync_space_members
database: all
fields:
space: OfflineSyncSpace?, relation(onDelete=Cascade)
userUuid: UuidValue
role: OfflineSyncSpaceRole
indexes:
offline_sync_space_member_unique_idx:
fields: userUuid, spaceId
unique: true
```
- The table is **`database: all`** but **not** in `syncTables`. It exists on
every node so membership resolves with the same code on both ends, but it is
never CRDT-replicated by this package. The server is the authoritative writer;
a client copy (once populated) is a read-only cache for UI and offline role
checks — see *The client membership table*.
- The index **leads with `userUuid`** so the per-cycle `WHERE userUuid = ?`
resolution is index-covered.
- Rows are managed through `session.offlineSync.spaces` (`OfflineSyncSpaces`). Invitations,
acceptance, and authorization policy remain app domain.
- **Personal-space membership is implicit.** By convention a user's personal
space has the user's own UUID (already true: the space UUID keys the chain),
so no `offline_sync_space_members` row is stored for it. Shared spaces are
`offline_sync_spaces` rows whose UUID is no user's id, with explicit membership rows.
- `role` is a required CRDT access enum. Shared memberships always store an
explicit role: `readWrite` allows writes, while `readOnly` denies
shared-space writes.
### Members vs. nodes
`offline_sync_space_members` and the `nodes` list on a space are orthogonal layers and
both stay:
- A **member** is an authorization fact: a user identity allowed to access a
space. It answers *who may sync, read, and write*.
- A **node** (`crdt_nodes`) is a stable CRDT replica identity. A space's
`nodes` relation points at `offline_sync_space_nodes`, which records that replica's
participation and checkpoint state in that space. Together they answer
*whose changes, up to which HLC* and are what per-`(space, node)` checkpoints
key on.
They do not map one-to-one and neither replaces the other. One member may have
many nodes (one per replica/database install). The **server is a node** in every
space it syncs but is not a member row. A freshly invited member has **zero
space-node rows** until their first device syncs. Nodes and space-node rows are
created implicitly by sync (`getOrCreate`, `recordSyncCheckpoint`); membership
rows are managed through `session.offlineSync.spaces` from application endpoints.
Removing the `nodes` list would break checkpointing and causal filtering — it is
the chain topology, not the access list.
### The client membership table
`offline_sync_space_members` is `database: all`, so the table exists on the client and
is **populated as a read-only projection** of the server's authoritative grants
— without being bidirectionally CRDT-synced:
- The client's projected `offline_sync_space_members` rows plus the implicit personal
space are its membership view for *membership gating*: a membership-wide read
filters over projected membership, and a local `offline_sync_spaces` row by itself
does not grant access. The members table also carries **roles**, which the
client consults before local writes.
- The authoritative `OfflineSyncSpaceSet` carries `OfflineSyncSpaceGrant`s — `(uuidSpaceId,
role)` — not bare UUIDs. Every grant has an explicit role; the personal space
is announced as `readWrite` but remains implicit in storage. When the follower
*receives* an announcement, `projectFollowerMembership` reconciles the local
table from it: upsert a row per shared grant (the personal space is skipped),
delete rows whose space is no longer granted. Since the authoritative peer
announces only when its grants change, projection runs only on change and an
idle session writes nothing.
- This is a **server-authored, client-read-only projection**, never a synced
members table: the client never writes membership the server reads, so it
cannot self-grant. The server stays the security boundary and re-verifies on
sync.
- A revoke takes effect for sync as soon as the server omits the space from a
cycle's `SpaceSet`: the client stops cycling it and discards in-memory
checkpoint state for it. The space row and domain rows may remain stored
locally until the deferred revocation cleanup policy decides whether to purge
or keep them, but membership-wide reads stop returning those rows.
- Writes are bounded by projected membership and by the projected role. They are
re-verified by the server on sync, which remains the security boundary.
Client-side checks are early-failure UX, not the enforcement point.
### Role write access
The package exposes a closed CRDT access role enum rather than free-form role
strings:
```dart
enum OfflineSyncSpaceRole {
readOnly,
readWrite,
}
```
Only `readWrite` allows writes. `readOnly` denies writes. The implicit personal
space has no stored membership row, but its authoritative grant and role lookup
resolve to `readWrite`.
Roles gate **writes only**. Reads stay membership-wide and role-independent.
The authoritative server enforces role writes before applying inbound changes:
for each accepted inbound space group, it resolves the authenticated user's
role from `offline_sync_space_members`; if the role is not `readWrite`, the server
records `OfflineSyncIntegrityViolation(type: unauthorizedWrite)` and fails the
sync session before opening the merge transaction for that space. The rejected
change is not applied and checkpoints do not advance past it.
The client uses the same enum as early UX: `transactionForUser(userId, fn,
spaceId: sharedSpace)` resolves the projected role and throws
`OfflineSyncSpaceRoleException` before any local write when the role cannot write. A
follower also omits non-writable shared spaces from outbound pending-change
collection while still receiving inbound changes for those spaces. This avoids
repeatedly streaming local read-only changes; the server enforcement remains the
security boundary for stale, misconfigured, or hostile clients.
### Space enumeration (server is the authority)
At the start of each sync cycle the server resolves the authoritative set:
```
memberGrants(userUuid) =
{ personal space = userUuid, role = readWrite }
∪ { (s.uuidSpaceId, m.role)
: offline_sync_space_members m
join offline_sync_spaces s on s.id = m.spaceId
where m.userUuid = <auth user> }
```
This grant set bounds everything the cycle may sync and carries the role used by
followers for outbound filtering. The client never expands it; it only learns of
additions through what the server announces.
## Sync: one call syncs every accessible space
### One `sync`, two cadences: bounded `once`, idle-silent `continuous`
Merge is inherently one space at a time: a merge runs in exactly one space's
transaction and lock, so the spaces always serialize regardless of framing. A
single `sync` method serves both `syncOnce` and `syncContinuously`, branching on
`once` only where they genuinely differ — read discipline and termination — so
there is no duplicated send/merge logic:
1. **Establishment (shared, lockstep).** After `Connect`, both peers exchange the
`SpaceSet`. Doing this *before* the loop is what removes the announce/adopt
offset that would otherwise make a follower act before it knows the agreed
set.
2. **Data loop (shared, idle-silent).** Each cycle is **one combined batch**: the
space announcement (only when this peer's grants changed), a `SinceHlc` for
each newly active space, and merge chunks whose changes carry their space,
all closed by a single `EndOfBatch`. Continuous mode omits the terminator when
nothing was sent. The receive phase demultiplexes the peer's batch by type
until that terminator or — when the peer was idle — an idle timeout. An idle
continuous cycle therefore sends nothing and idles out **once**, regardless of
space count.
`once` stays in the data loop until every active space has exchanged `SinceHlc`
and had one sendable data cycle, then performs the symmetric `Close` handshake.
`continuous` loops instead, with no per-cycle chatter; it handshakes its initial
spaces in the first cycle and absorbs membership changes as they are announced
(a newly granted space is announced, adopted, and established over the following
cycles — its data deferred until its `SinceHlc` round-trips).
### Protocol frames
`Connect` opens the session once. After that the frames are the same in both
cadences; only *when* they are sent differs (`once`: a fixed sequence per space;
`continuous`: only on change, inside one combined batch). Frame changes
(regenerated models + a migration on both client and server schemas):
- `OfflineSyncConnect { syncTablesHash, localNodeId }` — sent once by each peer. The
schema hash is validated once, before any space work. `localNodeId` identifies
this peer's CRDT node for the whole session. The space list is dynamic and
exchanged per cycle.
- `OfflineSyncSpaceSet { spaces: List<OfflineSyncSpaceGrant> }` (each grant a
`uuidSpaceId` + `role`) — the server's content is authoritative (resolved from
`offline_sync_space_members`, roles included). Followers send an empty set because the
authoritative peer never widens access from follower-reported state. In `once`
it is exchanged exactly once. In `continuous` a peer re-announces it
**only when its own grant set changed** since the last announcement — this is
the frame that carries access changes mid-session, and gating it on change is
what keeps an idle session silent while still letting a grant or revoke take
effect.
- `OfflineSyncSinceHlc { uuidSpaceId, nodeCheckpoints }` — per space. Carries that
space's per-node checkpoints. Exchanged when a space first becomes active
(and again if a dropped space is later re-adopted);
checkpoints then live in memory for the rest of the session.
- `OfflineSyncMergeChunk { changes }` — each change carries `uuidSpaceId`, letting
a combined batch carry every space's changes and regroup them on receive.
- `OfflineSyncEndOfBatch`, `OfflineSyncClose`, `OfflineSyncIdleTimeout` — unchanged. A
single `EndOfBatch` terminates a cycle's combined batch (and is itself omitted
when nothing was sent, so an idle peer resolves to an empty batch instead).
`Close` ends the streaming session. Because it is an intentional peer control
frame, it may arrive even after frames for a partial cycle; the receiver
discards that partial cycle and records no checkpoint progress for it.
The data cycle's receive uses `collectNextBatch`, which demultiplexes one
cycle's frames by type into `{ spaceSet?, sinceHlcs, changes }`,
returning on the single `EndOfBatch`, on `Close` with a closed result, or —
when the peer was idle — on the idle timeout. A raw transport close without a
`Close` frame is stricter: it is graceful only between continuous-session
batches and is treated as truncation once a partial batch has arrived.
### The session state machine
```
handshake (both cadences):
send Connect(syncTablesHash, localNodeId); read peer Connect; validate
syncTablesHash; store peer localNodeId
establishment (both cadences, lockstep):
send SpaceSet(myGrants); read peer SpaceSet → adopt → activeSpaces
follower: materialize activeSpaces, project membership
data loop (both cadences, one combined batch per cycle):
loop:
resolve activeSpaces (authoritative: my set; follower: adopted peerGrants)
discard state for spaces no longer active
# send phase — emit only what changed; track whether any frame was sent
if myGrants != announcedGrants: send SpaceSet(myGrants)
for space in activeSpaces not yet sinceHlcSent: send SinceHlc(space)
if peer checkpoints exist: send MergeChunk(changes)* # follower skips non-writable spaces
if anything was sent (or once): send EndOfBatch
# receive phase
batch = collectNextBatch(allowCloseBeforeBatch: !once) # {spaceSet?, sinceHlcs, changes} | idle | closed
if closed: return
adopt batch.spaceSet (follower: materialize + project); store sinceHlcs;
group changes by uuidSpaceId; mergeInboundBatch each group
if once: send Close; read Close; drain; return
delay continuousSyncInterval
```
The shared establishment exchanges the space set so a follower never acts before
it knows the agreed set. Both cadences handshake initial and newly added spaces
through the loop's own `SinceHlc` exchange.
`collectPendingChanges`, `mergeInboundBatch`, `recordSyncCheckpoint`, and the
ownership-violation internals (`_streamInserts`/`Updates`/`Deletes`,
`_fetchDomainRow`, the durable-violation path) are space-parameterized and merge
each inbound space group in its own `transactionForUser`/lock; any space's merge
failure (for example an ownership collision or unauthorized role write) records
the durable violation and fails the session, preserving today's fail-fast
semantics.
Because `continuous` defers a space's data until both peers have exchanged its
`SinceHlc`, a newly active space is established over the **following** cycles
rather than in lockstep — the deliberate trade for keeping the steady state
silent. A peer only honors inbound `SinceHlc`/`MergeChunk` frames for spaces it
authorizes (authoritative: its own membership; follower: the announced set), so
the combined batch tolerates a peer's informational extras without widening
access.
`onMergeSuccess` becomes space-aware: `Function(UuidValue spaceUuid, Hlc hlc)`.
This touches three test call sites and the example app's `demo_controller`.
### Space-set reconciliation and lockstep
Both peers must cycle the **same ordered space set** or send/receive desyncs.
Membership is asymmetric — the client trusts the server's membership claims, the
server trusts none of the client's — so the reconciliation is **directional**,
not a symmetric set operation:
- **Union is wrong** (insecure): a client could name a space the user is not a
member of and force the server to sync it.
- **Intersection is wrong** (cannot adopt): a brand-new share is not in the
client's set yet, so it would never enter the intersection and never sync —
the client cannot hold what it has not synced, and cannot sync what it does
not hold.
The correct operation is neither: **the server dictates its set; the client
conforms.** Each cycle:
- **Server**: `orderedSpaces = sort(memberGrants(userUuid).uuidSpaceId)`,
computed from `offline_sync_space_members` plus the implicit personal grant. It never
reads the client's `SpaceSet` for authority (constraint 4).
- **Client**: takes the server's `SpaceSet` as the set to cycle —
`getOrCreate` (`OfflineSyncSpaceManager`) any space it lacks, ignore any local space
the server omitted for this cycle — then `orderedSpaces = sort(peer spaceIds)`.
This asymmetry is expressed by a **`OfflineSyncPeerMode` enum**
(`authoritative` / `follower`) passed at the call site: the server endpoint
passes `authoritative`, the client driver passes `follower`. Membership is
resolved from the same shared `offline_sync_space_members` table (via
`OfflineSyncSpaceMembership`); the mode varies only the cycle *direction* —
authoritative dictates its own set, follower adopts the peer's.
Once the data is unified, a closed enum is the right shape: there is no
per-role enumeration callback left to inject. Security does not rest on the
enum — a malicious client passing `authoritative` only fails to adopt the
peer's set; writes are still gated server-side by
`OfflineSyncSpaceMembership.roleOf(...).canWrite` against the authoritative table.
In `once`, the client adopts straight from the cycle's `SpaceSet` *before* it
iterates, so a newly granted space syncs in that same round. In `continuous`,
adoption happens when the announcement arrives and the space's data follows over
the next cycles once its `SinceHlc` round-trips. Sorting by space UUID makes the
order deterministic on both sides with no extra negotiation. Frames carry
`uuidSpaceId` so a receiver groups each chunk into the right space and fails loud
on an unauthorized tag rather than merging a batch into the wrong space.
Re-resolving `memberGrants` every cycle is an indexed `offline_sync_space_members`
lookup on the server only; it may be throttled or invalidated by an
application hook if per-cycle querying becomes hot, at the cost of slower
propagation of access changes.
### Checkpoints, once vs. continuous
Checkpoint rows stay per `(space, node)` in `offline_sync_space_nodes`, while
`crdt_nodes` stores the stable replica identity. The same local replica node can
participate in multiple spaces, with separate `lastReceivedHlc` values per
space. `SinceHlc` is exchanged once per space on first visit; later cycles reuse
the in-memory
`nodeCheckpoints[space]`, advanced as chunks are sent and as inbound batches
merge — the multi-space analogue of today's single-space continuous loop, which
already keeps checkpoints in memory across rounds.
The continuous interval delay moves to **after each full cycle**, not after each
space, to keep per-space latency low while still preventing a busy loop when
idle.
### Access changes during a session
Because the authoritative space set is re-resolved every cycle and re-announced
whenever it changes (not pinned at `Connect`), a continuous session absorbs
membership changes without reconnecting:
- **Grant.** The server re-announces a `SpaceSet` that includes the new space;
the client adopts and materializes it, then handshakes and syncs it over the
next cycles. (The server detects the grant on its next cycle, so propagation
is bounded by one idle poll plus the handshake round-trip.)
- **Revoke.** The server re-announces a `SpaceSet` that omits the space; both
sides stop cycling it and discard its in-memory checkpoint state, and the
follower's `projectFollowerMembership` deletes its members-table row. The
device can still store the revoked space's rows locally, but membership-wide
reads no longer return them. Purging stored rows (and the fate of any unsynced
local changes) is the deferred revocation question, not a sync-loop concern.
- **Demote.** The server re-announces the same space with a different role; the
follower projection updates the cached role. The next local write consults
the new role and fails early if it is read-only. Pending local changes already
authored before demotion are not silently applied by the server: if streamed,
they are rejected and recorded as `unauthorizedWrite`. Whether the client
purges, surfaces, or keeps skipped pending changes is part of the deferred
revocation-cleanup design.
The authoritative peer originates every set change, so both sides converge on
the same active set from the same announcement and stay in step across it. A
one-shot `syncOnce` captures the set once; the next `syncOnce` call picks up any
change.
## Transaction API
`transactionForUser` keeps its name and user-first semantics and gains an
optional space:
```dart
db.transactionForUser(userId, fn); // acts in the personal space
db.transactionForUser(userId, fn, spaceId: listId); // acts in a shared space
```
- Without `spaceId`, the space resolves to the user's personal space —
`userId` itself by convention — which is exactly today's behavior.
- With `spaceId`, the package resolves the member role with
`OfflineSyncSpaceMembership.roleOf` — no injected validator. On the server that table
is authoritative; on a persistent client it is the server-projected membership
cache. A missing role throws `OfflineSyncSpaceMembershipException`; anything other
than `readWrite` throws `OfflineSyncSpaceRoleException` before the transaction starts.
- A transaction acts in **exactly one** space (constraint 5). The write path —
stamp-or-assert, the space-scoped `WHERE`, `spaceId` immutability — is unchanged
beyond which space is resolved.
## Read path: membership-wide
A user-space-scoped read in a sharing world means "rows in any space I am a member
of". The single-space resolver becomes a set:
- `mergeWhereWithTombstone` isolates space-scoped reads with `spaceId IN (<member
space ids>)` through `OfflineSyncSpaceIdsResolver`. For writes, the resolver supplies
the single acting space. For reads, both server sessions and persistent
clients resolve the set from `OfflineSyncSpaceMembership.memberSpaces`
(authoritative on the server, projected on the client); unscoped admin reads
still pass no space filter.
- The visibility probe is the row-keyed form for the multi-space case, so
sharing simplifies the predicate matrix rather than growing it: reads differ
only in their membership filter (one space, a user's spaces, or none for
admin).
- This is the shape of a Postgres row-level-security policy
(`spaceId IN (SELECT spaceId FROM offline_sync_space_members WHERE userUuid = …)`); the
package's application-level filter and RLS become two enforcements of one
predicate.
Writes stay space-pinned (above); only reads widen.
## What sharing does not change
Every enforcement rule in `row-ownership.md` survives unchanged:
`(table, uuidRowId)` stays globally identified; rows have exactly one owning
space; merge-insert ownership checks compare spaces; unique conflict groups form
within a space; ordinary global unique indexes stay rejected; stamp-or-assert
applies with the transaction's resolved space; the durable
`offline_sync_integrity_violations` contract now also records unauthorized role
writes. The merge, FK, and unique engines already key on the space the merge
runs in and need no ownership changes for sharing.
## Implementation status
The implementation landed in independently green phases. Tests should continue
to run with `--concurrency=1`.
- **Phase 1 — membership foundation.** Implemented `offline_sync_space_members`
(`database: all`, unsynced) and its migration. Added the shared
`OfflineSyncSpaceMembership` helpers (`memberGrants`, `memberSpaces`, `roleOf`,
`isMember`, and the follower projection path). No behavior change yet: a
personal-space-only set reproduces today's single-space sync exactly.
- **Phase 2 — sequential multi-space sync (the requirement).** Implemented
protocol model
changes (`Connect` with session `localNodeId`, new per-cycle
`OfflineSyncSpaceSet`, per-space `SinceHlc`, `MergeChunk.uuidSpaceId`) plus
generate/migrate. Outer cycle loop in `OfflineSyncEngine.sync` with per-cycle
`SpaceSet` exchange with role-carrying `OfflineSyncSpaceGrant`s, per-space
handshake-on-first-visit, in-memory per-space checkpoints, and the
once/continuous cadence. Reconciliation via a
`OfflineSyncPeerMode` enum: the server endpoint passes `authoritative` (cycles
`OfflineSyncSpaceMembership.memberGrants`); the client driver passes `follower`
(adopts the server's set). Space-aware `onMergeSuccess`. A personal-space-only
device behaves identically to today.
- **Phase 3 — transaction API.** Implemented
`transactionForUser(userId, fn, {spaceId})` with the membership assertion;
one space per transaction.
- **Phase 4 — membership-wide reads.** Implemented the space-scoped read filter as
`spaceId IN (…)` over the user's member spaces.
- **Phase 5 — lifecycle and docs.** Documented client adoption of newly
announced shares; reconcile with the space-purge semantics in
`sync-non-sync-relations.md`.
Revocation cleanup is deferred (see *Follow-ups*).
- **Phase 6 — idle-silent sync, one method.** Reworked the single `sync` method
(no separate once/continuous bodies) into a shared lockstep establishment plus
a shared combined data loop: each cycle coalesces into one space-scoped-change batch
terminated by a single `EndOfBatch`, with `SpaceSet`/`SinceHlc`/`MergeChunk`
sent only on change and read via `collectNextBatch`. An idle continuous
session now sends zero frames and idles out once per cycle regardless of space
count (Phase 2's loop re-announced the set and an `EndOfBatch` per space every
tick). `once` runs one data cycle then closes; mid-session continuous spaces are
deferred until their `SinceHlc` round-trips, and inbound frames are honored
only for authorized spaces.
- **Phase 7 — roles within a space.** Implemented the closed
`OfflineSyncSpaceRole` enum (`readOnly`, `readWrite`) on server and client.
`readWrite` is the only shared-space write grant. Added authoritative inbound
enforcement that records
`unauthorizedWrite` and fails before merge, client-side
`OfflineSyncSpaceRoleException` early rejection, and follower outbound skipping for
non-writable shared spaces while reads remain membership-wide.
- **Phase 8 — space management service.** Implemented the server-side
`session.offlineSync.spaces` service (`create`, `createFor`, `grant`, `grantAll`,
`revoke`, `members`) with transaction/savepoint threading. Single grants reuse
the bulk-grant path; unknown-space grants throw `OfflineSyncSpaceNotFoundException`.
Invitations, acceptance, and authorization policy stay in app endpoints.
## Test plan
Because no code branches on space count or deployment role, mechanisms are
tested once, not per combination.
- **Regression.** Every existing single-space suite passes unchanged on the
personal-space-only path.
- **Protocol.** `Connect`/`SinceHlc` round-trip a multi-space list; merge chunks
route to the correct space by `uuidSpaceId`; per-space checkpoints advance
independently across cycles.
- **One call, every space.** A single `syncOnce` converges N spaces in one pass;
a continuous session keeps cycling. Outbound changes for the active spaces are
collected in one pass and streamed in `syncBatchSize` chunks; the receive
groups a cycle's inbound chunks by space before merging each in its own
transaction.
- **Idle silence.** A continuous session with no pending changes settles and then
emits no further frames — neither peer re-announces the space set nor sends an
`EndOfBatch` while idle.
- **Membership.** The server syncs personal + shared spaces; a client-announced
non-member space is refused and never streamed; a newly granted space is
adopted and syncs within the same `syncOnce` call.
- **Roles.** `readOnly` can still read shared rows, local writes fail before
mutation, `readWrite` syncs normally, a follower skips outbound pending changes
for non-writable spaces while still receiving inbound changes, and a stale or
hostile client write is rejected and recorded by the server as
`unauthorizedWrite`. Personal spaces remain writable through their implicit
`readWrite` grant.
- **Access changes mid-session.** During a continuous session, a grant added to
`offline_sync_space_members` appears and syncs on the next cycle; a revoke stops
cycling that space without dropping the session.
- **Isolation under sharing.** Two users sharing space X both converge on X
while their personal spaces stay isolated; an ownership collision in one space
records a durable violation and fails the session without corrupting another
space's merge.
- **Smoke.** One `withServerpod` round-trip with one personal plus one shared
space, proving the end-to-end wiring (mirrors `sync_flow_test.dart`).
## Resolved during review
- **Combined sync batch:** merges serialize regardless, so combining spaces is
not a throughput optimization. Both cadences use the same combined batch shape:
each change carries its `uuidSpaceId`, and receive groups changes by space
before merging each space group. Continuous mode omits idle frames so idle
latency is constant in space count.
- **Membership authority:** the server, from `offline_sync_space_members`. The
reconciliation is directional — server dictates, client conforms — and is
neither union (insecure) nor intersection (cannot adopt). It is a
**`OfflineSyncPeerMode` enum** (`authoritative` / `follower`): membership still
resolves through `OfflineSyncSpaceMembership`, so the only residual variation is
cycle direction — a closed enum, not a callback. The enum is not the security
boundary; server-side
`OfflineSyncSpaceMembership.roleOf(...).canWrite` gates every applied write
regardless of the client's declared mode.
- **Space set is dynamic, not pinned at connect:** the authoritative peer
re-resolves its set every cycle and re-announces a `SpaceSet` whenever it
changed, so grants and revokes take effect mid-session without reconnecting.
In `syncOnce` the client adopts from the initial `SpaceSet`, exchanges
`SinceHlc`, then runs the sendable data cycle; in `syncContinuously` a newly
announced space is adopted then established over the next cycles as its
`SinceHlc` round-trips.
- **Members vs. nodes:** orthogonal layers, both kept. Members are authorization
(`who`); nodes are CRDT causality (`whose changes, up to which HLC`). One
member maps to many nodes; the server is a node but not a member; a new member
has no node until first sync.
- **Client membership table:** `database: all` puts the table on the client,
populated as a read-only projection. The `OfflineSyncSpaceSet` carries
`OfflineSyncSpaceGrant`s (`uuidSpaceId` + `role`), and each follower cycle reconciles
the local table from the announcement (upsert granted, delete revoked; only
when the set changed). The projected members table drives shared-space
membership gating and roles for UI and offline role-gated writes; local
`offline_sync_spaces` rows alone do not grant access. It is a server-authored,
client-read-only projection — never a
bidirectionally-synced members table, so a client cannot self-grant.
- **Iteration order:** sorted space UUIDs, so both peers stay in lockstep
without negotiation. Each merge change carries its space UUID, so receive can
regroup a combined batch deterministically.
- **One space per transaction:** kept. Cross-space writes need separate
transactions; remote replicas cannot observe a cross-space write atomically
anyway.
- **Membership table is `database: all` but unsynced:** the schema exists on
every node, yet it is server-authoritative app state, never CRDT-replicated.
- **Idle chatter removed:** continuous sync has one combined data loop per
cycle. When neither peer has grants, handshakes, or changes to send, it emits
no `SpaceSet`, `SinceHlc`, `MergeChunk`, or `EndOfBatch`; `collectNextBatch`
resolves the idle cycle from the timeout once per session cycle, not once per
space.
- **Initial-sync ordering is not a current concern:** a device joining many
spaces cycles them one at a time on first sync. Prioritizing recently active
spaces in the iteration order is a compatible refinement if cold-start latency
over large space counts ever matters, but it is not a present worry.
## Follow-ups
Deferred work needing its own design pass. It does not block the implemented
role enforcement above.
1. **Membership revocation.** When a user loses membership, the server omits the
space from the next cycle's `SpaceSet` and both sides stop syncing it
promptly. The device may still store the space's data and unsynced local
changes, though membership-wide reads no longer expose revoked rows.
Purge-vs-keep policy, and whether revoked or demoted unsynced changes are
surfaced, dropped, or kept pending locally, need their own design. (Carried
from `row-ownership.md` open question 3.)
2. **Outbound collection consistency.** Collection reads CRDT metadata and the
domain data separately and lock-free, so a concurrent delete — or an FK
reference to a row inserted after the insert snapshot — can fail a healthy
stream sync and record a durable violation for a race that the next round
would resolve. The single-pass multi-space collection widened this window.
Failing the session for a transient, self-correcting race is the weakness to
fix; the leading direction is a read-only snapshot (MVCC, not a lock). See
`outbound-collection-consistency.md`.

View file

@ -0,0 +1,215 @@
# Space management API
Status: implemented. Builds on the shared-space
membership model in `shared-spaces.md`; this document specifies the developer-
facing API for *managing* membership while leaving invitations, acceptance, and
authorization policy in app domain.
## Summary
The shared-space machinery is complete for **resolving and enforcing**
membership (`OfflineSyncSpaceMembership.memberSpaces` / `memberGrants` / `roleOf` /
`isMember`, the sync-loop enforcement, membership-wide reads), but those helpers
are package internals. The developer-facing API for **managing** membership is
the server-side `session.offlineSync.spaces` service: creating a shared space, adding a
member with a role, changing a role, revoking, and listing explicit members.
`shared-spaces.md` records the same boundary: rows are managed through
`session.offlineSync.spaces`, while invitations, acceptance, and authorization policy
remain app domain.
The service avoids making developers replicate what `space_membership_test.dart`
used to do — two non-atomic raw `insertRow` calls against the **server**
package's generated `OfflineSyncSpace` / `OfflineSyncSpaceMember`, leaking every internal
invariant (`spaceId` vs `uuidSpaceId`, the non-null `role` contract, the unique
index, personal-space-UUID == user-UUID):
```dart
final spaceId = await session.offlineSync.spaces.createFor(creatorUuid);
await session.offlineSync.spaces.grant(space: spaceId, user: bob, role: OfflineSyncSpaceRole.readWrite);
await session.offlineSync.spaces.revoke(space: spaceId, user: bob);
```
## Design decisions
These were settled in discussion; the reasoning is recorded so the
implementation issue does not relitigate them.
### A service, not an endpoint (the `AuthUsers` model)
The load-bearing constraint is `shared-spaces.md` #4: *membership cannot be
forged; the server never widens access from client-reported state*. If the
package auto-generated a management **endpoint**, any logged-in client could
self-grant into any space and break that.
So the package ships a **server-side business service** the developer calls from
their **own** endpoint, after applying their **own** authorization policy. This
is exactly Serverpod's `AuthUsers`: `AuthUsers.create(session, ...)` is a
capability, not a policy — it does not check "may the caller create a user".
The sync path enforces roles internally only because it is *directly reachable
by untrusted clients*; the management service is server-only (no endpoint), so
its only caller is the developer's own server code.
The package owns the **data invariants**; the app owns the **authorization
policy**. That is the same division that makes invitations/acceptance app domain.
### No authorization check inside the service
An earlier sketch gated writes with a `by:` (acting-user) parameter checked
against an admin role. This was dropped:
- **`by` is forgeable, so it is not enforcement.** It is a `UuidValue` the
caller supplies; the service cannot tell a real identity from a typed-in admin
UUID. It prevents a bug, not an attacker — and only if someone upstream binds
it to the true identity, which is the endpoint's job.
- The unforgeable identity is `session.authenticated` — the same source the sync
endpoint already trusts (`offline_sync_endpoint.dart`). It lives at the endpoint,
which is where the app's policy lives anyway.
- The internal check is what forced a system/backend **bypass**; removing the
check dissolves the bypass.
The service therefore performs **no** internal authorization. Authorization is
100% the endpoint's job, where the authenticated caller is unforgeable:
```dart
class TeamEndpoint extends Endpoint {
Future<void> invite(Session s, UuidValue space, UuidValue user, OfflineSyncSpaceRole role) async {
final caller = UuidValue.withValidation(s.authenticated!.userIdentifier); // unforgeable
if (!myApp.mayManage(caller, space)) throw ...; // app policy
await s.offlineSync.spaces.grant(space: space, user: user, role: role);
}
}
```
### No `admin` role in the package
The discussion considered an `admin` tier in `OfflineSyncSpaceRole` so "any admin can
grant". It was rejected on the **enforce-vs-define** line: *the package should
own only what it enforces.*
- `readOnly` / `readWrite` are **enforced by the sync engine against untrusted
clients** (the server rejects an unauthorized stream write and records
`unauthorizedWrite`). This check cannot live in the app; it must be in the
package.
- `admin` is **never consulted by the engine** — nothing in the sync path,
merge, or read filter asks "is this user an admin". It would be a permission
the package *defines but never acts on*, and it overlaps with Serverpod's own
`Space` mechanism (a second permission system the app must reconcile).
Everything uneasy traced back to `admin`: the forgeable `by`, the bypass, and a
"last-admin" orphan guard all evaporate once it leaves. `OfflineSyncSpaceRole` stays
`{ readOnly, readWrite }` — exactly what is already implemented — so this
implementation adds **no** enum churn.
Apps model "admin" with what they already have: a Serverpod `Space` on the auth
user for global staff, or their own domain data for per-space owners. If the
app's `TeamMember { teamId, userId, isAdmin }` is itself a synced CRDT table, an
"am I an admin" UI works offline for free. The two membership layers stay
separate: **access-membership** (`offline_sync_space_members`, server-authoritative,
package-enforced: who may sync/read/write) vs **domain-membership** (app data:
who is an admin, display name, joined-at). The app keeps them in step — adding a
domain member also calls `spaces.grant` — the same split Serverpod has between
`AuthUser` and an app's user profile.
## API
Server-side, hung off the `session.offlineSync` accessor (the session-bound
`OfflineSyncSession` facade) as `session.offlineSync.spaces`. A pure data layer: atomic,
transaction-threaded, no internal auth.
```dart
/// Creates a shared space and returns its UUID.
///
/// [grants], if given, adds each user as a member with its role, applied in
/// one transaction with the space insert. With no grants the space is created
/// dormant — it exists but no one can sync it until membership is granted.
Future<UuidValue> create({
Map<UuidValue, OfflineSyncSpaceRole>? grants,
Transaction? transaction,
});
/// Creates a shared space granting [user] the given [role], in one
/// transaction.
///
/// Pure sugar for [create] with a single-entry grants map — the member has no
/// special bond to the space and can be re-granted or revoked like any other.
Future<UuidValue> createFor(
UuidValue user, {
OfflineSyncSpaceRole role = OfflineSyncSpaceRole.readWrite,
Transaction? transaction,
});
/// Grants [user] the given [role] in [space], upserting an existing membership.
Future<void> grant({
required UuidValue space,
required UuidValue user,
required OfflineSyncSpaceRole role,
Transaction? transaction,
});
/// Grants several members in one transaction; the bulk twin of [grant].
Future<void> grantAll(
UuidValue space,
Map<UuidValue, OfflineSyncSpaceRole> grants, {
Transaction? transaction,
});
/// Removes [user]'s membership in [space]. A no-op if not a member.
Future<void> revoke({
required UuidValue space,
required UuidValue user,
Transaction? transaction,
});
/// The members of [space] and their roles.
Future<Map<UuidValue, OfflineSyncSpaceRole>> members(
UuidValue space, {
Transaction? transaction,
});
```
Notes:
- **There is no "owner".** An earlier sketch had an `owner` parameter on
`create`, but the name implied a durable bond the data model does not have —
the space row stores no owner, and the creator ends up an ordinary
`readWrite` member. It was replaced by `createFor`, which is honest sugar:
create a space and grant one member, atomically, with no special standing.
Removing `owner` also dissolved the owner-appears-in-`grants` `ArgumentError`
special case. A durable creator/owner concept is app domain, like `admin`.
- **`grants` may be empty or absent** — a space with no members is valid but
dormant. `create()` skips the membership insert entirely.
- **`create` and `grantAll` share one internal helper** so the create path and
the later bulk-grant path have a single implementation.
- **Unknown-space grants throw `OfflineSyncSpaceNotFoundException`** rather than a
generic argument error, so callers can distinguish a stale or invalid space
UUID from other failures.
- **`create` returns a server-generated UUID** — the model already defaults
`uuidSpaceId` to `random_v7`. An optional explicit `uuidSpaceId` param
(app-chosen or idempotent create) is a compatible later addition; left out
until there is a real need.
- Membership writes are plain `offline_sync_space_members` / `offline_sync_spaces` rows — **not**
CRDT-tracked, not part of the sync loop. Changes propagate to clients on the
next sync cycle through the existing `SpaceSet` re-announcement path.
## Client side
Unchanged. Management is server-only; the client never writes membership. A
client learns of new spaces and role changes purely through the sync `SpaceSet`
announcement and the follower projection already implemented in
`shared-spaces.md`. There is no client `create`/`grant`/`revoke`.
## Test plan
- **Create.** `createFor(u)` yields a space with `u` at `readWrite` and a
member-resolvable space; `createFor(u, role: ...)` stores the explicit role;
`create()` (no grants) yields a dormant space no user can sync;
`create(grants: {...})` stores every entry with its role.
- **Grant/revoke.** `grant` inserts and upserts (role change); `revoke` removes
and is a no-op for a non-member; `grantAll` applies a map atomically.
- **Reads.** `members` returns the explicit member-role map. Internal role
resolution still covers stored, missing, and implicit personal membership.
- **Propagation.** After a server-side `grant`, a continuous follower session
adopts the new space on the next cycle (covered by the existing access-change
suite); after `revoke`, it stops cycling it.
- **Transaction threading.** A `create` folded into an app domain transaction
rolls back atomically with the app write on failure.

View file

@ -13,7 +13,7 @@ the CRDT layer recording the corresponding facts.
There are only two allowed sync/non-sync relation shapes:
1. the package-owned ownership link from each synced domain row to
`crdt_scopes`;
`offline_sync_spaces`;
2. a synced row referencing a non-synced, server-authoritative reference table,
but only after the sync protocol can provision that reference data before
CRDT merge.
@ -23,15 +23,15 @@ The ownership link is:
```yaml
fields:
id: UuidValue?, defaultPersist=random_v7
### Owner scope of this row. Maintained by the CRDT sync layer.
scopeId: int?, relation(parent=crdt_scopes, onDelete=Cascade)
### Owner space of this row. Maintained by the CRDT sync layer.
spaceId: int?, relation(parent=offline_sync_spaces, onDelete=Cascade)
```
`scopeId == null` remains valid persisted state, but it means the row is
`spaceId == null` remains valid persisted state, but it means the row is
orphaned from CRDT ownership: it is never synced, never collected for outbound
sync, and not removed by deleting a scope. `scopeId != null` means the row is
owned by that scope, participates in CRDT, and is physically purged when the
scope is deleted.
sync, and not removed by deleting a space. `spaceId != null` means the row is
owned by that space, participates in CRDT, and is physically purged when the
space is deleted.
All other foreign keys between synced and non-synced tables must fail
`initialize()`. Until the reference-data sync phase exists, `initialize()` must
@ -49,10 +49,10 @@ also reject server-authoritative reference-table relations even when they use
protocol, applied before CRDT merge, and not removed while synced rows
reference them.
- **Package metadata table:** CRDT infrastructure tables owned by this package,
especially `crdt_scopes`, `crdt_nodes`, `crdt_data_rows`,
especially `offline_sync_spaces`, `crdt_nodes`, `crdt_data_rows`,
`crdt_data_fields`, `crdt_data_tombstone`, and `crdt_data_attempted_value`.
`crdt_scopes` is not a synced domain table. The `scopeId -> crdt_scopes.id`
`offline_sync_spaces` is not a synced domain table. The `spaceId -> offline_sync_spaces.id`
relation is a special package-owned purge boundary, not an application-level
foreign key.
@ -65,27 +65,27 @@ Allowed under the normal FK invariant rules:
- the child and parent are both in `syncTables`;
- supported FK shapes are single-column child references to single-column
parent references;
- local writes assert same-scope visibility;
- merge-time FK projection repairs hidden, missing, or foreign-scope targets.
- local writes assert same-space visibility;
- merge-time FK projection repairs hidden, missing, or foreign-space targets.
### Non-synced -> non-synced
Ignored by CRDT. These relations are normal application/database relations.
### Synced -> `crdt_scopes` through `scopeId`
### Synced -> `offline_sync_spaces` through `spaceId`
Allowed, and required once this proposal is implemented. The relation must be
exactly:
- child table is a synced domain table;
- child column is the reserved `scopeId` column;
- child column is the reserved `spaceId` column;
- child column type is nullable `int`;
- parent table is `crdt_scopes`;
- parent table is `offline_sync_spaces`;
- parent column is `id`;
- `onDelete` is `Cascade`;
- `onUpdate` is `NoAction` or unspecified database default.
This relation exists only to make terminal scope purge database-enforced. It
This relation exists only to make terminal space purge database-enforced. It
does not change normal CRDT deletes, which remain soft deletes recorded in
CRDT metadata.
@ -126,7 +126,7 @@ delta before CRDT merge.
This exception also does not expand the global unique-index exception. A unique
index involving a FK to a protocol-provisioned reference table is
application-level uniqueness and must include `scopeId` unless another explicit
application-level uniqueness and must include `spaceId` unless another explicit
policy is added.
## Reference-data sync phase
@ -153,7 +153,7 @@ through HLCs, unique conflict projection, or FK projection.
Always forbidden.
A non-synced child has no CRDT row, no scope ownership, and no merge history.
A non-synced child has no CRDT row, no space ownership, and no merge history.
If it references a synced parent, then a CRDT soft delete can hide the parent
while the database FK still sees the physical parent row. Delete actions and
visibility checks become incoherent, and the current recorder can also reach
@ -163,7 +163,7 @@ paths that assume the child table has CRDT schema metadata.
Forbidden, except for:
- the special `scopeId -> crdt_scopes.id` relation;
- the special `spaceId -> offline_sync_spaces.id` relation;
- the protocol-provisioned reference-table relation above.
The apparent safe use case is an app user/auth table, because that table is
@ -184,10 +184,10 @@ non-synced table is unsafe:
deferrable.
The app-user relation should be modeled outside synced domain rows. Account
deletion should revoke the user and purge the user's `CrdtScope`, not rely on
deletion should revoke the user and purge the user's `OfflineSyncSpace`, not rely on
arbitrary application FKs into synced data.
### Synced -> package metadata table other than `crdt_scopes`
### Synced -> package metadata table other than `offline_sync_spaces`
Forbidden. Domain rows must not reference CRDT metadata internals such as nodes,
schema rows, field rows, tombstones, or ownership violations.
@ -196,36 +196,36 @@ schema rows, field rows, tombstones, or ownership violations.
Forbidden unless the relation is part of a future explicit package API. An
application table must not depend on metadata rows whose lifecycle is controlled
by sync, merge, or scope purge.
by sync, merge, or space purge.
## Scope purge semantics
## Space purge semantics
Deleting a scope is a terminal GDPR/account-erasure operation, not a CRDT
Deleting a space is a terminal GDPR/account-erasure operation, not a CRDT
operation.
A safe purge workflow is:
1. Revoke the account or scope so no new sync session can start.
2. Mark the scope UUID as deleted, or otherwise make auth reject future
`getOrCreate(uuidScopeId)` attempts for that UUID.
3. Delete `crdt_scopes.id`.
1. Revoke the account or space so no new sync session can start.
2. Mark the space UUID as deleted, or otherwise make auth reject future
`getOrCreate(uuidSpaceId)` attempts for that UUID.
3. Delete `offline_sync_spaces.id`.
4. Let database cascades remove:
- synced domain rows with `scopeId = deletedScope.id`;
- `crdt_nodes` for that scope;
- `crdt_data_rows` for that scope;
- synced domain rows with `spaceId = deletedSpace.id`;
- `crdt_nodes` for that space;
- `crdt_data_rows` for that space;
- row fields, tombstones, and FK projection rows through their existing
metadata cascades.
5. Delete or anonymize diagnostics that intentionally outlive metadata
cascades, such as durable ownership violation rows. Those rows store
denormalized table names and scope UUIDs only; they do not reference
denormalized table names and space UUIDs only; they do not reference
purged metadata through foreign keys.
If an old offline client later reconnects with data for a purged scope, sync
must fail with a terminal "scope deleted/reset required" response. The server
must not recreate the scope and must not accept the old CRDT chain, or erased
If an old offline client later reconnects with data for a purged space, sync
must fail with a terminal "space deleted/reset required" response. The server
must not recreate the space and must not accept the old CRDT chain, or erased
data can be resurrected.
Rows with `scopeId == null` are not affected by scope purge. They are
Rows with `spaceId == null` are not affected by space purge. They are
application-owned orphan/admin rows and remain outside CRDT.
## Initialize validation
@ -238,7 +238,7 @@ Validation rules:
- `synced -> synced`: allowed only when the existing FK invariant support can
handle the FK shape.
- `non-synced -> non-synced`: ignored.
- `synced -> crdt_scopes` through reserved `scopeId`: allowed only when it
- `synced -> offline_sync_spaces` through reserved `spaceId`: allowed only when it
matches the exact allowed shape above.
- `synced -> protocol-provisioned reference table`: allowed only when the
reference-data sync phase is implemented, the parent table is declared as
@ -255,14 +255,14 @@ The error message should explain the allowed alternatives:
- remove the relation;
- use the future reference-data sync phase with `onDelete=Restrict` for
immutable server-authoritative reference data;
- for account deletion, use the package-owned scope purge relation instead of
- for account deletion, use the package-owned space purge relation instead of
an application FK.
## Tests
The implementation issue should add schema/initialize tests for:
- accepting `scopeId: int?, relation(parent=crdt_scopes,
- accepting `spaceId: int?, relation(parent=offline_sync_spaces,
onDelete=Cascade)` on every synced table;
- rejecting `sync -> non-sync` restricted reference-table FKs with
`onDelete=Restrict` while the reference-data sync phase is absent;
@ -276,9 +276,9 @@ The implementation issue should add schema/initialize tests for:
- rejecting `sync -> non-sync` application FKs with `NoAction`, `SetNull`,
`SetDefault`, and `Cascade`;
- rejecting global unique indexes over protocol-provisioned reference FK
columns unless the index includes `scopeId`;
columns unless the index includes `spaceId`;
- rejecting synced domain relations to package metadata tables other than
`crdt_scopes`;
- preserving orphan rows with `scopeId == null` across scope purge;
- deleting a scope removes owned domain rows and CRDT metadata for that scope;
- stale clients for a purged scope cannot recreate the scope or sync old data.
`offline_sync_spaces`;
- preserving orphan rows with `spaceId == null` across space purge;
- deleting a space removes owned domain rows and CRDT metadata for that space;
- stale clients for a purged space cannot recreate the space or sync old data.

View file

@ -17,7 +17,7 @@ This is the `flag` policy. It is intentionally simple:
- no business conflict object is synced;
- no foreign key retargeting is needed for unique conflicts.
This policy is the only unique-conflict policy in scope for the current
This policy is the only unique-conflict policy in space for the current
implementation.
## Current Policy: Flag
@ -47,9 +47,9 @@ For each conflict group among visible rows:
5. Keep every conflict-group row visible. Application code can decide whether
and how to surface the changed visible value as a business conflict.
For a composite unique index, fixed partition columns such as `scopeId` are not
For a composite unique index, fixed partition columns such as `spaceId` are not
released, and not every remaining indexed column has to be releasable. The index
is supported when at least one non-`scopeId` column can be released
is supported when at least one non-`spaceId` column can be released
deterministically. Stable discriminator columns may remain unchanged; changing
the releasable column or columns is enough to make the unique tuple conflict-free.
Supported candidate types are defined centrally under
@ -206,7 +206,7 @@ resolution.
when the loser is released, then the materialized UUID is deterministic from
the table, column, original value, and loser row id.
7. Given a scoped composite unique index contains a stable non-releasable
7. Given a space-scoped composite unique index contains a stable non-releasable
discriminator and a releasable text column, when a loser is released, then the
discriminator remains unchanged and the text column receives the deterministic
conflict suffix.
@ -239,6 +239,6 @@ projection model exists.
row hiding or FK retargeting.
- Convergence: same CRDT facts produce the same visible unique values.
- Idempotence: repeated merges do not keep rewriting the same conflict.
- Scope control: future `merge` and `overwrite` behavior is documented but not
- Space control: future `merge` and `overwrite` behavior is documented but not
treated as supported until the projection algorithm can handle second-order FK
effects.