refactor!: Rename sync APIs and ownership spaces (#128)
This commit is contained in:
parent
4331daf661
commit
b89725f6ea
9 changed files with 43650 additions and 9417 deletions
|
|
@ -54,7 +54,7 @@ soft-deleted instead of removed:
|
|||
- `ON DELETE SET DEFAULT`: visible child FK columns are updated to the column
|
||||
default when the default is legal, and those child FK fields receive ordinary
|
||||
CRDT field updates. A nullable null default is legal; a non-null default must
|
||||
remain visible in the same scope after the delete. A default in the same
|
||||
remain visible in the same space after the delete. A default in the same
|
||||
delete batch is unavailable. If a visible child needs this repair, an invalid
|
||||
default rejects the transaction, including any earlier repairs or tombstones
|
||||
in that transaction. An unused invalid default does not itself block a delete.
|
||||
|
|
@ -100,7 +100,7 @@ restrict edges are evaluated first internally.
|
|||
|
||||
Hidden children do not block parent deletions. They still participate in FK
|
||||
projection: when no legal repair exists and the authored parent remains
|
||||
physically present in the same scope, the FK candidate retains the authored
|
||||
physically present in the same space, the FK candidate retains the authored
|
||||
reference, even if that parent is hidden. This recomputes from authored values;
|
||||
it does not preserve an earlier projected fallback. Unique projection can then
|
||||
release that candidate, as described in the combined projection model below.
|
||||
|
|
@ -134,7 +134,7 @@ common case skips the dependency walk.
|
|||
|
||||
When a default is changed directly, or affected through a possible deletion,
|
||||
cascade, or missing parent, the loader queries sparse dependency seeds in the
|
||||
current scope: authored tombstones and hidden rows in the original-parent and
|
||||
current space: authored tombstones and hidden rows in the original-parent and
|
||||
cascade-ancestor tables, hidden children, and children with FK overrides. The
|
||||
ordinary reference walk then loads their children, parents, and unique claimants.
|
||||
Blocked deletions are included even when the original parent is still visible
|
||||
|
|
@ -145,7 +145,7 @@ starting from authored tombstones.
|
|||
This keeps ordinary merges on the existing row-closure path; there is no
|
||||
component-wide domain-row load. Default-specific invalidation can still visit
|
||||
many rows when many deletes, hidden rows, or repair claims depend on that default.
|
||||
The sparse metadata lookup itself may scan rows in the relevant scoped metadata
|
||||
The sparse metadata lookup itself may scan rows in the relevant space-scoped metadata
|
||||
tables, but it runs only for affected defaults. Full rebuilds still load the
|
||||
complete requested components.
|
||||
|
||||
|
|
|
|||
|
|
@ -38,10 +38,10 @@ The three steps below contain all required changes on a project to use the packa
|
|||
3. Wire the sync call on the client.
|
||||
```dart
|
||||
// Push local changes and merge remote ones once (i.e. pull-to-refresh).
|
||||
await client.crdt.syncOnce(session);
|
||||
await client.offlineSync.syncOnce(session);
|
||||
|
||||
// Or stream changes both ways until cancelled for a near-real-time sync.
|
||||
final syncSession = client.crdt.syncContinuously(session);
|
||||
final syncSession = client.offlineSync.syncContinuously(session);
|
||||
```
|
||||
|
||||
After this setup, the app can be developed using normal repository calls against the `session` instance.
|
||||
|
|
@ -50,7 +50,7 @@ The sync engine is a **black box that "just works"**: changes are tracked atomic
|
|||
|
||||
Understanding the behavior is also straightforward: conflict resolution respects unique and relational constraints, mirroring what would be expected if the merged data existed in a single database at the time of each operation.
|
||||
|
||||
Finally, the data is isolated per scope: each user has their own implicit personal scope, and can be a member of any number of shared scopes - with read-only or read-write access. On the server, database operations need to be wrapped in a special `session.db.transactionForUser` method to target the correct scope. On the client, it is possible to set a `persistentUserId` to target the personal scope by default, or use the `transactionForUser` to target a shared scope.
|
||||
Finally, the data is isolated per space: each user has their own implicit personal space, and can be a member of any number of shared spaces - with read-only or read-write access. On the server, database operations need to be wrapped in a special `session.db.transactionForUser` method to target the correct space. On the client, it is possible to set a `persistentUserId` to target the personal space by default, or use the `transactionForUser` to target a shared space.
|
||||
|
||||
#### Data modeling limitations
|
||||
|
||||
|
|
@ -58,12 +58,12 @@ Due to the nature of merge conflicts, the package imposes some limitations on th
|
|||
|
||||
- Every synced table must:
|
||||
- Have a UUID primary key.
|
||||
- Declare the field `scopeId: int?, relation(parent=crdt_scopes, onDelete=Cascade)`.
|
||||
- Unique indexes must include `scopeId` together with the target columns.
|
||||
- Declare the field `spaceId: int?, relation(parent=offline_sync_spaces, onDelete=Cascade)`.
|
||||
- Unique indexes must include `spaceId` together with the target columns.
|
||||
- Global unique indexes are unsupported, except for FK-only indexes.
|
||||
- Unique indexes are only supported with at least one `String`/`UuidValue`/nullable column.
|
||||
- All 1:1 relations must have the foreign-key column nullable (`optional` relation).
|
||||
- The only allowed non-synced-to-synced relation is `scopeId -> crdt_scopes.id`.
|
||||
- The only allowed non-synced-to-synced relation is `spaceId -> offline_sync_spaces.id`.
|
||||
- Foreign-key relations that foreign-key projection cannot repair — non-nullable columns and composite relations — must be `deferred`.
|
||||
- `onDelete=Restrict` is unsupported on synced tables; use `onDelete=NoAction`.
|
||||
|
||||
|
|
@ -103,9 +103,9 @@ Below is one of the last performance reports of the package (available at each p
|
|||
Delay per delete: 408.75 μs
|
||||
```
|
||||
```
|
||||
🔭 SELECT scope impact (100 extra users, 100,000 CRDT noise rows):
|
||||
find all (1,000 rows): 9.91 ms --> 10.91 ms scoped (+10.07%) / 11.19 ms unscoped (+12.96%)
|
||||
findById (per lookup): 295.04 μs --> 375.24 μs scoped (+27.19%) / 348.88 μs unscoped (+18.25%)
|
||||
🔭 SELECT space impact (100 extra users, 100,000 CRDT noise rows):
|
||||
find all (1,000 rows): 9.91 ms --> 10.91 ms space-scoped (+10.07%) / 11.19 ms unscoped (+12.96%)
|
||||
findById (per lookup): 295.04 μs --> 375.24 μs space-scoped (+27.19%) / 348.88 μs unscoped (+18.25%)
|
||||
```
|
||||
```
|
||||
💽 Storage impact (base footprint removed):
|
||||
|
|
@ -139,9 +139,9 @@ A report similar to the one above is generated per pull request as a comment on
|
|||
|
||||
**Visibility as a pure function.** Replicas never coordinate. Each merge applies remote facts monotonically, then derives the visible database — tombstone state, unique-conflict winners, and foreign-key repair — as a deterministic projection computed to a fixed point. This is the architecture introduced by *Synql* (Ignat et al., DAIS 2024), extended here to the full set of `ON DELETE` actions and to unique constraints over deeply relational schemas.
|
||||
|
||||
**Ownership & scopes.** A *scope* is the unit of replication and ownership; a *user* is an authentication identity. Each row is globally identified by `(table, rowId)` and owned by exactly one scope (`scopeId`), so merging is always one chain per row. Sharing is a membership layer in front of scopes, not a change to row ownership. Scopes can be purged for GDPR / account erasure via a database-enforced cascade. Ownership collisions (the same `(table, rowId)` claimed by two scopes, which is mostly a malicious attack vector) fail the sync and are recorded in a durable `crdt_sync_integrity_violations` table.
|
||||
**Ownership & spaces.** A *space* is the unit of replication and ownership; a *user* is an authentication identity. Each row is globally identified by `(table, rowId)` and owned by exactly one space (`spaceId`), so merging is always one chain per row. Sharing is a membership layer in front of spaces, not a change to row ownership. Spaces can be purged for GDPR / account erasure via a database-enforced cascade. Ownership collisions (the same `(table, rowId)` claimed by two spaces, which is mostly a malicious attack vector) fail the sync and are recorded in a durable `offline_sync_integrity_violations` table.
|
||||
|
||||
**Sync protocol.** A session handshakes once to validate both ends have the same table schema and ensure that a merge will reach the same result. Then, per cycle the server sends the authoritative scope set and all scopes that the user has access to are synced (data is transferred all at once, but merged one at a time). The data is transferred in chunks to avoid pressuring the bandwidth with too many small messages. This same stream-based implementation is used for both "one-shot" and "continuous" sync modes, so a one-shot sync with large amounts of data will be as performant as a continuous sync. It also means that all tests hold for both sync modes with minimal duplication.
|
||||
**Sync protocol.** A session handshakes once to validate both ends have the same table schema and ensure that a merge will reach the same result. Then, per cycle the server sends the authoritative space set and all spaces that the user has access to are synced (data is transferred all at once, but merged one at a time). The data is transferred in chunks to avoid pressuring the bandwidth with too many small messages. This same stream-based implementation is used for both "one-shot" and "continuous" sync modes, so a one-shot sync with large amounts of data will be as performant as a continuous sync. It also means that all tests hold for both sync modes with minimal duplication.
|
||||
|
||||
## Testing
|
||||
|
||||
|
|
@ -153,13 +153,13 @@ The package is functionally in a production-ready state, but some APIs need to i
|
|||
|
||||
### Technical debts
|
||||
|
||||
- **Rename the `scope` name for data isolation to `space`**. This is a small refactor to avoid confusion with the `scope` keyword used in the Serverpod Auth modules. Using `space` is also clearer for usage on an application level.
|
||||
- **Rename the `space` name for data isolation to `space`**. This is a small refactor to avoid confusion with the `space` keyword used in the Serverpod Auth modules. Using `space` is also clearer for usage on an application level.
|
||||
|
||||
### Post-integration UX improvements
|
||||
|
||||
The package already requires minimal configuration from the developer, but it can be even smoother if the `serverpod_cli` is made aware of the offline sync capabilities.
|
||||
|
||||
- **Add a fourth `sync` option for the `database` keyword to be used on models**. Models that use it will automatically receive the `scopeId` relation and be included in a generated list of synced tables on the server and client. This feature will enable all improvements below.
|
||||
- **Add a fourth `sync` option for the `database` keyword to be used on models**. Models that use it will automatically receive the `spaceId` relation and be included in a generated list of synced tables on the server and client. This feature will enable all improvements below.
|
||||
- **Simplify the `Serverpod` initialization by generating the class**. This is mapped on Serverpod's issue #4544 and will allow the CRDT database interceptor to be injected on the `Serverpod` constructor without the user having to call it manually whenever a `sync` model exists.
|
||||
- **Move the restrictions that are runtime-enforced to a static analysis step**. The package enforces the limitations of the CRDT layer at runtime (during the initialization), but a better UX would be to apply the restrictions to `sync` models during static analysis of `generate`.
|
||||
- **Inject the `serverpod_offline_sync` packages on the `pubspec.yaml` file**. This would remove the last manual step from the developer to make the solution work. Otherwise, the user will have to manually add the dependencies to their `pubspec.yaml` to generate the project.
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@
|
|||
|
||||
Outbound change collection reads CRDT metadata and the domain data it describes
|
||||
in **separate, non-atomic queries**. Under concurrent domain writes — real on
|
||||
the server, where co-members of a shared scope write while a sync runs — this
|
||||
the server, where co-members of a shared space write while a sync runs — this
|
||||
race can make an otherwise-healthy stream sync **fail and record a durable
|
||||
integrity violation** for a condition that the very next round would resolve on
|
||||
its own. Failing the session for a transient, self-correcting race is the design
|
||||
|
|
@ -12,12 +12,12 @@ weakness to fix; no data is ever applied incorrectly, so this is a
|
|||
robustness/liveness problem, not a corruption one.
|
||||
|
||||
This is **not yet implemented** — it is tracked design debt. The single-pass
|
||||
multi-scope collection (`collectPendingChanges`) did not introduce the race
|
||||
multi-space collection (`collectPendingChanges`) did not introduce the race
|
||||
but widened its window, which is what surfaced it.
|
||||
|
||||
## The mechanism
|
||||
|
||||
Collection is *snapshot-then-fetch*, and lock-free by design (the per-scope
|
||||
Collection is *snapshot-then-fetch*, and lock-free by design (the per-space
|
||||
`transactionForUser` lock guards the inbound merge, not outbound collection):
|
||||
|
||||
1. `_streamInserts` / `_streamUpdates` / `_streamDeletes` read the CRDT metadata
|
||||
|
|
@ -62,7 +62,7 @@ up the tombstone, the late insert, or the higher HLC, and replicas converge.
|
|||
Tearing down the stream — and persisting an integrity-violation record that reads
|
||||
like corruption — for a race that resolves itself next round converts an
|
||||
ordering skew into an outage plus a misleading audit trail. A durable
|
||||
`crdt_sync_integrity_violations` entry should mean a *genuine* invariant breach
|
||||
`offline_sync_integrity_violations` entry should mean a *genuine* invariant breach
|
||||
(an ownership collision, real corruption), not "two reads happened to straddle a
|
||||
concurrent commit."
|
||||
|
||||
|
|
@ -71,9 +71,9 @@ concurrent commit."
|
|||
- **Read a consistent snapshot (primary).** Run the whole collection — the three
|
||||
metadata queries *and* the domain reads — inside one read-only,
|
||||
snapshot-consistent transaction (repeatable-read). On Postgres this is MVCC:
|
||||
**a read snapshot, not a write lock**, so it adds no cross-scope write
|
||||
**a read snapshot, not a write lock**, so it adds no cross-space write
|
||||
contention (and is therefore the right answer to "do we have to lock all
|
||||
scopes?" — no, we take a snapshot, not a lock). A single snapshot closes the
|
||||
spaces?" — no, we take a snapshot, not a lock). A single snapshot closes the
|
||||
race outright:
|
||||
- The delete in (1) is invisible if it commits after the snapshot, and fully
|
||||
visible (metadata + domain) if before — never half-seen.
|
||||
|
|
@ -98,7 +98,7 @@ concurrent commit."
|
|||
|
||||
Correctness is preserved today (convergence + fail-safe: no wrong data is ever
|
||||
applied), so this does not block current use. But because it can fail a healthy
|
||||
long-lived continuous sync under concurrent shared-scope writes — the exact
|
||||
workload shared scopes enable — it should be addressed before such deployments
|
||||
long-lived continuous sync under concurrent shared-space writes — the exact
|
||||
workload shared spaces enable — it should be addressed before such deployments
|
||||
are relied upon. The consistent-snapshot approach is the smallest change that
|
||||
removes all three failure modes at once.
|
||||
|
|
|
|||
|
|
@ -160,7 +160,7 @@ Outbound collection substitutes `CrdtDataAttemptedValue.value` whenever it
|
|||
exists; otherwise it reads the domain value. It never exports an FK candidate,
|
||||
unique loser value, or temporary parking value as an authored update.
|
||||
|
||||
An empty replica merging a complete scope export must reproduce normalized
|
||||
An empty replica merging a complete space export must reproduce normalized
|
||||
domain rows, hidden state, authored values/HLCs, and derived projection. A
|
||||
rebuild may rewrite `projectionReason`, but it must preserve
|
||||
`CrdtDataAttemptedValue.value`: while the row exists, that is the only copy of
|
||||
|
|
@ -210,7 +210,7 @@ store an otherwise supported dynamic value.
|
|||
|
||||
### Initialization algorithm
|
||||
|
||||
Call `await crdtDatabase.initialize()` after application migrations and before
|
||||
Call `await offlineSyncDatabase.initialize()` after application migrations and before
|
||||
serving requests or starting sync. Reconciliation must plan before mutating:
|
||||
|
||||
1. Load the persisted CRDT registry and target `TableDefinition`s.
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
# Row ownership and scope isolation
|
||||
# Row ownership and space isolation
|
||||
|
||||
Status: implemented.
|
||||
|
||||
|
|
@ -14,26 +14,26 @@ different users silently overwrite each other across tenant boundaries.
|
|||
This document makes ownership explicit. The core decisions:
|
||||
|
||||
> **Every synced domain row is globally identified by `(table, uuidRowId)` and
|
||||
> owned by exactly one scope.** `scopeId` is not part of row identity; it is
|
||||
> ownership and isolation metadata on a globally identified row. Two scopes
|
||||
> never materialize or sync the same `(table, uuidRowId)`. Cross-scope UUID
|
||||
> owned by exactly one space.** `spaceId` is not part of row identity; it is
|
||||
> ownership and isolation metadata on a globally identified row. Two spaces
|
||||
> never materialize or sync the same `(table, uuidRowId)`. Cross-space UUID
|
||||
> reuse is a terminal ownership collision, not valid multi-tenant state. Sync
|
||||
> fails on that collision, records a durable violation, and does not advance
|
||||
> the sender past the rejected change. Sharing is modeled as shared scopes that
|
||||
> multiple identities subscribe to — never as two scopes aliasing one row.
|
||||
> the sender past the rejected change. Sharing is modeled as shared spaces that
|
||||
> multiple identities subscribe to — never as two spaces aliasing one row.
|
||||
|
||||
> **Every synced table declares `scopeId`, identically on server and
|
||||
> **Every synced table declares `spaceId`, identically on server and
|
||||
> client.** The column is a hard requirement, never `scope=serverOnly`, so
|
||||
> both databases run one schema shape and the package has exactly one code
|
||||
> path. "Single user" is a usage pattern (a database that happens to contain
|
||||
> one scope), not a mode: there are no mode flags, no schema branching, and
|
||||
> one space), not a mode: there are no mode flags, no schema branching, and
|
||||
> no per-mode test matrix.
|
||||
|
||||
> **The sync layer owns the column: stamp if null, assert if set — and the
|
||||
> value leaves the way it came in.** On every insert, a null `scopeId` is
|
||||
> stamped from the effective scope; a non-null one must equal it or the
|
||||
> value leaves the way it came in.** On every insert, a null `spaceId` is
|
||||
> stamped from the effective space; a non-null one must equal it or the
|
||||
> operation throws. Stamped values are stripped back to null on returned
|
||||
> rows and on scoped reads, so the database-local value never leaks into
|
||||
> rows and on space-scoped reads, so the database-local value never leaks into
|
||||
> endpoint payloads; explicitly provided values are kept on the way out.
|
||||
> One rule serves both personalities: users who never want to think about
|
||||
> tenancy never do, and users who pass it explicitly get verified
|
||||
|
|
@ -46,8 +46,8 @@ validated at `initialize()` with paste-ready error messages:
|
|||
```yaml
|
||||
fields:
|
||||
id: UuidValue?, defaultPersist=random_v7
|
||||
### Owner scope of this row. Maintained by the CRDT sync layer.
|
||||
scopeId: int?
|
||||
### Owner space of this row. Maintained by the CRDT sync layer.
|
||||
spaceId: int?
|
||||
```
|
||||
|
||||
No model inheritance is needed: like the UUID primary key requirement, the
|
||||
|
|
@ -56,7 +56,7 @@ by `CrdtSchemaRegistry`), not nominal (a base class), which is the only
|
|||
option anyway since Serverpod models cannot extend models from an imported
|
||||
module. The same constraint shapes the implementation: with no common
|
||||
interface to call, the package stamps the field through dynamic dispatch —
|
||||
`(row as dynamic).copyWith(scopeId: …)` (or the plain field setter, since
|
||||
`(row as dynamic).copyWith(spaceId: …)` (or the plain field setter, since
|
||||
generated fields are mutable). The field name is a compile-time constant, so
|
||||
the serialization manager's `patchTableRow` would be the wrong tool here: its
|
||||
serialize–patch–deserialize round-trip costs 50–200× a dynamic call and is
|
||||
|
|
@ -79,13 +79,13 @@ justified only where field names are data-driven, as in the merge path.
|
|||
`patchTableRow`.
|
||||
3. **Roles are not hard-coded.** Nothing in the package assumes the server
|
||||
is multi-user or the client single-user. Multi-account devices and
|
||||
single-tenant servers are the same code path with different scope counts.
|
||||
single-tenant servers are the same code path with different space counts.
|
||||
4. **Application code cannot get tenancy wrong.** The package must verify
|
||||
any user-supplied scope against the effective scope regardless (a tenant
|
||||
any user-supplied space against the effective space regardless (a tenant
|
||||
id from application code can never be trusted), so correctness never
|
||||
rests on call-site discipline.
|
||||
5. **Multi-device per user keeps working unchanged.** Devices of one user
|
||||
share the user's scope and merge into one chain.
|
||||
share the user's space and merge into one chain.
|
||||
6. **Sharing must not be foreclosed.** Collaborative data is a core
|
||||
local-first use case; the design must leave a clean path to it.
|
||||
|
||||
|
|
@ -93,8 +93,8 @@ justified only where field names are data-driven, as in the merge path.
|
|||
|
||||
The server holds one physical domain row per `(table, uuidRowId)`, while CRDT
|
||||
metadata (`crdt_data_rows` and its children) is keyed per
|
||||
`(scopeId, tblId, uuidRowId)`. Before this design was implemented, that
|
||||
metadata shape let multiple scopes track the same row even though the domain
|
||||
`(spaceId, tblId, uuidRowId)`. Before this design was implemented, that
|
||||
metadata shape let multiple spaces track the same row even though the domain
|
||||
table could only materialize one copy. Symptoms, in increasing severity:
|
||||
|
||||
1. **Tombstone masking.** A tombstone recorded by one user (or for another
|
||||
|
|
@ -125,31 +125,31 @@ All four share one root cause: the metadata says who owns a row, but nothing
|
|||
enforces that the answer is unique, and the domain storage assumes it is.
|
||||
|
||||
"Leave it up to users" is rejected explicitly: symptom 2 means the storage
|
||||
model cannot represent what multi-scope tracking promises, so allowing it is
|
||||
model cannot represent what multi-space tracking promises, so allowing it is
|
||||
not granting flexibility — it is permitting silent cross-tenant data
|
||||
corruption. Applications that need shared data use shared *scopes* (see
|
||||
`shared-scopes.md`), which preserve the one-chain-per-row property that makes
|
||||
corruption. Applications that need shared data use shared *spaces* (see
|
||||
`shared-spaces.md`), which preserve the one-chain-per-row property that makes
|
||||
LWW merging coherent.
|
||||
|
||||
## The `scopeId` column
|
||||
## The `spaceId` column
|
||||
|
||||
### Contract
|
||||
|
||||
`scopeId` is a reserved column name, required on every synced table:
|
||||
`spaceId` is a reserved column name, required on every synced table:
|
||||
|
||||
- **It is not part of row identity.** The domain primary key remains the
|
||||
required `id: UuidValue`, and `(table, uuidRowId)` remains globally unique
|
||||
in the physical domain table. A row with the same UUID in a different scope
|
||||
in the physical domain table. A row with the same UUID in a different space
|
||||
cannot be represented as a second domain row; it is treated as a corrupt or
|
||||
hostile ownership collision, the merge fails, and the violation is recorded.
|
||||
- The value references `crdt_scopes.id` — the scope that owns the row. Today
|
||||
a scope is a user; the name is chosen deliberately so shared scopes can be
|
||||
- The value references `offline_sync_spaces.id` — the space that owns the row. Today
|
||||
a space is a user; the name is chosen deliberately so shared spaces can be
|
||||
added later without renaming storage again.
|
||||
- **The value is database-local.** It is a normalization of the scope UUID
|
||||
into the local `crdt_scopes` table; the client's int for a scope is not
|
||||
the server's int for the same scope. The sync layer re-maps it on merge
|
||||
- **The value is database-local.** It is a normalization of the space UUID
|
||||
into the local `offline_sync_spaces` table; the client's int for a space is not
|
||||
the server's int for the same space. The sync layer re-maps it on merge
|
||||
and never ships it as data, and application code cannot ship it by
|
||||
accident either, because scoped reads return it stripped (see *Keeping
|
||||
accident either, because space-scoped reads return it stripped (see *Keeping
|
||||
the value off the wire*). This is also why the field is nullable: a model
|
||||
outside the database honestly carries "not mine to say" instead of a
|
||||
meaningless local id.
|
||||
|
|
@ -159,77 +159,77 @@ LWW merging coherent.
|
|||
- The column is **owned by the sync layer** and immutable after insert.
|
||||
The value round-trips the way it came in: callers that never provide it
|
||||
never receive it (stamped values are stripped from returned rows, and
|
||||
scoped reads return null), explicit callers keep seeing what they passed,
|
||||
space-scoped reads return null), explicit callers keep seeing what they passed,
|
||||
and unscoped admin reads always return it populated (see *Keeping the
|
||||
value off the wire*).
|
||||
|
||||
### Write semantics: stamp if null, assert if set
|
||||
|
||||
On every local insert (and batch insert), the database proxy resolves the
|
||||
effective scope — transaction scope, else persistent scope, else the
|
||||
effective space — transaction space, else persistent space, else the
|
||||
operation throws, exactly as recording already requires — and applies one
|
||||
rule per row:
|
||||
|
||||
- `scopeId == null` → the row is stamped with the effective scope before the
|
||||
- `spaceId == null` → the row is stamped with the effective space before the
|
||||
delegate insert (one write, not an UPDATE after the fact). Stamping uses a
|
||||
dynamic call, since no base interface can declare the field; this is the
|
||||
per-row hot path of every insert, which is why the heavier `patchTableRow`
|
||||
stays out of it. In-place mutation is preferred (generated fields are
|
||||
mutable, so no per-row allocation); dynamic `copyWith` is the fallback
|
||||
only if rows were ever immutable.
|
||||
- `scopeId != null` → it must equal the effective scope, or the operation
|
||||
- `spaceId != null` → it must equal the effective space, or the operation
|
||||
throws. Explicit callers get verification for free.
|
||||
|
||||
Consequences of the rule:
|
||||
|
||||
- **Cross-scope writes cannot succeed.** A model carrying another scope's id
|
||||
- **Cross-space writes cannot succeed.** A model carrying another space's id
|
||||
trips the assertion; a model carrying null is stamped with the acting
|
||||
scope and then the scoped `WHERE` filter (which includes
|
||||
`scopeId = <scope>`) matches no foreign row, so the update or delete fails
|
||||
space and then the space-scoped `WHERE` filter (which includes
|
||||
`spaceId = <space>`) matches no foreign row, so the update or delete fails
|
||||
with "no rows" instead of touching another tenant's data.
|
||||
- **The column never appears in update SET lists**, exactly like `id`
|
||||
(`_updateRowWithoutRecording` already filters `columnName != 'id'`).
|
||||
A tampered model cannot attempt to move a row across scopes.
|
||||
- **Merges stamp from the merging scope** (each merge acts in exactly one
|
||||
scope), and incoming payloads have the field stripped by
|
||||
A tampered model cannot attempt to move a row across spaces.
|
||||
- **Merges stamp from the merging space** (each merge acts in exactly one
|
||||
space), and incoming payloads have the field stripped by
|
||||
`_sanitizeMergeRowData` — it is not portable data.
|
||||
|
||||
The same effective-scope requirement applies to every CRUD entrypoint, not
|
||||
The same effective-space requirement applies to every CRUD entrypoint, not
|
||||
only inserts:
|
||||
|
||||
- `insert`/`insertRow`/batch insert: stamp-or-assert before the delegate write;
|
||||
- reinsert of a tombstoned row: only the same owning scope can revive it;
|
||||
a foreign-scope row id is a UUID ownership collision, not a reinsert target;
|
||||
- `update`/`updateRow`: the scoped `WHERE` includes `scopeId = <scope>`, and
|
||||
`scopeId` is removed from the SET list even if it appears in `columns`;
|
||||
- `updateWhere`/`updateById`: any `ColumnValue` for `scopeId` is rejected (or
|
||||
- reinsert of a tombstoned row: only the same owning space can revive it;
|
||||
a foreign-space row id is a UUID ownership collision, not a reinsert target;
|
||||
- `update`/`updateRow`: the space-scoped `WHERE` includes `spaceId = <space>`, and
|
||||
`spaceId` is removed from the SET list even if it appears in `columns`;
|
||||
- `updateWhere`/`updateById`: any `ColumnValue` for `spaceId` is rejected (or
|
||||
dropped before SQL generation, but rejection gives better caller feedback),
|
||||
and the scoped `WHERE` still includes `scopeId = <scope>`;
|
||||
and the space-scoped `WHERE` still includes `spaceId = <space>`;
|
||||
- `delete`/`deleteRow`/`deleteWhere`: deletes only see rows owned by the
|
||||
effective scope, so a foreign row id returns "no rows" instead of recording a
|
||||
effective space, so a foreign row id returns "no rows" instead of recording a
|
||||
tombstone for another owner;
|
||||
- future `upsert`: must first resolve ownership of the conflicting row and may
|
||||
update only when the existing row's `scopeId` equals the effective scope.
|
||||
update only when the existing row's `spaceId` equals the effective space.
|
||||
|
||||
### Keeping the value off the wire
|
||||
|
||||
The wire discipline is enforced by construction and respects the caller's
|
||||
choice: **the field leaves the package the way it came in.**
|
||||
|
||||
- **Stamped values stay internal.** When a model comes in with `scopeId`
|
||||
null, the proxy stamps the effective scope for the database write, and
|
||||
- **Stamped values stay internal.** When a model comes in with `spaceId`
|
||||
null, the proxy stamps the effective space for the database write, and
|
||||
the rows handed back by that operation carry null again. The same applies
|
||||
to scoped reads, which have no input model to respect. Stripping applies to
|
||||
to space-scoped reads, which have no input model to respect. Stripping applies to
|
||||
every synced model instance returned by the proxy, including nested
|
||||
`IncludeObject` and `IncludeList` result graphs. Callers who never provide
|
||||
the value never observe it, and cannot serialize the database-local int into
|
||||
endpoint payloads even by accident.
|
||||
- **Explicit values are respected.** When a model comes in with `scopeId`
|
||||
- **Explicit values are respected.** When a model comes in with `spaceId`
|
||||
set (and verified by the assert arm), the rows returned by that operation
|
||||
keep it — the package does not hide a value the caller knowingly manages.
|
||||
In batch operations the rule applies per row.
|
||||
- **Unscoped (admin) reads keep it populated.** Within a scoped read the
|
||||
value is redundant — it always equals the acting scope — but for admin
|
||||
- **Unscoped (admin) reads keep it populated.** Within a space-scoped read the
|
||||
value is redundant — it always equals the acting space — but for admin
|
||||
reads it is the only per-row ownership information, and those results are
|
||||
consumed server-side.
|
||||
|
||||
|
|
@ -244,14 +244,14 @@ This mirrors mature row-level-security practice, where the tenant column is
|
|||
populated from session context (`DEFAULT current_setting(...)`) precisely so
|
||||
application code cannot get it wrong, rather than passed through call sites.
|
||||
|
||||
A convenience accessor exposing the effective scope's normalized id (e.g. on
|
||||
A convenience accessor exposing the effective space's normalized id (e.g. on
|
||||
the transaction or the database) may be added for explicit callers and
|
||||
custom filtering; it is not required by any flow, and its documentation
|
||||
carries the database-local warning above.
|
||||
|
||||
### Sync-layer exclusions
|
||||
|
||||
`scopeId` is infrastructure that happens to live in the domain row. Keyed by
|
||||
`spaceId` is infrastructure that happens to live in the domain row. Keyed by
|
||||
the reserved name, it is excluded from:
|
||||
|
||||
- `crdt_schema_columns` registration and field-level CRDT metadata (it has no
|
||||
|
|
@ -259,38 +259,38 @@ the reserved name, it is excluded from:
|
|||
the synced-schema hash symmetrically on both sides;
|
||||
- update recording (`afterUpdate` ignores it) and merge payload sanitization;
|
||||
- outbound sync insert payloads and outbound sync update payloads. The sender
|
||||
must not serialize `scopeId` from the domain row; the receiver stamps the
|
||||
merge-local normalized scope instead.
|
||||
must not serialize `spaceId` from the domain row; the receiver stamps the
|
||||
merge-local normalized space instead.
|
||||
|
||||
### Validation
|
||||
|
||||
`CrdtSchemaRegistry` extends the existing UUID primary key check:
|
||||
|
||||
- every synced table must have a nullable `int` column named `scopeId`
|
||||
- every synced table must have a nullable `int` column named `spaceId`
|
||||
(missing or mistyped → error with the paste-ready yaml, including the
|
||||
"remove `scope=serverOnly`" hint when the server schema has the column and
|
||||
the client schema does not);
|
||||
- every non-primary unique index on a synced table must include `scopeId`
|
||||
- every non-primary unique index on a synced table must include `spaceId`
|
||||
(missing → error), except a foreign-key-only unique index whose referenced
|
||||
tables are also synced. Ordinary global unique indexes are forbidden for now
|
||||
because the package does not yet define deterministic cross-scope conflict
|
||||
because the package does not yet define deterministic cross-space conflict
|
||||
resolution.
|
||||
|
||||
## Unique indexes
|
||||
|
||||
Per-scope uniqueness is declared by the application as an ordinary composite
|
||||
Per-space uniqueness is declared by the application as an ordinary composite
|
||||
index, which now exists identically on both databases:
|
||||
|
||||
```yaml
|
||||
indexes:
|
||||
person_scope_name_unique_idx:
|
||||
fields: scopeId, name
|
||||
person_space_name_unique_idx:
|
||||
fields: spaceId, name
|
||||
unique: true
|
||||
```
|
||||
|
||||
Ordinary global unique indexes are rejected until the package has a
|
||||
deterministic cross-scope arbitration policy and a way to record the
|
||||
losing-side release in that scope's CRDT chain. "Whoever syncs first" is not
|
||||
deterministic cross-space arbitration policy and a way to record the
|
||||
losing-side release in that space's CRDT chain. "Whoever syncs first" is not
|
||||
an acceptable CRDT rule. The only permitted global unique indexes are
|
||||
foreign-key-only indexes: every index element must be a column, and the
|
||||
indexed columns must be composed only of complete foreign-key column sets whose
|
||||
|
|
@ -299,47 +299,47 @@ policy, every indexed FK column in that exception must be nullable. A required
|
|||
unique FK has no safe loser value: `null` violates the column, a synthetic UUID
|
||||
violates the FK constraint, and keeping the original value violates uniqueness.
|
||||
This covers optional Serverpod relation indexes, including optional one-to-one
|
||||
and composite FK relation indexes, without forcing duplicate per-scope indexes
|
||||
and composite FK relation indexes, without forcing duplicate per-space indexes
|
||||
for include-query performance. Those columns point at globally unique synced row
|
||||
ids, and FK enforcement already repairs or rejects cross-scope references, so
|
||||
ids, and FK enforcement already repairs or rejects cross-space references, so
|
||||
they are not treated as global application-level arbitration. The model must
|
||||
still declare scoped composite unique indexes for application uniqueness. On a
|
||||
deployment that happens to contain one scope, a composite unique with a constant
|
||||
`scopeId` accepts exactly the same rows as a plain unique; nothing degrades.
|
||||
still declare space-scoped composite unique indexes for application uniqueness. On a
|
||||
deployment that happens to contain one space, a composite unique with a constant
|
||||
`spaceId` accepts exactly the same rows as a plain unique; nothing degrades.
|
||||
|
||||
The unique-conflict resolver discovers indexes from the table definitions,
|
||||
uses `scopeId` as part of the lookup predicate for scoped indexes, and releases
|
||||
only releasable non-`scopeId` columns. For allowed FK-only global unique
|
||||
uses `spaceId` as part of the lookup predicate for space-scoped indexes, and releases
|
||||
only releasable non-`spaceId` columns. For allowed FK-only global unique
|
||||
indexes, nullable FK columns may be released to `null`; required FK columns are
|
||||
rejected during schema initialization. Same-scope application conflicts are
|
||||
handled by scoped composite indexes, and cross-scope references are handled by
|
||||
FK repair. Rows in different scopes never form a conflict group under the
|
||||
rejected during schema initialization. Same-space application conflicts are
|
||||
handled by space-scoped composite indexes, and cross-space references are handled by
|
||||
FK repair. Rows in different spaces never form a conflict group under the
|
||||
required composite index. The flag policy (see `unique-constraint-invariants.md`)
|
||||
is otherwise unchanged.
|
||||
|
||||
## Enforcement mechanics
|
||||
|
||||
1. **Local writes: stamp-or-assert** (see *Write semantics*), with the
|
||||
effective scope required exactly as it already is for CRDT recording.
|
||||
effective space required exactly as it already is for CRDT recording.
|
||||
2. **Merge-insert ownership check.** `_applyMergeInsertForMissingRow` treats
|
||||
an existing domain row with another `scopeId` as a UUID ownership collision.
|
||||
an existing domain row with another `spaceId` as a UUID ownership collision.
|
||||
This is a terminal sync error, not a recoverable conflict policy. The check
|
||||
is atomic with the physical write:
|
||||
1. Look up the domain row by `(table, uuidRowId)` before writing.
|
||||
2. If it exists with the merging scope, follow the same-scope
|
||||
2. If it exists with the merging space, follow the same-space
|
||||
update/reinsert path.
|
||||
3. If it exists with another scope, roll back the merge work for the
|
||||
3. If it exists with another space, roll back the merge work for the
|
||||
incoming change, record a durable integrity violation with denormalized
|
||||
operation, table, row, scope, node UUID, and HLC fields, then fail the
|
||||
operation, table, row, space, node UUID, and HLC fields, then fail the
|
||||
sync session.
|
||||
4. If it does not exist, insert with `scopeId` stamped from the merging
|
||||
scope.
|
||||
5. If the insert races another scope and hits the domain primary key,
|
||||
4. If it does not exist, insert with `spaceId` stamped from the merging
|
||||
space.
|
||||
5. If the insert races another space and hits the domain primary key,
|
||||
re-read the row and apply the same ownership decision before any CRDT
|
||||
metadata survives the transaction.
|
||||
|
||||
Step 5 carries an implementation constraint: today the merge records the
|
||||
scope's CRDT row (`_upsertMergeRow`) — and unique-conflict resolution and
|
||||
space's CRDT row (`_upsertMergeRow`) — and unique-conflict resolution and
|
||||
FK-safe handling, which may mutate neighboring rows — *before* the domain
|
||||
write where the collision surfaces. The insert application must therefore
|
||||
run inside a savepoint rolled back on collision (or be reordered
|
||||
|
|
@ -349,9 +349,9 @@ is otherwise unchanged.
|
|||
diagnostic record survives even though the bad change does not.
|
||||
|
||||
The durable record is sparse exception state, not a second ownership index.
|
||||
`crdt_sync_integrity_violations` stores one deduplicated row per observed
|
||||
`offline_sync_integrity_violations` stores one deduplicated row per observed
|
||||
violation key: `(type, operation, domainTableName, uuidRowId,
|
||||
ownerScopeUuid, incomingScopeUuid)`. All other stored fields are
|
||||
ownerSpaceUuid, incomingSpaceUuid)`. All other stored fields are
|
||||
denormalized scalars so the record remains useful after merge rollbacks and
|
||||
on databases that do not share the same local CRDT metadata rows. There are
|
||||
no foreign keys into CRDT metadata tables (`crdt_schema_tables`,
|
||||
|
|
@ -366,9 +366,9 @@ is otherwise unchanged.
|
|||
causal context; optional `crdtDataRowId` as a same-database hint to the
|
||||
local metadata row id when one was known (for example outbound sync with a
|
||||
stale tracker); `firstSeenAt`, `lastSeenAt`, and `occurrences`.
|
||||
`ownerScopeUuid` is nullable because a corrupt physical row can exist with
|
||||
`ownerSpaceUuid` is nullable because a corrupt physical row can exist with
|
||||
a null owner;
|
||||
`incomingScopeUuid` is required and globally locatable. `crdtDataRowId` is
|
||||
`incomingSpaceUuid` is required and globally locatable. `crdtDataRowId` is
|
||||
often null on merge-insert collisions and is not portable across devices.
|
||||
The record intentionally stores neither the rejected payload nor a payload
|
||||
hash. The sync response returns the violation identifier and checkpoints do
|
||||
|
|
@ -382,23 +382,23 @@ is otherwise unchanged.
|
|||
between honest clients are negligible; a collision is a bug or an attack,
|
||||
and failing the sync is the only response that preserves both the existing
|
||||
owner's data and the sender's still-unaccepted local change.
|
||||
3. **Foreign keys stay within a scope.** A synced row may only reference
|
||||
synced rows of its own scope. Locally,
|
||||
`_assertVisibleForeignKeyTargets` gains a scope-equality check next to
|
||||
its visibility check; in merges, a foreign-scope target is treated
|
||||
3. **Foreign keys stay within a space.** A synced row may only reference
|
||||
synced rows of its own space. Locally,
|
||||
`_assertVisibleForeignKeyTargets` gains a space-equality check next to
|
||||
its visibility check; in merges, a foreign-space target is treated
|
||||
exactly like an invisible target, so the existing attempt/override
|
||||
machinery (see `foreign-key-invariants.md`) repairs the reference instead
|
||||
of linking across scopes. Cascade deletes and SET NULL projections
|
||||
therefore never cross a scope boundary. References to non-synced tables
|
||||
of linking across spaces. Cascade deletes and SET NULL projections
|
||||
therefore never cross a space boundary. References to non-synced tables
|
||||
are unaffected.
|
||||
4. **No duplicated metadata lookup index.** An earlier draft added a unique
|
||||
index on `crdt_data_rows (tblId, uuidRowId)` as defense-in-depth. It is
|
||||
dropped because the hot CRDT paths need the existing scoped index
|
||||
`(scopeId, tblId, uuidRowId)`, and adding the unscoped index duplicates
|
||||
dropped because the hot CRDT paths need the existing space-scoped index
|
||||
`(spaceId, tblId, uuidRowId)`, and adding the unscoped index duplicates
|
||||
most of the key space on every row. Instead, every consumer of CRDT row
|
||||
metadata is required to prove ownership against the physical row: reads key
|
||||
visibility through `domain.scopeId`; outbound sync serializes a domain row
|
||||
only when `domain.scopeId` equals the collecting scope; merge, FK, and
|
||||
visibility through `domain.spaceId`; outbound sync serializes a domain row
|
||||
only when `domain.spaceId` equals the collecting space; merge, FK, and
|
||||
unique helpers reject metadata that attempts to materialize or mutate a
|
||||
foreign-owned row. A stray foreign tracker is therefore inert by consumer
|
||||
discipline, while an observed attempt to use it is a durable sync violation.
|
||||
|
|
@ -413,24 +413,24 @@ index-covered probe; the aggregate (`GROUP BY`/`MIN`) form, its full-scan
|
|||
cost (~95 ms per point lookup at 1M CRDT rows in the SQLite benchmark), and
|
||||
the covering index it wanted never ship.
|
||||
|
||||
- **Scoped query** (a `transactionForUser` read, or the persistent default
|
||||
scope): rows are filtered by `scopeId = <scope>` — the isolation promise,
|
||||
- **Space-scoped query** (a `transactionForUser` read, or the persistent default
|
||||
space): rows are filtered by `spaceId = <space>` — the isolation promise,
|
||||
and the central point of the design — plus the existing correlated
|
||||
`NOT EXISTS` visibility probe on `(scopeId, tblId, uuidRowId)`.
|
||||
- **Unscoped query** (no transaction scope, no persistent scope — admin
|
||||
`NOT EXISTS` visibility probe on `(spaceId, tblId, uuidRowId)`.
|
||||
- **Unscoped query** (no transaction space, no persistent space — admin
|
||||
reads): no isolation filter; the visibility probe is keyed by the row's
|
||||
own column — `NOT EXISTS (… WHERE c.scopeId = t."scopeId" AND tblId = ?
|
||||
own column — `NOT EXISTS (… WHERE c.spaceId = t."spaceId" AND tblId = ?
|
||||
AND uuidRowId = t.id AND visibility > …)` — fully covered by the existing
|
||||
unique index. A row is visible while its owner sees it.
|
||||
|
||||
On a database containing one scope, the two views return the same rows; no
|
||||
code anywhere branches on scope count. Rows with a null `scopeId` — created
|
||||
On a database containing one space, the two views return the same rows; no
|
||||
code anywhere branches on space count. Rows with a null `spaceId` — created
|
||||
behind the sync layer's back (raw SQL, seed fixtures) — are visible only to
|
||||
unscoped reads; seeding flows must stamp scopes, which ties into the
|
||||
unscoped reads; seeding flows must stamp spaces, which ties into the
|
||||
reference-table open question.
|
||||
|
||||
Multi-scope deployments additionally gain Postgres row-level security as an
|
||||
option, since policies can reference `scopeId` directly — the package's
|
||||
Multi-space deployments additionally gain Postgres row-level security as an
|
||||
option, since policies can reference `spaceId` directly — the package's
|
||||
isolation filter and RLS become two enforcements of one predicate.
|
||||
|
||||
## Sync path
|
||||
|
|
@ -438,71 +438,71 @@ isolation filter and RLS become two enforcements of one predicate.
|
|||
Sync also consumes CRDT metadata, so it follows the same row-ownership rule
|
||||
as reads and writes:
|
||||
|
||||
- **Outbound inserts** collect `crdt_data_rows` for one scope, then fetch the
|
||||
physical domain row only if `domain.scopeId` equals that same scope. A
|
||||
- **Outbound inserts** collect `crdt_data_rows` for one space, then fetch the
|
||||
physical domain row only if `domain.spaceId` equals that same space. A
|
||||
stale or hostile tracker for a foreign-owned row fails the sync and records
|
||||
a durable violation before any payload is produced.
|
||||
- **Outbound updates** fetch the column value only after the field's CRDT row
|
||||
is proven to match the domain row owner. `scopeId` itself is never emitted
|
||||
is proven to match the domain row owner. `spaceId` itself is never emitted
|
||||
as an update because it has no field metadata.
|
||||
- **Outbound deletes** are scoped by CRDT metadata; a foreign tracker can at
|
||||
worst try to produce a tombstone for a row the collecting scope never owned,
|
||||
- **Outbound deletes** are space-scoped by CRDT metadata; a foreign tracker can at
|
||||
worst try to produce a tombstone for a row the collecting space never owned,
|
||||
so delete collection also verifies the domain owner when the domain row still
|
||||
exists and fails with a durable violation on mismatch.
|
||||
- **Inbound inserts and updates** sanitize any incoming `scopeId` before merge
|
||||
and stamp from the merge scope. Incoming row UUIDs that already belong to
|
||||
another scope fail the sync session as ownership collisions, record durable
|
||||
- **Inbound inserts and updates** sanitize any incoming `spaceId` before merge
|
||||
and stamp from the merge space. Incoming row UUIDs that already belong to
|
||||
another space fail the sync session as ownership collisions, record durable
|
||||
violations, and do not advance checkpoints past the rejected change.
|
||||
|
||||
This is the audit rule for all future sync helpers: no path may serialize,
|
||||
materialize, resolve conflicts for, or project a synced domain row through
|
||||
CRDT metadata unless the domain row's `scopeId` equals the effective CRDT
|
||||
scope, except explicit unscoped admin reads.
|
||||
CRDT metadata unless the domain row's `spaceId` equals the effective CRDT
|
||||
space, except explicit unscoped admin reads.
|
||||
|
||||
## API surface
|
||||
|
||||
The package is unreleased, so renames land directly, with no aliases and no
|
||||
data migration. Public API stays user-first (that is the semantics
|
||||
developers expect to write); internals and storage are named after what they
|
||||
actually key — scopes:
|
||||
actually key — spaces:
|
||||
|
||||
- **`transactionForUser(userId, fn)`** keeps its name and shape: it resolves
|
||||
the user's UUID to the local scope and pins the transaction to it — writes
|
||||
the user's UUID to the local space and pins the transaction to it — writes
|
||||
stamp and assert against it, reads are isolated to it. The optional
|
||||
`scopeId` parameter is reserved for shared scopes (see *Future*).
|
||||
- **`persistentUserId`** remains the way to pin a default scope outside
|
||||
`spaceId` parameter is reserved for shared spaces (see *Future*).
|
||||
- **`persistentUserId`** remains the way to pin a default space outside
|
||||
explicit transactions — the standard single-user client configuration.
|
||||
- **`mergeChanges(scopeId: …)`** keys the chain being merged — the parameter
|
||||
- **`mergeChanges(spaceId: …)`** keys the chain being merged — the parameter
|
||||
is renamed from `userId` to what it always was.
|
||||
- **Internals and storage** adopt scope naming in one sweep: `crdt_users` →
|
||||
`crdt_scopes` (`CrdtUser` → `CrdtScope`, `uuidUserId` → `uuidScopeId`),
|
||||
`crdt_data_rows.userId` → `scopeId` (matching the domain column),
|
||||
- **Internals and storage** adopt space naming in one sweep: `crdt_users` →
|
||||
`offline_sync_spaces` (`CrdtUser` → `OfflineSyncSpace`, `uuidUserId` → `uuidSpaceId`),
|
||||
`crdt_data_rows.userId` → `spaceId` (matching the domain column),
|
||||
`userForTransaction`, the effective-user helpers, and the HLC manager keys.
|
||||
- **Unscoped reads** remain supported as admin reads.
|
||||
|
||||
Consequences for application code, stated as contract:
|
||||
|
||||
1. **Every synced table declares `id: UuidValue` and `scopeId: int?`.**
|
||||
1. **Every synced table declares `id: UuidValue` and `spaceId: int?`.**
|
||||
Startup fails with a paste-ready error otherwise.
|
||||
2. **`id` remains globally unique per synced table.** The same
|
||||
`(table, uuidRowId)` cannot exist in two scopes. If merge observes that
|
||||
UUID already owned by another scope, it records a durable ownership
|
||||
`(table, uuidRowId)` cannot exist in two spaces. If merge observes that
|
||||
UUID already owned by another space, it records a durable ownership
|
||||
violation and fails the sync.
|
||||
3. **`scopeId` round-trips the way it came in** — left null, the sync layer
|
||||
stamps it internally and returns it null (scoped reads return it null
|
||||
too); provided explicitly, it must match the effective scope or the
|
||||
3. **`spaceId` round-trips the way it came in** — left null, the sync layer
|
||||
stamps it internally and returns it null (space-scoped reads return it null
|
||||
too); provided explicitly, it must match the effective space or the
|
||||
operation throws, and it stays visible on the way out. The column is
|
||||
immutable after insert.
|
||||
4. **Scoped reads are isolated to their scope.** Reading across scopes is an
|
||||
4. **Space-scoped reads are isolated to their space.** Reading across spaces is an
|
||||
explicit unscoped (admin) read.
|
||||
5. **Merge-inserts colliding with another scope's row fail the sync**, never
|
||||
5. **Merge-inserts colliding with another space's row fail the sync**, never
|
||||
apply over the owner's data, and never advance the sender past the rejected
|
||||
change.
|
||||
6. **Foreign keys on synced tables target same-scope rows.** Cross-scope
|
||||
6. **Foreign keys on synced tables target same-space rows.** Cross-space
|
||||
references are repaired by the existing FK override machinery instead of
|
||||
being linked.
|
||||
7. **Application unique indexes on synced tables include `scopeId`.** Global
|
||||
unique indexes are rejected at startup until a deterministic cross-scope
|
||||
7. **Application unique indexes on synced tables include `spaceId`.** Global
|
||||
unique indexes are rejected at startup until a deterministic cross-space
|
||||
policy exists, except FK-only indexes whose referenced tables are synced.
|
||||
|
||||
## Implementation plan
|
||||
|
|
@ -510,165 +510,165 @@ Consequences for application code, stated as contract:
|
|||
Each phase lands independently green; later phases depend on earlier ones
|
||||
only where stated. No generator or Serverpod fork changes are required.
|
||||
|
||||
- **Phase 1 — naming, contract, validation.** The user→scope renames in
|
||||
- **Phase 1 — naming, contract, validation.** The user→space renames in
|
||||
internals and storage (above), done first so later phases build on the
|
||||
final names. `CrdtSchemaRegistry`: the `scopeId` presence/type check with
|
||||
final names. `CrdtSchemaRegistry`: the `spaceId` presence/type check with
|
||||
the `serverOnly` hint, and the global unique-index rejection. Reserved-name
|
||||
exclusions: `crdt_schema_columns` registration, field-level merge, update
|
||||
recording, inbound merge sanitization, and outbound sync serialization.
|
||||
Test models gain `scopeId`.
|
||||
Test models gain `spaceId`.
|
||||
- **Phase 2 — write-path enforcement.** Stamp-or-assert in the proxy insert
|
||||
paths (dynamic dispatch on the known field); `scopeId` rejected from
|
||||
paths (dynamic dispatch on the known field); `spaceId` rejected from
|
||||
`updateWhere`/`updateById` column values and excluded from model-update SET
|
||||
lists; ownership assertion on update, delete, reinsert, and upsert; atomic
|
||||
merge-insert ownership check with durable fail-fast collision handling,
|
||||
including primary-key race handling; merge stamping from the merging scope;
|
||||
FK same-scope assertion and merge repair routing; unique resolver limited
|
||||
to scoped indexes, with ordinary global indexes rejected by validation.
|
||||
- **Phase 3 — read paths.** Isolation filter for scoped reads; row-keyed
|
||||
visibility probe for unscoped reads; stripping of `scopeId` from scoped
|
||||
including primary-key race handling; merge stamping from the merging space;
|
||||
FK same-space assertion and merge repair routing; unique resolver limited
|
||||
to space-scoped indexes, with ordinary global indexes rejected by validation.
|
||||
- **Phase 3 — read paths.** Isolation filter for space-scoped reads; row-keyed
|
||||
visibility probe for unscoped reads; stripping of `spaceId` from space-scoped
|
||||
root and include-graph read results; outbound sync owner checks before
|
||||
serializing inserts, updates, or deletes; delete the `GROUP BY`/`MIN` form
|
||||
and the server-only-index TODO from `tombstone.dart`.
|
||||
- **Phase 4 — tests and benchmarks.** The plan below, plus updating
|
||||
`TombstoneScopeBenchmark` to ownership-conformant seeding (one tracker per
|
||||
`TombstoneSpaceBenchmark` to ownership-conformant seeding (one tracker per
|
||||
row); its current second-tracker fixtures survive only as the enforcement
|
||||
scenario.
|
||||
|
||||
### Test plan
|
||||
|
||||
Because no code branches on scope count or deployment role, mechanisms are
|
||||
Because no code branches on space count or deployment role, mechanisms are
|
||||
tested **once**, not per combination:
|
||||
|
||||
- **Mechanism suites (run once).** All existing CRUD, tombstone visibility,
|
||||
unique-conflict, FK-invariant, merge, and sync suites continue on the
|
||||
uniform model; test models declare `scopeId` and are otherwise unchanged
|
||||
uniform model; test models declare `spaceId` and are otherwise unchanged
|
||||
(the stamp rule means call sites do not change).
|
||||
- **Enforcement tests.** Stamping on insert; assert-on-mismatch throwing for
|
||||
insert and update; immutability of the column; round-trip symmetry (null
|
||||
in → null out, explicit in → kept, per row in batches); stripping on
|
||||
scoped root and included read results and retention on unscoped reads; the
|
||||
space-scoped root and included read results and retention on unscoped reads; the
|
||||
caller's input instance left unmutated; merge-insert collision fails sync,
|
||||
records a durable violation, and covers the primary-key race regression;
|
||||
stale foreign tracker fails before producing outbound insert/update/delete
|
||||
payloads and records a durable violation (the multi-user fixtures in
|
||||
`tombstone_scope_test.dart` become these tests); FK cross-scope repair;
|
||||
`tombstone_space_test.dart` become these tests); FK cross-space repair;
|
||||
schema validation rejects ordinary global unique indexes and accepts
|
||||
FK-only unique indexes whose referenced tables are synced.
|
||||
- **Combination smoke tests (one each, not per mechanism).** A two-scope
|
||||
database proving read isolation and per-scope uniqueness on SQLite; one
|
||||
- **Combination smoke tests (one each, not per mechanism).** A two-space
|
||||
database proving read isolation and per-space uniqueness on SQLite; one
|
||||
`withServerpod` sync roundtrip proving the wiring end to end. These exist
|
||||
to prove deployment shapes compose, not to re-verify mechanisms.
|
||||
|
||||
## Shared scopes
|
||||
## Shared spaces
|
||||
|
||||
`crdt_scopes` is the entity everything is keyed by: sync, locking, HLC
|
||||
`offline_sync_spaces` is the entity everything is keyed by: sync, locking, HLC
|
||||
chains, and visibility. Sharing becomes an indirection in front of it, not a
|
||||
change to row ownership:
|
||||
|
||||
- an authenticated identity maps to one personal scope plus any number of
|
||||
shared scopes;
|
||||
- a shared list/document/workspace is a scope that several identities
|
||||
- an authenticated identity maps to one personal space plus any number of
|
||||
shared spaces;
|
||||
- a shared list/document/workspace is a space that several identities
|
||||
subscribe to and sync;
|
||||
- every row still has exactly one owning scope (its `scopeId`), so merging
|
||||
- every row still has exactly one owning space (its `spaceId`), so merging
|
||||
stays one chain per row and nothing in this document's enforcement
|
||||
changes.
|
||||
|
||||
This is the standard shape of sharing in local-first systems, and it is the
|
||||
reason the column and concept are named `scope` rather than `user`.
|
||||
Implementation details and current limits live in `shared-scopes.md`; the rest
|
||||
reason the column and concept are named `space` rather than `user`.
|
||||
Implementation details and current limits live in `shared-spaces.md`; the rest
|
||||
of this section summarizes why the ownership rules remain compatible.
|
||||
|
||||
### Users authenticate, scopes act
|
||||
### Users authenticate, spaces act
|
||||
|
||||
The two concepts separate cleanly:
|
||||
|
||||
- a **user** is an authentication identity — what endpoints and the sync
|
||||
handshake verify;
|
||||
- a **scope** is the unit of replication and ownership — what transactions
|
||||
act in, what chains and tombstones are keyed by, what `scopeId`
|
||||
- a **space** is the unit of replication and ownership — what transactions
|
||||
act in, what chains and tombstones are keyed by, what `spaceId`
|
||||
references.
|
||||
|
||||
Two conventions bridge them with no storage migration:
|
||||
|
||||
1. **A user's personal scope has the user's own UUID.** This is already true
|
||||
(the scope UUID keys the chain today), so every deployment without
|
||||
sharing is the degenerate case "each user has exactly one scope: their
|
||||
personal one". Shared scopes are new `crdt_scopes` rows whose UUID is no
|
||||
1. **A user's personal space has the user's own UUID.** This is already true
|
||||
(the space UUID keys the chain today), so every deployment without
|
||||
sharing is the degenerate case "each user has exactly one space: their
|
||||
personal one". Shared spaces are new `offline_sync_spaces` rows whose UUID is no
|
||||
user's id.
|
||||
2. **Membership** is a server-side relation, `crdt_scope_members(scopeId,
|
||||
2. **Membership** is a server-side relation, `offline_sync_space_members(spaceId,
|
||||
userUuid, role?)`, managed by application code (invitations and roles are
|
||||
app domain). Personal-scope membership is implicit; no row is stored for
|
||||
app domain). Personal-space membership is implicit; no row is stored for
|
||||
it.
|
||||
|
||||
### The transaction API
|
||||
|
||||
`transactionForUser` keeps its name and user-first semantics — that is what
|
||||
developers expect to write — and gains an optional scope:
|
||||
developers expect to write — and gains an optional space:
|
||||
|
||||
```dart
|
||||
db.transactionForUser(userId, fn); // acts in the personal scope
|
||||
db.transactionForUser(userId, fn, scopeId: listId); // acts in a shared scope
|
||||
db.transactionForUser(userId, fn); // acts in the personal space
|
||||
db.transactionForUser(userId, fn, spaceId: listId); // acts in a shared space
|
||||
```
|
||||
|
||||
- Without `scopeId`, the scope resolves to the user's personal scope — by
|
||||
- Without `spaceId`, the space resolves to the user's personal space — by
|
||||
convention 1, simply `userId` itself, which is exactly today's behavior.
|
||||
- With `scopeId`, the `userId` argument is **not dead weight**: the package
|
||||
asserts `membership(userId, scopeId)` before acting. The parameter pair
|
||||
reads as "authenticated as `userId`, acting in `scopeId`, verified
|
||||
- With `spaceId`, the `userId` argument is **not dead weight**: the package
|
||||
asserts `membership(userId, spaceId)` before acting. The parameter pair
|
||||
reads as "authenticated as `userId`, acting in `spaceId`, verified
|
||||
member" — the same predicate row-level security policies mirror. The
|
||||
caller remains responsible for `userId` being authenticated, exactly as
|
||||
today.
|
||||
- A transaction acts in **exactly one scope**. Cross-scope writes need
|
||||
- A transaction acts in **exactly one space**. Cross-space writes need
|
||||
separate transactions; this is deliberate honesty about CRDT semantics,
|
||||
since two scopes' chains replicate independently and remote replicas can
|
||||
never observe a cross-scope write atomically anyway.
|
||||
since two spaces' chains replicate independently and remote replicas can
|
||||
never observe a cross-space write atomically anyway.
|
||||
|
||||
### Reads: membership-wide; writes: scope-pinned
|
||||
### Reads: membership-wide; writes: space-pinned
|
||||
|
||||
A user-scoped read in a sharing world naturally means "rows in any scope I am
|
||||
a member of": the isolation filter becomes `scopeId IN (<member scopes>)`
|
||||
instead of `scopeId = <scope>`. The visibility probe is the row-keyed form —
|
||||
A user-space-scoped read in a sharing world naturally means "rows in any space I am
|
||||
a member of": the isolation filter becomes `spaceId IN (<member spaces>)`
|
||||
instead of `spaceId = <space>`. The visibility probe is the row-keyed form —
|
||||
the same probe as unscoped admin reads — so sharing simplifies the predicate
|
||||
matrix rather than growing it: reads differ only in their membership filter
|
||||
(one scope, a user's scopes, or none for admin).
|
||||
(one space, a user's spaces, or none for admin).
|
||||
|
||||
This is also exactly the shape of a Postgres row-level security policy
|
||||
(`scopeId IN (SELECT scopeId FROM crdt_scope_members WHERE userUuid =
|
||||
(`spaceId IN (SELECT spaceId FROM offline_sync_space_members WHERE userUuid =
|
||||
current_setting(...))`), which is the standard practice for multi-tenant
|
||||
isolation and on the Serverpod backlog; the package's application-level
|
||||
filter and RLS become two enforcements of one predicate.
|
||||
|
||||
### Sync and clients
|
||||
|
||||
- A device syncs one chain per scope it holds: each sync call enumerates the
|
||||
authenticated user's member scopes, verified server-side against
|
||||
`crdt_scope_members` at the start of every sync cycle; the membership table
|
||||
itself is not synced. `CrdtScopeRole` is enforced as described in
|
||||
`shared-scopes.md`: reads stay membership-wide, writes are role-gated, and the
|
||||
- A device syncs one chain per space it holds: each sync call enumerates the
|
||||
authenticated user's member spaces, verified server-side against
|
||||
`offline_sync_space_members` at the start of every sync cycle; the membership table
|
||||
itself is not synced. `OfflineSyncSpaceRole` is enforced as described in
|
||||
`shared-spaces.md`: reads stay membership-wide, writes are role-gated, and the
|
||||
server records `unauthorizedWrite` for rejected inbound changes.
|
||||
- Every database is already sharing-capable schema-wise — the column is
|
||||
universal — so a sharing-enabled client simply holds several scopes' rows.
|
||||
One database file per scope remains a valid alternative layout.
|
||||
- Checkpoints stay per `(scope, node)`: `crdt_nodes` stores the stable replica
|
||||
identity, while `crdt_scope_nodes` stores each scope's checkpoint for that
|
||||
universal — so a sharing-enabled client simply holds several spaces' rows.
|
||||
One database file per space remains a valid alternative layout.
|
||||
- Checkpoints stay per `(space, node)`: `crdt_nodes` stores the stable replica
|
||||
identity, while `offline_sync_space_nodes` stores each space's checkpoint for that
|
||||
node.
|
||||
|
||||
### What sharing does not change
|
||||
|
||||
Every enforcement rule in this document survives unchanged: `(table,
|
||||
uuidRowId)` stays globally identified, rows have exactly one owning scope,
|
||||
merge-insert ownership checks compare scopes, unique conflict groups form
|
||||
within a scope, ordinary global unique indexes stay rejected, and
|
||||
stamp-or-assert applies with the transaction's resolved scope. Sharing adds a
|
||||
membership layer in front of scopes; it does not touch row ownership.
|
||||
uuidRowId)` stays globally identified, rows have exactly one owning space,
|
||||
merge-insert ownership checks compare spaces, unique conflict groups form
|
||||
within a space, ordinary global unique indexes stay rejected, and
|
||||
stamp-or-assert applies with the transaction's resolved space. Sharing adds a
|
||||
membership layer in front of spaces; it does not touch row ownership.
|
||||
|
||||
## Resolved during review
|
||||
|
||||
Decisions that were open questions in earlier drafts, kept here so they are
|
||||
not relitigated by accident:
|
||||
|
||||
- **Optional vs. required `scopeId`:** required, on every synced table and
|
||||
- **Optional vs. required `spaceId`:** required, on every synced table and
|
||||
in both schemas. The optional-column draft and the schema-presence-mode
|
||||
draft were both rejected: each created a second ownership architecture (or
|
||||
a second schema shape) that every mechanism would have to support and
|
||||
|
|
@ -677,8 +677,8 @@ not relitigated by accident:
|
|||
cost of ~1–2 bytes per row and a visible read-only field on domain models.
|
||||
- **Global UUID row identity:** retained. Serverpod's generated domain model
|
||||
and ORM expect one primary key column, so the package does not introduce
|
||||
composite `(scopeId, id)` identity or a surrogate physical row id. A
|
||||
duplicate `(table, uuidRowId)` in another scope is an ownership collision,
|
||||
composite `(spaceId, id)` identity or a surrogate physical row id. A
|
||||
duplicate `(table, uuidRowId)` in another space is an ownership collision,
|
||||
not a valid second row.
|
||||
- **`scope=serverOnly` on the column:** rejected (would reintroduce
|
||||
asymmetric schemas); validation forbids it.
|
||||
|
|
@ -694,7 +694,7 @@ not relitigated by accident:
|
|||
sync layer guarantees the value on every synced row.
|
||||
- **Wire transport prevented by construction, respecting the caller:** the
|
||||
value leaves the way it came in — stamped values are stripped from
|
||||
returned rows and scoped reads, including include graphs; explicitly
|
||||
returned rows and space-scoped reads, including include graphs; explicitly
|
||||
provided (and verified) values are kept on the way out; unscoped admin
|
||||
reads keep the value as the only per-row ownership information server-side;
|
||||
outbound sync never serializes it. Mechanism: in-place dynamic mutation on
|
||||
|
|
@ -703,7 +703,7 @@ not relitigated by accident:
|
|||
field names are data-driven and its 50–200× cost is justified), never
|
||||
leaving the caller's input observably mutated.
|
||||
- **Unscoped-read semantics:** an unscoped read is an admin view; a row is
|
||||
visible while its owner sees it. Scoped reads are isolated to their scope
|
||||
visible while its owner sees it. Space-scoped reads are isolated to their space
|
||||
— confirmed as the central goal of the design.
|
||||
- **Fail vs. skip vs. re-key on colliding inserts:** fail and record a
|
||||
durable violation. Skipping can silently consume a client's rejected change
|
||||
|
|
@ -711,9 +711,9 @@ not relitigated by accident:
|
|||
would break the incoming chain's own FK references.
|
||||
- **Metadata backstop index:** dropped only with the consumer audit rule:
|
||||
every read, write, sync, merge, FK, and unique path that consumes CRDT row
|
||||
metadata must prove `domain.scopeId` equals the effective metadata scope
|
||||
metadata must prove `domain.spaceId` equals the effective metadata space
|
||||
before materializing domain data.
|
||||
- **Audit surface:** a durable `crdt_sync_integrity_violations` table is part
|
||||
- **Audit surface:** a durable `offline_sync_integrity_violations` table is part
|
||||
of the terminal sync-integrity contract. Session logs may mirror it, but
|
||||
they are not the source of truth.
|
||||
- **No compatibility aliases:** the package is unreleased, so renames
|
||||
|
|
@ -724,23 +724,23 @@ not relitigated by accident:
|
|||
## Open questions
|
||||
|
||||
1. **Globally shared reference tables.** Read-only data synced to every user
|
||||
would violate "every row has one owning scope" and is out of scope; if
|
||||
would violate "every row has one owning space" and is out of space; if
|
||||
the use case materializes, it likely becomes a third table kind rather
|
||||
than a relaxation of the contract. Related: rows seeded behind the sync
|
||||
layer's back carry a null `scopeId` and are visible only to admin reads.
|
||||
2. **`scopeId` naming collisions.** The name is reserved by convention; an
|
||||
application with a pre-existing, semantically different `scopeId` column
|
||||
layer's back carry a null `spaceId` and are visible only to admin reads.
|
||||
2. **`spaceId` naming collisions.** The name is reserved by convention; an
|
||||
application with a pre-existing, semantically different `spaceId` column
|
||||
on a synced table has no escape hatch other than renaming its column.
|
||||
Acceptable for now; revisit if it bites real users.
|
||||
3. **Membership revocation semantics (sharing).** When a user loses membership
|
||||
of a scope, the server stops streaming it at the next sync cycle. The device
|
||||
may still store the scope's data and unsynced local changes, though
|
||||
of a space, the server stops streaming it at the next sync cycle. The device
|
||||
may still store the space's data and unsynced local changes, though
|
||||
membership-wide reads no longer expose revoked rows. Purge-vs-keep policy,
|
||||
and whether revoked unsynced changes are surfaced or dropped, need their own
|
||||
design.
|
||||
4. **Roles within a scope (sharing).** Resolved in `shared-scopes.md`.
|
||||
`crdt_scope_members.role` is enforced for writes through the closed
|
||||
`CrdtScopeRole` enum: the server rejects and records unauthorized inbound
|
||||
4. **Roles within a space (sharing).** Resolved in `shared-spaces.md`.
|
||||
`offline_sync_space_members.role` is enforced for writes through the closed
|
||||
`OfflineSyncSpaceRole` enum: the server rejects and records unauthorized inbound
|
||||
writes, the client rejects local writes from projected `readOnly` roles, and
|
||||
reads remain membership-wide. The remaining cleanup question for demoted or
|
||||
revoked unsynced local changes is part of question 3.
|
||||
|
|
|
|||
649
docs/shared-spaces.md
Normal file
649
docs/shared-spaces.md
Normal file
|
|
@ -0,0 +1,649 @@
|
|||
# Shared spaces
|
||||
|
||||
Status: implemented. This records the shared-space behavior built on top of
|
||||
the row-ownership work in `row-ownership.md`. Revocation cleanup is still a
|
||||
deferred follow-up; where the ownership summary and this document differ, this
|
||||
one wins.
|
||||
|
||||
## Summary
|
||||
|
||||
A user is an authentication identity; a space is the unit of replication and
|
||||
ownership. Each user has a personal space and may also belong to any number of
|
||||
shared spaces. The load-bearing
|
||||
requirement is:
|
||||
|
||||
> **A single `sync` call syncs every space the authenticated user has access
|
||||
> to.** One `syncOnce` cycles through all accessible spaces exactly once; a
|
||||
> continuous session keeps cycling forever. There is no separate stream or
|
||||
> separate call per space.
|
||||
|
||||
The row-ownership work keys storage, locking, HLC chains, visibility, and
|
||||
ownership by **space**, not user, so sharing is an indirection in front of
|
||||
`offline_sync_spaces`, not a re-architecture. The implemented pieces are a server-side
|
||||
membership relation and a sync protocol that iterates spaces.
|
||||
|
||||
> **Merges serialize one space at a time.** A merge runs in exactly one space's
|
||||
> transaction and lock, so spaces always serialize on the merge step regardless
|
||||
> of framing. A cycle's outbound changes are instead collected in a single pass
|
||||
> over the active spaces and carried in one combined batch, each change tagged
|
||||
> with its `uuidSpaceId`; the receiver regroups by space and merges each group
|
||||
> in its own transaction. The wire stays a near-verbatim repetition of today's
|
||||
> frames, now space-tagged.
|
||||
|
||||
> **The server is the authority on membership; the client follows.** The set of
|
||||
> spaces a session syncs is computed server-side from `offline_sync_space_members` and
|
||||
> re-resolved at the start of every sync cycle, so access changes take effect
|
||||
> mid-session. The server dictates the set; the client adopts spaces it
|
||||
> announces and stops cycling the ones it omits. The reconciliation is
|
||||
> directional — not a set operation toggled by a flag — and membership itself is
|
||||
> never synced. Local data cleanup for omitted spaces is a separate revocation
|
||||
> policy, not part of the sync loop.
|
||||
|
||||
> **Row ownership is unchanged.** Every row still has exactly one owning
|
||||
> `spaceId`, `(table, uuidRowId)` stays globally identified, and every
|
||||
> enforcement rule in `row-ownership.md` survives untouched. Sharing adds a
|
||||
> membership layer in front of spaces; it does not touch row ownership.
|
||||
|
||||
## Goals and constraints
|
||||
|
||||
1. **One call, every space.** The public client API (`syncOnce`,
|
||||
`syncContinuously`) does not change shape: callers pass a session, never a
|
||||
space. A device with only a personal space behaves exactly as today.
|
||||
2. **Smallest protocol delta.** The exchanged frames are today's, plus a
|
||||
per-cycle space-set exchange and a space tag on each frame. `collectNextBatch`
|
||||
demultiplexes a cycle's combined batch, `collectPendingChanges` collects every
|
||||
active space in one pass, and the per-space merge plus `recordSyncCheckpoint`
|
||||
run once per inbound space group.
|
||||
3. **Chunked streaming.** Outbound changes are collected in a single pass over
|
||||
the active spaces and streamed in `syncBatchSize` chunks, so the wire payload
|
||||
stays bounded regardless of space count. The collection step itself
|
||||
materializes a cycle's pending rows across the active spaces (one
|
||||
`spaceId IN (…)` query per change kind), so peak memory tracks a cycle's
|
||||
pending rows rather than being held strictly one space at a time. See
|
||||
`outbound-collection-consistency.md`.
|
||||
4. **Membership cannot be forged.** The server computes its own space set from
|
||||
`offline_sync_space_members` (plus the implicit personal space) and never widens it
|
||||
from anything the client sends. A space the user is not a member of is never
|
||||
synced, even if the client names it.
|
||||
5. **One space per transaction.** Reads are membership-wide; writes stay pinned
|
||||
to exactly one space. This is deliberate honesty about CRDT semantics: two
|
||||
spaces' chains replicate independently and a remote replica can never
|
||||
observe a cross-space write atomically.
|
||||
6. **Roles are closed CRDT access roles.** The package stores and projects
|
||||
non-null `OfflineSyncSpaceRole` values in space grants and shared memberships:
|
||||
`readWrite` allows CRDT writes and `readOnly` blocks them. The implicit
|
||||
personal space is announced as a `readWrite` grant, so single-tenant servers
|
||||
and multi-account devices remain the same code path with different space
|
||||
counts.
|
||||
|
||||
## Membership
|
||||
|
||||
### The `offline_sync_space_members` table
|
||||
|
||||
```yaml
|
||||
class: OfflineSyncSpaceMember
|
||||
table: offline_sync_space_members
|
||||
database: all
|
||||
fields:
|
||||
space: OfflineSyncSpace?, relation(onDelete=Cascade)
|
||||
userUuid: UuidValue
|
||||
role: OfflineSyncSpaceRole
|
||||
indexes:
|
||||
offline_sync_space_member_unique_idx:
|
||||
fields: userUuid, spaceId
|
||||
unique: true
|
||||
```
|
||||
|
||||
- The table is **`database: all`** but **not** in `syncTables`. It exists on
|
||||
every node so membership resolves with the same code on both ends, but it is
|
||||
never CRDT-replicated by this package. The server is the authoritative writer;
|
||||
a client copy (once populated) is a read-only cache for UI and offline role
|
||||
checks — see *The client membership table*.
|
||||
- The index **leads with `userUuid`** so the per-cycle `WHERE userUuid = ?`
|
||||
resolution is index-covered.
|
||||
- Rows are managed through `session.offlineSync.spaces` (`OfflineSyncSpaces`). Invitations,
|
||||
acceptance, and authorization policy remain app domain.
|
||||
- **Personal-space membership is implicit.** By convention a user's personal
|
||||
space has the user's own UUID (already true: the space UUID keys the chain),
|
||||
so no `offline_sync_space_members` row is stored for it. Shared spaces are
|
||||
`offline_sync_spaces` rows whose UUID is no user's id, with explicit membership rows.
|
||||
- `role` is a required CRDT access enum. Shared memberships always store an
|
||||
explicit role: `readWrite` allows writes, while `readOnly` denies
|
||||
shared-space writes.
|
||||
|
||||
### Members vs. nodes
|
||||
|
||||
`offline_sync_space_members` and the `nodes` list on a space are orthogonal layers and
|
||||
both stay:
|
||||
|
||||
- A **member** is an authorization fact: a user identity allowed to access a
|
||||
space. It answers *who may sync, read, and write*.
|
||||
- A **node** (`crdt_nodes`) is a stable CRDT replica identity. A space's
|
||||
`nodes` relation points at `offline_sync_space_nodes`, which records that replica's
|
||||
participation and checkpoint state in that space. Together they answer
|
||||
*whose changes, up to which HLC* and are what per-`(space, node)` checkpoints
|
||||
key on.
|
||||
|
||||
They do not map one-to-one and neither replaces the other. One member may have
|
||||
many nodes (one per replica/database install). The **server is a node** in every
|
||||
space it syncs but is not a member row. A freshly invited member has **zero
|
||||
space-node rows** until their first device syncs. Nodes and space-node rows are
|
||||
created implicitly by sync (`getOrCreate`, `recordSyncCheckpoint`); membership
|
||||
rows are managed through `session.offlineSync.spaces` from application endpoints.
|
||||
Removing the `nodes` list would break checkpointing and causal filtering — it is
|
||||
the chain topology, not the access list.
|
||||
|
||||
### The client membership table
|
||||
|
||||
`offline_sync_space_members` is `database: all`, so the table exists on the client and
|
||||
is **populated as a read-only projection** of the server's authoritative grants
|
||||
— without being bidirectionally CRDT-synced:
|
||||
|
||||
- The client's projected `offline_sync_space_members` rows plus the implicit personal
|
||||
space are its membership view for *membership gating*: a membership-wide read
|
||||
filters over projected membership, and a local `offline_sync_spaces` row by itself
|
||||
does not grant access. The members table also carries **roles**, which the
|
||||
client consults before local writes.
|
||||
- The authoritative `OfflineSyncSpaceSet` carries `OfflineSyncSpaceGrant`s — `(uuidSpaceId,
|
||||
role)` — not bare UUIDs. Every grant has an explicit role; the personal space
|
||||
is announced as `readWrite` but remains implicit in storage. When the follower
|
||||
*receives* an announcement, `projectFollowerMembership` reconciles the local
|
||||
table from it: upsert a row per shared grant (the personal space is skipped),
|
||||
delete rows whose space is no longer granted. Since the authoritative peer
|
||||
announces only when its grants change, projection runs only on change and an
|
||||
idle session writes nothing.
|
||||
- This is a **server-authored, client-read-only projection**, never a synced
|
||||
members table: the client never writes membership the server reads, so it
|
||||
cannot self-grant. The server stays the security boundary and re-verifies on
|
||||
sync.
|
||||
- A revoke takes effect for sync as soon as the server omits the space from a
|
||||
cycle's `SpaceSet`: the client stops cycling it and discards in-memory
|
||||
checkpoint state for it. The space row and domain rows may remain stored
|
||||
locally until the deferred revocation cleanup policy decides whether to purge
|
||||
or keep them, but membership-wide reads stop returning those rows.
|
||||
- Writes are bounded by projected membership and by the projected role. They are
|
||||
re-verified by the server on sync, which remains the security boundary.
|
||||
Client-side checks are early-failure UX, not the enforcement point.
|
||||
|
||||
### Role write access
|
||||
|
||||
The package exposes a closed CRDT access role enum rather than free-form role
|
||||
strings:
|
||||
|
||||
```dart
|
||||
enum OfflineSyncSpaceRole {
|
||||
readOnly,
|
||||
readWrite,
|
||||
}
|
||||
```
|
||||
|
||||
Only `readWrite` allows writes. `readOnly` denies writes. The implicit personal
|
||||
space has no stored membership row, but its authoritative grant and role lookup
|
||||
resolve to `readWrite`.
|
||||
|
||||
Roles gate **writes only**. Reads stay membership-wide and role-independent.
|
||||
The authoritative server enforces role writes before applying inbound changes:
|
||||
for each accepted inbound space group, it resolves the authenticated user's
|
||||
role from `offline_sync_space_members`; if the role is not `readWrite`, the server
|
||||
records `OfflineSyncIntegrityViolation(type: unauthorizedWrite)` and fails the
|
||||
sync session before opening the merge transaction for that space. The rejected
|
||||
change is not applied and checkpoints do not advance past it.
|
||||
|
||||
The client uses the same enum as early UX: `transactionForUser(userId, fn,
|
||||
spaceId: sharedSpace)` resolves the projected role and throws
|
||||
`OfflineSyncSpaceRoleException` before any local write when the role cannot write. A
|
||||
follower also omits non-writable shared spaces from outbound pending-change
|
||||
collection while still receiving inbound changes for those spaces. This avoids
|
||||
repeatedly streaming local read-only changes; the server enforcement remains the
|
||||
security boundary for stale, misconfigured, or hostile clients.
|
||||
|
||||
### Space enumeration (server is the authority)
|
||||
|
||||
At the start of each sync cycle the server resolves the authoritative set:
|
||||
|
||||
```
|
||||
memberGrants(userUuid) =
|
||||
{ personal space = userUuid, role = readWrite }
|
||||
∪ { (s.uuidSpaceId, m.role)
|
||||
: offline_sync_space_members m
|
||||
join offline_sync_spaces s on s.id = m.spaceId
|
||||
where m.userUuid = <auth user> }
|
||||
```
|
||||
|
||||
This grant set bounds everything the cycle may sync and carries the role used by
|
||||
followers for outbound filtering. The client never expands it; it only learns of
|
||||
additions through what the server announces.
|
||||
|
||||
## Sync: one call syncs every accessible space
|
||||
|
||||
### One `sync`, two cadences: bounded `once`, idle-silent `continuous`
|
||||
|
||||
Merge is inherently one space at a time: a merge runs in exactly one space's
|
||||
transaction and lock, so the spaces always serialize regardless of framing. A
|
||||
single `sync` method serves both `syncOnce` and `syncContinuously`, branching on
|
||||
`once` only where they genuinely differ — read discipline and termination — so
|
||||
there is no duplicated send/merge logic:
|
||||
|
||||
1. **Establishment (shared, lockstep).** After `Connect`, both peers exchange the
|
||||
`SpaceSet`. Doing this *before* the loop is what removes the announce/adopt
|
||||
offset that would otherwise make a follower act before it knows the agreed
|
||||
set.
|
||||
2. **Data loop (shared, idle-silent).** Each cycle is **one combined batch**: the
|
||||
space announcement (only when this peer's grants changed), a `SinceHlc` for
|
||||
each newly active space, and merge chunks whose changes carry their space,
|
||||
all closed by a single `EndOfBatch`. Continuous mode omits the terminator when
|
||||
nothing was sent. The receive phase demultiplexes the peer's batch by type
|
||||
until that terminator or — when the peer was idle — an idle timeout. An idle
|
||||
continuous cycle therefore sends nothing and idles out **once**, regardless of
|
||||
space count.
|
||||
|
||||
`once` stays in the data loop until every active space has exchanged `SinceHlc`
|
||||
and had one sendable data cycle, then performs the symmetric `Close` handshake.
|
||||
`continuous` loops instead, with no per-cycle chatter; it handshakes its initial
|
||||
spaces in the first cycle and absorbs membership changes as they are announced
|
||||
(a newly granted space is announced, adopted, and established over the following
|
||||
cycles — its data deferred until its `SinceHlc` round-trips).
|
||||
|
||||
### Protocol frames
|
||||
|
||||
`Connect` opens the session once. After that the frames are the same in both
|
||||
cadences; only *when* they are sent differs (`once`: a fixed sequence per space;
|
||||
`continuous`: only on change, inside one combined batch). Frame changes
|
||||
(regenerated models + a migration on both client and server schemas):
|
||||
|
||||
- `OfflineSyncConnect { syncTablesHash, localNodeId }` — sent once by each peer. The
|
||||
schema hash is validated once, before any space work. `localNodeId` identifies
|
||||
this peer's CRDT node for the whole session. The space list is dynamic and
|
||||
exchanged per cycle.
|
||||
- `OfflineSyncSpaceSet { spaces: List<OfflineSyncSpaceGrant> }` (each grant a
|
||||
`uuidSpaceId` + `role`) — the server's content is authoritative (resolved from
|
||||
`offline_sync_space_members`, roles included). Followers send an empty set because the
|
||||
authoritative peer never widens access from follower-reported state. In `once`
|
||||
it is exchanged exactly once. In `continuous` a peer re-announces it
|
||||
**only when its own grant set changed** since the last announcement — this is
|
||||
the frame that carries access changes mid-session, and gating it on change is
|
||||
what keeps an idle session silent while still letting a grant or revoke take
|
||||
effect.
|
||||
- `OfflineSyncSinceHlc { uuidSpaceId, nodeCheckpoints }` — per space. Carries that
|
||||
space's per-node checkpoints. Exchanged when a space first becomes active
|
||||
(and again if a dropped space is later re-adopted);
|
||||
checkpoints then live in memory for the rest of the session.
|
||||
- `OfflineSyncMergeChunk { changes }` — each change carries `uuidSpaceId`, letting
|
||||
a combined batch carry every space's changes and regroup them on receive.
|
||||
- `OfflineSyncEndOfBatch`, `OfflineSyncClose`, `OfflineSyncIdleTimeout` — unchanged. A
|
||||
single `EndOfBatch` terminates a cycle's combined batch (and is itself omitted
|
||||
when nothing was sent, so an idle peer resolves to an empty batch instead).
|
||||
`Close` ends the streaming session. Because it is an intentional peer control
|
||||
frame, it may arrive even after frames for a partial cycle; the receiver
|
||||
discards that partial cycle and records no checkpoint progress for it.
|
||||
|
||||
The data cycle's receive uses `collectNextBatch`, which demultiplexes one
|
||||
cycle's frames by type into `{ spaceSet?, sinceHlcs, changes }`,
|
||||
returning on the single `EndOfBatch`, on `Close` with a closed result, or —
|
||||
when the peer was idle — on the idle timeout. A raw transport close without a
|
||||
`Close` frame is stricter: it is graceful only between continuous-session
|
||||
batches and is treated as truncation once a partial batch has arrived.
|
||||
|
||||
### The session state machine
|
||||
|
||||
```
|
||||
handshake (both cadences):
|
||||
send Connect(syncTablesHash, localNodeId); read peer Connect; validate
|
||||
syncTablesHash; store peer localNodeId
|
||||
|
||||
establishment (both cadences, lockstep):
|
||||
send SpaceSet(myGrants); read peer SpaceSet → adopt → activeSpaces
|
||||
follower: materialize activeSpaces, project membership
|
||||
data loop (both cadences, one combined batch per cycle):
|
||||
loop:
|
||||
resolve activeSpaces (authoritative: my set; follower: adopted peerGrants)
|
||||
discard state for spaces no longer active
|
||||
# send phase — emit only what changed; track whether any frame was sent
|
||||
if myGrants != announcedGrants: send SpaceSet(myGrants)
|
||||
for space in activeSpaces not yet sinceHlcSent: send SinceHlc(space)
|
||||
if peer checkpoints exist: send MergeChunk(changes)* # follower skips non-writable spaces
|
||||
if anything was sent (or once): send EndOfBatch
|
||||
# receive phase
|
||||
batch = collectNextBatch(allowCloseBeforeBatch: !once) # {spaceSet?, sinceHlcs, changes} | idle | closed
|
||||
if closed: return
|
||||
adopt batch.spaceSet (follower: materialize + project); store sinceHlcs;
|
||||
group changes by uuidSpaceId; mergeInboundBatch each group
|
||||
if once: send Close; read Close; drain; return
|
||||
delay continuousSyncInterval
|
||||
```
|
||||
|
||||
The shared establishment exchanges the space set so a follower never acts before
|
||||
it knows the agreed set. Both cadences handshake initial and newly added spaces
|
||||
through the loop's own `SinceHlc` exchange.
|
||||
|
||||
`collectPendingChanges`, `mergeInboundBatch`, `recordSyncCheckpoint`, and the
|
||||
ownership-violation internals (`_streamInserts`/`Updates`/`Deletes`,
|
||||
`_fetchDomainRow`, the durable-violation path) are space-parameterized and merge
|
||||
each inbound space group in its own `transactionForUser`/lock; any space's merge
|
||||
failure (for example an ownership collision or unauthorized role write) records
|
||||
the durable violation and fails the session, preserving today's fail-fast
|
||||
semantics.
|
||||
|
||||
Because `continuous` defers a space's data until both peers have exchanged its
|
||||
`SinceHlc`, a newly active space is established over the **following** cycles
|
||||
rather than in lockstep — the deliberate trade for keeping the steady state
|
||||
silent. A peer only honors inbound `SinceHlc`/`MergeChunk` frames for spaces it
|
||||
authorizes (authoritative: its own membership; follower: the announced set), so
|
||||
the combined batch tolerates a peer's informational extras without widening
|
||||
access.
|
||||
|
||||
`onMergeSuccess` becomes space-aware: `Function(UuidValue spaceUuid, Hlc hlc)`.
|
||||
This touches three test call sites and the example app's `demo_controller`.
|
||||
|
||||
### Space-set reconciliation and lockstep
|
||||
|
||||
Both peers must cycle the **same ordered space set** or send/receive desyncs.
|
||||
Membership is asymmetric — the client trusts the server's membership claims, the
|
||||
server trusts none of the client's — so the reconciliation is **directional**,
|
||||
not a symmetric set operation:
|
||||
|
||||
- **Union is wrong** (insecure): a client could name a space the user is not a
|
||||
member of and force the server to sync it.
|
||||
- **Intersection is wrong** (cannot adopt): a brand-new share is not in the
|
||||
client's set yet, so it would never enter the intersection and never sync —
|
||||
the client cannot hold what it has not synced, and cannot sync what it does
|
||||
not hold.
|
||||
|
||||
The correct operation is neither: **the server dictates its set; the client
|
||||
conforms.** Each cycle:
|
||||
|
||||
- **Server**: `orderedSpaces = sort(memberGrants(userUuid).uuidSpaceId)`,
|
||||
computed from `offline_sync_space_members` plus the implicit personal grant. It never
|
||||
reads the client's `SpaceSet` for authority (constraint 4).
|
||||
- **Client**: takes the server's `SpaceSet` as the set to cycle —
|
||||
`getOrCreate` (`OfflineSyncSpaceManager`) any space it lacks, ignore any local space
|
||||
the server omitted for this cycle — then `orderedSpaces = sort(peer spaceIds)`.
|
||||
|
||||
This asymmetry is expressed by a **`OfflineSyncPeerMode` enum**
|
||||
(`authoritative` / `follower`) passed at the call site: the server endpoint
|
||||
passes `authoritative`, the client driver passes `follower`. Membership is
|
||||
resolved from the same shared `offline_sync_space_members` table (via
|
||||
`OfflineSyncSpaceMembership`); the mode varies only the cycle *direction* —
|
||||
authoritative dictates its own set, follower adopts the peer's.
|
||||
Once the data is unified, a closed enum is the right shape: there is no
|
||||
per-role enumeration callback left to inject. Security does not rest on the
|
||||
enum — a malicious client passing `authoritative` only fails to adopt the
|
||||
peer's set; writes are still gated server-side by
|
||||
`OfflineSyncSpaceMembership.roleOf(...).canWrite` against the authoritative table.
|
||||
|
||||
In `once`, the client adopts straight from the cycle's `SpaceSet` *before* it
|
||||
iterates, so a newly granted space syncs in that same round. In `continuous`,
|
||||
adoption happens when the announcement arrives and the space's data follows over
|
||||
the next cycles once its `SinceHlc` round-trips. Sorting by space UUID makes the
|
||||
order deterministic on both sides with no extra negotiation. Frames carry
|
||||
`uuidSpaceId` so a receiver groups each chunk into the right space and fails loud
|
||||
on an unauthorized tag rather than merging a batch into the wrong space.
|
||||
|
||||
Re-resolving `memberGrants` every cycle is an indexed `offline_sync_space_members`
|
||||
lookup on the server only; it may be throttled or invalidated by an
|
||||
application hook if per-cycle querying becomes hot, at the cost of slower
|
||||
propagation of access changes.
|
||||
|
||||
### Checkpoints, once vs. continuous
|
||||
|
||||
Checkpoint rows stay per `(space, node)` in `offline_sync_space_nodes`, while
|
||||
`crdt_nodes` stores the stable replica identity. The same local replica node can
|
||||
participate in multiple spaces, with separate `lastReceivedHlc` values per
|
||||
space. `SinceHlc` is exchanged once per space on first visit; later cycles reuse
|
||||
the in-memory
|
||||
`nodeCheckpoints[space]`, advanced as chunks are sent and as inbound batches
|
||||
merge — the multi-space analogue of today's single-space continuous loop, which
|
||||
already keeps checkpoints in memory across rounds.
|
||||
|
||||
The continuous interval delay moves to **after each full cycle**, not after each
|
||||
space, to keep per-space latency low while still preventing a busy loop when
|
||||
idle.
|
||||
|
||||
### Access changes during a session
|
||||
|
||||
Because the authoritative space set is re-resolved every cycle and re-announced
|
||||
whenever it changes (not pinned at `Connect`), a continuous session absorbs
|
||||
membership changes without reconnecting:
|
||||
|
||||
- **Grant.** The server re-announces a `SpaceSet` that includes the new space;
|
||||
the client adopts and materializes it, then handshakes and syncs it over the
|
||||
next cycles. (The server detects the grant on its next cycle, so propagation
|
||||
is bounded by one idle poll plus the handshake round-trip.)
|
||||
- **Revoke.** The server re-announces a `SpaceSet` that omits the space; both
|
||||
sides stop cycling it and discard its in-memory checkpoint state, and the
|
||||
follower's `projectFollowerMembership` deletes its members-table row. The
|
||||
device can still store the revoked space's rows locally, but membership-wide
|
||||
reads no longer return them. Purging stored rows (and the fate of any unsynced
|
||||
local changes) is the deferred revocation question, not a sync-loop concern.
|
||||
- **Demote.** The server re-announces the same space with a different role; the
|
||||
follower projection updates the cached role. The next local write consults
|
||||
the new role and fails early if it is read-only. Pending local changes already
|
||||
authored before demotion are not silently applied by the server: if streamed,
|
||||
they are rejected and recorded as `unauthorizedWrite`. Whether the client
|
||||
purges, surfaces, or keeps skipped pending changes is part of the deferred
|
||||
revocation-cleanup design.
|
||||
|
||||
The authoritative peer originates every set change, so both sides converge on
|
||||
the same active set from the same announcement and stay in step across it. A
|
||||
one-shot `syncOnce` captures the set once; the next `syncOnce` call picks up any
|
||||
change.
|
||||
|
||||
## Transaction API
|
||||
|
||||
`transactionForUser` keeps its name and user-first semantics and gains an
|
||||
optional space:
|
||||
|
||||
```dart
|
||||
db.transactionForUser(userId, fn); // acts in the personal space
|
||||
db.transactionForUser(userId, fn, spaceId: listId); // acts in a shared space
|
||||
```
|
||||
|
||||
- Without `spaceId`, the space resolves to the user's personal space —
|
||||
`userId` itself by convention — which is exactly today's behavior.
|
||||
- With `spaceId`, the package resolves the member role with
|
||||
`OfflineSyncSpaceMembership.roleOf` — no injected validator. On the server that table
|
||||
is authoritative; on a persistent client it is the server-projected membership
|
||||
cache. A missing role throws `OfflineSyncSpaceMembershipException`; anything other
|
||||
than `readWrite` throws `OfflineSyncSpaceRoleException` before the transaction starts.
|
||||
- A transaction acts in **exactly one** space (constraint 5). The write path —
|
||||
stamp-or-assert, the space-scoped `WHERE`, `spaceId` immutability — is unchanged
|
||||
beyond which space is resolved.
|
||||
|
||||
## Read path: membership-wide
|
||||
|
||||
A user-space-scoped read in a sharing world means "rows in any space I am a member
|
||||
of". The single-space resolver becomes a set:
|
||||
|
||||
- `mergeWhereWithTombstone` isolates space-scoped reads with `spaceId IN (<member
|
||||
space ids>)` through `OfflineSyncSpaceIdsResolver`. For writes, the resolver supplies
|
||||
the single acting space. For reads, both server sessions and persistent
|
||||
clients resolve the set from `OfflineSyncSpaceMembership.memberSpaces`
|
||||
(authoritative on the server, projected on the client); unscoped admin reads
|
||||
still pass no space filter.
|
||||
- The visibility probe is the row-keyed form for the multi-space case, so
|
||||
sharing simplifies the predicate matrix rather than growing it: reads differ
|
||||
only in their membership filter (one space, a user's spaces, or none for
|
||||
admin).
|
||||
- This is the shape of a Postgres row-level-security policy
|
||||
(`spaceId IN (SELECT spaceId FROM offline_sync_space_members WHERE userUuid = …)`); the
|
||||
package's application-level filter and RLS become two enforcements of one
|
||||
predicate.
|
||||
|
||||
Writes stay space-pinned (above); only reads widen.
|
||||
|
||||
## What sharing does not change
|
||||
|
||||
Every enforcement rule in `row-ownership.md` survives unchanged:
|
||||
`(table, uuidRowId)` stays globally identified; rows have exactly one owning
|
||||
space; merge-insert ownership checks compare spaces; unique conflict groups form
|
||||
within a space; ordinary global unique indexes stay rejected; stamp-or-assert
|
||||
applies with the transaction's resolved space; the durable
|
||||
`offline_sync_integrity_violations` contract now also records unauthorized role
|
||||
writes. The merge, FK, and unique engines already key on the space the merge
|
||||
runs in and need no ownership changes for sharing.
|
||||
|
||||
## Implementation status
|
||||
|
||||
The implementation landed in independently green phases. Tests should continue
|
||||
to run with `--concurrency=1`.
|
||||
|
||||
- **Phase 1 — membership foundation.** Implemented `offline_sync_space_members`
|
||||
(`database: all`, unsynced) and its migration. Added the shared
|
||||
`OfflineSyncSpaceMembership` helpers (`memberGrants`, `memberSpaces`, `roleOf`,
|
||||
`isMember`, and the follower projection path). No behavior change yet: a
|
||||
personal-space-only set reproduces today's single-space sync exactly.
|
||||
- **Phase 2 — sequential multi-space sync (the requirement).** Implemented
|
||||
protocol model
|
||||
changes (`Connect` with session `localNodeId`, new per-cycle
|
||||
`OfflineSyncSpaceSet`, per-space `SinceHlc`, `MergeChunk.uuidSpaceId`) plus
|
||||
generate/migrate. Outer cycle loop in `OfflineSyncEngine.sync` with per-cycle
|
||||
`SpaceSet` exchange with role-carrying `OfflineSyncSpaceGrant`s, per-space
|
||||
handshake-on-first-visit, in-memory per-space checkpoints, and the
|
||||
once/continuous cadence. Reconciliation via a
|
||||
`OfflineSyncPeerMode` enum: the server endpoint passes `authoritative` (cycles
|
||||
`OfflineSyncSpaceMembership.memberGrants`); the client driver passes `follower`
|
||||
(adopts the server's set). Space-aware `onMergeSuccess`. A personal-space-only
|
||||
device behaves identically to today.
|
||||
- **Phase 3 — transaction API.** Implemented
|
||||
`transactionForUser(userId, fn, {spaceId})` with the membership assertion;
|
||||
one space per transaction.
|
||||
- **Phase 4 — membership-wide reads.** Implemented the space-scoped read filter as
|
||||
`spaceId IN (…)` over the user's member spaces.
|
||||
- **Phase 5 — lifecycle and docs.** Documented client adoption of newly
|
||||
announced shares; reconcile with the space-purge semantics in
|
||||
`sync-non-sync-relations.md`.
|
||||
Revocation cleanup is deferred (see *Follow-ups*).
|
||||
- **Phase 6 — idle-silent sync, one method.** Reworked the single `sync` method
|
||||
(no separate once/continuous bodies) into a shared lockstep establishment plus
|
||||
a shared combined data loop: each cycle coalesces into one space-scoped-change batch
|
||||
terminated by a single `EndOfBatch`, with `SpaceSet`/`SinceHlc`/`MergeChunk`
|
||||
sent only on change and read via `collectNextBatch`. An idle continuous
|
||||
session now sends zero frames and idles out once per cycle regardless of space
|
||||
count (Phase 2's loop re-announced the set and an `EndOfBatch` per space every
|
||||
tick). `once` runs one data cycle then closes; mid-session continuous spaces are
|
||||
deferred until their `SinceHlc` round-trips, and inbound frames are honored
|
||||
only for authorized spaces.
|
||||
- **Phase 7 — roles within a space.** Implemented the closed
|
||||
`OfflineSyncSpaceRole` enum (`readOnly`, `readWrite`) on server and client.
|
||||
`readWrite` is the only shared-space write grant. Added authoritative inbound
|
||||
enforcement that records
|
||||
`unauthorizedWrite` and fails before merge, client-side
|
||||
`OfflineSyncSpaceRoleException` early rejection, and follower outbound skipping for
|
||||
non-writable shared spaces while reads remain membership-wide.
|
||||
- **Phase 8 — space management service.** Implemented the server-side
|
||||
`session.offlineSync.spaces` service (`create`, `createFor`, `grant`, `grantAll`,
|
||||
`revoke`, `members`) with transaction/savepoint threading. Single grants reuse
|
||||
the bulk-grant path; unknown-space grants throw `OfflineSyncSpaceNotFoundException`.
|
||||
Invitations, acceptance, and authorization policy stay in app endpoints.
|
||||
|
||||
## Test plan
|
||||
|
||||
Because no code branches on space count or deployment role, mechanisms are
|
||||
tested once, not per combination.
|
||||
|
||||
- **Regression.** Every existing single-space suite passes unchanged on the
|
||||
personal-space-only path.
|
||||
- **Protocol.** `Connect`/`SinceHlc` round-trip a multi-space list; merge chunks
|
||||
route to the correct space by `uuidSpaceId`; per-space checkpoints advance
|
||||
independently across cycles.
|
||||
- **One call, every space.** A single `syncOnce` converges N spaces in one pass;
|
||||
a continuous session keeps cycling. Outbound changes for the active spaces are
|
||||
collected in one pass and streamed in `syncBatchSize` chunks; the receive
|
||||
groups a cycle's inbound chunks by space before merging each in its own
|
||||
transaction.
|
||||
- **Idle silence.** A continuous session with no pending changes settles and then
|
||||
emits no further frames — neither peer re-announces the space set nor sends an
|
||||
`EndOfBatch` while idle.
|
||||
- **Membership.** The server syncs personal + shared spaces; a client-announced
|
||||
non-member space is refused and never streamed; a newly granted space is
|
||||
adopted and syncs within the same `syncOnce` call.
|
||||
- **Roles.** `readOnly` can still read shared rows, local writes fail before
|
||||
mutation, `readWrite` syncs normally, a follower skips outbound pending changes
|
||||
for non-writable spaces while still receiving inbound changes, and a stale or
|
||||
hostile client write is rejected and recorded by the server as
|
||||
`unauthorizedWrite`. Personal spaces remain writable through their implicit
|
||||
`readWrite` grant.
|
||||
- **Access changes mid-session.** During a continuous session, a grant added to
|
||||
`offline_sync_space_members` appears and syncs on the next cycle; a revoke stops
|
||||
cycling that space without dropping the session.
|
||||
- **Isolation under sharing.** Two users sharing space X both converge on X
|
||||
while their personal spaces stay isolated; an ownership collision in one space
|
||||
records a durable violation and fails the session without corrupting another
|
||||
space's merge.
|
||||
- **Smoke.** One `withServerpod` round-trip with one personal plus one shared
|
||||
space, proving the end-to-end wiring (mirrors `sync_flow_test.dart`).
|
||||
|
||||
## Resolved during review
|
||||
|
||||
- **Combined sync batch:** merges serialize regardless, so combining spaces is
|
||||
not a throughput optimization. Both cadences use the same combined batch shape:
|
||||
each change carries its `uuidSpaceId`, and receive groups changes by space
|
||||
before merging each space group. Continuous mode omits idle frames so idle
|
||||
latency is constant in space count.
|
||||
- **Membership authority:** the server, from `offline_sync_space_members`. The
|
||||
reconciliation is directional — server dictates, client conforms — and is
|
||||
neither union (insecure) nor intersection (cannot adopt). It is a
|
||||
**`OfflineSyncPeerMode` enum** (`authoritative` / `follower`): membership still
|
||||
resolves through `OfflineSyncSpaceMembership`, so the only residual variation is
|
||||
cycle direction — a closed enum, not a callback. The enum is not the security
|
||||
boundary; server-side
|
||||
`OfflineSyncSpaceMembership.roleOf(...).canWrite` gates every applied write
|
||||
regardless of the client's declared mode.
|
||||
- **Space set is dynamic, not pinned at connect:** the authoritative peer
|
||||
re-resolves its set every cycle and re-announces a `SpaceSet` whenever it
|
||||
changed, so grants and revokes take effect mid-session without reconnecting.
|
||||
In `syncOnce` the client adopts from the initial `SpaceSet`, exchanges
|
||||
`SinceHlc`, then runs the sendable data cycle; in `syncContinuously` a newly
|
||||
announced space is adopted then established over the next cycles as its
|
||||
`SinceHlc` round-trips.
|
||||
- **Members vs. nodes:** orthogonal layers, both kept. Members are authorization
|
||||
(`who`); nodes are CRDT causality (`whose changes, up to which HLC`). One
|
||||
member maps to many nodes; the server is a node but not a member; a new member
|
||||
has no node until first sync.
|
||||
- **Client membership table:** `database: all` puts the table on the client,
|
||||
populated as a read-only projection. The `OfflineSyncSpaceSet` carries
|
||||
`OfflineSyncSpaceGrant`s (`uuidSpaceId` + `role`), and each follower cycle reconciles
|
||||
the local table from the announcement (upsert granted, delete revoked; only
|
||||
when the set changed). The projected members table drives shared-space
|
||||
membership gating and roles for UI and offline role-gated writes; local
|
||||
`offline_sync_spaces` rows alone do not grant access. It is a server-authored,
|
||||
client-read-only projection — never a
|
||||
bidirectionally-synced members table, so a client cannot self-grant.
|
||||
- **Iteration order:** sorted space UUIDs, so both peers stay in lockstep
|
||||
without negotiation. Each merge change carries its space UUID, so receive can
|
||||
regroup a combined batch deterministically.
|
||||
- **One space per transaction:** kept. Cross-space writes need separate
|
||||
transactions; remote replicas cannot observe a cross-space write atomically
|
||||
anyway.
|
||||
- **Membership table is `database: all` but unsynced:** the schema exists on
|
||||
every node, yet it is server-authoritative app state, never CRDT-replicated.
|
||||
- **Idle chatter removed:** continuous sync has one combined data loop per
|
||||
cycle. When neither peer has grants, handshakes, or changes to send, it emits
|
||||
no `SpaceSet`, `SinceHlc`, `MergeChunk`, or `EndOfBatch`; `collectNextBatch`
|
||||
resolves the idle cycle from the timeout once per session cycle, not once per
|
||||
space.
|
||||
- **Initial-sync ordering is not a current concern:** a device joining many
|
||||
spaces cycles them one at a time on first sync. Prioritizing recently active
|
||||
spaces in the iteration order is a compatible refinement if cold-start latency
|
||||
over large space counts ever matters, but it is not a present worry.
|
||||
|
||||
## Follow-ups
|
||||
|
||||
Deferred work needing its own design pass. It does not block the implemented
|
||||
role enforcement above.
|
||||
|
||||
1. **Membership revocation.** When a user loses membership, the server omits the
|
||||
space from the next cycle's `SpaceSet` and both sides stop syncing it
|
||||
promptly. The device may still store the space's data and unsynced local
|
||||
changes, though membership-wide reads no longer expose revoked rows.
|
||||
Purge-vs-keep policy, and whether revoked or demoted unsynced changes are
|
||||
surfaced, dropped, or kept pending locally, need their own design. (Carried
|
||||
from `row-ownership.md` open question 3.)
|
||||
2. **Outbound collection consistency.** Collection reads CRDT metadata and the
|
||||
domain data separately and lock-free, so a concurrent delete — or an FK
|
||||
reference to a row inserted after the insert snapshot — can fail a healthy
|
||||
stream sync and record a durable violation for a race that the next round
|
||||
would resolve. The single-pass multi-space collection widened this window.
|
||||
Failing the session for a transient, self-correcting race is the weakness to
|
||||
fix; the leading direction is a read-only snapshot (MVCC, not a lock). See
|
||||
`outbound-collection-consistency.md`.
|
||||
215
docs/space-management-api.md
Normal file
215
docs/space-management-api.md
Normal file
|
|
@ -0,0 +1,215 @@
|
|||
# Space management API
|
||||
|
||||
Status: implemented. Builds on the shared-space
|
||||
membership model in `shared-spaces.md`; this document specifies the developer-
|
||||
facing API for *managing* membership while leaving invitations, acceptance, and
|
||||
authorization policy in app domain.
|
||||
|
||||
## Summary
|
||||
|
||||
The shared-space machinery is complete for **resolving and enforcing**
|
||||
membership (`OfflineSyncSpaceMembership.memberSpaces` / `memberGrants` / `roleOf` /
|
||||
`isMember`, the sync-loop enforcement, membership-wide reads), but those helpers
|
||||
are package internals. The developer-facing API for **managing** membership is
|
||||
the server-side `session.offlineSync.spaces` service: creating a shared space, adding a
|
||||
member with a role, changing a role, revoking, and listing explicit members.
|
||||
`shared-spaces.md` records the same boundary: rows are managed through
|
||||
`session.offlineSync.spaces`, while invitations, acceptance, and authorization policy
|
||||
remain app domain.
|
||||
|
||||
The service avoids making developers replicate what `space_membership_test.dart`
|
||||
used to do — two non-atomic raw `insertRow` calls against the **server**
|
||||
package's generated `OfflineSyncSpace` / `OfflineSyncSpaceMember`, leaking every internal
|
||||
invariant (`spaceId` vs `uuidSpaceId`, the non-null `role` contract, the unique
|
||||
index, personal-space-UUID == user-UUID):
|
||||
|
||||
```dart
|
||||
final spaceId = await session.offlineSync.spaces.createFor(creatorUuid);
|
||||
await session.offlineSync.spaces.grant(space: spaceId, user: bob, role: OfflineSyncSpaceRole.readWrite);
|
||||
await session.offlineSync.spaces.revoke(space: spaceId, user: bob);
|
||||
```
|
||||
|
||||
## Design decisions
|
||||
|
||||
These were settled in discussion; the reasoning is recorded so the
|
||||
implementation issue does not relitigate them.
|
||||
|
||||
### A service, not an endpoint (the `AuthUsers` model)
|
||||
|
||||
The load-bearing constraint is `shared-spaces.md` #4: *membership cannot be
|
||||
forged; the server never widens access from client-reported state*. If the
|
||||
package auto-generated a management **endpoint**, any logged-in client could
|
||||
self-grant into any space and break that.
|
||||
|
||||
So the package ships a **server-side business service** the developer calls from
|
||||
their **own** endpoint, after applying their **own** authorization policy. This
|
||||
is exactly Serverpod's `AuthUsers`: `AuthUsers.create(session, ...)` is a
|
||||
capability, not a policy — it does not check "may the caller create a user".
|
||||
The sync path enforces roles internally only because it is *directly reachable
|
||||
by untrusted clients*; the management service is server-only (no endpoint), so
|
||||
its only caller is the developer's own server code.
|
||||
|
||||
The package owns the **data invariants**; the app owns the **authorization
|
||||
policy**. That is the same division that makes invitations/acceptance app domain.
|
||||
|
||||
### No authorization check inside the service
|
||||
|
||||
An earlier sketch gated writes with a `by:` (acting-user) parameter checked
|
||||
against an admin role. This was dropped:
|
||||
|
||||
- **`by` is forgeable, so it is not enforcement.** It is a `UuidValue` the
|
||||
caller supplies; the service cannot tell a real identity from a typed-in admin
|
||||
UUID. It prevents a bug, not an attacker — and only if someone upstream binds
|
||||
it to the true identity, which is the endpoint's job.
|
||||
- The unforgeable identity is `session.authenticated` — the same source the sync
|
||||
endpoint already trusts (`offline_sync_endpoint.dart`). It lives at the endpoint,
|
||||
which is where the app's policy lives anyway.
|
||||
- The internal check is what forced a system/backend **bypass**; removing the
|
||||
check dissolves the bypass.
|
||||
|
||||
The service therefore performs **no** internal authorization. Authorization is
|
||||
100% the endpoint's job, where the authenticated caller is unforgeable:
|
||||
|
||||
```dart
|
||||
class TeamEndpoint extends Endpoint {
|
||||
Future<void> invite(Session s, UuidValue space, UuidValue user, OfflineSyncSpaceRole role) async {
|
||||
final caller = UuidValue.withValidation(s.authenticated!.userIdentifier); // unforgeable
|
||||
if (!myApp.mayManage(caller, space)) throw ...; // app policy
|
||||
await s.offlineSync.spaces.grant(space: space, user: user, role: role);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### No `admin` role in the package
|
||||
|
||||
The discussion considered an `admin` tier in `OfflineSyncSpaceRole` so "any admin can
|
||||
grant". It was rejected on the **enforce-vs-define** line: *the package should
|
||||
own only what it enforces.*
|
||||
|
||||
- `readOnly` / `readWrite` are **enforced by the sync engine against untrusted
|
||||
clients** (the server rejects an unauthorized stream write and records
|
||||
`unauthorizedWrite`). This check cannot live in the app; it must be in the
|
||||
package.
|
||||
- `admin` is **never consulted by the engine** — nothing in the sync path,
|
||||
merge, or read filter asks "is this user an admin". It would be a permission
|
||||
the package *defines but never acts on*, and it overlaps with Serverpod's own
|
||||
`Space` mechanism (a second permission system the app must reconcile).
|
||||
|
||||
Everything uneasy traced back to `admin`: the forgeable `by`, the bypass, and a
|
||||
"last-admin" orphan guard all evaporate once it leaves. `OfflineSyncSpaceRole` stays
|
||||
`{ readOnly, readWrite }` — exactly what is already implemented — so this
|
||||
implementation adds **no** enum churn.
|
||||
|
||||
Apps model "admin" with what they already have: a Serverpod `Space` on the auth
|
||||
user for global staff, or their own domain data for per-space owners. If the
|
||||
app's `TeamMember { teamId, userId, isAdmin }` is itself a synced CRDT table, an
|
||||
"am I an admin" UI works offline for free. The two membership layers stay
|
||||
separate: **access-membership** (`offline_sync_space_members`, server-authoritative,
|
||||
package-enforced: who may sync/read/write) vs **domain-membership** (app data:
|
||||
who is an admin, display name, joined-at). The app keeps them in step — adding a
|
||||
domain member also calls `spaces.grant` — the same split Serverpod has between
|
||||
`AuthUser` and an app's user profile.
|
||||
|
||||
## API
|
||||
|
||||
Server-side, hung off the `session.offlineSync` accessor (the session-bound
|
||||
`OfflineSyncSession` facade) as `session.offlineSync.spaces`. A pure data layer: atomic,
|
||||
transaction-threaded, no internal auth.
|
||||
|
||||
```dart
|
||||
/// Creates a shared space and returns its UUID.
|
||||
///
|
||||
/// [grants], if given, adds each user as a member with its role, applied in
|
||||
/// one transaction with the space insert. With no grants the space is created
|
||||
/// dormant — it exists but no one can sync it until membership is granted.
|
||||
Future<UuidValue> create({
|
||||
Map<UuidValue, OfflineSyncSpaceRole>? grants,
|
||||
Transaction? transaction,
|
||||
});
|
||||
|
||||
/// Creates a shared space granting [user] the given [role], in one
|
||||
/// transaction.
|
||||
///
|
||||
/// Pure sugar for [create] with a single-entry grants map — the member has no
|
||||
/// special bond to the space and can be re-granted or revoked like any other.
|
||||
Future<UuidValue> createFor(
|
||||
UuidValue user, {
|
||||
OfflineSyncSpaceRole role = OfflineSyncSpaceRole.readWrite,
|
||||
Transaction? transaction,
|
||||
});
|
||||
|
||||
/// Grants [user] the given [role] in [space], upserting an existing membership.
|
||||
Future<void> grant({
|
||||
required UuidValue space,
|
||||
required UuidValue user,
|
||||
required OfflineSyncSpaceRole role,
|
||||
Transaction? transaction,
|
||||
});
|
||||
|
||||
/// Grants several members in one transaction; the bulk twin of [grant].
|
||||
Future<void> grantAll(
|
||||
UuidValue space,
|
||||
Map<UuidValue, OfflineSyncSpaceRole> grants, {
|
||||
Transaction? transaction,
|
||||
});
|
||||
|
||||
/// Removes [user]'s membership in [space]. A no-op if not a member.
|
||||
Future<void> revoke({
|
||||
required UuidValue space,
|
||||
required UuidValue user,
|
||||
Transaction? transaction,
|
||||
});
|
||||
|
||||
/// The members of [space] and their roles.
|
||||
Future<Map<UuidValue, OfflineSyncSpaceRole>> members(
|
||||
UuidValue space, {
|
||||
Transaction? transaction,
|
||||
});
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- **There is no "owner".** An earlier sketch had an `owner` parameter on
|
||||
`create`, but the name implied a durable bond the data model does not have —
|
||||
the space row stores no owner, and the creator ends up an ordinary
|
||||
`readWrite` member. It was replaced by `createFor`, which is honest sugar:
|
||||
create a space and grant one member, atomically, with no special standing.
|
||||
Removing `owner` also dissolved the owner-appears-in-`grants` `ArgumentError`
|
||||
special case. A durable creator/owner concept is app domain, like `admin`.
|
||||
- **`grants` may be empty or absent** — a space with no members is valid but
|
||||
dormant. `create()` skips the membership insert entirely.
|
||||
- **`create` and `grantAll` share one internal helper** so the create path and
|
||||
the later bulk-grant path have a single implementation.
|
||||
- **Unknown-space grants throw `OfflineSyncSpaceNotFoundException`** rather than a
|
||||
generic argument error, so callers can distinguish a stale or invalid space
|
||||
UUID from other failures.
|
||||
- **`create` returns a server-generated UUID** — the model already defaults
|
||||
`uuidSpaceId` to `random_v7`. An optional explicit `uuidSpaceId` param
|
||||
(app-chosen or idempotent create) is a compatible later addition; left out
|
||||
until there is a real need.
|
||||
- Membership writes are plain `offline_sync_space_members` / `offline_sync_spaces` rows — **not**
|
||||
CRDT-tracked, not part of the sync loop. Changes propagate to clients on the
|
||||
next sync cycle through the existing `SpaceSet` re-announcement path.
|
||||
|
||||
## Client side
|
||||
|
||||
Unchanged. Management is server-only; the client never writes membership. A
|
||||
client learns of new spaces and role changes purely through the sync `SpaceSet`
|
||||
announcement and the follower projection already implemented in
|
||||
`shared-spaces.md`. There is no client `create`/`grant`/`revoke`.
|
||||
|
||||
## Test plan
|
||||
|
||||
- **Create.** `createFor(u)` yields a space with `u` at `readWrite` and a
|
||||
member-resolvable space; `createFor(u, role: ...)` stores the explicit role;
|
||||
`create()` (no grants) yields a dormant space no user can sync;
|
||||
`create(grants: {...})` stores every entry with its role.
|
||||
- **Grant/revoke.** `grant` inserts and upserts (role change); `revoke` removes
|
||||
and is a no-op for a non-member; `grantAll` applies a map atomically.
|
||||
- **Reads.** `members` returns the explicit member-role map. Internal role
|
||||
resolution still covers stored, missing, and implicit personal membership.
|
||||
- **Propagation.** After a server-side `grant`, a continuous follower session
|
||||
adopts the new space on the next cycle (covered by the existing access-change
|
||||
suite); after `revoke`, it stops cycling it.
|
||||
- **Transaction threading.** A `create` folded into an app domain transaction
|
||||
rolls back atomically with the app write on failure.
|
||||
|
|
@ -13,7 +13,7 @@ the CRDT layer recording the corresponding facts.
|
|||
There are only two allowed sync/non-sync relation shapes:
|
||||
|
||||
1. the package-owned ownership link from each synced domain row to
|
||||
`crdt_scopes`;
|
||||
`offline_sync_spaces`;
|
||||
2. a synced row referencing a non-synced, server-authoritative reference table,
|
||||
but only after the sync protocol can provision that reference data before
|
||||
CRDT merge.
|
||||
|
|
@ -23,15 +23,15 @@ The ownership link is:
|
|||
```yaml
|
||||
fields:
|
||||
id: UuidValue?, defaultPersist=random_v7
|
||||
### Owner scope of this row. Maintained by the CRDT sync layer.
|
||||
scopeId: int?, relation(parent=crdt_scopes, onDelete=Cascade)
|
||||
### Owner space of this row. Maintained by the CRDT sync layer.
|
||||
spaceId: int?, relation(parent=offline_sync_spaces, onDelete=Cascade)
|
||||
```
|
||||
|
||||
`scopeId == null` remains valid persisted state, but it means the row is
|
||||
`spaceId == null` remains valid persisted state, but it means the row is
|
||||
orphaned from CRDT ownership: it is never synced, never collected for outbound
|
||||
sync, and not removed by deleting a scope. `scopeId != null` means the row is
|
||||
owned by that scope, participates in CRDT, and is physically purged when the
|
||||
scope is deleted.
|
||||
sync, and not removed by deleting a space. `spaceId != null` means the row is
|
||||
owned by that space, participates in CRDT, and is physically purged when the
|
||||
space is deleted.
|
||||
|
||||
All other foreign keys between synced and non-synced tables must fail
|
||||
`initialize()`. Until the reference-data sync phase exists, `initialize()` must
|
||||
|
|
@ -49,10 +49,10 @@ also reject server-authoritative reference-table relations even when they use
|
|||
protocol, applied before CRDT merge, and not removed while synced rows
|
||||
reference them.
|
||||
- **Package metadata table:** CRDT infrastructure tables owned by this package,
|
||||
especially `crdt_scopes`, `crdt_nodes`, `crdt_data_rows`,
|
||||
especially `offline_sync_spaces`, `crdt_nodes`, `crdt_data_rows`,
|
||||
`crdt_data_fields`, `crdt_data_tombstone`, and `crdt_data_attempted_value`.
|
||||
|
||||
`crdt_scopes` is not a synced domain table. The `scopeId -> crdt_scopes.id`
|
||||
`offline_sync_spaces` is not a synced domain table. The `spaceId -> offline_sync_spaces.id`
|
||||
relation is a special package-owned purge boundary, not an application-level
|
||||
foreign key.
|
||||
|
||||
|
|
@ -65,27 +65,27 @@ Allowed under the normal FK invariant rules:
|
|||
- the child and parent are both in `syncTables`;
|
||||
- supported FK shapes are single-column child references to single-column
|
||||
parent references;
|
||||
- local writes assert same-scope visibility;
|
||||
- merge-time FK projection repairs hidden, missing, or foreign-scope targets.
|
||||
- local writes assert same-space visibility;
|
||||
- merge-time FK projection repairs hidden, missing, or foreign-space targets.
|
||||
|
||||
### Non-synced -> non-synced
|
||||
|
||||
Ignored by CRDT. These relations are normal application/database relations.
|
||||
|
||||
### Synced -> `crdt_scopes` through `scopeId`
|
||||
### Synced -> `offline_sync_spaces` through `spaceId`
|
||||
|
||||
Allowed, and required once this proposal is implemented. The relation must be
|
||||
exactly:
|
||||
|
||||
- child table is a synced domain table;
|
||||
- child column is the reserved `scopeId` column;
|
||||
- child column is the reserved `spaceId` column;
|
||||
- child column type is nullable `int`;
|
||||
- parent table is `crdt_scopes`;
|
||||
- parent table is `offline_sync_spaces`;
|
||||
- parent column is `id`;
|
||||
- `onDelete` is `Cascade`;
|
||||
- `onUpdate` is `NoAction` or unspecified database default.
|
||||
|
||||
This relation exists only to make terminal scope purge database-enforced. It
|
||||
This relation exists only to make terminal space purge database-enforced. It
|
||||
does not change normal CRDT deletes, which remain soft deletes recorded in
|
||||
CRDT metadata.
|
||||
|
||||
|
|
@ -126,7 +126,7 @@ delta before CRDT merge.
|
|||
|
||||
This exception also does not expand the global unique-index exception. A unique
|
||||
index involving a FK to a protocol-provisioned reference table is
|
||||
application-level uniqueness and must include `scopeId` unless another explicit
|
||||
application-level uniqueness and must include `spaceId` unless another explicit
|
||||
policy is added.
|
||||
|
||||
## Reference-data sync phase
|
||||
|
|
@ -153,7 +153,7 @@ through HLCs, unique conflict projection, or FK projection.
|
|||
|
||||
Always forbidden.
|
||||
|
||||
A non-synced child has no CRDT row, no scope ownership, and no merge history.
|
||||
A non-synced child has no CRDT row, no space ownership, and no merge history.
|
||||
If it references a synced parent, then a CRDT soft delete can hide the parent
|
||||
while the database FK still sees the physical parent row. Delete actions and
|
||||
visibility checks become incoherent, and the current recorder can also reach
|
||||
|
|
@ -163,7 +163,7 @@ paths that assume the child table has CRDT schema metadata.
|
|||
|
||||
Forbidden, except for:
|
||||
|
||||
- the special `scopeId -> crdt_scopes.id` relation;
|
||||
- the special `spaceId -> offline_sync_spaces.id` relation;
|
||||
- the protocol-provisioned reference-table relation above.
|
||||
|
||||
The apparent safe use case is an app user/auth table, because that table is
|
||||
|
|
@ -184,10 +184,10 @@ non-synced table is unsafe:
|
|||
deferrable.
|
||||
|
||||
The app-user relation should be modeled outside synced domain rows. Account
|
||||
deletion should revoke the user and purge the user's `CrdtScope`, not rely on
|
||||
deletion should revoke the user and purge the user's `OfflineSyncSpace`, not rely on
|
||||
arbitrary application FKs into synced data.
|
||||
|
||||
### Synced -> package metadata table other than `crdt_scopes`
|
||||
### Synced -> package metadata table other than `offline_sync_spaces`
|
||||
|
||||
Forbidden. Domain rows must not reference CRDT metadata internals such as nodes,
|
||||
schema rows, field rows, tombstones, or ownership violations.
|
||||
|
|
@ -196,36 +196,36 @@ schema rows, field rows, tombstones, or ownership violations.
|
|||
|
||||
Forbidden unless the relation is part of a future explicit package API. An
|
||||
application table must not depend on metadata rows whose lifecycle is controlled
|
||||
by sync, merge, or scope purge.
|
||||
by sync, merge, or space purge.
|
||||
|
||||
## Scope purge semantics
|
||||
## Space purge semantics
|
||||
|
||||
Deleting a scope is a terminal GDPR/account-erasure operation, not a CRDT
|
||||
Deleting a space is a terminal GDPR/account-erasure operation, not a CRDT
|
||||
operation.
|
||||
|
||||
A safe purge workflow is:
|
||||
|
||||
1. Revoke the account or scope so no new sync session can start.
|
||||
2. Mark the scope UUID as deleted, or otherwise make auth reject future
|
||||
`getOrCreate(uuidScopeId)` attempts for that UUID.
|
||||
3. Delete `crdt_scopes.id`.
|
||||
1. Revoke the account or space so no new sync session can start.
|
||||
2. Mark the space UUID as deleted, or otherwise make auth reject future
|
||||
`getOrCreate(uuidSpaceId)` attempts for that UUID.
|
||||
3. Delete `offline_sync_spaces.id`.
|
||||
4. Let database cascades remove:
|
||||
- synced domain rows with `scopeId = deletedScope.id`;
|
||||
- `crdt_nodes` for that scope;
|
||||
- `crdt_data_rows` for that scope;
|
||||
- synced domain rows with `spaceId = deletedSpace.id`;
|
||||
- `crdt_nodes` for that space;
|
||||
- `crdt_data_rows` for that space;
|
||||
- row fields, tombstones, and FK projection rows through their existing
|
||||
metadata cascades.
|
||||
5. Delete or anonymize diagnostics that intentionally outlive metadata
|
||||
cascades, such as durable ownership violation rows. Those rows store
|
||||
denormalized table names and scope UUIDs only; they do not reference
|
||||
denormalized table names and space UUIDs only; they do not reference
|
||||
purged metadata through foreign keys.
|
||||
|
||||
If an old offline client later reconnects with data for a purged scope, sync
|
||||
must fail with a terminal "scope deleted/reset required" response. The server
|
||||
must not recreate the scope and must not accept the old CRDT chain, or erased
|
||||
If an old offline client later reconnects with data for a purged space, sync
|
||||
must fail with a terminal "space deleted/reset required" response. The server
|
||||
must not recreate the space and must not accept the old CRDT chain, or erased
|
||||
data can be resurrected.
|
||||
|
||||
Rows with `scopeId == null` are not affected by scope purge. They are
|
||||
Rows with `spaceId == null` are not affected by space purge. They are
|
||||
application-owned orphan/admin rows and remain outside CRDT.
|
||||
|
||||
## Initialize validation
|
||||
|
|
@ -238,7 +238,7 @@ Validation rules:
|
|||
- `synced -> synced`: allowed only when the existing FK invariant support can
|
||||
handle the FK shape.
|
||||
- `non-synced -> non-synced`: ignored.
|
||||
- `synced -> crdt_scopes` through reserved `scopeId`: allowed only when it
|
||||
- `synced -> offline_sync_spaces` through reserved `spaceId`: allowed only when it
|
||||
matches the exact allowed shape above.
|
||||
- `synced -> protocol-provisioned reference table`: allowed only when the
|
||||
reference-data sync phase is implemented, the parent table is declared as
|
||||
|
|
@ -255,14 +255,14 @@ The error message should explain the allowed alternatives:
|
|||
- remove the relation;
|
||||
- use the future reference-data sync phase with `onDelete=Restrict` for
|
||||
immutable server-authoritative reference data;
|
||||
- for account deletion, use the package-owned scope purge relation instead of
|
||||
- for account deletion, use the package-owned space purge relation instead of
|
||||
an application FK.
|
||||
|
||||
## Tests
|
||||
|
||||
The implementation issue should add schema/initialize tests for:
|
||||
|
||||
- accepting `scopeId: int?, relation(parent=crdt_scopes,
|
||||
- accepting `spaceId: int?, relation(parent=offline_sync_spaces,
|
||||
onDelete=Cascade)` on every synced table;
|
||||
- rejecting `sync -> non-sync` restricted reference-table FKs with
|
||||
`onDelete=Restrict` while the reference-data sync phase is absent;
|
||||
|
|
@ -276,9 +276,9 @@ The implementation issue should add schema/initialize tests for:
|
|||
- rejecting `sync -> non-sync` application FKs with `NoAction`, `SetNull`,
|
||||
`SetDefault`, and `Cascade`;
|
||||
- rejecting global unique indexes over protocol-provisioned reference FK
|
||||
columns unless the index includes `scopeId`;
|
||||
columns unless the index includes `spaceId`;
|
||||
- rejecting synced domain relations to package metadata tables other than
|
||||
`crdt_scopes`;
|
||||
- preserving orphan rows with `scopeId == null` across scope purge;
|
||||
- deleting a scope removes owned domain rows and CRDT metadata for that scope;
|
||||
- stale clients for a purged scope cannot recreate the scope or sync old data.
|
||||
`offline_sync_spaces`;
|
||||
- preserving orphan rows with `spaceId == null` across space purge;
|
||||
- deleting a space removes owned domain rows and CRDT metadata for that space;
|
||||
- stale clients for a purged space cannot recreate the space or sync old data.
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@ This is the `flag` policy. It is intentionally simple:
|
|||
- no business conflict object is synced;
|
||||
- no foreign key retargeting is needed for unique conflicts.
|
||||
|
||||
This policy is the only unique-conflict policy in scope for the current
|
||||
This policy is the only unique-conflict policy in space for the current
|
||||
implementation.
|
||||
|
||||
## Current Policy: Flag
|
||||
|
|
@ -47,9 +47,9 @@ For each conflict group among visible rows:
|
|||
5. Keep every conflict-group row visible. Application code can decide whether
|
||||
and how to surface the changed visible value as a business conflict.
|
||||
|
||||
For a composite unique index, fixed partition columns such as `scopeId` are not
|
||||
For a composite unique index, fixed partition columns such as `spaceId` are not
|
||||
released, and not every remaining indexed column has to be releasable. The index
|
||||
is supported when at least one non-`scopeId` column can be released
|
||||
is supported when at least one non-`spaceId` column can be released
|
||||
deterministically. Stable discriminator columns may remain unchanged; changing
|
||||
the releasable column or columns is enough to make the unique tuple conflict-free.
|
||||
Supported candidate types are defined centrally under
|
||||
|
|
@ -206,7 +206,7 @@ resolution.
|
|||
when the loser is released, then the materialized UUID is deterministic from
|
||||
the table, column, original value, and loser row id.
|
||||
|
||||
7. Given a scoped composite unique index contains a stable non-releasable
|
||||
7. Given a space-scoped composite unique index contains a stable non-releasable
|
||||
discriminator and a releasable text column, when a loser is released, then the
|
||||
discriminator remains unchanged and the text column receives the deterministic
|
||||
conflict suffix.
|
||||
|
|
@ -239,6 +239,6 @@ projection model exists.
|
|||
row hiding or FK retargeting.
|
||||
- Convergence: same CRDT facts produce the same visible unique values.
|
||||
- Idempotence: repeated merges do not keep rewriting the same conflict.
|
||||
- Scope control: future `merge` and `overwrite` behavior is documented but not
|
||||
- Space control: future `merge` and `overwrite` behavior is documented but not
|
||||
treated as supported until the projection algorithm can handle second-order FK
|
||||
effects.
|
||||
|
|
|
|||
Loading…
Reference in a new issue