284 lines
12 KiB
Markdown
284 lines
12 KiB
Markdown
# Sync and non-sync foreign key boundaries
|
|
|
|
Status: proposal for a separate implementation issue.
|
|
|
|
## Summary
|
|
|
|
Foreign keys that cross the sync boundary are unsafe by default. A synced row is
|
|
owned, collected, merged, hidden, and repaired through CRDT metadata. A
|
|
non-synced row has none of that state. Letting arbitrary database relations
|
|
cross that boundary lets the database mutate or constrain synced rows without
|
|
the CRDT layer recording the corresponding facts.
|
|
|
|
There are only two allowed sync/non-sync relation shapes:
|
|
|
|
1. the package-owned ownership link from each synced domain row to
|
|
`crdt_scopes`;
|
|
2. a synced row referencing a non-synced, server-authoritative reference table,
|
|
but only after the sync protocol can provision that reference data before
|
|
CRDT merge.
|
|
|
|
The ownership link is:
|
|
|
|
```yaml
|
|
fields:
|
|
id: UuidValue?, defaultPersist=random_v7
|
|
### Owner scope of this row. Maintained by the CRDT sync layer.
|
|
scopeId: int?, relation(parent=crdt_scopes, onDelete=Cascade)
|
|
```
|
|
|
|
`scopeId == null` remains valid persisted state, but it means the row is
|
|
orphaned from CRDT ownership: it is never synced, never collected for outbound
|
|
sync, and not removed by deleting a scope. `scopeId != null` means the row is
|
|
owned by that scope, participates in CRDT, and is physically purged when the
|
|
scope is deleted.
|
|
|
|
All other foreign keys between synced and non-synced tables must fail
|
|
`initialize()`. Until the reference-data sync phase exists, `initialize()` must
|
|
also reject server-authoritative reference-table relations even when they use
|
|
`onDelete=Restrict`.
|
|
|
|
## Table classes
|
|
|
|
- **Synced table:** any application table passed to the CRDT `syncTables`
|
|
configuration.
|
|
- **Non-synced table:** any other application table known to the Serverpod
|
|
serialization manager.
|
|
- **Protocol-provisioned reference table:** a non-synced table whose rows are
|
|
authored by the server, sent through a reference-data phase in the sync
|
|
protocol, applied before CRDT merge, and not removed while synced rows
|
|
reference them.
|
|
- **Package metadata table:** CRDT infrastructure tables owned by this package,
|
|
especially `crdt_scopes`, `crdt_nodes`, `crdt_data_rows`,
|
|
`crdt_data_fields`, `crdt_data_tombstone`, and FK projection tables.
|
|
|
|
`crdt_scopes` is not a synced domain table. The `scopeId -> crdt_scopes.id`
|
|
relation is a special package-owned purge boundary, not an application-level
|
|
foreign key.
|
|
|
|
## Allowed relations
|
|
|
|
### Synced -> synced
|
|
|
|
Allowed under the normal FK invariant rules:
|
|
|
|
- the child and parent are both in `syncTables`;
|
|
- supported FK shapes are single-column child references to single-column
|
|
parent references;
|
|
- local writes assert same-scope visibility;
|
|
- merge-time FK projection repairs hidden, missing, or foreign-scope targets.
|
|
|
|
### Non-synced -> non-synced
|
|
|
|
Ignored by CRDT. These relations are normal application/database relations.
|
|
|
|
### Synced -> `crdt_scopes` through `scopeId`
|
|
|
|
Allowed, and required once this proposal is implemented. The relation must be
|
|
exactly:
|
|
|
|
- child table is a synced domain table;
|
|
- child column is the reserved `scopeId` column;
|
|
- child column type is nullable `int`;
|
|
- parent table is `crdt_scopes`;
|
|
- parent column is `id`;
|
|
- `onDelete` is `Cascade`;
|
|
- `onUpdate` is `NoAction` or unspecified database default.
|
|
|
|
This relation exists only to make terminal scope purge database-enforced. It
|
|
does not change normal CRDT deletes, which remain soft deletes recorded in
|
|
CRDT metadata.
|
|
|
|
### Future: synced -> protocol-provisioned reference table
|
|
|
|
Allowed when every referenced row is server-authoritative reference data, such
|
|
as a global category table used to group synced rows, and when the sync
|
|
protocol provisions that table before applying CRDT changes. The relation must
|
|
be strictly constraining: it may prevent reference-table deletion, but it must
|
|
never cause the database to mutate synced rows behind the CRDT recorder.
|
|
|
|
This exception must not be implemented as a schema-only relaxation. The same
|
|
implementation issue that permits this relation must also add the reference-data
|
|
sync phase described below.
|
|
|
|
The allowed shape is:
|
|
|
|
- child table is a synced domain table;
|
|
- parent table is a non-synced application table;
|
|
- parent table is declared in sync configuration as protocol-provisioned
|
|
reference data;
|
|
- the FK is single-column child reference to single-column parent reference;
|
|
- `onDelete` is exactly `Restrict`;
|
|
- `onUpdate` is `Restrict`, `NoAction`, or unspecified database default;
|
|
- the referenced parent key is treated as immutable by application policy;
|
|
- referenced parent rows are applied before any local insert, local update, or
|
|
incoming merge can use them.
|
|
|
|
`NoAction` is intentionally not accepted for `onDelete`: the delete must fail
|
|
immediately rather than being deferrable to transaction end. `Cascade`,
|
|
`SetNull`, and `SetDefault` are forbidden because they can physically delete or
|
|
mutate synced rows without CRDT tombstones or field HLCs.
|
|
|
|
This exception does not make the reference table offline-first. Creating,
|
|
renaming, deleting, or otherwise syncing reference rows is outside CRDT. The
|
|
server authors reference data, and clients cache the latest server snapshot or
|
|
delta before CRDT merge.
|
|
|
|
This exception also does not expand the global unique-index exception. A unique
|
|
index involving a FK to a protocol-provisioned reference table is
|
|
application-level uniqueness and must include `scopeId` unless another explicit
|
|
policy is added.
|
|
|
|
## Reference-data sync phase
|
|
|
|
The sync protocol must treat server-authoritative reference data as an ordered
|
|
precondition for CRDT merge:
|
|
|
|
1. The sync handshake includes the client's reference-data versions or hashes.
|
|
2. The server sends missing reference-data snapshots or deltas before CRDT
|
|
changes that may reference them.
|
|
3. The client applies those reference rows to local non-synced tables.
|
|
4. Only after the reference-data phase succeeds does the client apply incoming
|
|
CRDT operations.
|
|
5. If an incoming CRDT row references missing reference data after that phase,
|
|
the sync fails as a protocol violation. FK projection must not repair it.
|
|
|
|
Reference data has server-authoritative replacement/delta semantics, not CRDT
|
|
semantics. Clients do not author reference rows, and conflicts are not resolved
|
|
through HLCs, unique conflict projection, or FK projection.
|
|
|
|
## Forbidden relations
|
|
|
|
### Non-synced -> synced
|
|
|
|
Always forbidden.
|
|
|
|
A non-synced child has no CRDT row, no scope ownership, and no merge history.
|
|
If it references a synced parent, then a CRDT soft delete can hide the parent
|
|
while the database FK still sees the physical parent row. Delete actions and
|
|
visibility checks become incoherent, and the current recorder can also reach
|
|
paths that assume the child table has CRDT schema metadata.
|
|
|
|
### Synced -> other non-synced application table
|
|
|
|
Forbidden, except for:
|
|
|
|
- the special `scopeId -> crdt_scopes.id` relation;
|
|
- the protocol-provisioned reference-table relation above.
|
|
|
|
The apparent safe use case is an app user/auth table, because that table is
|
|
server-authoritative and must exist before offline data can be synced. That can
|
|
fit the protocol-provisioned reference-table rule only when user rows are
|
|
provisioned by the sync reference-data phase and never deleted while synced rows
|
|
reference them. If account deletion must erase synced data, it should not rely
|
|
on a direct app-user FK cascade. A direct unrestricted FK from synced rows to a
|
|
non-synced table is unsafe:
|
|
|
|
- merge inserts and updates cannot project or repair missing non-synced
|
|
parents;
|
|
- database FK failures become raw merge failures, not deterministic CRDT
|
|
projection;
|
|
- deleting the non-synced parent with `CASCADE`, `SET NULL`, or `SET DEFAULT`
|
|
can change synced child rows without CRDT tombstones or field HLCs;
|
|
- `NoAction` is not strict enough for this boundary because it can be
|
|
deferrable.
|
|
|
|
The app-user relation should be modeled outside synced domain rows. Account
|
|
deletion should revoke the user and purge the user's `CrdtScope`, not rely on
|
|
arbitrary application FKs into synced data.
|
|
|
|
### Synced -> package metadata table other than `crdt_scopes`
|
|
|
|
Forbidden. Domain rows must not reference CRDT metadata internals such as nodes,
|
|
schema rows, field rows, tombstones, or ownership violations.
|
|
|
|
### Non-synced -> package metadata table
|
|
|
|
Forbidden unless the relation is part of a future explicit package API. An
|
|
application table must not depend on metadata rows whose lifecycle is controlled
|
|
by sync, merge, or scope purge.
|
|
|
|
## Scope purge semantics
|
|
|
|
Deleting a scope is a terminal GDPR/account-erasure operation, not a CRDT
|
|
operation.
|
|
|
|
A safe purge workflow is:
|
|
|
|
1. Revoke the account or scope so no new sync session can start.
|
|
2. Mark the scope UUID as deleted, or otherwise make auth reject future
|
|
`getOrCreate(uuidScopeId)` attempts for that UUID.
|
|
3. Delete `crdt_scopes.id`.
|
|
4. Let database cascades remove:
|
|
- synced domain rows with `scopeId = deletedScope.id`;
|
|
- `crdt_nodes` for that scope;
|
|
- `crdt_data_rows` for that scope;
|
|
- row fields, tombstones, and FK projection rows through their existing
|
|
metadata cascades.
|
|
5. Delete or anonymize diagnostics that intentionally outlive metadata
|
|
cascades, such as durable ownership violation rows. Those rows store
|
|
denormalized table names and scope UUIDs only; they do not reference
|
|
purged metadata through foreign keys.
|
|
|
|
If an old offline client later reconnects with data for a purged scope, sync
|
|
must fail with a terminal "scope deleted/reset required" response. The server
|
|
must not recreate the scope and must not accept the old CRDT chain, or erased
|
|
data can be resurrected.
|
|
|
|
Rows with `scopeId == null` are not affected by scope purge. They are
|
|
application-owned orphan/admin rows and remain outside CRDT.
|
|
|
|
## Initialize validation
|
|
|
|
`initialize()` must inspect all generated table definitions and classify every
|
|
foreign key by child and parent table class.
|
|
|
|
Validation rules:
|
|
|
|
- `synced -> synced`: allowed only when the existing FK invariant support can
|
|
handle the FK shape.
|
|
- `non-synced -> non-synced`: ignored.
|
|
- `synced -> crdt_scopes` through reserved `scopeId`: allowed only when it
|
|
matches the exact allowed shape above.
|
|
- `synced -> protocol-provisioned reference table`: allowed only when the
|
|
reference-data sync phase is implemented, the parent table is declared as
|
|
protocol-provisioned reference data, and the relation matches the exact
|
|
allowed shape above. Without that protocol support, reject it even when the
|
|
FK uses `onDelete=Restrict`.
|
|
- every other `synced <-> non-synced` or `domain <-> package metadata` FK:
|
|
fail initialization with an error naming the offending constraint.
|
|
|
|
The error message should explain the allowed alternatives:
|
|
|
|
- make both tables synced;
|
|
- make both tables non-synced;
|
|
- remove the relation;
|
|
- use the future reference-data sync phase with `onDelete=Restrict` for
|
|
immutable server-authoritative reference data;
|
|
- for account deletion, use the package-owned scope purge relation instead of
|
|
an application FK.
|
|
|
|
## Tests
|
|
|
|
The implementation issue should add schema/initialize tests for:
|
|
|
|
- accepting `scopeId: int?, relation(parent=crdt_scopes,
|
|
onDelete=Cascade)` on every synced table;
|
|
- rejecting `sync -> non-sync` restricted reference-table FKs with
|
|
`onDelete=Restrict` while the reference-data sync phase is absent;
|
|
- accepting `sync -> non-sync` protocol-provisioned reference-table FKs with
|
|
`onDelete=Restrict` after the reference-data sync phase is implemented;
|
|
- applying reference-data deltas before CRDT merge when incoming rows reference
|
|
those rows;
|
|
- failing sync as a protocol violation when incoming CRDT rows reference
|
|
missing reference data after the reference-data phase;
|
|
- rejecting `non-sync -> sync`;
|
|
- rejecting `sync -> non-sync` application FKs with `NoAction`, `SetNull`,
|
|
`SetDefault`, and `Cascade`;
|
|
- rejecting global unique indexes over protocol-provisioned reference FK
|
|
columns unless the index includes `scopeId`;
|
|
- rejecting synced domain relations to package metadata tables other than
|
|
`crdt_scopes`;
|
|
- preserving orphan rows with `scopeId == null` across scope purge;
|
|
- deleting a scope removes owned domain rows and CRDT metadata for that scope;
|
|
- stale clients for a purged scope cannot recreate the scope or sync old data.
|