serverpod-offline-sync/docs/sync-non-sync-relations.md

# Sync and non-sync foreign key boundaries

Status: proposal for a separate implementation issue.

## Summary

Foreign keys that cross the sync boundary are unsafe by default. A synced row is
owned, collected, merged, hidden, and repaired through CRDT metadata. A
non-synced row has none of that state. Letting arbitrary database relations
cross that boundary lets the database mutate or constrain synced rows without
the CRDT layer recording the corresponding facts.

There are only two allowed sync/non-sync relation shapes:

1. the package-owned ownership link from each synced domain row to
   `crdt_scopes`;
2. a synced row referencing a non-synced, server-authoritative reference table,
   but only after the sync protocol can provision that reference data before
   CRDT merge.

The ownership link is:

```yaml
fields:
  id: UuidValue?, defaultPersist=random_v7
  ### Owner scope of this row. Maintained by the CRDT sync layer.
  scopeId: int?, relation(parent=crdt_scopes, onDelete=Cascade)
```

`scopeId == null` remains valid persisted state, but it means the row is
orphaned from CRDT ownership: it is never synced, never collected for outbound
sync, and not removed by deleting a scope. `scopeId != null` means the row is
owned by that scope, participates in CRDT, and is physically purged when the
scope is deleted.

All other foreign keys between synced and non-synced tables must fail
`initialize()`. Until the reference-data sync phase exists, `initialize()` must
also reject server-authoritative reference-table relations even when they use
`onDelete=Restrict`.

## Table classes

- **Synced table:** any application table passed to the CRDT `syncTables`
  configuration.
- **Non-synced table:** any other application table known to the Serverpod
  serialization manager.
- **Protocol-provisioned reference table:** a non-synced table whose rows are
  authored by the server, sent through a reference-data phase in the sync
  protocol, applied before CRDT merge, and not removed while synced rows
  reference them.
- **Package metadata table:** CRDT infrastructure tables owned by this package,
  especially `crdt_scopes`, `crdt_nodes`, `crdt_data_rows`,
  `crdt_data_fields`, `crdt_data_tombstone`, and FK projection tables.

`crdt_scopes` is not a synced domain table. The `scopeId -> crdt_scopes.id`
relation is a special package-owned purge boundary, not an application-level
foreign key.

## Allowed relations

### Synced -> synced

Allowed under the normal FK invariant rules:

- the child and parent are both in `syncTables`;
- supported FK shapes are single-column child references to single-column
  parent references;
- local writes assert same-scope visibility;
- merge-time FK projection repairs hidden, missing, or foreign-scope targets.

### Non-synced -> non-synced

Ignored by CRDT. These relations are normal application/database relations.

### Synced -> `crdt_scopes` through `scopeId`

Allowed, and required once this proposal is implemented. The relation must be
exactly:

- child table is a synced domain table;
- child column is the reserved `scopeId` column;
- child column type is nullable `int`;
- parent table is `crdt_scopes`;
- parent column is `id`;
- `onDelete` is `Cascade`;
- `onUpdate` is `NoAction` or unspecified database default.

This relation exists only to make terminal scope purge database-enforced. It
does not change normal CRDT deletes, which remain soft deletes recorded in
CRDT metadata.

### Future: synced -> protocol-provisioned reference table

Allowed when every referenced row is server-authoritative reference data, such
as a global category table used to group synced rows, and when the sync
protocol provisions that table before applying CRDT changes. The relation must
be strictly constraining: it may prevent reference-table deletion, but it must
never cause the database to mutate synced rows behind the CRDT recorder.

This exception must not be implemented as a schema-only relaxation. The same
implementation issue that permits this relation must also add the reference-data
sync phase described below.

The allowed shape is:

- child table is a synced domain table;
- parent table is a non-synced application table;
- parent table is declared in sync configuration as protocol-provisioned
  reference data;
- the FK is single-column child reference to single-column parent reference;
- `onDelete` is exactly `Restrict`;
- `onUpdate` is `Restrict`, `NoAction`, or unspecified database default;
- the referenced parent key is treated as immutable by application policy;
- referenced parent rows are applied before any local insert, local update, or
  incoming merge can use them.

`NoAction` is intentionally not accepted for `onDelete`: the delete must fail
immediately rather than being deferrable to transaction end. `Cascade`,
`SetNull`, and `SetDefault` are forbidden because they can physically delete or
mutate synced rows without CRDT tombstones or field HLCs.

This exception does not make the reference table offline-first. Creating,
renaming, deleting, or otherwise syncing reference rows is outside CRDT. The
server authors reference data, and clients cache the latest server snapshot or
delta before CRDT merge.

This exception also does not expand the global unique-index exception. A unique
index involving a FK to a protocol-provisioned reference table is
application-level uniqueness and must include `scopeId` unless another explicit
policy is added.

## Reference-data sync phase

The sync protocol must treat server-authoritative reference data as an ordered
precondition for CRDT merge:

1. The sync handshake includes the client's reference-data versions or hashes.
2. The server sends missing reference-data snapshots or deltas before CRDT
   changes that may reference them.
3. The client applies those reference rows to local non-synced tables.
4. Only after the reference-data phase succeeds does the client apply incoming
   CRDT operations.
5. If an incoming CRDT row references missing reference data after that phase,
   the sync fails as a protocol violation. FK projection must not repair it.

Reference data has server-authoritative replacement/delta semantics, not CRDT
semantics. Clients do not author reference rows, and conflicts are not resolved
through HLCs, unique conflict projection, or FK projection.

## Forbidden relations

### Non-synced -> synced

Always forbidden.

A non-synced child has no CRDT row, no scope ownership, and no merge history.
If it references a synced parent, then a CRDT soft delete can hide the parent
while the database FK still sees the physical parent row. Delete actions and
visibility checks become incoherent, and the current recorder can also reach
paths that assume the child table has CRDT schema metadata.

### Synced -> other non-synced application table

Forbidden, except for:

- the special `scopeId -> crdt_scopes.id` relation;
- the protocol-provisioned reference-table relation above.

The apparent safe use case is an app user/auth table, because that table is
server-authoritative and must exist before offline data can be synced. That can
fit the protocol-provisioned reference-table rule only when user rows are
provisioned by the sync reference-data phase and never deleted while synced rows
reference them. If account deletion must erase synced data, it should not rely
on a direct app-user FK cascade. A direct unrestricted FK from synced rows to a
non-synced table is unsafe:

- merge inserts and updates cannot project or repair missing non-synced
  parents;
- database FK failures become raw merge failures, not deterministic CRDT
  projection;
- deleting the non-synced parent with `CASCADE`, `SET NULL`, or `SET DEFAULT`
  can change synced child rows without CRDT tombstones or field HLCs;
- `NoAction` is not strict enough for this boundary because it can be
  deferrable.

The app-user relation should be modeled outside synced domain rows. Account
deletion should revoke the user and purge the user's `CrdtScope`, not rely on
arbitrary application FKs into synced data.

### Synced -> package metadata table other than `crdt_scopes`

Forbidden. Domain rows must not reference CRDT metadata internals such as nodes,
schema rows, field rows, tombstones, or ownership violations.

### Non-synced -> package metadata table

Forbidden unless the relation is part of a future explicit package API. An
application table must not depend on metadata rows whose lifecycle is controlled
by sync, merge, or scope purge.

## Scope purge semantics

Deleting a scope is a terminal GDPR/account-erasure operation, not a CRDT
operation.

A safe purge workflow is:

1. Revoke the account or scope so no new sync session can start.
2. Mark the scope UUID as deleted, or otherwise make auth reject future
   `getOrCreate(uuidScopeId)` attempts for that UUID.
3. Delete `crdt_scopes.id`.
4. Let database cascades remove:
   - synced domain rows with `scopeId = deletedScope.id`;
   - `crdt_nodes` for that scope;
   - `crdt_data_rows` for that scope;
   - row fields, tombstones, and FK projection rows through their existing
     metadata cascades.
5. Delete or anonymize diagnostics that intentionally outlive metadata
   cascades, such as durable ownership violation rows. Those rows store
   denormalized table names and scope UUIDs only; they do not reference
   purged metadata through foreign keys.

If an old offline client later reconnects with data for a purged scope, sync
must fail with a terminal "scope deleted/reset required" response. The server
must not recreate the scope and must not accept the old CRDT chain, or erased
data can be resurrected.

Rows with `scopeId == null` are not affected by scope purge. They are
application-owned orphan/admin rows and remain outside CRDT.

## Initialize validation

`initialize()` must inspect all generated table definitions and classify every
foreign key by child and parent table class.

Validation rules:

- `synced -> synced`: allowed only when the existing FK invariant support can
  handle the FK shape.
- `non-synced -> non-synced`: ignored.
- `synced -> crdt_scopes` through reserved `scopeId`: allowed only when it
  matches the exact allowed shape above.
- `synced -> protocol-provisioned reference table`: allowed only when the
  reference-data sync phase is implemented, the parent table is declared as
  protocol-provisioned reference data, and the relation matches the exact
  allowed shape above. Without that protocol support, reject it even when the
  FK uses `onDelete=Restrict`.
- every other `synced <-> non-synced` or `domain <-> package metadata` FK:
  fail initialization with an error naming the offending constraint.

The error message should explain the allowed alternatives:

- make both tables synced;
- make both tables non-synced;
- remove the relation;
- use the future reference-data sync phase with `onDelete=Restrict` for
  immutable server-authoritative reference data;
- for account deletion, use the package-owned scope purge relation instead of
  an application FK.

## Tests

The implementation issue should add schema/initialize tests for:

- accepting `scopeId: int?, relation(parent=crdt_scopes,
  onDelete=Cascade)` on every synced table;
- rejecting `sync -> non-sync` restricted reference-table FKs with
  `onDelete=Restrict` while the reference-data sync phase is absent;
- accepting `sync -> non-sync` protocol-provisioned reference-table FKs with
  `onDelete=Restrict` after the reference-data sync phase is implemented;
- applying reference-data deltas before CRDT merge when incoming rows reference
  those rows;
- failing sync as a protocol violation when incoming CRDT rows reference
  missing reference data after the reference-data phase;
- rejecting `non-sync -> sync`;
- rejecting `sync -> non-sync` application FKs with `NoAction`, `SetNull`,
  `SetDefault`, and `Cascade`;
- rejecting global unique indexes over protocol-provisioned reference FK
  columns unless the index includes `scopeId`;
- rejecting synced domain relations to package metadata tables other than
  `crdt_scopes`;
- preserving orphan rows with `scopeId == null` across scope purge;
- deleting a scope removes owned domain rows and CRDT metadata for that scope;
- stale clients for a purged scope cannot recreate the scope or sync old data.