serverpod-offline-sync/docs/foreign-key-invariants.md

Preserving foreign key invariants while merging

Summary

After merging remote CRDT operations, the visible database must not contain foreign key violations. The merge result must be associative, commutative, and idempotent: the same merged CRDT facts must produce the same visible rows and visible FK values regardless of arrival order, batching, or replay.

The safe merge model is projection-based. Local ORM operations are still authored user actions: when a local delete is intercepted for soft delete, the database-style ON DELETE side effects are materialized as CRDT facts in the same local transaction. FK repairs created only because independently authored facts were merged are deterministic materialized projections over those facts, not new user field updates emitted opportunistically during merge.

Core Policy

• Keep every row as a CRDT fact. Visibility is derived from row tombstones and schema-driven FK projection.
• Supported foreign keys are single-column child references to single-column parent references. Schemas with composite foreign keys must fail recorder initialization instead of silently bypassing projection.
• Merge input is causally complete: the sync protocol delivers each batch as a full causal snapshot of the sender, so every merged FK attempt's parent facts precede or accompany the child fact. Input that violates this precondition (for example, a non-nullable FK attempt whose parent fact is absent and whose default target is missing) is outside the merge contract and fails the merge transaction atomically against the database foreign key constraints.
• Local ON DELETE actions are authored CRDT facts. RESTRICT and NO ACTION reject the local delete while visible children exist; SET NULL and SET DEFAULT update child FK fields and their ordinary CrdtDataField HLCs; CASCADE records user-delete tombstones for the visible cascade descendants.
• Keep the user/attempted FK value as the CRDT field value for authored operations.
• During merge projection, materialize a separate visible FK value only when FK repair requires it. CrdtDataForeignKey.attemptedValue is the durable FK field-value payload for FK columns because CrdtDataField stores only HLC metadata. visibleValue, hasOverride, and overrideReason are projection metadata, not business conflict classes.
• Do not create ordinary user field updates for SET NULL or SET DEFAULT repairs during merge. Doing so makes repair HLCs depend on merge timing. This does not apply to locally initiated SET NULL or SET DEFAULT actions, which are authored user field updates.
• For FK-affecting operations, recompute FK projection to the same fixed point as the affected FK closure. A full graph recomputation is a valid implementation strategy and oracle as long as it produces the same projection. Operations that touch no child FK column, parent reference column, insert, or tombstone must not trigger FK projection.

Local Action Semantics

Local deletes preserve database action semantics even though the physical row is soft-deleted instead of removed:

• ON DELETE RESTRICT and NO ACTION: the delete fails if any visible child still references the parent. No parent tombstone is recorded for the failed operation.
• ON DELETE SET NULL: visible child FK columns are updated to null when the FK is nullable, and those child FK fields receive ordinary CRDT field updates.
• ON DELETE SET DEFAULT: visible child FK columns are updated to the column default when the default is legal, and those child FK fields receive ordinary CRDT field updates.
• ON DELETE CASCADE: visible cascade descendants receive synced user-delete tombstones. Local cascade descendants are not hidden only as foreignKeyCascade projection rows.

These local side effects are the facts that other replicas merge.

Merge-Time Action Semantics

For each child FK whose attempted value points to a hidden or missing parent:

• ON DELETE SET NULL: the visible FK value is null when the FK is nullable. If the FK is non-nullable, this action cannot repair the row and must fall through to the deterministic blocking policy.
• ON DELETE SET DEFAULT: the visible FK value is the column default only if that default points to a visible parent or is a legal nullable default. If the default target is hidden or missing, this action cannot repair the row and must fall through to the deterministic blocking policy.
• ON DELETE RESTRICT and NO ACTION: the parent delete loses while a visible child still references the parent. The parent remains visible in the derived projection.
• ON DELETE CASCADE: the child can be hidden only if the whole cascade closure is valid. If any descendant blocks the cascade through RESTRICT, NO ACTION, invalid SET NULL, or invalid SET DEFAULT, the ancestor delete loses and the cascade is not partially applied.

This makes mixed chains atomic for visibility. For example, if deleting A would cascade-delete B, but B has a visible RESTRICT child C, then A, B, and C remain visible. The result must not depend on whether cascade or restrict edges are evaluated first internally.

Projection Metadata

CrdtDataForeignKey should represent the FK field payload and materialized FK projection for one CrdtDataField:

• attemptedValue: the current user-attempted FK value carried by the CRDT field. It may be null for nullable FKs. This is durable FK field-value storage and is part of the CRDT facts for FK columns.
• visibleValue: the value currently materialized into the domain row when an override is active (overrideReason is non-null).
• overrideReason: the authoritative stored indicator that an override is active, and the cause. Non-null means an override is active; null means no override (the domain row carries attemptedValue). Values: setNull — the parent was hidden/deleted and the column was set null by the FK action; setDefault — the column was set to a default FK value; missingParent — the parent is hidden or missing and cannot be repaired, so the row is hidden by projection. The derived hasOverride bool (overrideReason != null) is available as a convenience getter.

visibleValue is meaningful only when overrideReason is non-null. When overrideReason is null the domain row carries attemptedValue directly; when overrideReason is non-null and visibleValue is null it means "materialize null".

The resolver may use visibleValue and overrideReason for efficient materialization and write deduplication. It must not use those projection fields as the authority for FK conflict decisions; the authority is the deterministic resolver over merged CRDT row, field, tombstone, and FK attempt facts.

A local full-row update (no column narrowing) that writes an FK value equal to the materialized projected value is a passthrough of the repair, not an authored FK change: the existing override and attempted value are preserved and the FK field HLC does not advance. Writing a different FK value, or updating the FK column with narrowed columns (even when the written value equals the projected value), authors the change as an ordinary user field fact: the FK field HLC advances, attemptedValue becomes the new value, and overrideReason becomes null.

Merge Pipeline

1. Merge CRDT row, field, and tombstone facts by their existing HLC/CLFlag rules.
2. Identify whether inserts, updates, deletes, FK columns, parent reference columns, or existing FK projection records can affect FK state. Skip FK projection when the merge touches none of them.
3. For FK-affecting work, expand to the FK closure in both directions: parents needed by children and children affected by parent visibility. A full graph recomputation may be used instead of the minimal closure.
4. Compute effective row visibility and visible FK values to a fixed point.
5. Apply only the materialized differences:
   • update domain FK columns when projection differs from attempted value;
   • update CrdtDataForeignKey projection metadata;
   • update system tombstones only for deterministic derived visibility where that is part of the chosen storage model.
6. Re-run unique conflict projection after FK projection if FK repairs can change unique columns, or define the opposite stage order and prove it with the full recomputation oracle.

Invariants

• No visible row references a hidden or missing parent.
• No cascade delete is partially applied across a blocked nested relation.
• Merge-time SET NULL and SET DEFAULT repairs are pure functions of merged facts. Local SET NULL and SET DEFAULT actions are authored field facts.
• Local cascade deletes produce synced user-delete tombstones for cascade descendants; merge-time cascade projection may derive foreignKeyCascade visibility for descendants without authored tombstones.
• Replaying the same merge batch produces no metadata churn.
• Splitting a merge into smaller batches converges to the same final projection as merging the same operations in one batch.
• FK-affecting recomputation over the affected closure matches full global recomputation; unrelated non-FK updates do not trigger FK projection.

Test Cases

Local CRUD Actions

• Given a visible RESTRICT or NO ACTION child, when the parent is deleted locally, then the delete fails and no parent tombstone is recorded.
• Given a nullable SET NULL child, when the parent is deleted locally, then the child FK is set to null as an authored field update.
• Given a valid SET DEFAULT child, when the parent is deleted locally, then the child FK is set to the default as an authored field update.
• Given a valid cascade chain, when the root is deleted locally, then every visible cascade descendant receives a user-delete tombstone.

Restrict and No Action

1. Given a parent and a visible child with ON DELETE RESTRICT, when one replica deletes the parent and another concurrently updates the child, then the parent remains visible after convergence and no visible orphan appears.

2. Given a parent with multiple visible restrict children, when the parent is deleted concurrently with updates to different children, then any visible child is enough to keep the parent visible on every replica.

3. Given a parent kept visible because of restrict, when all restrict children are later deleted or detached, then the parent visibility follows the same deterministic recomputation rule on every replica.

Set Null and Set Default

4. Given a nullable FK with ON DELETE SET NULL, when the parent is deleted on another replica, then the child remains visible with a materialized null FK and the user/attempted FK value remains unchanged as a CRDT fact.

5. Given a child whose FK was materialized to null by SET NULL, when the user later changes the FK to a visible parent, then the projection override becomes inactive and the domain row materializes the new user value.

6. Given ON DELETE SET DEFAULT and a visible default target, when the attempted parent is deleted, then the child remains visible with the default FK materialized.

7. Given ON DELETE SET DEFAULT and a hidden or missing default target, when the attempted parent is deleted, then the action cannot repair the child and the deterministic blocking policy keeps the necessary parent visible.

Cascade

8. Given a parent delete fact merged from another replica with children under ON DELETE CASCADE, then all cascade descendants in the valid cascade closure become hidden and no visible descendant references a hidden ancestor.

9. Given a child already hidden by a cascade projection, when the same delete facts are merged again, then no additional metadata churn occurs.

Mixed Nested Relations

10. Given A -> B is cascade and B <- C is restrict, when A is deleted concurrently with an update to C, then the cascade is blocked and A, B, and C remain visible.

11. Given a fourth-degree chain with alternating cascade, restrict, set-null, and set-default edges, when a merge touches both a root and a leaf, then the affected-closure result and a full fixed-point recomputation produce the same visible graph and FK projection.

12. Given an FK cycle that the engine permits, when concurrent deletes and updates happen around the cycle, then fixed-point evaluation terminates and every replica computes the same projection.

Cross-Cutting

13. Given an FK repair changes a unique-indexed column, when unique projection and FK projection are recomputed, then the documented stage order prevents visible unique and FK violations.

14. Given the same remote operations arrive in one batch or several smaller batches, when all operations have merged, then visible rows, visible FK values, FK projection metadata, unique projection, and tombstones are the same.

15. Given a merge touches only a non-indexed, non-FK field that is not a parent reference column, then FK projection is skipped and the visible FK graph is unchanged.

What These Tests Prove

• Integrity: no surviving visible FK violation.
• Convergence: same merged facts produce the same projection on every replica.
• ACI merge behavior: batching, replay, and merge order do not change the final visible result.
• Nested safety: cascade cannot bypass restrict/no-action descendants.
• Local action fidelity: local deletes preserve database-style FK side effects as authored CRDT facts.
• Projection discipline: merge-time SET NULL and SET DEFAULT repairs do not become ordinary user CRDT writes.