261 lines
13 KiB
Markdown
261 lines
13 KiB
Markdown
# Preserving foreign key invariants while merging
|
|
|
|
## Summary
|
|
|
|
After merging remote CRDT operations, the visible database must not contain
|
|
foreign key violations. The merge result must be associative, commutative, and
|
|
idempotent: the same merged CRDT facts must produce the same visible rows and
|
|
visible FK values regardless of arrival order, batching, or replay.
|
|
|
|
The safe merge model is projection-based. Local ORM operations are still
|
|
authored user actions: when a local delete is intercepted for soft delete, the
|
|
database-style `ON DELETE` side effects are materialized as CRDT facts in the
|
|
same local transaction. FK repairs created only because independently authored
|
|
facts were merged are deterministic materialized projections over those facts,
|
|
not new user field updates emitted opportunistically during merge.
|
|
|
|
## Core Policy
|
|
|
|
- Keep every row as a CRDT fact. Visibility is derived from row tombstones and
|
|
schema-driven FK projection.
|
|
- Supported foreign keys are single-column child references to single-column
|
|
parent references. Schemas with composite foreign keys must fail recorder
|
|
initialization instead of silently bypassing projection.
|
|
- Merge input is causally complete: the sync protocol delivers each batch as a
|
|
full causal snapshot of the sender, so every merged FK attempt's parent facts
|
|
precede or accompany the child fact. Input that violates this precondition
|
|
(for example, a non-nullable FK attempt whose parent fact is absent and whose
|
|
default target is missing) is outside the merge contract and fails the merge
|
|
transaction atomically against the database foreign key constraints.
|
|
- Local `ON DELETE` actions are authored CRDT facts. `RESTRICT` and `NO ACTION`
|
|
reject the local delete while visible children exist; `SET NULL` and
|
|
`SET DEFAULT` update child FK fields and their ordinary `CrdtDataField` HLCs;
|
|
`CASCADE` records user-delete tombstones for the visible cascade descendants.
|
|
- Keep the user/attempted FK value as the CRDT field value for authored
|
|
operations.
|
|
- During merge projection, materialize a separate visible FK value only when FK
|
|
repair requires it. `CrdtDataForeignKey.attemptedValue` is the durable FK
|
|
field-value payload for FK columns because `CrdtDataField` stores only HLC
|
|
metadata. `visibleValue`, `hasOverride`, and `overrideReason` are projection
|
|
metadata, not business conflict classes.
|
|
- Do not create ordinary user field updates for `SET NULL` or `SET DEFAULT`
|
|
repairs during merge. Doing so makes repair HLCs depend on merge timing. This
|
|
does not apply to locally initiated `SET NULL` or `SET DEFAULT` actions,
|
|
which are authored user field updates.
|
|
- For FK-affecting operations, recompute FK projection to the same fixed point
|
|
as the affected FK closure. A full graph recomputation is a valid
|
|
implementation strategy and oracle as long as it produces the same projection.
|
|
Operations that touch no child FK column, parent reference column, insert, or
|
|
tombstone must not trigger FK projection.
|
|
|
|
## Local Action Semantics
|
|
|
|
Local deletes preserve database action semantics even though the physical row is
|
|
soft-deleted instead of removed:
|
|
|
|
- `ON DELETE RESTRICT` and `NO ACTION`: the delete fails if any visible child
|
|
still references the parent. No parent tombstone is recorded for the failed
|
|
operation.
|
|
- `ON DELETE SET NULL`: visible child FK columns are updated to `null` when the
|
|
FK is nullable, and those child FK fields receive ordinary CRDT field updates.
|
|
- `ON DELETE SET DEFAULT`: visible child FK columns are updated to the column
|
|
default when the default is legal, and those child FK fields receive ordinary
|
|
CRDT field updates.
|
|
- `ON DELETE CASCADE`: visible cascade descendants receive synced
|
|
user-delete tombstones. Local cascade descendants are not hidden only as
|
|
`foreignKeyCascade` projection rows.
|
|
|
|
These local side effects are the facts that other replicas merge.
|
|
|
|
## Merge-Time Action Semantics
|
|
|
|
For each child FK whose attempted value points to a hidden or missing parent:
|
|
|
|
- `ON DELETE SET NULL`: the visible FK value is `null` when the FK is nullable.
|
|
If the FK is non-nullable, this action cannot repair the row and must fall
|
|
through to the deterministic blocking policy.
|
|
- `ON DELETE SET DEFAULT`: the visible FK value is the column default only if
|
|
that default points to a visible parent or is a legal nullable default. If the
|
|
default target is hidden or missing, this action cannot repair the row and
|
|
must fall through to the deterministic blocking policy.
|
|
- `ON DELETE RESTRICT` and `NO ACTION`: the parent delete loses while a visible
|
|
child still references the parent. The parent remains visible in the derived
|
|
projection.
|
|
- `ON DELETE CASCADE`: the child can be hidden only if the whole cascade closure
|
|
is valid. If any descendant blocks the cascade through `RESTRICT`, `NO ACTION`,
|
|
invalid `SET NULL`, or invalid `SET DEFAULT`, the ancestor delete loses and
|
|
the cascade is not partially applied.
|
|
|
|
This makes mixed chains atomic for visibility. For example, if deleting `A`
|
|
would cascade-delete `B`, but `B` has a visible `RESTRICT` child `C`, then `A`,
|
|
`B`, and `C` remain visible. The result must not depend on whether cascade or
|
|
restrict edges are evaluated first internally.
|
|
|
|
## Projection Metadata
|
|
|
|
`CrdtDataForeignKey` should represent the FK field payload and materialized FK
|
|
projection for one `CrdtDataField`:
|
|
|
|
- `attemptedValue`: the current user-attempted FK value carried by the CRDT
|
|
field. It may be null for nullable FKs. This is durable FK field-value storage
|
|
and is part of the CRDT facts for FK columns.
|
|
- `visibleValue`: the value currently materialized into the domain row when an
|
|
override is active (`overrideReason` is non-null).
|
|
- `overrideReason`: the authoritative stored indicator that an override is
|
|
active, and the cause. Non-null means an override is active; null means no
|
|
override (the domain row carries `attemptedValue`). Values: `setNull` — the
|
|
parent was hidden/deleted and the column was set null by the FK action;
|
|
`setDefault` — the column was set to a default FK value; `missingParent` —
|
|
the parent is hidden or missing and cannot be repaired, so the row is hidden
|
|
by projection. The derived `hasOverride` bool (`overrideReason != null`) is
|
|
available as a convenience getter.
|
|
|
|
`visibleValue` is meaningful only when `overrideReason` is non-null. When
|
|
`overrideReason` is null the domain row carries `attemptedValue` directly; when
|
|
`overrideReason` is non-null and `visibleValue` is null it means "materialize
|
|
null".
|
|
|
|
The resolver may use `visibleValue` and `overrideReason` for efficient
|
|
materialization and write deduplication. It must not use those projection fields
|
|
as the authority for FK conflict decisions; the authority is the deterministic
|
|
resolver over merged CRDT row, field, tombstone, and FK attempt facts.
|
|
|
|
A local full-row update (no column narrowing) that writes an FK value equal to
|
|
the materialized projected value is a passthrough of the repair, not an authored
|
|
FK change: the existing override and attempted value are preserved and the FK
|
|
field HLC does not advance. Writing a different FK value, or updating the FK
|
|
column with narrowed columns (even when the written value equals the projected
|
|
value), authors the change as an ordinary user field fact: the FK field HLC
|
|
advances, `attemptedValue` becomes the new value, and `overrideReason` becomes
|
|
null.
|
|
|
|
## Merge Pipeline
|
|
|
|
1. Merge CRDT row, field, and tombstone facts by their existing HLC/CLFlag
|
|
rules.
|
|
2. Identify whether inserts, updates, deletes, FK columns, parent reference
|
|
columns, or existing FK projection records can affect FK state. Skip FK
|
|
projection when the merge touches none of them.
|
|
3. For FK-affecting work, expand to the FK closure in both directions: parents
|
|
needed by children and children affected by parent visibility. A full graph
|
|
recomputation may be used instead of the minimal closure.
|
|
4. Compute effective row visibility and visible FK values to a fixed point.
|
|
5. Apply only the materialized differences:
|
|
- update domain FK columns when projection differs from attempted value;
|
|
- update `CrdtDataForeignKey` projection metadata;
|
|
- update system tombstones only for deterministic derived visibility where
|
|
that is part of the chosen storage model.
|
|
6. Re-run unique conflict projection after FK projection if FK repairs can
|
|
change unique columns, or define the opposite stage order and prove it with
|
|
the full recomputation oracle.
|
|
|
|
## Invariants
|
|
|
|
- No visible row references a hidden or missing parent.
|
|
- No cascade delete is partially applied across a blocked nested relation.
|
|
- Merge-time `SET NULL` and `SET DEFAULT` repairs are pure functions of merged
|
|
facts. Local `SET NULL` and `SET DEFAULT` actions are authored field facts.
|
|
- Local cascade deletes produce synced user-delete tombstones for cascade
|
|
descendants; merge-time cascade projection may derive `foreignKeyCascade`
|
|
visibility for descendants without authored tombstones.
|
|
- Replaying the same merge batch produces no metadata churn.
|
|
- Splitting a merge into smaller batches converges to the same final projection
|
|
as merging the same operations in one batch.
|
|
- FK-affecting recomputation over the affected closure matches full global
|
|
recomputation; unrelated non-FK updates do not trigger FK projection.
|
|
|
|
## Test Cases
|
|
|
|
### Local CRUD Actions
|
|
|
|
- Given a visible `RESTRICT` or `NO ACTION` child, when the parent is deleted
|
|
locally, then the delete fails and no parent tombstone is recorded.
|
|
- Given a nullable `SET NULL` child, when the parent is deleted locally, then the
|
|
child FK is set to `null` as an authored field update.
|
|
- Given a valid `SET DEFAULT` child, when the parent is deleted locally, then
|
|
the child FK is set to the default as an authored field update.
|
|
- Given a valid cascade chain, when the root is deleted locally, then every
|
|
visible cascade descendant receives a user-delete tombstone.
|
|
|
|
### Restrict and No Action
|
|
|
|
1. Given a parent and a visible child with `ON DELETE RESTRICT`, when one replica
|
|
deletes the parent and another concurrently updates the child, then the parent
|
|
remains visible after convergence and no visible orphan appears.
|
|
|
|
2. Given a parent with multiple visible restrict children, when the parent is
|
|
deleted concurrently with updates to different children, then any visible
|
|
child is enough to keep the parent visible on every replica.
|
|
|
|
3. Given a parent kept visible because of restrict, when all restrict children
|
|
are later deleted or detached, then the parent visibility follows the same
|
|
deterministic recomputation rule on every replica.
|
|
|
|
### Set Null and Set Default
|
|
|
|
4. Given a nullable FK with `ON DELETE SET NULL`, when the parent is deleted on
|
|
another replica, then the child remains visible with a materialized null FK
|
|
and the user/attempted FK value remains unchanged as a CRDT fact.
|
|
|
|
5. Given a child whose FK was materialized to null by `SET NULL`, when the user
|
|
later changes the FK to a visible parent, then the projection override becomes
|
|
inactive and the domain row materializes the new user value.
|
|
|
|
6. Given `ON DELETE SET DEFAULT` and a visible default target, when the attempted
|
|
parent is deleted, then the child remains visible with the default FK
|
|
materialized.
|
|
|
|
7. Given `ON DELETE SET DEFAULT` and a hidden or missing default target, when the
|
|
attempted parent is deleted, then the action cannot repair the child and the
|
|
deterministic blocking policy keeps the necessary parent visible.
|
|
|
|
### Cascade
|
|
|
|
8. Given a parent delete fact merged from another replica with children under
|
|
`ON DELETE CASCADE`, then all cascade descendants in the valid cascade
|
|
closure become hidden and no visible descendant references a hidden ancestor.
|
|
|
|
9. Given a child already hidden by a cascade projection, when the same delete
|
|
facts are merged again, then no additional metadata churn occurs.
|
|
|
|
### Mixed Nested Relations
|
|
|
|
10. Given `A -> B` is cascade and `B <- C` is restrict, when `A` is deleted
|
|
concurrently with an update to `C`, then the cascade is blocked and `A`,
|
|
`B`, and `C` remain visible.
|
|
|
|
11. Given a fourth-degree chain with alternating cascade, restrict, set-null,
|
|
and set-default edges, when a merge touches both a root and a leaf, then the
|
|
affected-closure result and a full fixed-point recomputation produce the
|
|
same visible graph and FK projection.
|
|
|
|
12. Given an FK cycle that the engine permits, when concurrent deletes and
|
|
updates happen around the cycle, then fixed-point evaluation terminates and
|
|
every replica computes the same projection.
|
|
|
|
### Cross-Cutting
|
|
|
|
13. Given an FK repair changes a unique-indexed column, when unique projection
|
|
and FK projection are recomputed, then the documented stage order prevents
|
|
visible unique and FK violations.
|
|
|
|
14. Given the same remote operations arrive in one batch or several smaller
|
|
batches, when all operations have merged, then visible rows, visible FK
|
|
values, FK projection metadata, unique projection, and tombstones are the
|
|
same.
|
|
|
|
15. Given a merge touches only a non-indexed, non-FK field that is not a parent
|
|
reference column, then FK projection is skipped and the visible FK graph is
|
|
unchanged.
|
|
|
|
## What These Tests Prove
|
|
|
|
- Integrity: no surviving visible FK violation.
|
|
- Convergence: same merged facts produce the same projection on every replica.
|
|
- ACI merge behavior: batching, replay, and merge order do not change the final
|
|
visible result.
|
|
- Nested safety: cascade cannot bypass restrict/no-action descendants.
|
|
- Local action fidelity: local deletes preserve database-style FK side effects
|
|
as authored CRDT facts.
|
|
- Projection discipline: merge-time `SET NULL` and `SET DEFAULT` repairs do not
|
|
become ordinary user CRDT writes.
|