halkyon-days/serverpod-offline-sync
Watch
1
0
Fork
You've already forked serverpod-offline-sync
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat: Implements foreign key handling (#68)

Browse source
This commit is contained in:
Marcelo Mendonça Soares 2026-06-10 17:31:05 -03:00 • committed by GitHub
commit 4580326956
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
39 changed files with 6630 additions and 189 deletions
Download patch file Download diff file Expand all files Collapse all files

237
docs/foreign-key-invariants-divergence-report.md Normal file
View file

@ -0,0 +1,237 @@
# Foreign key invariant implementation review
Reviewed against `docs/foreign-key-invariants.md` on branch `foreign-key-handling` at `6834d7e`.
Updated on the current branch after applying the local/merge semantic
clarification and the FK projection fixes.
## Summary for distribution
The branch now aligns with the clarified spec on the major FK invariant points:
1. **Local and merge FK semantics are intentionally different and implemented.** Local `RESTRICT` / `NO ACTION`, `SET NULL`, `SET DEFAULT`, and `CASCADE` are material user actions. Merge-time FK repairs remain projection-only and do not advance ordinary `CrdtDataField` HLCs.
2. **Missing-parent merge cases are handled before physical FK writes.** Incoming FK attempts are recorded in `CrdtDataForeignKey.attemptedValue`, while the constrained domain column receives a safe visible value before the write.
3. **FK attempted-value storage is explicit.** `attemptedValue` is durable FK field-value storage; `visibleValue`, `hasOverride`, and `overrideReason` are projection metadata.
4. **Projection recomputation is clarified.** FK-affecting work may use full fixed-point recomputation as a valid strategy/oracle, while unrelated non-FK updates skip FK projection.
5. **The targeted spec coverage has been expanded.** The suite now covers local cascade user tombstones, multiple restrict children, invalid non-null `SET NULL`, missing-parent repairs, unique FK repair, replay metadata churn, batching convergence, and a permitted FK cycle.
6. **The main easy performance cleanups are implemented.** CRDT field upsert logic is shared, projection rows are upserted in batches, parent/child projection lookups are indexed, loaded CRDT rows are reused for visibility writes, FK materialization writes are batched, FK edges are precomputed by parent/child table, list updates project once, and sync avoids FK projection N+1 lookups.
The detailed items below retain the original review structure but now record the updated status.
## Correctness work item status
### FK-1: Local cascade should be material; merge projection should stay non-authoring
**Spec reference:** Core Policy and Local Action Semantics: local `ON DELETE` actions are authored CRDT facts. `RESTRICT` / `NO ACTION` reject the local delete, `SET NULL` / `SET DEFAULT` update child FK fields and ordinary field HLCs, and `CASCADE` records user-delete tombstones for visible cascade descendants. Merge-time FK repairs remain deterministic projection and must not create ordinary user field updates.
**Current code after fix:**
- `CrdtMutationRecorder.insteadOfDelete` calls `_softDeleteRowsByTable`, then `_projectForeignKeys` (`packages/serverpod_offline_sync_client/lib/src/database/recorder.dart:620-634`).
- `_softDeleteRowsByTable` calls `_applyForeignKeyDeleteActions` before marking the parent deleted, releases unique conflicts, records the parent tombstone, and recursively soft-deletes cascade descendants with the same user-delete reason.
- `_applyForeignKeyDeleteActions` throws for `RESTRICT` / `NO ACTION`, mutates child FK columns for `SET NULL` / `SET DEFAULT`, records ordinary CRDT field updates through `recordFieldsUpdatedByTable`, and returns `CASCADE` descendants for recursive user tombstones. Under the revised spec, all of that local behavior is intended.
- CRUD tests assert local cascade user-delete tombstones while preserving local restrict/set-null/set-default material-action expectations (`test/serverpod_offline_sync_test_server/test/integration/crud/delete_test.dart`).
**Status after fix:**
Resolved. The reviewed branch already treated local `SET NULL`, `SET DEFAULT`, `RESTRICT`, and `NO ACTION` as material database-style actions. Local `CASCADE` now also authors synced `CrdtDataDeletedReason.userDelete` tombstones for visible cascade descendants instead of relying on projection-only `foreignKeyCascade` visibility.
The inverse risk also matters: when projection runs after a remote merge, its `SET NULL` / `SET DEFAULT` materialization must not call `recordFieldsUpdatedByTable` or otherwise mint new user field HLCs. Those repairs are merge-time projections, not local authored actions.
**Scenarios verified:**
- Local delete of `City` with visible `Organization` and `Person` descendants under `ON DELETE CASCADE` should leave descendant domain rows present but attach `CrdtDataDeletedReason.userDelete` tombstones to every cascade descendant and set their visibility to `userDelete`.
- Local delete of `Person` referenced by `Town.mayorId ON DELETE SET NULL` should continue to leave `Town` visible with `mayorId == null` and a newer ordinary `Town.mayorId` `CrdtDataField` HLC.
- Local delete of `Town` referenced by `Company.townId ON DELETE SET DEFAULT` should continue to materialize the default as an ordinary child FK field update.
- Local delete of a `RESTRICT` / `NO ACTION` parent should continue to fail while visible children exist and should not record a parent tombstone for the failed operation.
- Remote/merge-time `SET NULL` and `SET DEFAULT` repairs should still update only the materialized domain FK value and `CrdtDataForeignKey` projection metadata, without bumping ordinary child `CrdtDataField` HLCs.
**Implemented direction:**
Keep local `RESTRICT`, `SET NULL`, and `SET DEFAULT` as material actions. Add recursive local cascade handling that records user-delete tombstones for visible cascade descendants. Keep merge-time projection pure: do not record ordinary field updates from `_materializeForeignKeyValues`, and keep derived cascade visibility as `foreignKeyCascade` only for rows hidden by merged facts rather than the current local transaction.
**Stash `e77b65745ee6277d7e695417547ac47ec60acd5f` evaluation:** not sufficient as-is. Directionally, the stash moves local cascade toward authored tombstones by adding `_cascadeUserDeleteFacts` and updating the cascade CRUD expectations, but it regresses two clarified requirements:
- It removes the existing local `SET NULL` / `SET DEFAULT` material actions from the soft-delete path instead of preserving them.
- It adds `recordFieldsUpdatedByTable` calls inside `_materializeForeignKeyValues`, which turns remote/merge-time `SET NULL` and `SET DEFAULT` projection repairs into ordinary user field updates and makes repair HLCs depend on merge timing.
It also removes `_foreignKeysByReferencedTable`, which removes the existing composite-FK initialization guard and leaves composite FKs silently ignored by `_foreignKeyEdges`.
### FK-2: Missing-parent merge operations can fail before projection can repair them
**Spec reference:** Merge-Time Action Semantics and Invariants: for attempted FK values that point to a hidden or missing parent, projection must produce no visible orphan; merge-time `SET NULL` / `SET DEFAULT` repairs are pure functions of merged facts.
**Current code after fix:**
- Merge insert/update paths call `_safeIncomingForeignKeyData` before writing FK columns to constrained domain tables.
- The original incoming FK value is recorded as an attempted value for `CrdtDataForeignKey.attemptedValue`.
- Repairable missing-parent `SET NULL` / `SET DEFAULT` writes use a safe visible value before the physical write. Nullable unrepairable `RESTRICT` / `NO ACTION` / `CASCADE` missing-parent writes use `null` as the physical value and projection hides the row so no visible orphan remains.
- Local user updates are still blocked by `_assertVisibleForeignKeyTargets`, which throws when the updated FK points at a missing or hidden parent.
**Status after fix:**
Resolved. The projection model now accepts the merged attempted FK fact without forcing that unsafe value into the constrained domain column first. Repairable actions materialize a safe visible value; unrepairable missing-parent attempts are hidden deterministically.
**Scenarios verified:**
- Remote `Town` insert/update with `mayorId` pointing at a parent row that is absent locally should converge to a visible town with a materialized `null` mayor and the attempted id preserved.
- Remote `Company` insert/update with `townId` pointing at an absent attempted town and a visible default town should converge to the visible default, not throw during the write.
- Remote child insert/update under `RESTRICT` / `NO ACTION` / `CASCADE` with a missing parent must follow an explicit deterministic policy that satisfies “no visible row references a hidden or missing parent.” The current spec implies no visible orphan, but the implementation needs a concrete policy for the “parent fact is absent” case.
**Implemented direction:**
Do not write an unsafe attempted FK into the physical FK column before projection. Options include computing the visible FK value before the domain write for FK columns, staging attempted values outside the constrained domain column, deferring constraints where supported, or making the attempted value durable in CRDT metadata and writing only the projected visible value to the domain table.
### FK-3: FK attempted-value storage must be explicit
**Spec reference:** Projection Metadata: `CrdtDataForeignKey.attemptedValue` is the durable FK field-value payload for FK columns because `CrdtDataField` stores only HLC metadata. `visibleValue`, `hasOverride`, and `overrideReason` are projection metadata and must not be used as business conflict authority.
**Current code:**
- `_attemptedForeignKeyValue` returns `projection.attemptedValue` whenever an override is active, otherwise it reads the current domain column (`packages/serverpod_offline_sync_client/lib/src/database/merge_utils/foreign_key_projector.dart:730-746`).
- `_childrenReferencingParent` and closure/blocking decisions use `_attemptedForeignKeyValue` (`foreign_key_projector.dart:798-811`).
- Sync also relies on projection metadata to reconstruct attempted values: `_applyProjectedForeignKeyAttempts` replaces row insert payload columns with `projection.attemptedValue`, and streamed update fields include `foreignKey` projection rows to return attempted values when an override is active.
**Status after spec clarification:**
Once `SET NULL` or `SET DEFAULT` is materialized, the physical domain column no longer contains the attempted FK. `CrdtDataField` only stores HLC metadata, not the attempted value. Therefore `CrdtDataForeignKey.attemptedValue` is explicitly the durable attempted FK value. This removes the earlier spec/code contradiction.
The remaining correctness requirement is that merge projection only updates the materialized domain value plus projection fields. It must not advance the ordinary `CrdtDataField` HLC for a repaired FK. The test suite now includes a set-null merge repair regression that snapshots the child FK field HLC and verifies projection does not advance it.
**Documented direction:**
Keep this storage model documented in the spec and model comment. Treat `CrdtDataForeignKey.attemptedValue` as FK field-value storage and keep `visibleValue`, `hasOverride`, and `overrideReason` as recomputable projection metadata.
### FK-4: `visibleValue` is populated when `hasOverride == false`
**Spec reference:** Projection Metadata: `visibleValue` is meaningful only with `hasOverride`; when `hasOverride` is false, it means “same as attempted.”
**Current code after fix:**
- `_recordForeignKeyAttemptsForRows` and `_materializeForeignKeyValues` write `visibleValue == null` when `hasOverride == false`.
- Tests now assert inactive projections have `visibleValue == null`.
**Status after fix:**
Resolved. Inactive projections no longer duplicate the attempted value in `visibleValue`; `visibleValue == null` is interpreted with `hasOverride`.
**Implemented direction:**
Store `visibleValue == null` whenever `hasOverride == false`; tests assert that inactive projections leave `visibleValue` empty.
### FK-5: FK-affecting projection uses full fixed-point recomputation
**Spec reference:** Merge Pipeline steps 2-4 and invariant/test 15: FK-affecting work must compute the same fixed point as the affected FK closure. A full graph recomputation is valid for FK-affecting work; unrelated non-FK updates must skip FK projection.
**Current code after fix:**
- `_projectForeignKeys` still computes a full fixed point for FK-affecting operations, which is now an allowed implementation strategy and oracle.
- `_tableColumnsMayAffectForeignKeys` and `_mergeOperationsMayAffectForeignKeys` skip projection for updates that touch no child FK column or parent reference column.
- `Database.update(List<T>)` performs row writes in one transaction and calls `afterUpdate` once, so list updates trigger at most one projection pass.
- Projection state now indexes parent and child lookups and batches projection/domain writes, reducing the cost of full fixed-point recomputation.
**Status after spec clarification and fix:**
Resolved against the clarified spec. The implementation uses full fixed-point recomputation only when an operation can affect FK state, and skips FK projection for unrelated non-FK updates. A narrower affected-closure loader remains a possible optimization, not a correctness requirement.
**Implemented direction:**
Gate projection by FK-affecting tables/columns; keep full recomputation as the fixed-point oracle for FK-affecting work; reduce full recomputation cost with indexed state and batched writes.
### FK-6: Composite foreign keys must be explicitly unsupported
**Spec reference:** Core Policy: supported foreign keys are single-column child references to single-column parent references. Schemas with composite foreign keys must fail recorder initialization instead of silently bypassing projection.
**Current code:**
- `_foreignKeysByReferencedTable` throws during recorder initialization if any target table definition contains a composite FK (`packages/serverpod_offline_sync_client/lib/src/database/recorder.dart:95-106`).
- `_foreignKeyEdges` silently includes only FKs whose `columns.length == 1` and `referenceColumns.length == 1` (`recorder.dart:119-135`).
**Status after spec clarification:**
A schema with a composite FK fails during recorder initialization through `_foreignKeysByReferencedTable`. This is now documented as an explicit limitation rather than a spec divergence. Later refactors must keep that guard so composite FKs are not silently ignored by `_foreignKeyEdges`.
**Documented direction:**
Keep the initialization validation and add/keep a schema test that proves composite FKs fail fast if such a model is introduced.
### FK-7: Unique/FK stage order is only partially implemented and not proven by tests
**Spec reference:** Merge Pipeline step 6 and test 13: if FK repairs can change unique columns, recompute unique projection after FK projection or define/prove the opposite order.
**Current code after fix:**
- Unique conflict resolution runs before FK projection for incoming inserts/updates (`merge.dart:273-345`).
- `_materializeForeignKeyValues` runs `_resolveUniqueConflictsAfterForeignKeyProjection` only for rows whose visible FK value changed (`foreign_key_projector.dart:588-679`).
- The test schema now includes a unique nullable `SET NULL` FK via `UniqueSetNullChild.parentId`.
**Status after fix:**
Resolved for the covered nullable unique `SET NULL` case. The code keeps the pre-FK unique pass and the post-FK unique pass for rows whose FK projection changed; the integration suite now proves the repaired unique FK becomes visible as `null` with the attempted value preserved.
**Implemented direction:**
Added a test model with a unique nullable `SET NULL` FK and covered the post-FK unique pass for that repair path. Batching convergence is covered separately for the repaired `SET NULL` graph.
## Spec coverage status in tests
The integration suite now covers the high-risk cases from the review:
- **Test 2:** multiple visible restrict children where any one child keeps the parent visible.
- **Local cascade:** local cascade deletes assert synced user-delete tombstones on cascade descendants.
- **Test 3:** restrict child detachment after a blocked parent delete restores the preserved delete projection.
- **Test 4/6/7 merge projection discipline:** remote `SET NULL` repairs assert that ordinary child `CrdtDataField` HLCs are not advanced by projection.
- **Invalid `SET NULL`:** a non-nullable `ON DELETE SET NULL` relation proves deterministic blocking.
- **Missing-parent repair:** remote insert/update references to absent parents cover repairable `SET NULL` / `SET DEFAULT` and unrepairable `RESTRICT` behavior before domain writes.
- **Replay metadata churn:** cascade replay snapshots both `CrdtDataRow.visibility` and `CrdtDataForeignKey` projection rows.
- **Test 12:** an existing permitted data cycle across `Person`, `Company`, and `Town` proves fixed-point evaluation terminates.
- **Test 13:** a unique-indexed nullable `SET NULL` FK repair is covered.
- **Batching convergence:** one-batch and split-batch merges converge for a repaired `SET NULL` graph.
Remaining optional coverage candidates:
- A dedicated fourth-degree chain comparing a hand-minimized affected closure to full recomputation. The current spec allows full recomputation as the implementation strategy.
- A synthetic composite-FK schema test. Composite FKs are explicitly unsupported and the initialization guard is retained, but the test server does not include a composite FK model.
## Duplication and easy performance cleanup items
### PERF-1: Extract duplicated CRDT field upsert logic
Resolved. `_recordUpdatedFields` and `recordFieldsUpdatedByTable` now share `_upsertCrdtFieldsForRows(tableName, crdtRows, schemaColumns, skippedFields?)`. This also keeps merge-time FK projection away from ordinary field-HLC authoring.
### PERF-2: Batch projection upserts instead of querying one row at a time
Resolved. `_ForeignKeyProjectionRow` now carries the projection row id and override reason, and `_upsertForeignKeyProjections` batches inserts/updates using either loaded projection state or one bulk lookup for missing field ids.
### PERF-3: Index parent and child lookups inside projection state
Resolved. `_ForeignKeyProjectionState` now builds `(parentTable, parentColumn, value) -> row` and `(childTable, childColumn, attemptedValue) -> children` maps once, and hot closure/blocking/materialization paths use those indexes.
### PERF-4: Avoid re-querying CRDT rows already present in projection state
Resolved. `_ProjectedForeignKeyRow` now carries the loaded `CrdtDataRow` and current visibility, so `_setProjectedRowVisibility` can update without re-querying rows.
### PERF-5: Batch domain FK materialization writes
Resolved for equal update sets. `_materializeForeignKeyValues` collects domain FK changes, groups rows by table and identical update map, writes each group once, then runs the post-FK unique pass.
### PERF-6: Precompute foreign-key edges by parent and child table
Resolved. The recorder now precomputes `_foreignKeyEdgesByParentTable` and `_foreignKeyEdgesByChildTable`, and projector paths use them for child-table and parent-table loops.
### PERF-7: Avoid full projection per row in `Database.update(List<T>)`
Resolved at the recorder/projection layer. `CrdtDatabase.update(List<T>)` still applies the tombstone-aware row update predicate per row, but it records CRDT updates once for the full list and therefore runs FK projection at most once.
### PERF-8: Avoid N+1 sync projection lookups
Resolved for FK projection lookups. `_streamUpdates` includes `foreignKey` with each `CrdtDataField`, and `_streamInserts` preloads active projected FK attempt fields for all streamed CRDT rows.
## Validation notes
- Ran `dart analyze packages/serverpod_offline_sync_client test/serverpod_offline_sync_test_server --fatal-infos --fatal-warnings`; no issues found.
- Ran `dart test test/serverpod_offline_sync_test_server/test/integration/merge/foreign_key_invariant_test.dart --reporter=expanded --concurrency=1`; it passed all 36 tests.
- Ran `dart test test/serverpod_offline_sync_test_server/test/integration/crud/delete_test.dart --reporter=expanded --concurrency=1`; it passed all 20 tests.
- Ran `dart test test/serverpod_offline_sync_test_server/test/integration/crud --reporter=expanded --concurrency=1`; it passed all 58 tests.
- Ran `dart test test/serverpod_offline_sync_test_server/test/integration/merge --reporter=expanded --concurrency=1`; it passed all 71 tests.
- Ran `dart test test/serverpod_offline_sync_test_server/test/integration/database --reporter=expanded --concurrency=1`; it passed all 3 tests.
- Ran `dart test test/serverpod_offline_sync_test_server/test/integration/sync/sync_roundtrip_test.dart --reporter=expanded --concurrency=1`; it passed all 6 tests, including projected FK attempted-value insert/update payload roundtrips.
- Attempted `dart test test/serverpod_offline_sync_test_server/test/integration/sync --reporter=expanded --concurrency=1`; `sync_roundtrip_test.dart`, `sync_merge_helpers_test.dart`, and `sync_hash_test.dart` passed, but `sync_flow_test.dart` and `sync_unique_conflict_test.dart` failed during load with `Database is not available in this session` from the `withServerpod` test harness before test bodies executed.

161
docs/foreign-key-invariants.md
View file

@ -7,23 +7,67 @@ foreign key violations. The merge result must be associative, commutative, and
idempotent: the same merged CRDT facts must produce the same visible rows and
visible FK values regardless of arrival order, batching, or replay.
The safe model is projection-based. User rows, fields, and tombstones remain the
CRDT facts. FK repairs are deterministic materialized projections over those
facts, not new user field updates emitted opportunistically during merge.
The safe merge model is projection-based. Local ORM operations are still
authored user actions: when a local delete is intercepted for soft delete, the
database-style `ON DELETE` side effects are materialized as CRDT facts in the
same local transaction. FK repairs created only because independently authored
facts were merged are deterministic materialized projections over those facts,
not new user field updates emitted opportunistically during merge.
## Core Policy
- Keep every row as a CRDT fact. Visibility is derived from row tombstones and
schema-driven FK projection.
- Keep the user/attempted FK value as the CRDT field value.
- Materialize a separate visible FK value only when FK repair requires it. Use
`CrdtDataForeignKey` as projection metadata, not as a business conflict class.
- Supported foreign keys are single-column child references to single-column
parent references. Schemas with composite foreign keys must fail recorder
initialization instead of silently bypassing projection.
- Merge input is causally complete: the sync protocol delivers each batch as a
full causal snapshot of the sender, so every merged FK attempt's parent facts
precede or accompany the child fact. Input that violates this precondition
(for example, a non-nullable FK attempt whose parent fact is absent and whose
default target is missing) is outside the merge contract and fails the merge
transaction atomically against the database foreign key constraints.
- Local `ON DELETE` actions are authored CRDT facts. `RESTRICT` and `NO ACTION`
reject the local delete while visible children exist; `SET NULL` and
`SET DEFAULT` update child FK fields and their ordinary `CrdtDataField` HLCs;
`CASCADE` records user-delete tombstones for the visible cascade descendants.
- Keep the user/attempted FK value as the CRDT field value for authored
operations.
- During merge projection, materialize a separate visible FK value only when FK
repair requires it. `CrdtDataForeignKey.attemptedValue` is the durable FK
field-value payload for FK columns because `CrdtDataField` stores only HLC
metadata. `visibleValue`, `hasOverride`, and `overrideReason` are projection
metadata, not business conflict classes.
- Do not create ordinary user field updates for `SET NULL` or `SET DEFAULT`
repairs during merge. Doing so makes repair HLCs depend on merge timing.
- Recompute FK projection from merged facts over the affected FK closure. The
incremental closure result must match a full recomputation oracle.
repairs during merge. Doing so makes repair HLCs depend on merge timing. This
does not apply to locally initiated `SET NULL` or `SET DEFAULT` actions,
which are authored user field updates.
- For FK-affecting operations, recompute FK projection to the same fixed point
as the affected FK closure. A full graph recomputation is a valid
implementation strategy and oracle as long as it produces the same projection.
Operations that touch no child FK column, parent reference column, insert, or
tombstone must not trigger FK projection.
## Action Semantics
## Local Action Semantics
Local deletes preserve database action semantics even though the physical row is
soft-deleted instead of removed:
- `ON DELETE RESTRICT` and `NO ACTION`: the delete fails if any visible child
still references the parent. No parent tombstone is recorded for the failed
operation.
- `ON DELETE SET NULL`: visible child FK columns are updated to `null` when the
FK is nullable, and those child FK fields receive ordinary CRDT field updates.
- `ON DELETE SET DEFAULT`: visible child FK columns are updated to the column
default when the default is legal, and those child FK fields receive ordinary
CRDT field updates.
- `ON DELETE CASCADE`: visible cascade descendants receive synced
user-delete tombstones. Local cascade descendants are not hidden only as
`foreignKeyCascade` projection rows.
These local side effects are the facts that other replicas merge.
## Merge-Time Action Semantics
For each child FK whose attempted value points to a hidden or missing parent:
@ -49,32 +93,52 @@ restrict edges are evaluated first internally.
## Projection Metadata
`CrdtDataForeignKey` should represent the materialized FK projection for one
`CrdtDataField`:
`CrdtDataForeignKey` should represent the FK field payload and materialized FK
projection for one `CrdtDataField`:
- `attemptedValue`: the current user-attempted FK value carried by the CRDT
field. It may be null for nullable FKs.
field. It may be null for nullable FKs. This is durable FK field-value storage
and is part of the CRDT facts for FK columns.
- `visibleValue`: the value currently materialized into the domain row when an
override is active.
- `hasOverride`: whether the projection overrides `attemptedValue` and
materializes `visibleValue` instead.
override is active (`overrideReason` is non-null).
- `overrideReason`: the authoritative stored indicator that an override is
active, and the cause. Non-null means an override is active; null means no
override (the domain row carries `attemptedValue`). Values: `setNull` — the
parent was hidden/deleted and the column was set null by the FK action;
`setDefault` — the column was set to a default FK value; `missingParent` —
the parent is hidden or missing and cannot be repaired, so the row is hidden
by projection. The derived `hasOverride` bool (`overrideReason != null`) is
available as a convenience getter.
`visibleValue == null` is meaningful only with `hasOverride`. When
`hasOverride` is false it means "same as attempted"; when `hasOverride` is true
it means "materialize null".
`visibleValue` is meaningful only when `overrideReason` is non-null. When
`overrideReason` is null the domain row carries `attemptedValue` directly; when
`overrideReason` is non-null and `visibleValue` is null it means "materialize
null".
The resolver may use this metadata for efficient materialization and write
deduplication. It must not use it as the authority for FK conflict decisions;
the authority is the deterministic resolver over merged CRDT facts.
The resolver may use `visibleValue` and `overrideReason` for efficient
materialization and write deduplication. It must not use those projection fields
as the authority for FK conflict decisions; the authority is the deterministic
resolver over merged CRDT row, field, tombstone, and FK attempt facts.
A local full-row update (no column narrowing) that writes an FK value equal to
the materialized projected value is a passthrough of the repair, not an authored
FK change: the existing override and attempted value are preserved and the FK
field HLC does not advance. Writing a different FK value, or updating the FK
column with narrowed columns (even when the written value equals the projected
value), authors the change as an ordinary user field fact: the FK field HLC
advances, `attemptedValue` becomes the new value, and `overrideReason` becomes
null.
## Merge Pipeline
1. Merge CRDT row, field, and tombstone facts by their existing HLC/CLFlag
rules.
2. Identify rows touched by inserts, updates, deletes, FK columns, and existing
FK projection records.
3. Expand to the FK closure in both directions: parents needed by children and
children affected by parent visibility.
2. Identify whether inserts, updates, deletes, FK columns, parent reference
columns, or existing FK projection records can affect FK state. Skip FK
projection when the merge touches none of them.
3. For FK-affecting work, expand to the FK closure in both directions: parents
needed by children and children affected by parent visibility. A full graph
recomputation may be used instead of the minimal closure.
4. Compute effective row visibility and visible FK values to a fixed point.
5. Apply only the materialized differences:
- update domain FK columns when projection differs from attempted value;
@ -89,15 +153,30 @@ the authority is the deterministic resolver over merged CRDT facts.
- No visible row references a hidden or missing parent.
- No cascade delete is partially applied across a blocked nested relation.
- `SET NULL` and `SET DEFAULT` repairs are pure functions of merged facts.
- Merge-time `SET NULL` and `SET DEFAULT` repairs are pure functions of merged
facts. Local `SET NULL` and `SET DEFAULT` actions are authored field facts.
- Local cascade deletes produce synced user-delete tombstones for cascade
descendants; merge-time cascade projection may derive `foreignKeyCascade`
visibility for descendants without authored tombstones.
- Replaying the same merge batch produces no metadata churn.
- Splitting a merge into smaller batches converges to the same final projection
as merging the same operations in one batch.
- Incremental recomputation over the affected closure matches full global
recomputation.
- FK-affecting recomputation over the affected closure matches full global
recomputation; unrelated non-FK updates do not trigger FK projection.
## Test Cases
### Local CRUD Actions
- Given a visible `RESTRICT` or `NO ACTION` child, when the parent is deleted
locally, then the delete fails and no parent tombstone is recorded.
- Given a nullable `SET NULL` child, when the parent is deleted locally, then the
child FK is set to `null` as an authored field update.
- Given a valid `SET DEFAULT` child, when the parent is deleted locally, then
the child FK is set to the default as an authored field update.
- Given a valid cascade chain, when the root is deleted locally, then every
visible cascade descendant receives a user-delete tombstone.
### Restrict and No Action
1. Given a parent and a visible child with `ON DELETE RESTRICT`, when one replica
@ -132,9 +211,9 @@ the authority is the deterministic resolver over merged CRDT facts.
### Cascade
8. Given a parent with children under `ON DELETE CASCADE`, when the parent is
deleted, then all cascade descendants in the valid cascade closure become
hidden and no visible descendant references a hidden ancestor.
8. Given a parent delete fact merged from another replica with children under
`ON DELETE CASCADE`, then all cascade descendants in the valid cascade
closure become hidden and no visible descendant references a hidden ancestor.
9. Given a child already hidden by a cascade projection, when the same delete
facts are merged again, then no additional metadata churn occurs.
@ -146,9 +225,9 @@ the authority is the deterministic resolver over merged CRDT facts.
`B`, and `C` remain visible.
11. Given a fourth-degree chain with alternating cascade, restrict, set-null,
and set-default edges, when a merge touches both a root and a leaf, then
incremental recomputation produces the same visible graph and FK projection
as full recomputation.
and set-default edges, when a merge touches both a root and a leaf, then the
affected-closure result and a full fixed-point recomputation produce the
same visible graph and FK projection.
12. Given an FK cycle that the engine permits, when concurrent deletes and
updates happen around the cycle, then fixed-point evaluation terminates and
@ -165,9 +244,9 @@ the authority is the deterministic resolver over merged CRDT facts.
values, FK projection metadata, unique projection, and tombstones are the
same.
15. Given a merge touches only a non-indexed, non-FK field, when FK projection
runs, then unrelated rows are not reconsidered and the visible state outside
the affected closure remains unchanged.
15. Given a merge touches only a non-indexed, non-FK field that is not a parent
reference column, then FK projection is skipped and the visible FK graph is
unchanged.
## What These Tests Prove
@ -176,5 +255,7 @@ the authority is the deterministic resolver over merged CRDT facts.
- ACI merge behavior: batching, replay, and merge order do not change the final
visible result.
- Nested safety: cascade cannot bypass restrict/no-action descendants.
- Projection discipline: `SET NULL` and `SET DEFAULT` do not become ordinary
user CRDT writes.
- Local action fidelity: local deletes preserve database-style FK side effects
as authored CRDT facts.
- Projection discipline: merge-time `SET NULL` and `SET DEFAULT` repairs do not
become ordinary user CRDT writes.

35
packages/serverpod_offline_sync_client/lib/src/crdt/extensions.dart
View file

@ -2,6 +2,21 @@ import 'package:serverpod_offline_sync_shared/serverpod_offline_sync_shared.dart
import '../protocol/protocol.dart';
/// Last serialized index that still represents a visible CRDT row.
///
/// The [CrdtDataRowVisibility] is ordered by visibility, so the index is used
/// to compare the visibility states. The last visible state in the enum is
/// [CrdtDataRowVisibility.foreignKeyRestrictRestore].
final crdtRowLastVisibleVisibilityIndex =
CrdtDataRowVisibility.foreignKeyRestrictRestore.index;
/// Extensions for the [CrdtDataRowVisibility] enum.
extension CrdtDataRowVisibilityExtension on CrdtDataRowVisibility {
/// Whether this visibility state hides the row from domain queries.
///
bool get isHidden => index > crdtRowLastVisibleVisibilityIndex;
}
/// Extensions for the [CrdtDataField] class.
extension CrdtDataFieldExtension on CrdtDataField {
/// The HLC represented by this field.
@ -14,6 +29,9 @@ extension CrdtDataFieldExtension on CrdtDataField {
/// Extensions for the [CrdtDataRow] class.
extension CrdtDataRowExtension on CrdtDataRow {
/// Whether this row is currently hidden from domain queries.
bool get isHidden => visibility.isHidden;
/// The HLC represented by this row.
Hlc get hlc => node == null
? (throw StateError(
@ -34,3 +52,20 @@ extension CrdtDataDeletedExtension on CrdtDataDeleted {
))
: toHlcForNode(node!.uuidNodeId);
}
/// Extensions for the [CrdtDataForeignKey] class.
extension CrdtDataForeignKeyExtension on CrdtDataForeignKey {
/// Whether an active projection override exists for this FK field.
///
/// Derived from [CrdtDataForeignKey.overrideReason]: an override is active if
/// and only if [overrideReason] is non-null. This is the authoritative test
/// for whether the domain row's FK value differs from the attempted value.
bool get hasOverride => overrideReason != null;
}
/// Extensions for the [CrdtDataDeletedReason] enum.
extension CrdtDataDeletedReasonExtension on CrdtDataDeletedReason {
/// Whether this tombstone reason is synced across replicas. All reasons that
/// start with 'user' are synced.
bool get isSynced => name.startsWith('user');
}

108
packages/serverpod_offline_sync_client/lib/src/crdt/sync.dart
View file

@ -9,6 +9,7 @@ import '../managers/user.dart';
import '../protocol/protocol.dart';
import '../utils/case_when.dart' show Case;
import 'exceptions.dart';
import 'extensions.dart';
import 'merge.dart';
/// Callback function for when a merge is successful.
@ -128,6 +129,11 @@ class CrdtSync {
/// Changes are emitted in insert, update, then delete order. Domain row and
/// column payloads are resolved incrementally as each change is yielded.
///
/// Foreign-key columns with an active projection override are sent with their
/// durable [CrdtDataForeignKey.attemptedValue], not the visible value stored
/// in the domain table. Peers need the attempted fact to converge; local FK
/// projection materializes only the safe visible value into domain tables.
///
/// All changes for nodes that are not present in the [nodeCheckpoints] list
/// are collected and emitted. Passing an empty list will collect all changes.
Stream<CrdtMergeChange> collectPendingChanges(
@ -171,6 +177,9 @@ class CrdtSync {
/// Merges a remote [mergeSet] and records the sync checkpoint for [otherNodeId].
///
/// Inbound merge applies each remote change, then materializes foreign-key
/// projection into domain tables via [CrdtDatabase.mergeChanges].
///
/// Throws if the merge fails. The sync stream should be closed so the next
/// attempt resumes from the last persisted checkpoint.
///
@ -368,6 +377,11 @@ class CrdtSync {
),
);
final foreignKeyAttemptFieldsByRowId = await _loadProjectedForeignKeyAttemptFields(
session,
rows,
);
for (final row in rows) {
final tableName = row.tbl!.name;
if (!_syncTablesByName.containsKey(tableName)) continue;
@ -382,6 +396,7 @@ class CrdtSync {
row.uuidRowId,
table,
dartName,
foreignKeyAttemptFieldsByRowId[row.id!],
);
if (domainRow == null) continue;
@ -408,12 +423,18 @@ class CrdtSync {
row: CrdtDataRow.include(tbl: CrdtSchemaTable.include()),
column: CrdtSchemaColumn.include(),
node: CrdtNode.include(),
foreignKey: CrdtDataForeignKey.include(),
),
);
for (final field in fields) {
final tableName = field.row!.tbl!.name;
if (!_syncTablesByName.containsKey(tableName)) continue;
if (field.hlcDatetime == field.row!.hlcDatetime &&
field.hlcCounter == field.row!.hlcCounter &&
field.nodeId == field.row!.nodeId) {
continue;
}
final columnName = field.column!.name;
final decodedValue = await _fetchColumnValue(
@ -421,6 +442,7 @@ class CrdtSync {
tableName,
field.row!.uuidRowId,
columnName,
field.foreignKey,
);
yield CrdtMergeUpdate(
@ -450,6 +472,8 @@ class CrdtSync {
);
for (final tombstone in tombstones) {
if (!tombstone.reason.isSynced) continue;
final tableName = tombstone.row!.tbl!.name;
if (!_syncTablesByName.containsKey(tableName)) continue;
@ -504,41 +528,97 @@ class CrdtSync {
nodeCheckpoints,
);
/// Loads a domain row for outbound insert sync.
///
/// Reads the materialized row from the domain table, then swaps any FK columns
/// with an active override back to [CrdtDataForeignKey.attemptedValue] before
/// deserializing. This is the inverse of inbound FK materialization: the wire
/// payload carries attempted facts, not locally projected visible values.
Future<dynamic> _fetchDomainRow(
DatabaseSession session,
String tableName,
UuidValue rowId,
Table table,
String dartName,
List<CrdtDataField>? foreignKeyAttemptFields,
) async {
final cols = table.columns.map((column) => '"${column.columnName}"').join(', ');
final cols = table.columns
.map((column) => '"${_escapeIdentifier(column.columnName)}"')
.join(', ');
final encodedRowId = ValueEncoder.instance.convert(rowId);
final result = await session.db.unsafeQuery(
'SELECT $cols FROM "$tableName" WHERE "id" = $encodedRowId',
'SELECT $cols FROM "${_escapeIdentifier(tableName)}" '
'WHERE "id" = $encodedRowId',
);
if (result.isEmpty) return null;
final columnMap = result.first.toColumnMap();
final columnMap = result.first.toColumnMap()
// Domain columns hold visible/materialized FK values; restore attempted
// values for override columns before building the outbound merge payload.
..applyProjectedForeignKeyAttempts(foreignKeyAttemptFields);
return session.db.serializationManager.deserializeByClassName({
'className': dartName,
'data': columnMap,
});
}
/// Resolves a column value for outbound update sync.
///
/// When [projection] has an active override, returns the attempted value
/// from [CrdtDataForeignKey.attemptedValue] instead of the materialized
/// domain column value. Non-FK columns and FK columns without an override
/// are read directly from the domain table.
Future<dynamic> _fetchColumnValue(
DatabaseSession session,
String tableName,
UuidValue rowId,
String columnName,
CrdtDataForeignKey? projection,
) async {
if (projection != null && projection.hasOverride) {
return _decodeColumnValue(tableName, columnName, projection.attemptedValue);
}
final encodedValue = ValueEncoder.instance.convert(rowId);
final result = await session.db.unsafeQuery(
'SELECT "$columnName" FROM "$tableName" WHERE "id" = $encodedValue',
'SELECT "${_escapeIdentifier(columnName)}" '
'FROM "${_escapeIdentifier(tableName)}" '
'WHERE "id" = $encodedValue',
);
if (result.isEmpty) return null;
return _decodeColumnValue(tableName, columnName, result.first[0]);
}
/// Loads FK projection metadata for outbound insert sync.
///
/// Returns only fields whose [CrdtDataForeignKey.overrideReason] is non-null,
/// keyed by CRDT row id. These are the columns whose domain-table value
/// differs from the durable attempted FK fact.
Future<Map<int, List<CrdtDataField>>> _loadProjectedForeignKeyAttemptFields(
DatabaseSession session,
List<CrdtDataRow> rows,
) async {
final rowIds = {for (final row in rows) ?row.id};
if (rowIds.isEmpty) return {};
final fields = await CrdtDataField.db.find(
session,
where: (t) => t.rowId.inSet(rowIds) & t.foreignKey.overrideReason.notEquals(null),
include: CrdtDataField.include(
column: CrdtSchemaColumn.include(),
foreignKey: CrdtDataForeignKey.include(),
),
);
final fieldsByRowId = <int, List<CrdtDataField>>{};
for (final field in fields) {
fieldsByRowId.putIfAbsent(field.rowId, () => []).add(field);
}
return fieldsByRowId;
}
dynamic _decodeColumnValue(String tableName, String columnName, Object? value) {
if (value == null) return null;
@ -563,6 +643,8 @@ class CrdtSync {
return withoutNullable.split(':').last;
}
String _escapeIdentifier(String identifier) => identifier.replaceAll('"', '""');
static String _computeCanonicalSyncTablesSignature(
List<Table> syncTables, {
required List<TableDefinition> tableDefinitions,
@ -650,3 +732,21 @@ extension on CrdtNodeTable {
return caseExpression.orElse(Constant.bool(true));
}
}
extension on Map<String, dynamic> {
/// Replaces materialized FK column values with attempted values for sync.
///
/// After local FK projection (for example `SET NULL` or `SET DEFAULT`), the
/// domain table stores the safe visible value while
/// [CrdtDataForeignKey.attemptedValue] preserves what was actually tried.
/// Outbound sync must send the attempted value so peers can apply their own
/// projection from the same fact.
void applyProjectedForeignKeyAttempts(List<CrdtDataField>? foreignKeyAttemptFields) {
if (foreignKeyAttemptFields == null) return;
for (final field in foreignKeyAttemptFields) {
final projection = field.foreignKey;
if (projection == null || !projection.hasOverride) continue;
this[field.column!.name] = projection.attemptedValue;
}
}
}

33
packages/serverpod_offline_sync_client/lib/src/database/database.dart
View file

@ -296,10 +296,13 @@ class CrdtDatabase implements Database {
_delegate,
transaction,
(tx) async {
return [
final updatedRows = [
for (final row in rows)
await updateRow<T>(row, columns: columns, transaction: tx),
await _updateRowWithoutRecording(row, columns: columns, transaction: tx),
];
await _recorder.afterUpdate(updatedRows, columns, tx);
return updatedRows;
},
);
}
@ -309,6 +312,27 @@ class CrdtDatabase implements Database {
T row, {
List<Column>? columns,
Transaction? transaction,
}) async {
return DatabaseUtil.runInTransactionOrSavepoint(
_delegate,
transaction,
(tx) async {
final updatedRow = await _updateRowWithoutRecording(
row,
columns: columns,
transaction: tx,
);
await _recorder.afterUpdate([updatedRow], columns, tx);
return updatedRow;
},
);
}
Future<T> _updateRowWithoutRecording<T extends TableRow>(
T row, {
required Transaction transaction,
List<Column>? columns,
}) async {
final values = row.toJsonForDatabase() as Map<String, dynamic>;
final columnValues = (columns ?? row.table.managedColumns)
@ -316,9 +340,10 @@ class CrdtDatabase implements Database {
.map((c) => ColumnValue(c, values[c.columnName]))
.toList();
final updatedRows = await updateWhere<T>(
final where = row.table.id.equals(row.id);
final updatedRows = await _delegate.updateWhere<T>(
columnValues: columnValues,
where: row.table.id.equals(row.id),
where: mergeWhereWithTombstone<T>(serializationManager, where, null)!,
transaction: transaction,
);

85
packages/serverpod_offline_sync_client/lib/src/database/merge.dart
View file

@ -36,6 +36,9 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
if (mergeSet.isEmpty) return;
final operations = mergeSet.causallyOrderedChanges;
final shouldProjectForeignKeys = _mergeOperationsMayAffectForeignKeys(
operations,
);
final currentUser = _getEffectiveUser(transaction);
final remoteNodes = await _findOrCreateNodesForMerge(
currentUser.id!,
@ -74,6 +77,9 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
}
}
if (shouldProjectForeignKeys) {
await _projectForeignKeys(transaction);
}
await _updateHlcFromIncomingOperations(
operations,
remoteNodes,
@ -221,16 +227,13 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
);
}
final (_, columnsByName) = _schema[tableName]!;
final columnIds = <int>{
for (final columnName in columnNames)
if (columnsByName[columnName] != null) columnsByName[columnName]!.id!,
};
if (columnIds.isEmpty) continue;
final columnIdsByName = _schemaColumnIds(_schema, tableName, columnNames);
if (columnIdsByName.isEmpty) continue;
final loadedFields = await CrdtDataField.db.find(
_session,
where: (t) => t.rowId.inSet(rowPks) & t.columnId.inSet(columnIds),
where: (t) =>
t.rowId.inSet(rowPks) & t.columnId.inSet(columnIdsByName.values.toSet()),
include: CrdtDataField.include(
column: CrdtSchemaColumn.include(),
node: CrdtNode.include(),
@ -277,13 +280,26 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
transaction,
);
final safeInsert = await _safeIncomingForeignKeyData(
insert.tableName,
insert.uuidRowId,
resolvedInsert.data,
transaction,
);
context.rows[rowKey] = await _applyMergeInsertForMissingRow(
insert,
remoteNode,
incomingHlc,
resolvedInsert.data,
safeInsert.data,
transaction,
);
await _recordForeignKeyInsertAttempts(
insert.tableName,
{insert.uuidRowId},
transaction,
safeInsert.attempts,
);
} else {
await _applyMergeInsertForExistingRow(
insert,
@ -326,12 +342,25 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
context,
transaction,
);
final safeUpdate = await _safeIncomingForeignKeyData(
update.tableName,
update.uuidRowId,
resolvedUpdates,
transaction,
);
await _updateDomainRows(
update.tableName,
{update.uuidRowId},
resolvedUpdates,
safeUpdate.data,
transaction,
);
await _recordForeignKeyAttemptsForRows(
update.tableName,
{update.uuidRowId},
resolvedUpdates.keys.toSet(),
transaction,
attemptedValues: safeUpdate.attempts,
);
}
Future<void> _applyMergeDelete(
@ -343,6 +372,7 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
final rowKey = (delete.tableName, delete.uuidRowId);
final row = context.rows[rowKey];
if (row == null) return;
if (!delete.reason.isSynced) return;
final currentTombstone = context.tombstones[rowKey];
final currentClFlag = currentTombstone?.clFlag ?? 1;
@ -359,6 +389,14 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
currentTombstone,
transaction,
);
if (delete.reason.isSynced) {
await _applyRowVisibilityFromUserTombstone(
row,
context.tombstones[rowKey]!,
transaction,
);
}
}
Future<CrdtDataRow> _upsertMergeRow(
@ -575,12 +613,25 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
}
if (updatesToApply.isNotEmpty) {
final safeUpdate = await _safeIncomingForeignKeyData(
insert.tableName,
insert.uuidRowId,
updatesToApply,
transaction,
);
await _updateDomainRows(
insert.tableName,
{insert.uuidRowId},
updatesToApply,
safeUpdate.data,
transaction,
);
await _recordForeignKeyAttemptsForRows(
insert.tableName,
{insert.uuidRowId},
updatesToApply.keys.toSet(),
transaction,
attemptedValues: safeUpdate.attempts,
);
}
final rowKey = (insert.tableName, insert.uuidRowId);
@ -596,6 +647,11 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
currentTombstone,
transaction,
);
await _applyRowVisibilityFromUserTombstone(
currentRow,
context.tombstones[rowKey]!,
transaction,
);
}
}
@ -610,8 +666,8 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
required Transaction transaction,
CrdtSchemaColumn? schemaColumn,
}) async {
final (_, columnsByName) = _schema[tableName]!;
final resolvedSchemaColumn = schemaColumn ?? columnsByName[columnName];
final resolvedSchemaColumn =
schemaColumn ?? _schemaColumn(_schema, tableName, columnName);
if (resolvedSchemaColumn == null) return false;
final fieldKey = (tableName, rowId, columnName);
@ -659,7 +715,10 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
return {
for (final MapEntry(key: columnName, value: value) in data.entries)
if (columnName != 'id' && columns.containsKey(columnName)) columnName: value,
if (columnName != 'id' && columns.containsKey(columnName))
columnName: columns[columnName]!.columnType == ColumnType.uuid
? _uuidValueFromDatabase(value)
: value,
};
}
}

1199
packages/serverpod_offline_sync_client/lib/src/database/merge_utils/foreign_key_projector.dart Normal file
View file

File diff suppressed because it is too large Load diff

7
packages/serverpod_offline_sync_client/lib/src/database/merge_utils/unique_resolver.dart
View file

@ -306,7 +306,6 @@ class CrdtUniqueConflictResolver {
required Map<_MergeFieldKey, CrdtDataField> fields,
required Transaction transaction,
}) async {
final (_, columnsByName) = _recorder._schema[tableName]!;
final uniqueColumnNames = {
for (final column in uniqueIndex.columns) column.columnName,
};
@ -315,7 +314,7 @@ class CrdtUniqueConflictResolver {
for (final columnName in uniqueColumnNames) {
final field = fields[(tableName, row.uuidRowId, columnName)];
if (field == null) {
final columnId = columnsByName[columnName]?.id;
final columnId = _schemaColumn(_recorder._schema, tableName, columnName)?.id;
if (columnId != null) missingColumnIds.add(columnId);
continue;
}
@ -356,8 +355,8 @@ class CrdtUniqueConflictResolver {
tableName: tableName,
predicates: [
for (final MapEntry(key: columnName, value: value) in values.entries)
'd."${_escapeIdentifier(columnName)}" = ${_sqlLiteral(value)}',
'd."id" <> ${_sqlLiteral(rowId)}',
_domainColumnPredicate(columnName, value),
_domainColumnNotPredicate('id', rowId),
],
transaction: transaction,
);

501
packages/serverpod_offline_sync_client/lib/src/database/recorder.dart
View file

@ -12,6 +12,7 @@ import 'schema.dart';
import 'session.dart';
part 'merge.dart';
part 'merge_utils/foreign_key_projector.dart';
part 'merge_utils/unique_resolver.dart';
typedef _ReferencingForeignKey = ({
@ -21,6 +22,18 @@ typedef _ReferencingForeignKey = ({
ForeignKeyAction action,
});
typedef _ForeignKeyEdge = ({
String childTableName,
String childColumn,
String parentTableName,
String parentColumn,
ForeignKeyAction action,
bool childNullable,
Object? defaultValue,
});
typedef _ForeignKeyValueKey = (String tableName, String columnName, String value);
typedef _UniqueIndexConflictRelease = ({
List<_UniqueColumnConflictRelease> columns,
});
@ -98,6 +111,45 @@ class CrdtMutationRecorder {
return result;
}();
late final List<_ForeignKeyEdge> _foreignKeyEdges = [
for (final table in _tableDefinitionsByName.values)
if (_isCrdtTrackedTableName(table.name))
for (final foreignKey in table.foreignKeys)
if (foreignKey.columns.length == 1 &&
foreignKey.referenceColumns.length == 1 &&
_isCrdtTrackedTableName(foreignKey.referenceTable))
(
childTableName: table.name,
childColumn: foreignKey.columns.single,
parentTableName: foreignKey.referenceTable,
parentColumn: foreignKey.referenceColumns.single,
action: foreignKey.onDelete ?? ForeignKeyAction.noAction,
childNullable:
_columnsByTableAndName[table.name]![foreignKey.columns.single]!
.isNullable,
defaultValue: _defaultValueForColumn(
table.name,
foreignKey.columns.single,
),
),
];
late final Map<String, List<_ForeignKeyEdge>> _foreignKeyEdgesByParentTable = {
for (final tableName in _tableDefinitionsByName.keys)
tableName: [
for (final edge in _foreignKeyEdges)
if (edge.parentTableName == tableName) edge,
],
};
late final Map<String, List<_ForeignKeyEdge>> _foreignKeyEdgesByChildTable = {
for (final tableName in _tableDefinitionsByName.keys)
tableName: [
for (final edge in _foreignKeyEdges)
if (edge.childTableName == tableName) edge,
],
};
final _uniqueIndexesByTableName = <String, List<_UniqueIndexConflictRelease>>{};
late final Map<String, Map<String, ColumnDefinition>> _columnsByTableAndName = {
@ -220,6 +272,11 @@ class CrdtMutationRecorder {
transaction: transaction,
ignoreConflicts: true,
);
await _recordForeignKeyInsertAttempts(tableName, rowIds, transaction, {});
if (_tableColumnsMayAffectForeignKeys(tableName, null)) {
await _projectForeignKeys(transaction);
}
});
}
@ -248,6 +305,15 @@ class CrdtMutationRecorder {
transaction,
);
await _recordUpdatedFields(reinsertedRows, crdtDataRows, null, transaction);
await _recordForeignKeyAttemptsForRows(
tableName,
rowIds,
null,
transaction,
);
if (_tableColumnsMayAffectForeignKeys(tableName, null)) {
await _projectForeignKeys(transaction);
}
});
}
@ -270,7 +336,33 @@ class CrdtMutationRecorder {
'updated',
transaction,
);
await _recordUpdatedFields(updatedRows, crdtDataRows, columns, transaction);
final implicitForeignKeyRepairFields = columns == null
? await _findImplicitForeignKeyRepairFields(
tableName: tableName,
rowIds: rowIds,
transaction: transaction,
)
: const <_MergeFieldKey>{};
await _recordUpdatedFields(
updatedRows,
crdtDataRows,
columns,
transaction,
skippedFields: implicitForeignKeyRepairFields,
);
await _recordForeignKeyAttemptsForRows(
tableName,
rowIds,
columns?.map((column) => column.columnName).toSet(),
transaction,
skippedFields: implicitForeignKeyRepairFields,
);
if (_tableColumnsMayAffectForeignKeys(
tableName,
columns?.map((column) => column.columnName).toSet(),
)) {
await _projectForeignKeys(transaction);
}
});
}
@ -349,11 +441,9 @@ LEFT JOIN "$parentTable" p
ON p."$parentColumn" = c."$childColumn"
LEFT JOIN "crdt_data_rows" r
ON r."userId" = $userId AND r."tblId" = $parentTableId AND r."uuidRowId" = p."id"
LEFT JOIN "crdt_data_tombstone" tomb
ON tomb."rowId" = r."id"
WHERE c."id" IN ($whereRowIds)
AND c."$childColumn" IS NOT NULL
AND (p."id" IS NULL OR tomb."clFlag" % 2 = 0)
AND (p."id" IS NULL OR r."visibility" > $crdtRowLastVisibleVisibilityIndex)
''';
});
@ -451,8 +541,9 @@ WHERE c."id" IN ($whereRowIds)
List<T> updatedRows,
List<CrdtDataRow> crdtDataRows,
List<Column>? columns,
Transaction transaction,
) async {
Transaction transaction, {
Set<_MergeFieldKey> skippedFields = const {},
}) async {
if (updatedRows.isEmpty || crdtDataRows.isEmpty) return;
final crdtDataRowByUuid = {
@ -473,6 +564,26 @@ WHERE c."id" IN ($whereRowIds)
'No CRDT schema column for ${table.tableName}.${column.columnName}',
)),
];
await _upsertCrdtFieldsForRows(
table.tableName,
[
for (final row in updatedRows) crdtDataRowByUuid[row.id as UuidValue]!,
],
schemaColumns,
transaction,
skippedFields: skippedFields,
);
}
Future<void> _upsertCrdtFieldsForRows(
String tableName,
List<CrdtDataRow> crdtDataRows,
List<CrdtSchemaColumn> schemaColumns,
Transaction transaction, {
Set<_MergeFieldKey> skippedFields = const {},
}) async {
if (crdtDataRows.isEmpty || schemaColumns.isEmpty) return;
final rowPks = crdtDataRows.map((r) => r.id!).toSet();
final columnPks = schemaColumns.map((c) => c.id!).toSet();
final existingFields = await CrdtDataField.db.find(
@ -485,159 +596,16 @@ WHERE c."id" IN ($whereRowIds)
for (final f in existingFields) (f.rowId, f.columnId): f,
};
final hlcManager = _getHlcManager(transaction);
final toInsert = <CrdtDataField>[];
final toUpdate = <CrdtDataField>[];
for (final row in updatedRows) {
final crdtDataRow = crdtDataRowByUuid[row.id as UuidValue]!;
for (final schemaCol in schemaColumns) {
final hlc = hlcManager.increment();
final rowPk = crdtDataRow.id!;
final colPk = schemaCol.id!;
final existing = fieldByRowAndColumn[(rowPk, colPk)];
if (existing == null) {
toInsert.add(
CrdtDataField(
rowId: rowPk,
columnId: colPk,
nodeId: hlcManager.normalizedNodeId,
hlcDatetime: hlc.datetime,
hlcCounter: hlc.counter,
),
);
} else {
toUpdate.add(
existing.copyWith(
nodeId: hlcManager.normalizedNodeId,
hlcDatetime: hlc.datetime,
hlcCounter: hlc.counter,
),
);
}
}
}
if (toInsert.isNotEmpty) {
await CrdtDataField.db.insert(
_session,
toInsert,
transaction: transaction,
);
}
if (toUpdate.isNotEmpty) {
await CrdtDataField.db.update(
_session,
toUpdate,
transaction: transaction,
);
}
}
/// Soft-deletes rows by recording a tombstone instead of removing them.
///
/// Expects a matching [CrdtDataRow] per domain row (`uuidRowId` = domain id).
Future<void> insteadOfDelete<T extends TableRow>(
List<T> deletedRows,
Transaction transaction,
) async {
if (deletedRows.isEmpty) return;
if (!isCrdtTracked<T>(deletedRows.first.table)) return;
await _softDeleteRowsByTable(
deletedRows.first.table.tableName,
deletedRows.uuidRowIds,
transaction,
null,
CrdtDataDeletedReason.userDelete,
);
}
Future<void> _softDeleteRowsByTable(
String tableName,
Set<UuidValue> rowIds,
Transaction transaction,
Set<String>? processedRowIds,
CrdtDataDeletedReason reason,
) async {
if (rowIds.isEmpty) return;
if (!_isCrdtTrackedTableName(tableName)) return;
final processing = processedRowIds ?? {};
final unprocessedRowIds = rowIds
.where((rowId) => processing.add('$tableName:$rowId'))
.toSet();
if (unprocessedRowIds.isEmpty) return;
final crdtDataRows = await _findRequiredCrdtRows(
tableName,
unprocessedRowIds,
'deleted',
transaction,
includeDeleted: true,
);
try {
final visibleCrdtRows = crdtDataRows
.where((row) => row.deleted?.isDeleted != true)
.toList();
if (visibleCrdtRows.isEmpty) return;
final visibleRowIds = visibleCrdtRows.map((row) => row.uuidRowId).toSet();
await _applyForeignKeyDeleteActions(
tableName,
visibleRowIds,
transaction,
processing,
);
await _releaseUniqueConflicts(tableName, visibleRowIds, transaction);
await _markCrdtRowsDeleted(visibleCrdtRows, true, reason, transaction);
} finally {
for (final rowId in unprocessedRowIds) {
processing.remove('$tableName:$rowId');
}
}
}
/// Records field updates by table name and UUID ids.
Future<void> recordFieldsUpdatedByTable(
String tableName,
Set<UuidValue> rowIds,
List<String> columnNames,
Transaction transaction,
) async {
if (rowIds.isEmpty || columnNames.isEmpty) return;
if (!_isCrdtTrackedTableName(tableName)) return;
final crdtDataRows = await _findCrdtRows(tableName, rowIds, transaction);
if (crdtDataRows.isEmpty) return;
final (_, colMap) = _schema[tableName]!;
final schemaColumns = [
for (final columnName in columnNames)
if (colMap[columnName] != null) colMap[columnName]!,
];
if (schemaColumns.isEmpty) return;
final rowPks = crdtDataRows.map((r) => r.id!).toSet();
final columnPks = schemaColumns.map((c) => c.id!).toSet();
final existingFields = await CrdtDataField.db.find(
_session,
where: (t) => t.rowId.inSet(rowPks) & t.columnId.inSet(columnPks),
transaction: transaction,
);
final fieldByRowAndColumn = {
for (final f in existingFields) (f.rowId, f.columnId): f,
};
final hlcManager = _getHlcManager(transaction);
final toInsert = <CrdtDataField>[];
final toUpdate = <CrdtDataField>[];
for (final row in crdtDataRows) {
for (final schemaCol in schemaColumns) {
if (skippedFields.contains((tableName, row.uuidRowId, schemaCol.name))) {
continue;
}
final hlc = hlcManager.increment();
final existing = fieldByRowAndColumn[(row.id!, schemaCol.id!)];
if (existing == null) {
@ -670,6 +638,114 @@ WHERE c."id" IN ($whereRowIds)
}
}
/// Soft-deletes rows by recording a tombstone instead of removing them.
///
/// Expects a matching [CrdtDataRow] per domain row (`uuidRowId` = domain id).
Future<void> insteadOfDelete<T extends TableRow>(
List<T> deletedRows,
Transaction transaction,
) async {
if (deletedRows.isEmpty) return;
if (!isCrdtTracked<T>(deletedRows.first.table)) return;
await _softDeleteRowsByTable(
deletedRows.first.table.tableName,
deletedRows.uuidRowIds,
transaction,
null,
CrdtDataDeletedReason.userDelete,
);
if (_tableColumnsMayAffectForeignKeys(
deletedRows.first.table.tableName,
null,
)) {
await _projectForeignKeys(transaction);
}
}
Future<void> _softDeleteRowsByTable(
String tableName,
Set<UuidValue> rowIds,
Transaction transaction,
Set<String>? processedRowIds,
CrdtDataDeletedReason reason,
) async {
if (rowIds.isEmpty) return;
if (!_isCrdtTrackedTableName(tableName)) return;
final processing = processedRowIds ?? {};
final unprocessedRowIds = rowIds
.where((rowId) => processing.add('$tableName:$rowId'))
.toSet();
if (unprocessedRowIds.isEmpty) return;
final crdtDataRows = await _findRequiredCrdtRows(
tableName,
unprocessedRowIds,
'deleted',
transaction,
includeDeleted: true,
);
try {
final visibleCrdtRows = crdtDataRows.where((row) => !row.isHidden).toList();
if (visibleCrdtRows.isEmpty) return;
final visibleRowIds = visibleCrdtRows.map((row) => row.uuidRowId).toSet();
final cascadeDeletes = await _applyForeignKeyDeleteActions(
tableName,
visibleRowIds,
transaction,
processing,
);
await _releaseUniqueConflicts(tableName, visibleRowIds, transaction);
await _markCrdtRowsDeleted(visibleCrdtRows, true, reason, transaction);
for (final MapEntry(key: childTableName, value: childIds)
in cascadeDeletes.entries) {
await _softDeleteRowsByTable(
childTableName,
childIds,
transaction,
processing,
CrdtDataDeletedReason.userCascadeDelete,
);
}
} finally {
for (final rowId in unprocessedRowIds) {
processing.remove('$tableName:$rowId');
}
}
}
/// Records field updates by table name and UUID ids.
Future<void> recordFieldsUpdatedByTable(
String tableName,
Set<UuidValue> rowIds,
List<String> columnNames,
Transaction transaction,
) async {
if (rowIds.isEmpty || columnNames.isEmpty) return;
if (!_isCrdtTrackedTableName(tableName)) return;
final crdtDataRows = await _findCrdtRows(tableName, rowIds, transaction);
if (crdtDataRows.isEmpty) return;
final (_, colMap) = _schema[tableName]!;
final schemaColumns = [
for (final columnName in columnNames)
if (colMap[columnName] != null) colMap[columnName]!,
];
if (schemaColumns.isEmpty) return;
await _upsertCrdtFieldsForRows(
tableName,
crdtDataRows,
schemaColumns,
transaction,
);
}
Future<void> _markCrdtRowsDeleted(
List<CrdtDataRow> crdtDataRows,
bool isDeleted,
@ -731,6 +807,43 @@ WHERE c."id" IN ($whereRowIds)
transaction: transaction,
);
}
await _applyRowVisibility(
crdtDataRows,
visibility: reason.toVisibility(isDeleted: isDeleted),
transaction: transaction,
);
}
Future<void> _applyRowVisibility(
List<CrdtDataRow> rows, {
required CrdtDataRowVisibility visibility,
required Transaction transaction,
}) async {
final toUpdate = rows
.where((row) => row.visibility != visibility)
.map((row) => row.copyWith(visibility: visibility))
.toList();
if (toUpdate.isEmpty) return;
await CrdtDataRow.db.update(
_session,
toUpdate,
columns: (t) => [t.visibility],
transaction: transaction,
);
}
Future<void> _applyRowVisibilityFromUserTombstone(
CrdtDataRow row,
CrdtDataDeleted tombstone,
Transaction transaction,
) async {
await _applyRowVisibility(
[row],
visibility: tombstone.reason.toVisibility(isDeleted: tombstone.isDeleted),
transaction: transaction,
);
}
/// Whether the given row is soft-deleted.
@ -748,12 +861,7 @@ WHERE c."id" IN ($whereRowIds)
);
if (crdtRows.isEmpty) return false;
final tombstone = await CrdtDataDeleted.db.findFirstRow(
_session,
where: (t) => t.rowId.equals(crdtRows.single.id),
transaction: transaction,
);
return tombstone?.isDeleted ?? false;
return crdtRows.single.isHidden;
}
Future<List<CrdtDataRow>> _findCrdtRows(
@ -773,14 +881,15 @@ WHERE c."id" IN ($whereRowIds)
);
}
Future<void> _applyForeignKeyDeleteActions(
Future<Map<String, Set<UuidValue>>> _applyForeignKeyDeleteActions(
String parentTableName,
Set<UuidValue> parentIds,
Transaction transaction,
Set<String> processing,
) async {
if (parentIds.isEmpty) return;
if (parentIds.isEmpty) return {};
final cascadeDeletes = <String, Set<UuidValue>>{};
final foreignKeys =
_foreignKeysByReferencedTable[parentTableName] ??
const <_ReferencingForeignKey>[];
@ -850,16 +959,14 @@ WHERE c."id" IN ($whereRowIds)
transaction,
);
case ForeignKeyAction.cascade:
await _softDeleteRowsByTable(
reference.childTableName,
childIds,
transaction,
processing,
CrdtDataDeletedReason.cascadeDelete,
);
cascadeDeletes
.putIfAbsent(reference.childTableName, () => {})
.addAll(childIds);
}
}
}
return cascadeDeletes;
}
Future<void> _releaseUniqueConflicts(
@ -930,7 +1037,7 @@ WHERE c."id" IN ($whereRowIds)
}) async {
return _findVisibleDomainRowIdsWhere(
tableName: tableName,
predicates: ['d."${_escapeIdentifier(columnName)}" = ${_sqlLiteral(value)}'],
predicates: [_domainColumnPredicate(columnName, value)],
transaction: transaction,
);
}
@ -948,10 +1055,8 @@ SELECT d."id"
FROM "${_escapeIdentifier(tableName)}" d
LEFT JOIN "crdt_data_rows" r
ON r."userId" = $userId AND r."tblId" = $tableId AND r."uuidRowId" = d."id"
LEFT JOIN "crdt_data_tombstone" tomb
ON tomb."rowId" = r."id"
WHERE (${predicates.join(') AND (')})
AND (tomb."id" IS NULL OR tomb."clFlag" % 2 = 1)
AND (r."id" IS NULL OR r."visibility" <= $crdtRowLastVisibleVisibilityIndex)
''',
transaction: transaction,
);
@ -1158,6 +1263,58 @@ String _sqlLiteral(Object? value) => ValueEncoder.instance.convert(value);
String _sqlLiteralList(Iterable<Object?> values) =>
values.map(ValueEncoder.instance.convert).join(', ');
String _domainColumnPredicate(
String columnName,
Object? value, {
String alias = 'd',
}) => '$alias."${_escapeIdentifier(columnName)}" = ${_sqlLiteral(value)}';
String _domainColumnNotPredicate(
String columnName,
Object? value, {
String alias = 'd',
}) => '$alias."${_escapeIdentifier(columnName)}" <> ${_sqlLiteral(value)}';
UuidValue? _uuidValueFromDatabase(Object? value) {
if (value == null) return null;
if (value is UuidValue) return value;
if (value is String) return UuidValue.withValidation(value);
return UuidValueJsonExtension.fromJson(value);
}
CrdtSchemaColumn? _schemaColumn(
Map<String, (int, Map<String, CrdtSchemaColumn>)> schema,
String tableName,
String columnName,
) => schema[tableName]?.$2[columnName];
Map<String, int> _schemaColumnIds(
Map<String, (int, Map<String, CrdtSchemaColumn>)> schema,
String tableName,
Iterable<String> columnNames,
) => {
for (final columnName in columnNames)
columnName: ?_schemaColumn(schema, tableName, columnName)?.id,
};
extension on CrdtDataDeletedReason {
CrdtDataRowVisibility toVisibility({required bool isDeleted}) {
if (!isDeleted) {
return switch (this) {
CrdtDataDeletedReason.userReinsert => CrdtDataRowVisibility.userReinsert,
_ => CrdtDataRowVisibility.userInsert,
};
}
return switch (this) {
CrdtDataDeletedReason.userDelete => CrdtDataRowVisibility.userDelete,
CrdtDataDeletedReason.userCascadeDelete =>
CrdtDataRowVisibility.userCascadeDelete,
_ => CrdtDataRowVisibility.userDelete,
};
}
}
extension on List<TableRow> {
Set<UuidValue> get uuidRowIds {
if (isEmpty) return {};

36
packages/serverpod_offline_sync_client/lib/src/database/tombstone.dart
View file

@ -1,16 +1,17 @@
import 'package:meta/meta.dart';
import 'package:serverpod_database/serverpod_database.dart';
import '../crdt/extensions.dart';
import '../protocol/protocol.dart';
/// Merges [where] with CRDT tombstone predicates and mutates [include] in place.
/// Merges [where] with CRDT visibility predicates and mutates [include] in place.
///
/// This method ensures that no rows marked as deleted by the CLFlag are
/// This method ensures that no rows with a hidden [CrdtDataRow.visibility] are
/// returned for queries with [where] clauses or [include] graphs. If [include]
/// is null, only the root table and [where] are merged.
///
/// We do **not** add a root predicate for keys whose nested include is an
/// [IncludeList] - tombstone filtering for that path is already handled via the
/// [IncludeList] - visibility filtering for that path is already handled via the
/// list subquery [where] (and many-relations are not the same join shape as a
/// single [IncludeObject]).
@internal
@ -24,7 +25,7 @@ Expression? mergeWhereWithTombstone<T extends TableRow>(
final rootTable = serializationManager.getTableForType(T);
var merged = _mergeWhereOptional(
where,
rootTable?.whereNotDeletedOnTombstone,
rootTable?.whereVisibleOnCrdtRow,
);
merged = _mergeWhereOptional(merged, includeObjectPredicates);
return merged;
@ -41,7 +42,7 @@ Expression? _walkIncludeGraphForTombstone(
// List relation: filter inside the list subquery only.
inc.where = _mergeWhereOptional(
inc.where,
inc.table.whereNotDeletedOnTombstone,
inc.table.whereVisibleOnCrdtRow,
);
return _walkIncludeGraphForTombstone(inc.include, includeObjectPredicates);
}
@ -51,7 +52,7 @@ Expression? _walkIncludeGraphForTombstone(
for (final entry in obj.includes.entries) {
final nested = entry.value;
if (nested is IncludeList) {
// e.g. one-to-many: tombstone applies via IncludeList.where, not the main WHERE.
// e.g. one-to-many: visibility applies via IncludeList.where, not the main WHERE.
acc = _walkIncludeGraphForTombstone(nested, acc);
continue;
}
@ -61,7 +62,7 @@ Expression? _walkIncludeGraphForTombstone(
if (childTable != null) {
acc = _mergeWhereOptional(
acc,
childTable.whereNotDeletedOnTombstone,
childTable.whereVisibleOnCrdtRow,
);
}
acc = _walkIncludeGraphForTombstone(nested, acc);
@ -77,17 +78,16 @@ Expression? _mergeWhereOptional(Expression? where, Expression? addition) {
}
extension on Table {
/// Creates a predicate that filters out tombstone rows for the given [Table]
/// similarly to the generated field [CrdtDataRow.deleted].
/// Creates a predicate that filters out hidden rows for the given [Table]
/// using materialized [CrdtDataRow.visibility] metadata.
///
/// Join path:
/// - UserModel.[id] = [CrdtDataRow.uuidRowId],
/// - [CrdtDataRow.id] = [CrdtDataDeleted.rowId].
/// - UserModel.[id] = [CrdtDataRow.uuidRowId].
///
/// Returns null when [Table] does not use a UUID primary key. Keeps rows
/// when there is no tombstone row or [CrdtDataDeleted.clFlag] is odd
/// (LEFT JOIN null-safe).
Expression? get whereNotDeletedOnTombstone {
/// Returns null when [Table] does not use a UUID primary key. Keeps rows when
/// there is no CRDT row or [CrdtDataRow.visibility] is visible (LEFT JOIN
/// null-safe).
Expression? get whereVisibleOnCrdtRow {
if (id is! ColumnUuid) return null;
final crdtRowTable = createRelationTable<CrdtDataRowTable>(
relationFieldName: '${tableName}_crdt_row',
@ -97,7 +97,9 @@ extension on Table {
createTable: (foreignTableRelation) =>
CrdtDataRowTable(tableRelation: foreignTableRelation),
);
final tomb = crdtRowTable.deleted;
return (tomb.id.equals(null)) | Expression('${tomb.clFlag} % 2 = 1');
return (crdtRowTable.id.equals(null)) |
Expression(
'${crdtRowTable.visibility} <= $crdtRowLastVisibleVisibilityIndex',
);
}
}

7
packages/serverpod_offline_sync_server/lib/src/models/data/deleted_reason.spy.yaml
View file

@ -1,10 +1,9 @@
### Why a CRDT row entered its current visibility generation.
### Why a synced user tombstone changed row visibility.
enum: CrdtDataDeletedReason
serialized: byIndex
values:
- userInsert
- userDelete
- userReinsert
- restrictRestore
- cascadeDelete
- userDelete
- userCascadeDelete
- uniqueLoser

4
packages/serverpod_offline_sync_server/lib/src/models/data/field.spy.yaml
View file

@ -16,6 +16,10 @@ fields:
### The node that updated the field.
node: CrdtNode?, relation
### The foreign key information for the field, if this field holds a
### reference to another row.
foreignKey: CrdtDataForeignKey?, relation(optional, name=field_foreign_key)
indexes:
crdt_data_fields_row_column_idx:
fields: rowId, columnId

40
packages/serverpod_offline_sync_server/lib/src/models/data/foreign_key.spy.yaml Normal file
View file

@ -0,0 +1,40 @@
### CRDT field foreign key value and projection table.
###
### This table stores the user/attempted foreign key value for a CRDT field and
### the materialized foreign key projection for that value. CrdtDataField stores
### only HLC metadata, so FK columns store their durable attempted value here.
###
### Repairs must be recomputable from merged CRDT row, field, and tombstone
### facts plus this attempted FK value. The projection columns record what was
### materialized into the domain row; they do not decide business semantics or
### create user field updates during merge.
class: CrdtDataForeignKey
table: crdt_data_foreign_key
database: all
fields:
### The field that the foreign key belongs to.
field: CrdtDataField?, relation(name=field_foreign_key, field=fieldId, onDelete=Cascade)
fieldId: int
### The user-attempted foreign key value that the CRDT field currently carries.
###
### This may be null for nullable foreign keys. This is durable FK field-value
### storage for CRDT columns whose CrdtDataField row stores only HLC metadata.
attemptedValue: UuidValue?, required
### The foreign key value that is currently visible in the actual data row
### when an override is active (overrideReason is non-null).
visibleValue: UuidValue?
### The authoritative indicator and reason for an active projection override.
###
### This is the single source of truth for whether an override is active:
### non-null means an override is active; null means no override. It also
### names the cause of the override. The FK resolver recomputes it from merged
### row, field, and tombstone facts.
overrideReason: CrdtForeignKeyOverrideReason?
indexes:
crdt_data_foreign_key_field_idx:
fields: fieldId
unique: true

15
packages/serverpod_offline_sync_server/lib/src/models/data/foreign_key_override_reason.spy.yaml Normal file
View file

@ -0,0 +1,15 @@
### Why a foreign key projection override is active.
enum: CrdtForeignKeyOverrideReason
serialized: byIndex
values:
### The attempted parent target is hidden or missing and the FK projection
### set the column to null.
- setNull
### The attempted parent target is hidden or missing and the FK projection
### set the column to the default value.
- setDefault
### The attempted parent target is hidden or missing and the FK projection
### cannot repair the value (restrict, no action, cascade, or the column is
### non-nullable), so the row is hidden by projection while the attempt is
### preserved.
- missingParent

12
packages/serverpod_offline_sync_server/lib/src/models/data/row.spy.yaml
View file

@ -22,10 +22,16 @@ fields:
### The node that inserted the row with the HLC timestamp.
node: CrdtNode?, relation
### The visibility CLFlag for this row.
### Materialized visibility state for domain queries.
###
### Only present after the first visibility change event has been recorded.
### If null, the row is visible and has never been deleted/hidden.
### Updated by user tombstone writes and foreign key projection. Visible
### states are listed before hidden states in CrdtDataRowVisibility.
visibility: CrdtDataRowVisibility, default=userInsert
### The synced user tombstone for this row.
###
### Only present after the first user visibility change event has been recorded.
### Projection may keep a visible [visibility] while a user delete tombstone exists.
deleted: CrdtDataDeleted?, relation(name=row_deleted, optional)
### The fields for this row.

13
packages/serverpod_offline_sync_server/lib/src/models/data/row_visibility.spy.yaml Normal file
View file

@ -0,0 +1,13 @@
### Materialized visibility state for a CRDT row.
###
### Visible states are listed first, hidden states last. A row is visible when
### its serialized index is less than or equal to [foreignKeyRestrictRestore].
enum: CrdtDataRowVisibility
serialized: byIndex
values:
- userInsert
- userReinsert
- foreignKeyRestrictRestore
- userDelete
- userCascadeDelete
- foreignKeyCascade

1
test/serverpod_offline_sync_test_server/config/test.yaml
View file

@ -26,3 +26,4 @@ database:
sessionLogs:
persistentEnabled: false
consoleEnabled: true
consoleLogFormat: text

8
test/serverpod_offline_sync_test_server/lib/src/models/fk_chain/cascade_middle.spy.yaml Normal file
View file

@ -0,0 +1,8 @@
class: FkChainCascadeMiddle
table: fk_chain_cascade_middle
database: all
fields:
id: UuidValue?, defaultPersist=random_v7
name: String
root: FkChainRoot?, relation(field=rootId, onDelete=Cascade)
rootId: UuidValue?

8
test/serverpod_offline_sync_test_server/lib/src/models/fk_chain/middle_cascade_child.spy.yaml Normal file
View file

@ -0,0 +1,8 @@
class: FkChainMiddleCascadeChild
table: fk_chain_middle_cascade_child
database: all
fields:
id: UuidValue?, defaultPersist=random_v7
name: String
restrictBlocker: FkChainRestrictBlocker?, relation(field=restrictBlockerId, onDelete=Cascade)
restrictBlockerId: UuidValue?

8
test/serverpod_offline_sync_test_server/lib/src/models/fk_chain/middle_set_null_child.spy.yaml Normal file
View file

@ -0,0 +1,8 @@
class: FkChainMiddleSetNullChild
table: fk_chain_middle_set_null_child
database: all
fields:
id: UuidValue?, defaultPersist=random_v7
name: String
restrictBlocker: FkChainRestrictBlocker?, relation(field=restrictBlockerId, onDelete=SetNull)
restrictBlockerId: UuidValue?

Some files were not shown because too many files have changed in this diff Show more