feat: Implement unique conflict handling to preserve invariants (#64)
This commit is contained in:
parent
60cca9cc85
commit
e74caa1400
10 changed files with 1347 additions and 66 deletions
180
docs/foreign-key-invariants.md
Normal file
180
docs/foreign-key-invariants.md
Normal file
|
|
@ -0,0 +1,180 @@
|
|||
# Preserving foreign key invariants while merging
|
||||
|
||||
## Summary
|
||||
|
||||
After merging remote CRDT operations, the visible database must not contain
|
||||
foreign key violations. The merge result must be associative, commutative, and
|
||||
idempotent: the same merged CRDT facts must produce the same visible rows and
|
||||
visible FK values regardless of arrival order, batching, or replay.
|
||||
|
||||
The safe model is projection-based. User rows, fields, and tombstones remain the
|
||||
CRDT facts. FK repairs are deterministic materialized projections over those
|
||||
facts, not new user field updates emitted opportunistically during merge.
|
||||
|
||||
## Core Policy
|
||||
|
||||
- Keep every row as a CRDT fact. Visibility is derived from row tombstones and
|
||||
schema-driven FK projection.
|
||||
- Keep the user/attempted FK value as the CRDT field value.
|
||||
- Materialize a separate visible FK value only when FK repair requires it. Use
|
||||
`CrdtDataForeignKey` as projection metadata, not as a business conflict class.
|
||||
- Do not create ordinary user field updates for `SET NULL` or `SET DEFAULT`
|
||||
repairs during merge. Doing so makes repair HLCs depend on merge timing.
|
||||
- Recompute FK projection from merged facts over the affected FK closure. The
|
||||
incremental closure result must match a full recomputation oracle.
|
||||
|
||||
## Action Semantics
|
||||
|
||||
For each child FK whose attempted value points to a hidden or missing parent:
|
||||
|
||||
- `ON DELETE SET NULL`: the visible FK value is `null` when the FK is nullable.
|
||||
If the FK is non-nullable, this action cannot repair the row and must fall
|
||||
through to the deterministic blocking policy.
|
||||
- `ON DELETE SET DEFAULT`: the visible FK value is the column default only if
|
||||
that default points to a visible parent or is a legal nullable default. If the
|
||||
default target is hidden or missing, this action cannot repair the row and
|
||||
must fall through to the deterministic blocking policy.
|
||||
- `ON DELETE RESTRICT` and `NO ACTION`: the parent delete loses while a visible
|
||||
child still references the parent. The parent remains visible in the derived
|
||||
projection.
|
||||
- `ON DELETE CASCADE`: the child can be hidden only if the whole cascade closure
|
||||
is valid. If any descendant blocks the cascade through `RESTRICT`, `NO ACTION`,
|
||||
invalid `SET NULL`, or invalid `SET DEFAULT`, the ancestor delete loses and
|
||||
the cascade is not partially applied.
|
||||
|
||||
This makes mixed chains atomic for visibility. For example, if deleting `A`
|
||||
would cascade-delete `B`, but `B` has a visible `RESTRICT` child `C`, then `A`,
|
||||
`B`, and `C` remain visible. The result must not depend on whether cascade or
|
||||
restrict edges are evaluated first internally.
|
||||
|
||||
## Projection Metadata
|
||||
|
||||
`CrdtDataForeignKey` should represent the materialized FK projection for one
|
||||
`CrdtDataField`:
|
||||
|
||||
- `attemptedValue`: the current user-attempted FK value carried by the CRDT
|
||||
field. It may be null for nullable FKs.
|
||||
- `visibleValue`: the value currently materialized into the domain row when an
|
||||
override is active.
|
||||
- `hasOverride`: whether the projection overrides `attemptedValue` and
|
||||
materializes `visibleValue` instead.
|
||||
|
||||
`visibleValue == null` is meaningful only with `hasOverride`. When
|
||||
`hasOverride` is false it means "same as attempted"; when `hasOverride` is true
|
||||
it means "materialize null".
|
||||
|
||||
The resolver may use this metadata for efficient materialization and write
|
||||
deduplication. It must not use it as the authority for FK conflict decisions;
|
||||
the authority is the deterministic resolver over merged CRDT facts.
|
||||
|
||||
## Merge Pipeline
|
||||
|
||||
1. Merge CRDT row, field, and tombstone facts by their existing HLC/CLFlag
|
||||
rules.
|
||||
2. Identify rows touched by inserts, updates, deletes, FK columns, and existing
|
||||
FK projection records.
|
||||
3. Expand to the FK closure in both directions: parents needed by children and
|
||||
children affected by parent visibility.
|
||||
4. Compute effective row visibility and visible FK values to a fixed point.
|
||||
5. Apply only the materialized differences:
|
||||
- update domain FK columns when projection differs from attempted value;
|
||||
- update `CrdtDataForeignKey` projection metadata;
|
||||
- update system tombstones only for deterministic derived visibility where
|
||||
that is part of the chosen storage model.
|
||||
6. Re-run unique conflict projection after FK projection if FK repairs can
|
||||
change unique columns, or define the opposite stage order and prove it with
|
||||
the full recomputation oracle.
|
||||
|
||||
## Invariants
|
||||
|
||||
- No visible row references a hidden or missing parent.
|
||||
- No cascade delete is partially applied across a blocked nested relation.
|
||||
- `SET NULL` and `SET DEFAULT` repairs are pure functions of merged facts.
|
||||
- Replaying the same merge batch produces no metadata churn.
|
||||
- Splitting a merge into smaller batches converges to the same final projection
|
||||
as merging the same operations in one batch.
|
||||
- Incremental recomputation over the affected closure matches full global
|
||||
recomputation.
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Restrict and No Action
|
||||
|
||||
1. Given a parent and a visible child with `ON DELETE RESTRICT`, when one replica
|
||||
deletes the parent and another concurrently updates the child, then the parent
|
||||
remains visible after convergence and no visible orphan appears.
|
||||
|
||||
2. Given a parent with multiple visible restrict children, when the parent is
|
||||
deleted concurrently with updates to different children, then any visible
|
||||
child is enough to keep the parent visible on every replica.
|
||||
|
||||
3. Given a parent kept visible because of restrict, when all restrict children
|
||||
are later deleted or detached, then the parent visibility follows the same
|
||||
deterministic recomputation rule on every replica.
|
||||
|
||||
### Set Null and Set Default
|
||||
|
||||
4. Given a nullable FK with `ON DELETE SET NULL`, when the parent is deleted on
|
||||
another replica, then the child remains visible with a materialized null FK
|
||||
and the user/attempted FK value remains unchanged as a CRDT fact.
|
||||
|
||||
5. Given a child whose FK was materialized to null by `SET NULL`, when the user
|
||||
later changes the FK to a visible parent, then the projection override becomes
|
||||
inactive and the domain row materializes the new user value.
|
||||
|
||||
6. Given `ON DELETE SET DEFAULT` and a visible default target, when the attempted
|
||||
parent is deleted, then the child remains visible with the default FK
|
||||
materialized.
|
||||
|
||||
7. Given `ON DELETE SET DEFAULT` and a hidden or missing default target, when the
|
||||
attempted parent is deleted, then the action cannot repair the child and the
|
||||
deterministic blocking policy keeps the necessary parent visible.
|
||||
|
||||
### Cascade
|
||||
|
||||
8. Given a parent with children under `ON DELETE CASCADE`, when the parent is
|
||||
deleted, then all cascade descendants in the valid cascade closure become
|
||||
hidden and no visible descendant references a hidden ancestor.
|
||||
|
||||
9. Given a child already hidden by a cascade projection, when the same delete
|
||||
facts are merged again, then no additional metadata churn occurs.
|
||||
|
||||
### Mixed Nested Relations
|
||||
|
||||
10. Given `A -> B` is cascade and `B <- C` is restrict, when `A` is deleted
|
||||
concurrently with an update to `C`, then the cascade is blocked and `A`,
|
||||
`B`, and `C` remain visible.
|
||||
|
||||
11. Given a fourth-degree chain with alternating cascade, restrict, set-null,
|
||||
and set-default edges, when a merge touches both a root and a leaf, then
|
||||
incremental recomputation produces the same visible graph and FK projection
|
||||
as full recomputation.
|
||||
|
||||
12. Given an FK cycle that the engine permits, when concurrent deletes and
|
||||
updates happen around the cycle, then fixed-point evaluation terminates and
|
||||
every replica computes the same projection.
|
||||
|
||||
### Cross-Cutting
|
||||
|
||||
13. Given an FK repair changes a unique-indexed column, when unique projection
|
||||
and FK projection are recomputed, then the documented stage order prevents
|
||||
visible unique and FK violations.
|
||||
|
||||
14. Given the same remote operations arrive in one batch or several smaller
|
||||
batches, when all operations have merged, then visible rows, visible FK
|
||||
values, FK projection metadata, unique projection, and tombstones are the
|
||||
same.
|
||||
|
||||
15. Given a merge touches only a non-indexed, non-FK field, when FK projection
|
||||
runs, then unrelated rows are not reconsidered and the visible state outside
|
||||
the affected closure remains unchanged.
|
||||
|
||||
## What These Tests Prove
|
||||
|
||||
- Integrity: no surviving visible FK violation.
|
||||
- Convergence: same merged facts produce the same projection on every replica.
|
||||
- ACI merge behavior: batching, replay, and merge order do not change the final
|
||||
visible result.
|
||||
- Nested safety: cascade cannot bypass restrict/no-action descendants.
|
||||
- Projection discipline: `SET NULL` and `SET DEFAULT` do not become ordinary
|
||||
user CRDT writes.
|
||||
216
docs/unique-constraint-invariants.md
Normal file
216
docs/unique-constraint-invariants.md
Normal file
|
|
@ -0,0 +1,216 @@
|
|||
# Preserving unique constraint invariants while merging
|
||||
|
||||
## Summary
|
||||
|
||||
Concurrent inserts or updates can make multiple CRDT rows claim the same unique
|
||||
value. The current merge model resolves this without hiding rows: every row
|
||||
remains visible, and the losing unique claims are materialized with
|
||||
deterministic conflict-free visible values.
|
||||
|
||||
This is the `flag` policy. It is intentionally simple:
|
||||
|
||||
- all rows remain CRDT facts;
|
||||
- all rows remain visible domain rows;
|
||||
- one deterministic winner keeps the claimed unique value;
|
||||
- each loser keeps its own row identity and receives a conflict-free visible
|
||||
unique value;
|
||||
- no business conflict object is synced;
|
||||
- no foreign key retargeting is needed for unique conflicts.
|
||||
|
||||
This policy is the only unique-conflict policy in scope for the current
|
||||
implementation.
|
||||
|
||||
## Current Policy: Flag
|
||||
|
||||
For each unique index conflict group:
|
||||
|
||||
1. Compute each row's unique claim timestamp from the HLC of the unique-indexed
|
||||
field or fields. For an insert, use the insert HLC and any incoming field HLCs
|
||||
for the indexed fields.
|
||||
2. Pick the winner deterministically by oldest unique-claim HLC, using the same
|
||||
full HLC ordering as the rest of CRDT merge. This includes node id as the HLC
|
||||
tie-break. Row id is only a final fallback for identical HLCs.
|
||||
3. Leave the winner's unique value unchanged.
|
||||
4. Rewrite each loser's visible unique-indexed value to a deterministic
|
||||
conflict-free value:
|
||||
- nullable unique values may be released to `null`;
|
||||
- text values receive a stable `__conflict__<rowId>` suffix;
|
||||
- UUID values receive a deterministic synthetic UUID.
|
||||
5. Keep all rows visible. Application code can decide whether and how to surface
|
||||
the changed visible value as a business conflict.
|
||||
|
||||
The rewritten value is a materialized data value, not a synced conflict class.
|
||||
The core CRDT protocol does not need to expose unique conflict objects to users.
|
||||
|
||||
## Why Flag Is The Current Default
|
||||
|
||||
The `flag` policy avoids the hard second-order effects caused by hiding or
|
||||
merging rows:
|
||||
|
||||
- no child row needs to be moved from one parent row id to another;
|
||||
- `ON DELETE RESTRICT` children do not accidentally keep a unique loser alive;
|
||||
- cascade chains are not triggered by unique conflict resolution;
|
||||
- non-key field updates on both rows remain attached to their original rows;
|
||||
- the algorithm stays local to the conflicting unique index and affected rows.
|
||||
|
||||
This keeps merge behavior associative, commutative, and idempotent without
|
||||
requiring a combined unique/FK fixed-point projection.
|
||||
|
||||
## Deferred Policies
|
||||
|
||||
The following policies are plausible future schema-level options, but they are
|
||||
not part of the current implementation because they require a substantially more
|
||||
complex projection algorithm.
|
||||
|
||||
Both policies should be projections over the canonical `flag` result, not
|
||||
separate unique-resolution algorithms. The `flag` layer computes conflict
|
||||
components, winners, losers, and released loser values once from CRDT facts:
|
||||
|
||||
```text
|
||||
unique facts -> conflict components -> winner + losers + released loser values
|
||||
```
|
||||
|
||||
Future policies then interpret that canonical result:
|
||||
|
||||
```text
|
||||
flag = show all rows; losers show released unique values
|
||||
overwrite = use flag winner; alias/hide losers in the visible projection
|
||||
merge = overwrite projection + deterministic field folding into winner
|
||||
```
|
||||
|
||||
`merge` and `overwrite` must use the original unique claims and conflict
|
||||
components from the `flag` projection. They must not infer conflicts from only
|
||||
the already-released materialized values.
|
||||
|
||||
### Overwrite
|
||||
|
||||
`overwrite` would keep a deterministic winner visible and hide the losers.
|
||||
|
||||
To be safe, this cannot simply tombstone the loser. A hidden loser may already
|
||||
be referenced by visible children, including children using `ON DELETE RESTRICT`
|
||||
or deeper mixed cascade/restrict chains. A safe overwrite policy therefore needs
|
||||
an internal row-alias projection:
|
||||
|
||||
```text
|
||||
loserRowId -> winnerRowId
|
||||
```
|
||||
|
||||
Foreign key projection would then consume the alias map and materialize child FK
|
||||
values through the winner where legal. Without this aliasing step, overwrite can
|
||||
produce visible orphans or order-dependent outcomes.
|
||||
|
||||
### Merge
|
||||
|
||||
`merge` would also pick a deterministic winner and alias loser rows to it, but
|
||||
would additionally fold selected loser fields into the winner.
|
||||
|
||||
This needs explicit deterministic per-column policies, such as:
|
||||
|
||||
- `winnerWins`
|
||||
- `latestFieldWinsAcrossMergedRows`
|
||||
- `preferNonNullLatest`
|
||||
- `ignore`
|
||||
|
||||
These policies must be schema-defined, deterministic on every replica, and
|
||||
included in schema compatibility checks. Arbitrary app callbacks inside core
|
||||
merge would not be safe unless they are deterministic, versioned, and available
|
||||
identically on every replica.
|
||||
|
||||
## Why Merge And Overwrite Are Deferred
|
||||
|
||||
`merge` and `overwrite` combine unique projection with foreign key projection.
|
||||
The resolver would need to compute, at minimum:
|
||||
|
||||
- unique conflict groups;
|
||||
- winners and losers for each group;
|
||||
- row aliases from loser ids to winner ids;
|
||||
- hidden-row projection;
|
||||
- field projections for merged rows;
|
||||
- foreign key repairs over aliases;
|
||||
- nested cascade/restrict/set-null/set-default effects;
|
||||
- fixed-point recomputation when FK repairs touch unique-indexed values.
|
||||
|
||||
That interaction can grow quickly with graph depth, shared children, cycles, and
|
||||
rows participating in multiple unique indexes. The implementation would need a
|
||||
full recomputation oracle and incremental-closure equivalence tests before it is
|
||||
safe enough for real data.
|
||||
|
||||
For now, the `flag` policy gives a safer and smaller invariant:
|
||||
|
||||
```text
|
||||
unique projection changes values, not row identity or row visibility
|
||||
```
|
||||
|
||||
Foreign key repair can then be developed independently as a projection over row
|
||||
visibility and FK values, without also needing row aliasing and field folding.
|
||||
|
||||
## Invariants
|
||||
|
||||
- No visible duplicate remains for a unique index after projection.
|
||||
- Unique conflict resolution is deterministic from merged CRDT facts.
|
||||
- Merge replay does not create additional metadata churn.
|
||||
- Splitting the same operations into different sync batches converges to the
|
||||
same visible result.
|
||||
- Unique projection does not emit app-visible CRDT conflict objects.
|
||||
- Current `flag` projection does not hide rows or retarget foreign keys.
|
||||
|
||||
## Test Cases
|
||||
|
||||
### Current Flag Policy
|
||||
|
||||
1. Given two replicas concurrently insert different rows with the same unique
|
||||
value, when they merge, then both rows remain visible, the deterministic
|
||||
winner keeps the original value, and the loser receives a deterministic
|
||||
conflict-free value.
|
||||
|
||||
2. Given an older insert conflicts with a newer update on another row, when they
|
||||
merge, then the row with the older unique-field claim keeps the unique value,
|
||||
regardless of the other row's insert time.
|
||||
|
||||
3. Given two rows concurrently update the same unique field to the same value,
|
||||
when they merge, then both rows remain visible and only the deterministic
|
||||
winner keeps the claimed value.
|
||||
|
||||
4. Given the same conflict arrives in one batch or several smaller batches, when
|
||||
all operations have merged, then the same row keeps the unique value and the
|
||||
same rows receive the same conflict-free values.
|
||||
|
||||
5. Given a conflicting unique value is a nullable unique column, when the loser
|
||||
can be made conflict-free by setting the materialized value to null, then the
|
||||
loser remains visible and the unique index remains valid.
|
||||
|
||||
6. Given a conflicting unique value is a UUID column that is not an FK column,
|
||||
when the loser is released, then the materialized UUID is deterministic from
|
||||
the table, column, original value, and loser row id.
|
||||
|
||||
### Deferred Merge And Overwrite Policies
|
||||
|
||||
7. Given overwrite is enabled for a unique index, when a loser has visible
|
||||
restrict children, then FK projection retargets those children through the
|
||||
row alias or blocks the projection deterministically; no visible orphan is
|
||||
produced.
|
||||
|
||||
8. Given merge is enabled for a unique index, when loser rows contain newer
|
||||
non-key field values, then field folding follows documented per-column
|
||||
policies and every replica computes the same winner row.
|
||||
|
||||
9. Given a row participates in multiple unique conflict groups, when merge or
|
||||
overwrite projection runs, then row aliases are deterministic and do not form
|
||||
cycles.
|
||||
|
||||
10. Given FK repair changes a unique-indexed visible value, when combined unique
|
||||
and FK projection runs, then the fixed-point result matches a full
|
||||
recomputation oracle.
|
||||
|
||||
These deferred tests should not be implemented until the row-alias and combined
|
||||
projection model exists.
|
||||
|
||||
## What These Tests Prove
|
||||
|
||||
- Current implementation safety: the `flag` policy preserves uniqueness without
|
||||
row hiding or FK retargeting.
|
||||
- Convergence: same CRDT facts produce the same visible unique values.
|
||||
- Idempotence: repeated merges do not keep rewriting the same conflict.
|
||||
- Scope control: future `merge` and `overwrite` behavior is documented but not
|
||||
treated as supported until the projection algorithm can handle second-order FK
|
||||
effects.
|
||||
|
|
@ -2,9 +2,18 @@ part of 'recorder.dart';
|
|||
|
||||
typedef _MergeRowKey = (String, UuidValue);
|
||||
typedef _MergeFieldKey = (String, UuidValue, String);
|
||||
typedef _MergeContext = ({
|
||||
Map<_MergeRowKey, CrdtDataRow> rows,
|
||||
Map<_MergeFieldKey, CrdtDataField> fields,
|
||||
Map<_MergeFieldKey, Hlc> incomingFieldHlcs,
|
||||
Map<_MergeRowKey, CrdtDataDeleted> tombstones,
|
||||
});
|
||||
|
||||
/// Adds merge-specific behavior to [CrdtMutationRecorder].
|
||||
extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
||||
CrdtUniqueConflictResolver get _uniqueConflictResolver =>
|
||||
CrdtUniqueConflictResolver(this);
|
||||
|
||||
/// Locks the current user row so merges can serialize with other work.
|
||||
Future<void> lockCurrentUser(Transaction transaction) async {
|
||||
final user = _getEffectiveUser(transaction);
|
||||
|
|
@ -33,7 +42,7 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
|||
{for (final change in operations) change.uuidNodeId},
|
||||
transaction,
|
||||
);
|
||||
final metadata = await _loadMergeMetadata(mergeSet, transaction);
|
||||
final context = await _loadMergeContext(mergeSet, transaction);
|
||||
|
||||
for (final operation in operations) {
|
||||
if (!_isCrdtTrackedTableName(operation.tableName)) {
|
||||
|
|
@ -45,25 +54,21 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
|||
await _applyMergeInsert(
|
||||
insert,
|
||||
remoteNodes,
|
||||
metadata.rows,
|
||||
metadata.fields,
|
||||
metadata.tombstones,
|
||||
context,
|
||||
transaction,
|
||||
);
|
||||
case final CrdtMergeUpdate update:
|
||||
await _applyMergeUpdate(
|
||||
update,
|
||||
remoteNodes,
|
||||
metadata.rows,
|
||||
metadata.fields,
|
||||
context,
|
||||
transaction,
|
||||
);
|
||||
case final CrdtMergeDelete delete:
|
||||
await _applyMergeDelete(
|
||||
delete,
|
||||
remoteNodes,
|
||||
metadata.rows,
|
||||
metadata.tombstones,
|
||||
context,
|
||||
transaction,
|
||||
);
|
||||
}
|
||||
|
|
@ -171,14 +176,7 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
|||
};
|
||||
}
|
||||
|
||||
Future<
|
||||
({
|
||||
Map<_MergeRowKey, CrdtDataRow> rows,
|
||||
Map<_MergeFieldKey, CrdtDataField> fields,
|
||||
Map<_MergeRowKey, CrdtDataDeleted> tombstones,
|
||||
})
|
||||
>
|
||||
_loadMergeMetadata(
|
||||
Future<_MergeContext> _loadMergeContext(
|
||||
CrdtMergeSet mergeSet,
|
||||
Transaction transaction,
|
||||
) async {
|
||||
|
|
@ -188,6 +186,10 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
|||
|
||||
final rows = <_MergeRowKey, CrdtDataRow>{};
|
||||
final fields = <_MergeFieldKey, CrdtDataField>{};
|
||||
final incomingFieldHlcs = {
|
||||
for (final update in mergeSet.whereType<CrdtMergeUpdate>())
|
||||
(update.tableName, update.uuidRowId, update.columnName): update.hlc,
|
||||
};
|
||||
final tombstones = <_MergeRowKey, CrdtDataDeleted>{};
|
||||
|
||||
for (final MapEntry(key: tableName, value: rowIds) in rowIdsByTable.entries) {
|
||||
|
|
@ -245,6 +247,7 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
|||
return (
|
||||
rows: rows,
|
||||
fields: fields,
|
||||
incomingFieldHlcs: incomingFieldHlcs,
|
||||
tombstones: tombstones,
|
||||
);
|
||||
}
|
||||
|
|
@ -252,16 +255,14 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
|||
Future<void> _applyMergeInsert(
|
||||
CrdtMergeInsert insert,
|
||||
Map<UuidValue, CrdtNode> remoteNodes,
|
||||
Map<_MergeRowKey, CrdtDataRow> rows,
|
||||
Map<_MergeFieldKey, CrdtDataField> fields,
|
||||
Map<_MergeRowKey, CrdtDataDeleted> tombstones,
|
||||
_MergeContext context,
|
||||
Transaction transaction,
|
||||
) async {
|
||||
final rowKey = (insert.tableName, insert.uuidRowId);
|
||||
final incomingHlc = insert.hlc;
|
||||
final remoteNode = _requireRemoteNode(remoteNodes, insert.uuidNodeId);
|
||||
|
||||
final currentRow = rows[rowKey];
|
||||
final currentRow = context.rows[rowKey];
|
||||
if (currentRow != null && incomingHlc <= currentRow.hlc) {
|
||||
return;
|
||||
}
|
||||
|
|
@ -269,11 +270,18 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
|||
final data = _sanitizeMergeRowData(insert.tableName, insert.databaseColumns);
|
||||
|
||||
if (currentRow == null) {
|
||||
rows[rowKey] = await _applyMergeInsertForMissingRow(
|
||||
final resolvedInsert = await _uniqueConflictResolver._resolveForIncomingInsert(
|
||||
insert,
|
||||
data,
|
||||
context,
|
||||
transaction,
|
||||
);
|
||||
|
||||
context.rows[rowKey] = await _applyMergeInsertForMissingRow(
|
||||
insert,
|
||||
remoteNode,
|
||||
incomingHlc,
|
||||
data,
|
||||
resolvedInsert.data,
|
||||
transaction,
|
||||
);
|
||||
} else {
|
||||
|
|
@ -283,8 +291,7 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
|||
remoteNode,
|
||||
incomingHlc,
|
||||
data,
|
||||
fields,
|
||||
tombstones,
|
||||
context,
|
||||
transaction,
|
||||
);
|
||||
}
|
||||
|
|
@ -293,12 +300,11 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
|||
Future<void> _applyMergeUpdate(
|
||||
CrdtMergeUpdate update,
|
||||
Map<UuidValue, CrdtNode> remoteNodes,
|
||||
Map<_MergeRowKey, CrdtDataRow> rows,
|
||||
Map<_MergeFieldKey, CrdtDataField> fields,
|
||||
_MergeContext context,
|
||||
Transaction transaction,
|
||||
) async {
|
||||
final rowKey = (update.tableName, update.uuidRowId);
|
||||
final row = rows[rowKey];
|
||||
final row = context.rows[rowKey];
|
||||
if (row == null) return;
|
||||
|
||||
final shouldApply = await _shouldMergeFieldMetadataIfNewer(
|
||||
|
|
@ -308,15 +314,22 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
|||
row: row,
|
||||
remoteNode: _requireRemoteNode(remoteNodes, update.uuidNodeId),
|
||||
incomingHlc: update.hlc,
|
||||
fields: fields,
|
||||
fields: context.fields,
|
||||
transaction: transaction,
|
||||
);
|
||||
|
||||
if (!shouldApply) return;
|
||||
|
||||
final resolvedUpdates = await _uniqueConflictResolver._resolveForIncomingUpdate(
|
||||
update,
|
||||
row,
|
||||
context,
|
||||
transaction,
|
||||
);
|
||||
await _updateDomainRows(
|
||||
update.tableName,
|
||||
{update.uuidRowId},
|
||||
{update.columnName: update.value},
|
||||
resolvedUpdates,
|
||||
transaction,
|
||||
);
|
||||
}
|
||||
|
|
@ -324,21 +337,20 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
|||
Future<void> _applyMergeDelete(
|
||||
CrdtMergeDelete delete,
|
||||
Map<UuidValue, CrdtNode> remoteNodes,
|
||||
Map<_MergeRowKey, CrdtDataRow> rows,
|
||||
Map<_MergeRowKey, CrdtDataDeleted> tombstones,
|
||||
_MergeContext context,
|
||||
Transaction transaction,
|
||||
) async {
|
||||
final rowKey = (delete.tableName, delete.uuidRowId);
|
||||
final row = rows[rowKey];
|
||||
final row = context.rows[rowKey];
|
||||
if (row == null) return;
|
||||
|
||||
final currentTombstone = tombstones[rowKey];
|
||||
final currentTombstone = context.tombstones[rowKey];
|
||||
final currentClFlag = currentTombstone?.clFlag ?? 1;
|
||||
final currentHlc = currentTombstone?.hlc ?? row.hlc;
|
||||
if (delete.clFlag < currentClFlag) return;
|
||||
if (delete.clFlag == currentClFlag && delete.hlc <= currentHlc) return;
|
||||
|
||||
tombstones[rowKey] = await _upsertMergeTombstone(
|
||||
context.tombstones[rowKey] = await _upsertMergeTombstone(
|
||||
row,
|
||||
_requireRemoteNode(remoteNodes, delete.uuidNodeId),
|
||||
delete.hlc,
|
||||
|
|
@ -503,17 +515,18 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
|||
transaction,
|
||||
);
|
||||
|
||||
final row = _requireMergeInsertTableRow(insert);
|
||||
final tableRow = _requireMergeInsertTableRow(insert);
|
||||
final patchedRow = _db.serializationManager.patchTableRow(tableRow, data);
|
||||
try {
|
||||
await _db.updateRow(
|
||||
row,
|
||||
columns: row.table.managedColumns
|
||||
patchedRow,
|
||||
columns: patchedRow.table.managedColumns
|
||||
.where((c) => data.containsKey(c.columnName))
|
||||
.toList(),
|
||||
transaction: transaction,
|
||||
);
|
||||
} on DatabaseUpdateRowException {
|
||||
await _db.insertRow(row, transaction: transaction);
|
||||
await _db.insertRow(patchedRow, transaction: transaction);
|
||||
}
|
||||
|
||||
return insertedCrdtRow;
|
||||
|
|
@ -525,8 +538,7 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
|||
CrdtNode remoteNode,
|
||||
Hlc incomingHlc,
|
||||
Map<String, Object?> data,
|
||||
Map<_MergeFieldKey, CrdtDataField> fields,
|
||||
Map<_MergeRowKey, CrdtDataDeleted> tombstones,
|
||||
_MergeContext context,
|
||||
Transaction transaction,
|
||||
) async {
|
||||
final (_, columnsByName) = _schema[insert.tableName]!;
|
||||
|
|
@ -542,7 +554,7 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
|||
row: currentRow,
|
||||
remoteNode: remoteNode,
|
||||
incomingHlc: incomingHlc,
|
||||
fields: fields,
|
||||
fields: context.fields,
|
||||
transaction: transaction,
|
||||
schemaColumn: schemaColumn,
|
||||
);
|
||||
|
|
@ -551,20 +563,31 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
|||
updatedValues[columnName] = data[columnName];
|
||||
}
|
||||
|
||||
if (updatedValues.isNotEmpty) {
|
||||
var updatesToApply = updatedValues;
|
||||
if (updatesToApply.isNotEmpty) {
|
||||
updatesToApply = await _uniqueConflictResolver._resolveForIncomingUpdates(
|
||||
tableName: insert.tableName,
|
||||
row: currentRow,
|
||||
updates: updatesToApply,
|
||||
context: context,
|
||||
transaction: transaction,
|
||||
);
|
||||
}
|
||||
|
||||
if (updatesToApply.isNotEmpty) {
|
||||
await _updateDomainRows(
|
||||
insert.tableName,
|
||||
{insert.uuidRowId},
|
||||
updatedValues,
|
||||
updatesToApply,
|
||||
transaction,
|
||||
);
|
||||
}
|
||||
|
||||
final rowKey = (insert.tableName, insert.uuidRowId);
|
||||
final currentTombstone = tombstones[rowKey];
|
||||
final currentTombstone = context.tombstones[rowKey];
|
||||
if ((currentTombstone?.clFlag ?? 1) == 1 &&
|
||||
incomingHlc > (currentTombstone?.hlc ?? currentRow.hlc)) {
|
||||
tombstones[rowKey] = await _upsertMergeTombstone(
|
||||
context.tombstones[rowKey] = await _upsertMergeTombstone(
|
||||
currentRow,
|
||||
remoteNode,
|
||||
incomingHlc,
|
||||
|
|
@ -640,3 +663,16 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
|||
};
|
||||
}
|
||||
}
|
||||
|
||||
extension on DatabaseSerializationManager {
|
||||
/// Patches a [TableRow] with the given data.
|
||||
T patchTableRow<T extends TableRow>(T row, Map<String, Object?> data) {
|
||||
final rowJson = Map<String, dynamic>.from(row.toJson() as Map);
|
||||
for (final column in row.table.managedColumns) {
|
||||
if (column.columnName != 'id' && data.containsKey(column.columnName)) {
|
||||
rowJson[column.fieldName] = data[column.columnName];
|
||||
}
|
||||
}
|
||||
return deserialize<T>(rowJson);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,379 @@
|
|||
part of '../recorder.dart';
|
||||
|
||||
typedef _UniqueConflict = ({
|
||||
CrdtDataRow row,
|
||||
_UniqueIndexConflictRelease uniqueIndex,
|
||||
});
|
||||
|
||||
/// Resolves unique constraint conflicts that appear while merging CRDT changes.
|
||||
class CrdtUniqueConflictResolver {
|
||||
/// Creates a resolver for the merge recorder.
|
||||
CrdtUniqueConflictResolver(this._recorder);
|
||||
|
||||
final CrdtMutationRecorder _recorder;
|
||||
|
||||
/// Resolves unique conflicts for an incoming insert.
|
||||
Future<({Map<String, Object?> data, bool changed})> _resolveForIncomingInsert(
|
||||
CrdtMergeInsert insert,
|
||||
Map<String, Object?> data,
|
||||
_MergeContext context,
|
||||
Transaction transaction,
|
||||
) async {
|
||||
final resolvedData = Map<String, Object?>.from(data);
|
||||
var changed = false;
|
||||
final conflicts = await _findVisibleUniqueConflicts(
|
||||
tableName: insert.tableName,
|
||||
rowId: insert.uuidRowId,
|
||||
values: data,
|
||||
transaction: transaction,
|
||||
);
|
||||
|
||||
for (final conflict in conflicts) {
|
||||
final incomingClaimHlc = _incomingInsertUniqueClaim(
|
||||
insert: insert,
|
||||
uniqueIndex: conflict.uniqueIndex,
|
||||
context: context,
|
||||
);
|
||||
final conflictClaimHlc = await _uniqueIndexHlc(
|
||||
tableName: insert.tableName,
|
||||
row: conflict.row,
|
||||
uniqueIndex: conflict.uniqueIndex,
|
||||
fields: context.fields,
|
||||
transaction: transaction,
|
||||
);
|
||||
|
||||
if (incomingClaimHlc > conflictClaimHlc) {
|
||||
resolvedData.addAll(
|
||||
_uniqueConflictFreeValuesForData(
|
||||
insert.tableName,
|
||||
insert.uuidRowId,
|
||||
resolvedData,
|
||||
conflict.uniqueIndex,
|
||||
),
|
||||
);
|
||||
changed = true;
|
||||
} else {
|
||||
await _releaseUniqueConflictForRow(
|
||||
tableName: insert.tableName,
|
||||
rowId: conflict.row.uuidRowId,
|
||||
uniqueIndex: conflict.uniqueIndex,
|
||||
transaction: transaction,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
return (data: resolvedData, changed: changed);
|
||||
}
|
||||
|
||||
Hlc _incomingInsertUniqueClaim({
|
||||
required CrdtMergeInsert insert,
|
||||
required _UniqueIndexConflictRelease uniqueIndex,
|
||||
required _MergeContext context,
|
||||
}) {
|
||||
var uniqueHlc = insert.hlc;
|
||||
for (final column in uniqueIndex.columns) {
|
||||
uniqueHlc = uniqueHlc.maxBetween(
|
||||
context.incomingFieldHlcs[(
|
||||
insert.tableName,
|
||||
insert.uuidRowId,
|
||||
column.columnName,
|
||||
)],
|
||||
);
|
||||
}
|
||||
return uniqueHlc;
|
||||
}
|
||||
|
||||
/// Resolves unique conflicts for a single incoming field update.
|
||||
Future<Map<String, Object?>> _resolveForIncomingUpdate(
|
||||
CrdtMergeUpdate update,
|
||||
CrdtDataRow row,
|
||||
_MergeContext context,
|
||||
Transaction transaction,
|
||||
) {
|
||||
return _resolveForIncomingUpdates(
|
||||
tableName: update.tableName,
|
||||
row: row,
|
||||
updates: {update.columnName: update.value},
|
||||
context: context,
|
||||
transaction: transaction,
|
||||
);
|
||||
}
|
||||
|
||||
/// Resolves unique conflicts for one or more incoming field updates.
|
||||
Future<Map<String, Object?>> _resolveForIncomingUpdates({
|
||||
required String tableName,
|
||||
required CrdtDataRow row,
|
||||
required Map<String, Object?> updates,
|
||||
required _MergeContext context,
|
||||
required Transaction transaction,
|
||||
}) async {
|
||||
if (updates.keys.every(
|
||||
(columnName) => !_isUniqueIndexedColumn(tableName, columnName),
|
||||
)) {
|
||||
return updates;
|
||||
}
|
||||
|
||||
final tableDefinition = _recorder._tableDefinitionsByName[tableName];
|
||||
if (tableDefinition == null) {
|
||||
return updates;
|
||||
}
|
||||
|
||||
final uniqueColumnNamesToRead = {
|
||||
for (final uniqueIndex in _recorder._uniqueIndexesForTable(tableDefinition))
|
||||
for (final column in uniqueIndex.columns) column.columnName,
|
||||
};
|
||||
if (uniqueColumnNamesToRead.isEmpty) {
|
||||
return updates;
|
||||
}
|
||||
|
||||
final currentValues = await _recorder._readDomainColumnValues(
|
||||
tableName,
|
||||
{row.uuidRowId},
|
||||
uniqueColumnNamesToRead.toList(),
|
||||
transaction,
|
||||
);
|
||||
final values = {
|
||||
...?currentValues[row.uuidRowId],
|
||||
...updates,
|
||||
};
|
||||
|
||||
final conflicts = await _findVisibleUniqueConflicts(
|
||||
tableName: tableName,
|
||||
rowId: row.uuidRowId,
|
||||
values: values,
|
||||
transaction: transaction,
|
||||
);
|
||||
|
||||
final resolvedUpdates = Map<String, Object?>.from(updates);
|
||||
for (final conflict in conflicts) {
|
||||
final rowClaimHlc = await _uniqueIndexHlc(
|
||||
tableName: tableName,
|
||||
row: row,
|
||||
uniqueIndex: conflict.uniqueIndex,
|
||||
fields: context.fields,
|
||||
transaction: transaction,
|
||||
);
|
||||
final conflictClaimHlc = await _uniqueIndexHlc(
|
||||
tableName: tableName,
|
||||
row: conflict.row,
|
||||
uniqueIndex: conflict.uniqueIndex,
|
||||
fields: context.fields,
|
||||
transaction: transaction,
|
||||
);
|
||||
|
||||
if (rowClaimHlc > conflictClaimHlc) {
|
||||
final releasedValues = _uniqueConflictFreeValuesForData(
|
||||
tableName,
|
||||
row.uuidRowId,
|
||||
values,
|
||||
conflict.uniqueIndex,
|
||||
);
|
||||
for (final columnName in conflict.uniqueIndex.columnNames) {
|
||||
if (releasedValues.containsKey(columnName)) {
|
||||
resolvedUpdates[columnName] = releasedValues[columnName];
|
||||
values[columnName] = releasedValues[columnName];
|
||||
}
|
||||
}
|
||||
} else {
|
||||
await _releaseUniqueConflictForRow(
|
||||
tableName: tableName,
|
||||
rowId: conflict.row.uuidRowId,
|
||||
uniqueIndex: conflict.uniqueIndex,
|
||||
transaction: transaction,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
return resolvedUpdates;
|
||||
}
|
||||
|
||||
Map<String, Object?> _uniqueConflictFreeValuesForData(
|
||||
String tableName,
|
||||
UuidValue rowId,
|
||||
Map<String, Object?> data,
|
||||
_UniqueIndexConflictRelease uniqueIndex,
|
||||
) {
|
||||
final tableDefinition = _recorder._tableDefinitionsByName[tableName];
|
||||
if (tableDefinition == null) return data;
|
||||
|
||||
final released = Map<String, Object?>.from(data);
|
||||
final values = {
|
||||
for (final column in uniqueIndex.columns)
|
||||
column.columnName: released[column.columnName],
|
||||
};
|
||||
if (values.values.any((value) => value == null)) return released;
|
||||
|
||||
for (final column in uniqueIndex.columns) {
|
||||
released[column.columnName] = _recorder._conflictFreeValue(
|
||||
column,
|
||||
values[column.columnName],
|
||||
tableName,
|
||||
rowId,
|
||||
'conflict',
|
||||
);
|
||||
}
|
||||
return released;
|
||||
}
|
||||
|
||||
Future<void> _releaseUniqueConflictForRow({
|
||||
required String tableName,
|
||||
required UuidValue rowId,
|
||||
required _UniqueIndexConflictRelease uniqueIndex,
|
||||
required Transaction transaction,
|
||||
}) async {
|
||||
final columnNames = uniqueIndex.columnNames.toList();
|
||||
final valuesByRowId = await _recorder._readDomainColumnValues(
|
||||
tableName,
|
||||
{rowId},
|
||||
columnNames,
|
||||
transaction,
|
||||
);
|
||||
final values = valuesByRowId[rowId];
|
||||
if (values == null || values.values.any((value) => value == null)) return;
|
||||
|
||||
final releasedValues = _uniqueConflictFreeValuesForData(
|
||||
tableName,
|
||||
rowId,
|
||||
values,
|
||||
uniqueIndex,
|
||||
);
|
||||
await _recorder._updateDomainRows(tableName, {rowId}, releasedValues, transaction);
|
||||
}
|
||||
|
||||
Future<List<_UniqueConflict>> _findVisibleUniqueConflicts({
|
||||
required String tableName,
|
||||
required UuidValue rowId,
|
||||
required Map<String, Object?> values,
|
||||
required Transaction transaction,
|
||||
}) async {
|
||||
final tableDefinition = _recorder._tableDefinitionsByName[tableName];
|
||||
if (tableDefinition == null) return const [];
|
||||
|
||||
final uniqueIndexes = _recorder._uniqueIndexesForTable(tableDefinition);
|
||||
if (uniqueIndexes.isEmpty) return const [];
|
||||
|
||||
final uniqueIndexesByConflictId = <UuidValue, Set<_UniqueIndexConflictRelease>>{};
|
||||
for (final uniqueIndex in uniqueIndexes) {
|
||||
final columnValues = {
|
||||
for (final column in uniqueIndex.columns)
|
||||
if (values.containsKey(column.columnName))
|
||||
column.columnName: values[column.columnName],
|
||||
};
|
||||
if (columnValues.length != uniqueIndex.columns.length) continue;
|
||||
if (columnValues.values.any((value) => value == null)) continue;
|
||||
|
||||
final uniqueConflictIds = await _findVisibleRowsByUniqueValues(
|
||||
tableName: tableName,
|
||||
rowId: rowId,
|
||||
values: columnValues,
|
||||
transaction: transaction,
|
||||
);
|
||||
for (final conflictId in uniqueConflictIds) {
|
||||
uniqueIndexesByConflictId.putIfAbsent(conflictId, () => {}).add(uniqueIndex);
|
||||
}
|
||||
}
|
||||
|
||||
if (uniqueIndexesByConflictId.isEmpty) return const [];
|
||||
final conflictRows = await _recorder._findCrdtRows(
|
||||
tableName,
|
||||
uniqueIndexesByConflictId.keys.toSet(),
|
||||
transaction,
|
||||
include: CrdtDataRow.include(
|
||||
node: CrdtNode.include(),
|
||||
deleted: CrdtDataDeleted.include(node: CrdtNode.include()),
|
||||
),
|
||||
);
|
||||
return [
|
||||
for (final conflictRow in conflictRows)
|
||||
for (final uniqueIndex in uniqueIndexesByConflictId[conflictRow.uuidRowId]!)
|
||||
(
|
||||
row: conflictRow,
|
||||
uniqueIndex: uniqueIndex,
|
||||
),
|
||||
];
|
||||
}
|
||||
|
||||
/// Returns the HLC for this row's current claim on [uniqueIndex].
|
||||
///
|
||||
/// Composite indexes use the newest field HLC among their columns; rows with
|
||||
/// no field metadata fall back to the row insert HLC. Missing field metadata
|
||||
/// is loaded lazily because conflicting rows may not be part of the preloaded
|
||||
/// merge context.
|
||||
Future<Hlc> _uniqueIndexHlc({
|
||||
required String tableName,
|
||||
required CrdtDataRow row,
|
||||
required _UniqueIndexConflictRelease uniqueIndex,
|
||||
required Map<_MergeFieldKey, CrdtDataField> fields,
|
||||
required Transaction transaction,
|
||||
}) async {
|
||||
final (_, columnsByName) = _recorder._schema[tableName]!;
|
||||
final uniqueColumnNames = {
|
||||
for (final column in uniqueIndex.columns) column.columnName,
|
||||
};
|
||||
final missingColumnIds = <int>{};
|
||||
final maxFieldHlc = <Hlc>[];
|
||||
for (final columnName in uniqueColumnNames) {
|
||||
final field = fields[(tableName, row.uuidRowId, columnName)];
|
||||
if (field == null) {
|
||||
final columnId = columnsByName[columnName]?.id;
|
||||
if (columnId != null) missingColumnIds.add(columnId);
|
||||
continue;
|
||||
}
|
||||
|
||||
maxFieldHlc.add(field.hlc);
|
||||
}
|
||||
|
||||
if (missingColumnIds.isNotEmpty && row.id != null) {
|
||||
final loadedFields = await CrdtDataField.db.find(
|
||||
_recorder._session,
|
||||
where: (t) => t.rowId.equals(row.id) & t.columnId.inSet(missingColumnIds),
|
||||
include: CrdtDataField.include(
|
||||
column: CrdtSchemaColumn.include(),
|
||||
node: CrdtNode.include(),
|
||||
),
|
||||
transaction: transaction,
|
||||
);
|
||||
|
||||
for (final field in loadedFields) {
|
||||
final columnName = field.column?.name;
|
||||
if (columnName == null) continue;
|
||||
fields[(tableName, row.uuidRowId, columnName)] = field;
|
||||
maxFieldHlc.add(field.hlc);
|
||||
}
|
||||
}
|
||||
|
||||
if (maxFieldHlc.isEmpty) return row.hlc;
|
||||
return maxFieldHlc.reduce((left, right) => left.maxBetween(right));
|
||||
}
|
||||
|
||||
Future<Set<UuidValue>> _findVisibleRowsByUniqueValues({
|
||||
required String tableName,
|
||||
required UuidValue rowId,
|
||||
required Map<String, Object?> values,
|
||||
required Transaction transaction,
|
||||
}) async {
|
||||
return _recorder._findVisibleDomainRowIdsWhere(
|
||||
tableName: tableName,
|
||||
predicates: [
|
||||
for (final MapEntry(key: columnName, value: value) in values.entries)
|
||||
'd."${_escapeIdentifier(columnName)}" = ${_sqlLiteral(value)}',
|
||||
'd."id" <> ${_sqlLiteral(rowId)}',
|
||||
],
|
||||
transaction: transaction,
|
||||
);
|
||||
}
|
||||
|
||||
bool _isUniqueIndexedColumn(String tableName, String columnName) {
|
||||
final tableDefinition = _recorder._tableDefinitionsByName[tableName];
|
||||
if (tableDefinition == null) return false;
|
||||
return _recorder
|
||||
._uniqueIndexesForTable(tableDefinition)
|
||||
.any((i) => i.columns.any((c) => c.columnName == columnName));
|
||||
}
|
||||
}
|
||||
|
||||
extension on _UniqueIndexConflictRelease {
|
||||
Set<String> get columnNames => {
|
||||
for (final column in columns) column.columnName,
|
||||
};
|
||||
}
|
||||
|
|
@ -12,6 +12,7 @@ import 'schema.dart';
|
|||
import 'session.dart';
|
||||
|
||||
part 'merge.dart';
|
||||
part 'merge_utils/unique_resolver.dart';
|
||||
|
||||
typedef _ReferencingForeignKey = ({
|
||||
String childTableName,
|
||||
|
|
@ -893,6 +894,7 @@ WHERE c."id" IN ($whereRowIds)
|
|||
values[column.columnName],
|
||||
tableDefinition.name,
|
||||
rowId,
|
||||
'deleted',
|
||||
);
|
||||
}
|
||||
|
||||
|
|
@ -925,6 +927,18 @@ WHERE c."id" IN ($whereRowIds)
|
|||
required String columnName,
|
||||
required Object? value,
|
||||
required Transaction transaction,
|
||||
}) async {
|
||||
return _findVisibleDomainRowIdsWhere(
|
||||
tableName: tableName,
|
||||
predicates: ['d."${_escapeIdentifier(columnName)}" = ${_sqlLiteral(value)}'],
|
||||
transaction: transaction,
|
||||
);
|
||||
}
|
||||
|
||||
Future<Set<UuidValue>> _findVisibleDomainRowIdsWhere({
|
||||
required String tableName,
|
||||
required List<String> predicates,
|
||||
required Transaction transaction,
|
||||
}) async {
|
||||
final (tableId, _) = _schema[tableName]!;
|
||||
final userId = _getHlcManager(transaction).normalizedUserId;
|
||||
|
|
@ -936,7 +950,7 @@ LEFT JOIN "crdt_data_rows" r
|
|||
ON r."userId" = $userId AND r."tblId" = $tableId AND r."uuidRowId" = d."id"
|
||||
LEFT JOIN "crdt_data_tombstone" tomb
|
||||
ON tomb."rowId" = r."id"
|
||||
WHERE d."${_escapeIdentifier(columnName)}" = ${_sqlLiteral(value)}
|
||||
WHERE (${predicates.join(') AND (')})
|
||||
AND (tomb."id" IS NULL OR tomb."clFlag" % 2 = 1)
|
||||
''',
|
||||
transaction: transaction,
|
||||
|
|
@ -1017,21 +1031,20 @@ WHERE "id" IN (${_sqlLiteralList(rowIds)})
|
|||
Object? value,
|
||||
String tableName,
|
||||
UuidValue conflictingId,
|
||||
String releaseSuffix,
|
||||
) {
|
||||
switch (column.kind) {
|
||||
case _UniqueConflictReleaseKind.setNull:
|
||||
return null;
|
||||
case _UniqueConflictReleaseKind.textSuffix:
|
||||
if (value is String) {
|
||||
return '${value}__deleted__${conflictingId.uuid}';
|
||||
return '${value}__${releaseSuffix}__${conflictingId.uuid}';
|
||||
}
|
||||
case _UniqueConflictReleaseKind.syntheticUuid:
|
||||
if (value != null) {
|
||||
return _syntheticDeletedUuid(
|
||||
tableName,
|
||||
column.columnName,
|
||||
UuidValueJsonExtension.fromJson(value),
|
||||
conflictingId,
|
||||
return const Uuid().v5obj(
|
||||
Namespace.oid.value,
|
||||
'$tableName.${column.columnName}:${value}__${releaseSuffix}__$conflictingId',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -1140,20 +1153,6 @@ bool _isForeignKeyColumn(TableDefinition table, String columnName) {
|
|||
);
|
||||
}
|
||||
|
||||
UuidValue _syntheticDeletedUuid(
|
||||
String tableName,
|
||||
String columnName,
|
||||
UuidValue value,
|
||||
UuidValue conflictingId,
|
||||
) {
|
||||
return UuidValue.withValidation(
|
||||
const Uuid().v5(
|
||||
Namespace.oid.value,
|
||||
'$tableName.$columnName:${value.uuid}:${conflictingId.uuid}',
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
String _sqlLiteral(Object? value) => ValueEncoder.instance.convert(value);
|
||||
|
||||
String _sqlLiteralList(Iterable<Object?> values) =>
|
||||
|
|
|
|||
|
|
@ -74,6 +74,10 @@ melos:
|
|||
- packages/*
|
||||
|
||||
scripts:
|
||||
cleanup: |
|
||||
find . -type d \( -name generated -o -name protocol \) -print0 |
|
||||
xargs -0 -I@ find @ -mindepth 1 -maxdepth 1 ! -name analysis_options.yaml -print0 |
|
||||
xargs -0 rm -rf
|
||||
setup:
|
||||
run: dart pub get && melos generate && melos create-migration
|
||||
lint:
|
||||
|
|
|
|||
|
|
@ -0,0 +1,12 @@
|
|||
class: UniqueComposite
|
||||
table: unique_composite
|
||||
database: all
|
||||
fields:
|
||||
id: UuidValue?, defaultPersist=random_v7
|
||||
scope: String
|
||||
value: String
|
||||
|
||||
indexes:
|
||||
unique_composite_scope_value_idx:
|
||||
fields: scope, value
|
||||
unique: true
|
||||
|
|
@ -215,6 +215,159 @@ void main() {
|
|||
});
|
||||
},
|
||||
);
|
||||
|
||||
group(
|
||||
'Given a remote update that loses a composite unique conflict, ',
|
||||
() {
|
||||
late UniqueComposite loser;
|
||||
late CrdtMergeUpdate remoteUpdate;
|
||||
|
||||
setUp(() async {
|
||||
await session.db.transactionForUser(testCrdtUserId, (tx) async {
|
||||
await UniqueComposite.db.insertRow(
|
||||
session,
|
||||
UniqueComposite(
|
||||
id: const Uuid().v7obj(),
|
||||
scope: 'left-scope',
|
||||
value: 'shared-value',
|
||||
),
|
||||
transaction: tx,
|
||||
);
|
||||
loser = await UniqueComposite.db.insertRow(
|
||||
session,
|
||||
UniqueComposite(
|
||||
id: const Uuid().v7obj(),
|
||||
scope: 'right-scope',
|
||||
value: 'shared-value',
|
||||
),
|
||||
transaction: tx,
|
||||
);
|
||||
});
|
||||
|
||||
final loserCrdtRow = await CrdtDataRow.db.findFirstRow(
|
||||
session,
|
||||
where: (t) => t.uuidRowId.equals(loser.id),
|
||||
include: CrdtDataRow.include(node: CrdtNode.include()),
|
||||
);
|
||||
|
||||
remoteUpdate = CrdtMergeUpdate(
|
||||
tableName: UniqueComposite.t.tableName,
|
||||
uuidRowId: loser.id!,
|
||||
uuidNodeId: const Uuid().v7obj(),
|
||||
hlcDatetime: loserCrdtRow!.hlc.datetime.advance(),
|
||||
hlcCounter: loserCrdtRow.hlc.counter,
|
||||
columnName: UniqueComposite.t.scope.columnName,
|
||||
value: 'left-scope',
|
||||
);
|
||||
|
||||
mergeSet = [remoteUpdate];
|
||||
});
|
||||
|
||||
group('when merging, ', () {
|
||||
setUp(() async {
|
||||
await session.db.mergeChanges(
|
||||
mergeSet,
|
||||
userId: testCrdtUserId,
|
||||
);
|
||||
});
|
||||
|
||||
test(
|
||||
'then the losing row receives conflict-free composite unique values.',
|
||||
() async {
|
||||
final row = await UniqueComposite.db.findById(session, loser.id!);
|
||||
|
||||
expect(row, isNotNull);
|
||||
expect(row!.scope, 'left-scope__conflict__${loser.id!.uuid}');
|
||||
expect(row.value, 'shared-value__conflict__${loser.id!.uuid}');
|
||||
},
|
||||
);
|
||||
|
||||
test(
|
||||
'then the silently released unique column does not receive CRDT field metadata.',
|
||||
() async {
|
||||
final fields = await CrdtDataField.db.find(
|
||||
session,
|
||||
where: (t) => t.row.uuidRowId.equals(loser.id),
|
||||
include: CrdtDataField.include(
|
||||
column: CrdtSchemaColumn.include(),
|
||||
node: CrdtNode.include(),
|
||||
),
|
||||
);
|
||||
|
||||
expect(fields, hasLength(1));
|
||||
expect(fields.first.column!.name, UniqueComposite.t.scope.columnName);
|
||||
expect(fields.first.hlc, remoteUpdate.hlc);
|
||||
},
|
||||
);
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
group(
|
||||
'Given a remote update that loses a UUID unique conflict, ',
|
||||
() {
|
||||
final sharedValue = const Uuid().v7obj();
|
||||
final originalLoserValue = const Uuid().v7obj();
|
||||
late UniqueUuid loser;
|
||||
|
||||
setUp(() async {
|
||||
await session.db.transactionForUser(testCrdtUserId, (tx) async {
|
||||
await UniqueUuid.db.insertRow(
|
||||
session,
|
||||
UniqueUuid(id: const Uuid().v7obj(), value: sharedValue),
|
||||
transaction: tx,
|
||||
);
|
||||
loser = await UniqueUuid.db.insertRow(
|
||||
session,
|
||||
UniqueUuid(id: const Uuid().v7obj(), value: originalLoserValue),
|
||||
transaction: tx,
|
||||
);
|
||||
});
|
||||
|
||||
final loserCrdtRow = await CrdtDataRow.db.findFirstRow(
|
||||
session,
|
||||
where: (t) => t.uuidRowId.equals(loser.id),
|
||||
include: CrdtDataRow.include(node: CrdtNode.include()),
|
||||
);
|
||||
|
||||
final remoteUpdate = CrdtMergeUpdate(
|
||||
tableName: UniqueUuid.t.tableName,
|
||||
uuidRowId: loser.id!,
|
||||
uuidNodeId: const Uuid().v7obj(),
|
||||
hlcDatetime: loserCrdtRow!.hlc.datetime.advance(),
|
||||
hlcCounter: loserCrdtRow.hlc.counter,
|
||||
columnName: UniqueUuid.t.value.columnName,
|
||||
value: sharedValue,
|
||||
);
|
||||
|
||||
mergeSet = [remoteUpdate];
|
||||
});
|
||||
|
||||
group('when merging, ', () {
|
||||
setUp(() async {
|
||||
await session.db.mergeChanges(
|
||||
mergeSet,
|
||||
userId: testCrdtUserId,
|
||||
);
|
||||
});
|
||||
|
||||
test(
|
||||
'then the losing row receives a deterministic conflict-free UUID unique value.',
|
||||
() async {
|
||||
final row = await UniqueUuid.db.findById(session, loser.id!);
|
||||
final expectedConflictValue = const Uuid().v5obj(
|
||||
Namespace.oid.value,
|
||||
'${UniqueUuid.t.tableName}.${UniqueUuid.t.value.columnName}:'
|
||||
'${sharedValue.uuid}__conflict__${loser.id!.uuid}',
|
||||
);
|
||||
|
||||
expect(row, isNotNull);
|
||||
expect(row!.value, expectedConflictValue);
|
||||
},
|
||||
);
|
||||
});
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
extension on DateTime {
|
||||
|
|
|
|||
|
|
@ -0,0 +1,301 @@
|
|||
import 'package:serverpod/serverpod.dart';
|
||||
import 'package:serverpod_offline_sync_server/serverpod_offline_sync_server.dart';
|
||||
import 'package:serverpod_offline_sync_test_client/serverpod_offline_sync_test_client.dart'
|
||||
as client;
|
||||
import 'package:serverpod_offline_sync_test_server/src/generated/protocol.dart'
|
||||
as server;
|
||||
import 'package:test/test.dart';
|
||||
|
||||
import '../test_tools/client_session.dart';
|
||||
import '../test_tools/serverpod_test_tools.dart';
|
||||
|
||||
void main() {
|
||||
initTestClientSession(withPersistentUser: true);
|
||||
|
||||
final clientSyncTables = [client.Unique.t];
|
||||
final serverSyncTables = [server.Unique.t];
|
||||
|
||||
late client.Client syncHttpClient;
|
||||
late CrdtDatabaseSession serverSession;
|
||||
late CrdtDatabaseSession firstClientSession;
|
||||
late CrdtDatabaseSession secondClientSession;
|
||||
|
||||
withServerpod(
|
||||
'Given a server and two client CRDT sessions for the same user that are initialized with the unique sync table',
|
||||
rollbackDatabase: RollbackDatabase.disabled,
|
||||
(sessionBuilder, _) {
|
||||
final rawServerSession = sessionBuilder.build();
|
||||
|
||||
rawServerSession.serverpod
|
||||
..initializeCrdtSync(
|
||||
syncTables: serverSyncTables,
|
||||
)
|
||||
..authenticationHandler = (session, token) async => AuthenticationInfo(
|
||||
testCrdtUserId.toString(),
|
||||
<Scope>{},
|
||||
authId: const Uuid().v4(),
|
||||
);
|
||||
|
||||
setUp(() async {
|
||||
syncHttpClient = client.Client(
|
||||
'http://localhost:${rawServerSession.server.port}',
|
||||
)..authKeyProvider = TestClientAuthKeyProvider();
|
||||
|
||||
firstClientSession = CrdtDatabaseSession.wraps(
|
||||
testSession,
|
||||
syncTables: clientSyncTables,
|
||||
persistentUserId: testCrdtUserId,
|
||||
);
|
||||
await firstClientSession.db.initialize();
|
||||
|
||||
secondClientSession = CrdtDatabaseSession.wraps(
|
||||
await createAdditionalTestSession(),
|
||||
syncTables: clientSyncTables,
|
||||
persistentUserId: testCrdtUserId,
|
||||
);
|
||||
await secondClientSession.db.initialize();
|
||||
|
||||
serverSession = CrdtDatabaseSession.wraps(
|
||||
rawServerSession,
|
||||
syncTables: serverSyncTables,
|
||||
);
|
||||
await serverSession.db.initialize();
|
||||
});
|
||||
|
||||
tearDown(() async {
|
||||
await serverSession.clearUserTables();
|
||||
});
|
||||
|
||||
group('and an older server row conflicts with a newer incoming row,', () {
|
||||
late client.Unique olderRow;
|
||||
late client.Unique newerRow;
|
||||
|
||||
setUp(() async {
|
||||
olderRow = await client.Unique.db.insertRow(
|
||||
firstClientSession,
|
||||
client.Unique(id: const Uuid().v7obj(), name: 'shared-name'),
|
||||
);
|
||||
|
||||
await Future<void>.delayed(const Duration(milliseconds: 2));
|
||||
newerRow = await client.Unique.db.insertRow(
|
||||
secondClientSession,
|
||||
client.Unique(id: const Uuid().v7obj(), name: 'shared-name'),
|
||||
);
|
||||
|
||||
await syncHttpClient.crdt.syncOnce(firstClientSession);
|
||||
});
|
||||
|
||||
group('when the incoming row synchronizes,', () {
|
||||
setUp(() async {
|
||||
await syncHttpClient.crdt.syncOnce(secondClientSession);
|
||||
await syncHttpClient.crdt.syncOnce(firstClientSession);
|
||||
});
|
||||
|
||||
test(
|
||||
'then both rows remain visible and the older row keeps the unique value.',
|
||||
() async {
|
||||
await _expectVisibleUniqueRowsOnAllNodes(
|
||||
firstClientSession: firstClientSession,
|
||||
secondClientSession: secondClientSession,
|
||||
serverSession: serverSession,
|
||||
originalValueId: olderRow.id!,
|
||||
conflictValueId: newerRow.id!,
|
||||
);
|
||||
},
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
group('and a newer server row conflicts with an older incoming row,', () {
|
||||
late client.Unique olderRow;
|
||||
late client.Unique newerRow;
|
||||
|
||||
setUp(() async {
|
||||
olderRow = await client.Unique.db.insertRow(
|
||||
secondClientSession,
|
||||
client.Unique(id: const Uuid().v7obj(), name: 'shared-name'),
|
||||
);
|
||||
|
||||
await Future<void>.delayed(const Duration(milliseconds: 2));
|
||||
newerRow = await client.Unique.db.insertRow(
|
||||
firstClientSession,
|
||||
client.Unique(id: const Uuid().v7obj(), name: 'shared-name'),
|
||||
);
|
||||
|
||||
await syncHttpClient.crdt.syncOnce(firstClientSession);
|
||||
});
|
||||
|
||||
group('when the incoming row synchronizes,', () {
|
||||
setUp(() async {
|
||||
await syncHttpClient.crdt.syncOnce(secondClientSession);
|
||||
await syncHttpClient.crdt.syncOnce(firstClientSession);
|
||||
});
|
||||
|
||||
test(
|
||||
'then both rows remain visible and the older row keeps the unique value.',
|
||||
() async {
|
||||
await _expectVisibleUniqueRowsOnAllNodes(
|
||||
firstClientSession: firstClientSession,
|
||||
secondClientSession: secondClientSession,
|
||||
serverSession: serverSession,
|
||||
originalValueId: olderRow.id!,
|
||||
conflictValueId: newerRow.id!,
|
||||
);
|
||||
},
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
group('and a newer server update conflicts with an older incoming update,', () {
|
||||
late client.Unique olderUpdatedRow;
|
||||
late client.Unique newerUpdatedRow;
|
||||
|
||||
setUp(() async {
|
||||
olderUpdatedRow = await client.Unique.db.insertRow(
|
||||
firstClientSession,
|
||||
client.Unique(id: const Uuid().v7obj(), name: 'first-name'),
|
||||
);
|
||||
newerUpdatedRow = await client.Unique.db.insertRow(
|
||||
firstClientSession,
|
||||
client.Unique(id: const Uuid().v7obj(), name: 'second-name'),
|
||||
);
|
||||
|
||||
// Ensures the row and update are visible on all nodes.
|
||||
await syncHttpClient.crdt.syncOnce(firstClientSession);
|
||||
await syncHttpClient.crdt.syncOnce(secondClientSession);
|
||||
|
||||
// The `firstClientSession` holds the older update.
|
||||
await client.Unique.db.updateRow(
|
||||
firstClientSession,
|
||||
olderUpdatedRow.copyWith(name: 'shared-name'),
|
||||
);
|
||||
|
||||
// The `secondClientSession` holds the newer update.
|
||||
await Future<void>.delayed(const Duration(milliseconds: 2));
|
||||
await client.Unique.db.updateRow(
|
||||
secondClientSession,
|
||||
newerUpdatedRow.copyWith(name: 'shared-name'),
|
||||
);
|
||||
|
||||
// The `secondClientSession` syncs first, so the newer update
|
||||
// reaches the server first.
|
||||
await syncHttpClient.crdt.syncOnce(secondClientSession);
|
||||
});
|
||||
|
||||
group('when the incoming update synchronizes,', () {
|
||||
setUp(() async {
|
||||
await syncHttpClient.crdt.syncOnce(firstClientSession);
|
||||
await syncHttpClient.crdt.syncOnce(secondClientSession);
|
||||
});
|
||||
|
||||
test(
|
||||
'then both rows remain visible and the row with the older unique update keeps the unique value.',
|
||||
() async {
|
||||
await _expectVisibleUniqueRowsOnAllNodes(
|
||||
firstClientSession: firstClientSession,
|
||||
secondClientSession: secondClientSession,
|
||||
serverSession: serverSession,
|
||||
originalValueId: olderUpdatedRow.id!,
|
||||
conflictValueId: newerUpdatedRow.id!,
|
||||
);
|
||||
},
|
||||
timeout: const Timeout(Duration(seconds: 60)),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
group('and an older incoming insert conflicts with a newer server update,', () {
|
||||
late client.Unique insertedRow;
|
||||
late client.Unique updatedRow;
|
||||
|
||||
setUp(() async {
|
||||
updatedRow = await client.Unique.db.insertRow(
|
||||
firstClientSession,
|
||||
client.Unique(id: const Uuid().v7obj(), name: 'first-name'),
|
||||
);
|
||||
|
||||
await Future<void>.delayed(const Duration(milliseconds: 2));
|
||||
insertedRow = await client.Unique.db.insertRow(
|
||||
secondClientSession,
|
||||
client.Unique(id: const Uuid().v7obj(), name: 'shared-name'),
|
||||
);
|
||||
|
||||
await Future<void>.delayed(const Duration(milliseconds: 2));
|
||||
await client.Unique.db.updateRow(
|
||||
firstClientSession,
|
||||
updatedRow.copyWith(name: 'shared-name'),
|
||||
);
|
||||
|
||||
await syncHttpClient.crdt.syncOnce(firstClientSession);
|
||||
});
|
||||
|
||||
group('when the incoming insert synchronizes,', () {
|
||||
setUp(() async {
|
||||
await syncHttpClient.crdt.syncOnce(secondClientSession);
|
||||
await syncHttpClient.crdt.syncOnce(firstClientSession);
|
||||
});
|
||||
|
||||
test(
|
||||
'then both rows remain visible and the row with the older unique value keeps it on every node.',
|
||||
() async {
|
||||
await _expectVisibleUniqueRowsOnAllNodes(
|
||||
firstClientSession: firstClientSession,
|
||||
secondClientSession: secondClientSession,
|
||||
serverSession: serverSession,
|
||||
originalValueId: insertedRow.id!,
|
||||
conflictValueId: updatedRow.id!,
|
||||
);
|
||||
},
|
||||
timeout: const Timeout(Duration(seconds: 60)),
|
||||
);
|
||||
});
|
||||
});
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
Future<void> _expectVisibleUniqueRowsOnAllNodes({
|
||||
required CrdtDatabaseSession firstClientSession,
|
||||
required CrdtDatabaseSession secondClientSession,
|
||||
required CrdtDatabaseSession serverSession,
|
||||
required UuidValue originalValueId,
|
||||
required UuidValue conflictValueId,
|
||||
}) async {
|
||||
final firstClientRows = await client.Unique.db.find(firstClientSession);
|
||||
expect(firstClientRows, hasLength(2));
|
||||
expect(
|
||||
firstClientRows.singleWhere((row) => row.id == originalValueId).name,
|
||||
'shared-name',
|
||||
);
|
||||
expect(
|
||||
firstClientRows.singleWhere((row) => row.id == conflictValueId).name,
|
||||
'shared-name__conflict__${conflictValueId.uuid}',
|
||||
);
|
||||
|
||||
final secondClientRows = await client.Unique.db.find(secondClientSession);
|
||||
expect(secondClientRows, hasLength(2));
|
||||
expect(
|
||||
secondClientRows.singleWhere((row) => row.id == originalValueId).name,
|
||||
'shared-name',
|
||||
);
|
||||
expect(
|
||||
secondClientRows.singleWhere((row) => row.id == conflictValueId).name,
|
||||
'shared-name__conflict__${conflictValueId.uuid}',
|
||||
);
|
||||
|
||||
final serverRows = await server.Unique.db.find(serverSession);
|
||||
expect(serverRows, hasLength(2));
|
||||
expect(
|
||||
serverRows.singleWhere((row) => row.id == originalValueId).name,
|
||||
'shared-name',
|
||||
);
|
||||
expect(
|
||||
serverRows.singleWhere((row) => row.id == conflictValueId).name,
|
||||
'shared-name__conflict__${conflictValueId.uuid}',
|
||||
);
|
||||
}
|
||||
|
||||
class TestClientAuthKeyProvider implements ClientAuthKeyProvider {
|
||||
@override
|
||||
Future<String?> get authHeaderValue async => 'Bearer token';
|
||||
}
|
||||
|
|
@ -86,6 +86,7 @@ Future<void> _initialize() async {
|
|||
Person.t,
|
||||
Town.t,
|
||||
Unique.t,
|
||||
UniqueComposite.t,
|
||||
UniqueUuid.t,
|
||||
],
|
||||
persistentUserId: _withPersistentUser ? _testCrdtUserId : null,
|
||||
|
|
|
|||
Loading…
Reference in a new issue