feat: Implement unique conflict handling to preserve invariants (#64)
This commit is contained in:
parent
60cca9cc85
commit
e74caa1400
10 changed files with 1347 additions and 66 deletions
180
docs/foreign-key-invariants.md
Normal file
180
docs/foreign-key-invariants.md
Normal file
|
|
@ -0,0 +1,180 @@
|
||||||
|
# Preserving foreign key invariants while merging
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
After merging remote CRDT operations, the visible database must not contain
|
||||||
|
foreign key violations. The merge result must be associative, commutative, and
|
||||||
|
idempotent: the same merged CRDT facts must produce the same visible rows and
|
||||||
|
visible FK values regardless of arrival order, batching, or replay.
|
||||||
|
|
||||||
|
The safe model is projection-based. User rows, fields, and tombstones remain the
|
||||||
|
CRDT facts. FK repairs are deterministic materialized projections over those
|
||||||
|
facts, not new user field updates emitted opportunistically during merge.
|
||||||
|
|
||||||
|
## Core Policy
|
||||||
|
|
||||||
|
- Keep every row as a CRDT fact. Visibility is derived from row tombstones and
|
||||||
|
schema-driven FK projection.
|
||||||
|
- Keep the user/attempted FK value as the CRDT field value.
|
||||||
|
- Materialize a separate visible FK value only when FK repair requires it. Use
|
||||||
|
`CrdtDataForeignKey` as projection metadata, not as a business conflict class.
|
||||||
|
- Do not create ordinary user field updates for `SET NULL` or `SET DEFAULT`
|
||||||
|
repairs during merge. Doing so makes repair HLCs depend on merge timing.
|
||||||
|
- Recompute FK projection from merged facts over the affected FK closure. The
|
||||||
|
incremental closure result must match a full recomputation oracle.
|
||||||
|
|
||||||
|
## Action Semantics
|
||||||
|
|
||||||
|
For each child FK whose attempted value points to a hidden or missing parent:
|
||||||
|
|
||||||
|
- `ON DELETE SET NULL`: the visible FK value is `null` when the FK is nullable.
|
||||||
|
If the FK is non-nullable, this action cannot repair the row and must fall
|
||||||
|
through to the deterministic blocking policy.
|
||||||
|
- `ON DELETE SET DEFAULT`: the visible FK value is the column default only if
|
||||||
|
that default points to a visible parent or is a legal nullable default. If the
|
||||||
|
default target is hidden or missing, this action cannot repair the row and
|
||||||
|
must fall through to the deterministic blocking policy.
|
||||||
|
- `ON DELETE RESTRICT` and `NO ACTION`: the parent delete loses while a visible
|
||||||
|
child still references the parent. The parent remains visible in the derived
|
||||||
|
projection.
|
||||||
|
- `ON DELETE CASCADE`: the child can be hidden only if the whole cascade closure
|
||||||
|
is valid. If any descendant blocks the cascade through `RESTRICT`, `NO ACTION`,
|
||||||
|
invalid `SET NULL`, or invalid `SET DEFAULT`, the ancestor delete loses and
|
||||||
|
the cascade is not partially applied.
|
||||||
|
|
||||||
|
This makes mixed chains atomic for visibility. For example, if deleting `A`
|
||||||
|
would cascade-delete `B`, but `B` has a visible `RESTRICT` child `C`, then `A`,
|
||||||
|
`B`, and `C` remain visible. The result must not depend on whether cascade or
|
||||||
|
restrict edges are evaluated first internally.
|
||||||
|
|
||||||
|
## Projection Metadata
|
||||||
|
|
||||||
|
`CrdtDataForeignKey` should represent the materialized FK projection for one
|
||||||
|
`CrdtDataField`:
|
||||||
|
|
||||||
|
- `attemptedValue`: the current user-attempted FK value carried by the CRDT
|
||||||
|
field. It may be null for nullable FKs.
|
||||||
|
- `visibleValue`: the value currently materialized into the domain row when an
|
||||||
|
override is active.
|
||||||
|
- `hasOverride`: whether the projection overrides `attemptedValue` and
|
||||||
|
materializes `visibleValue` instead.
|
||||||
|
|
||||||
|
`visibleValue == null` is meaningful only with `hasOverride`. When
|
||||||
|
`hasOverride` is false it means "same as attempted"; when `hasOverride` is true
|
||||||
|
it means "materialize null".
|
||||||
|
|
||||||
|
The resolver may use this metadata for efficient materialization and write
|
||||||
|
deduplication. It must not use it as the authority for FK conflict decisions;
|
||||||
|
the authority is the deterministic resolver over merged CRDT facts.
|
||||||
|
|
||||||
|
## Merge Pipeline
|
||||||
|
|
||||||
|
1. Merge CRDT row, field, and tombstone facts by their existing HLC/CLFlag
|
||||||
|
rules.
|
||||||
|
2. Identify rows touched by inserts, updates, deletes, FK columns, and existing
|
||||||
|
FK projection records.
|
||||||
|
3. Expand to the FK closure in both directions: parents needed by children and
|
||||||
|
children affected by parent visibility.
|
||||||
|
4. Compute effective row visibility and visible FK values to a fixed point.
|
||||||
|
5. Apply only the materialized differences:
|
||||||
|
- update domain FK columns when projection differs from attempted value;
|
||||||
|
- update `CrdtDataForeignKey` projection metadata;
|
||||||
|
- update system tombstones only for deterministic derived visibility where
|
||||||
|
that is part of the chosen storage model.
|
||||||
|
6. Re-run unique conflict projection after FK projection if FK repairs can
|
||||||
|
change unique columns, or define the opposite stage order and prove it with
|
||||||
|
the full recomputation oracle.
|
||||||
|
|
||||||
|
## Invariants
|
||||||
|
|
||||||
|
- No visible row references a hidden or missing parent.
|
||||||
|
- No cascade delete is partially applied across a blocked nested relation.
|
||||||
|
- `SET NULL` and `SET DEFAULT` repairs are pure functions of merged facts.
|
||||||
|
- Replaying the same merge batch produces no metadata churn.
|
||||||
|
- Splitting a merge into smaller batches converges to the same final projection
|
||||||
|
as merging the same operations in one batch.
|
||||||
|
- Incremental recomputation over the affected closure matches full global
|
||||||
|
recomputation.
|
||||||
|
|
||||||
|
## Test Cases
|
||||||
|
|
||||||
|
### Restrict and No Action
|
||||||
|
|
||||||
|
1. Given a parent and a visible child with `ON DELETE RESTRICT`, when one replica
|
||||||
|
deletes the parent and another concurrently updates the child, then the parent
|
||||||
|
remains visible after convergence and no visible orphan appears.
|
||||||
|
|
||||||
|
2. Given a parent with multiple visible restrict children, when the parent is
|
||||||
|
deleted concurrently with updates to different children, then any visible
|
||||||
|
child is enough to keep the parent visible on every replica.
|
||||||
|
|
||||||
|
3. Given a parent kept visible because of restrict, when all restrict children
|
||||||
|
are later deleted or detached, then the parent visibility follows the same
|
||||||
|
deterministic recomputation rule on every replica.
|
||||||
|
|
||||||
|
### Set Null and Set Default
|
||||||
|
|
||||||
|
4. Given a nullable FK with `ON DELETE SET NULL`, when the parent is deleted on
|
||||||
|
another replica, then the child remains visible with a materialized null FK
|
||||||
|
and the user/attempted FK value remains unchanged as a CRDT fact.
|
||||||
|
|
||||||
|
5. Given a child whose FK was materialized to null by `SET NULL`, when the user
|
||||||
|
later changes the FK to a visible parent, then the projection override becomes
|
||||||
|
inactive and the domain row materializes the new user value.
|
||||||
|
|
||||||
|
6. Given `ON DELETE SET DEFAULT` and a visible default target, when the attempted
|
||||||
|
parent is deleted, then the child remains visible with the default FK
|
||||||
|
materialized.
|
||||||
|
|
||||||
|
7. Given `ON DELETE SET DEFAULT` and a hidden or missing default target, when the
|
||||||
|
attempted parent is deleted, then the action cannot repair the child and the
|
||||||
|
deterministic blocking policy keeps the necessary parent visible.
|
||||||
|
|
||||||
|
### Cascade
|
||||||
|
|
||||||
|
8. Given a parent with children under `ON DELETE CASCADE`, when the parent is
|
||||||
|
deleted, then all cascade descendants in the valid cascade closure become
|
||||||
|
hidden and no visible descendant references a hidden ancestor.
|
||||||
|
|
||||||
|
9. Given a child already hidden by a cascade projection, when the same delete
|
||||||
|
facts are merged again, then no additional metadata churn occurs.
|
||||||
|
|
||||||
|
### Mixed Nested Relations
|
||||||
|
|
||||||
|
10. Given `A -> B` is cascade and `B <- C` is restrict, when `A` is deleted
|
||||||
|
concurrently with an update to `C`, then the cascade is blocked and `A`,
|
||||||
|
`B`, and `C` remain visible.
|
||||||
|
|
||||||
|
11. Given a fourth-degree chain with alternating cascade, restrict, set-null,
|
||||||
|
and set-default edges, when a merge touches both a root and a leaf, then
|
||||||
|
incremental recomputation produces the same visible graph and FK projection
|
||||||
|
as full recomputation.
|
||||||
|
|
||||||
|
12. Given an FK cycle that the engine permits, when concurrent deletes and
|
||||||
|
updates happen around the cycle, then fixed-point evaluation terminates and
|
||||||
|
every replica computes the same projection.
|
||||||
|
|
||||||
|
### Cross-Cutting
|
||||||
|
|
||||||
|
13. Given an FK repair changes a unique-indexed column, when unique projection
|
||||||
|
and FK projection are recomputed, then the documented stage order prevents
|
||||||
|
visible unique and FK violations.
|
||||||
|
|
||||||
|
14. Given the same remote operations arrive in one batch or several smaller
|
||||||
|
batches, when all operations have merged, then visible rows, visible FK
|
||||||
|
values, FK projection metadata, unique projection, and tombstones are the
|
||||||
|
same.
|
||||||
|
|
||||||
|
15. Given a merge touches only a non-indexed, non-FK field, when FK projection
|
||||||
|
runs, then unrelated rows are not reconsidered and the visible state outside
|
||||||
|
the affected closure remains unchanged.
|
||||||
|
|
||||||
|
## What These Tests Prove
|
||||||
|
|
||||||
|
- Integrity: no surviving visible FK violation.
|
||||||
|
- Convergence: same merged facts produce the same projection on every replica.
|
||||||
|
- ACI merge behavior: batching, replay, and merge order do not change the final
|
||||||
|
visible result.
|
||||||
|
- Nested safety: cascade cannot bypass restrict/no-action descendants.
|
||||||
|
- Projection discipline: `SET NULL` and `SET DEFAULT` do not become ordinary
|
||||||
|
user CRDT writes.
|
||||||
216
docs/unique-constraint-invariants.md
Normal file
216
docs/unique-constraint-invariants.md
Normal file
|
|
@ -0,0 +1,216 @@
|
||||||
|
# Preserving unique constraint invariants while merging
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Concurrent inserts or updates can make multiple CRDT rows claim the same unique
|
||||||
|
value. The current merge model resolves this without hiding rows: every row
|
||||||
|
remains visible, and the losing unique claims are materialized with
|
||||||
|
deterministic conflict-free visible values.
|
||||||
|
|
||||||
|
This is the `flag` policy. It is intentionally simple:
|
||||||
|
|
||||||
|
- all rows remain CRDT facts;
|
||||||
|
- all rows remain visible domain rows;
|
||||||
|
- one deterministic winner keeps the claimed unique value;
|
||||||
|
- each loser keeps its own row identity and receives a conflict-free visible
|
||||||
|
unique value;
|
||||||
|
- no business conflict object is synced;
|
||||||
|
- no foreign key retargeting is needed for unique conflicts.
|
||||||
|
|
||||||
|
This policy is the only unique-conflict policy in scope for the current
|
||||||
|
implementation.
|
||||||
|
|
||||||
|
## Current Policy: Flag
|
||||||
|
|
||||||
|
For each unique index conflict group:
|
||||||
|
|
||||||
|
1. Compute each row's unique claim timestamp from the HLC of the unique-indexed
|
||||||
|
field or fields. For an insert, use the insert HLC and any incoming field HLCs
|
||||||
|
for the indexed fields.
|
||||||
|
2. Pick the winner deterministically by oldest unique-claim HLC, using the same
|
||||||
|
full HLC ordering as the rest of CRDT merge. This includes node id as the HLC
|
||||||
|
tie-break. Row id is only a final fallback for identical HLCs.
|
||||||
|
3. Leave the winner's unique value unchanged.
|
||||||
|
4. Rewrite each loser's visible unique-indexed value to a deterministic
|
||||||
|
conflict-free value:
|
||||||
|
- nullable unique values may be released to `null`;
|
||||||
|
- text values receive a stable `__conflict__<rowId>` suffix;
|
||||||
|
- UUID values receive a deterministic synthetic UUID.
|
||||||
|
5. Keep all rows visible. Application code can decide whether and how to surface
|
||||||
|
the changed visible value as a business conflict.
|
||||||
|
|
||||||
|
The rewritten value is a materialized data value, not a synced conflict class.
|
||||||
|
The core CRDT protocol does not need to expose unique conflict objects to users.
|
||||||
|
|
||||||
|
## Why Flag Is The Current Default
|
||||||
|
|
||||||
|
The `flag` policy avoids the hard second-order effects caused by hiding or
|
||||||
|
merging rows:
|
||||||
|
|
||||||
|
- no child row needs to be moved from one parent row id to another;
|
||||||
|
- `ON DELETE RESTRICT` children do not accidentally keep a unique loser alive;
|
||||||
|
- cascade chains are not triggered by unique conflict resolution;
|
||||||
|
- non-key field updates on both rows remain attached to their original rows;
|
||||||
|
- the algorithm stays local to the conflicting unique index and affected rows.
|
||||||
|
|
||||||
|
This keeps merge behavior associative, commutative, and idempotent without
|
||||||
|
requiring a combined unique/FK fixed-point projection.
|
||||||
|
|
||||||
|
## Deferred Policies
|
||||||
|
|
||||||
|
The following policies are plausible future schema-level options, but they are
|
||||||
|
not part of the current implementation because they require a substantially more
|
||||||
|
complex projection algorithm.
|
||||||
|
|
||||||
|
Both policies should be projections over the canonical `flag` result, not
|
||||||
|
separate unique-resolution algorithms. The `flag` layer computes conflict
|
||||||
|
components, winners, losers, and released loser values once from CRDT facts:
|
||||||
|
|
||||||
|
```text
|
||||||
|
unique facts -> conflict components -> winner + losers + released loser values
|
||||||
|
```
|
||||||
|
|
||||||
|
Future policies then interpret that canonical result:
|
||||||
|
|
||||||
|
```text
|
||||||
|
flag = show all rows; losers show released unique values
|
||||||
|
overwrite = use flag winner; alias/hide losers in the visible projection
|
||||||
|
merge = overwrite projection + deterministic field folding into winner
|
||||||
|
```
|
||||||
|
|
||||||
|
`merge` and `overwrite` must use the original unique claims and conflict
|
||||||
|
components from the `flag` projection. They must not infer conflicts from only
|
||||||
|
the already-released materialized values.
|
||||||
|
|
||||||
|
### Overwrite
|
||||||
|
|
||||||
|
`overwrite` would keep a deterministic winner visible and hide the losers.
|
||||||
|
|
||||||
|
To be safe, this cannot simply tombstone the loser. A hidden loser may already
|
||||||
|
be referenced by visible children, including children using `ON DELETE RESTRICT`
|
||||||
|
or deeper mixed cascade/restrict chains. A safe overwrite policy therefore needs
|
||||||
|
an internal row-alias projection:
|
||||||
|
|
||||||
|
```text
|
||||||
|
loserRowId -> winnerRowId
|
||||||
|
```
|
||||||
|
|
||||||
|
Foreign key projection would then consume the alias map and materialize child FK
|
||||||
|
values through the winner where legal. Without this aliasing step, overwrite can
|
||||||
|
produce visible orphans or order-dependent outcomes.
|
||||||
|
|
||||||
|
### Merge
|
||||||
|
|
||||||
|
`merge` would also pick a deterministic winner and alias loser rows to it, but
|
||||||
|
would additionally fold selected loser fields into the winner.
|
||||||
|
|
||||||
|
This needs explicit deterministic per-column policies, such as:
|
||||||
|
|
||||||
|
- `winnerWins`
|
||||||
|
- `latestFieldWinsAcrossMergedRows`
|
||||||
|
- `preferNonNullLatest`
|
||||||
|
- `ignore`
|
||||||
|
|
||||||
|
These policies must be schema-defined, deterministic on every replica, and
|
||||||
|
included in schema compatibility checks. Arbitrary app callbacks inside core
|
||||||
|
merge would not be safe unless they are deterministic, versioned, and available
|
||||||
|
identically on every replica.
|
||||||
|
|
||||||
|
## Why Merge And Overwrite Are Deferred
|
||||||
|
|
||||||
|
`merge` and `overwrite` combine unique projection with foreign key projection.
|
||||||
|
The resolver would need to compute, at minimum:
|
||||||
|
|
||||||
|
- unique conflict groups;
|
||||||
|
- winners and losers for each group;
|
||||||
|
- row aliases from loser ids to winner ids;
|
||||||
|
- hidden-row projection;
|
||||||
|
- field projections for merged rows;
|
||||||
|
- foreign key repairs over aliases;
|
||||||
|
- nested cascade/restrict/set-null/set-default effects;
|
||||||
|
- fixed-point recomputation when FK repairs touch unique-indexed values.
|
||||||
|
|
||||||
|
That interaction can grow quickly with graph depth, shared children, cycles, and
|
||||||
|
rows participating in multiple unique indexes. The implementation would need a
|
||||||
|
full recomputation oracle and incremental-closure equivalence tests before it is
|
||||||
|
safe enough for real data.
|
||||||
|
|
||||||
|
For now, the `flag` policy gives a safer and smaller invariant:
|
||||||
|
|
||||||
|
```text
|
||||||
|
unique projection changes values, not row identity or row visibility
|
||||||
|
```
|
||||||
|
|
||||||
|
Foreign key repair can then be developed independently as a projection over row
|
||||||
|
visibility and FK values, without also needing row aliasing and field folding.
|
||||||
|
|
||||||
|
## Invariants
|
||||||
|
|
||||||
|
- No visible duplicate remains for a unique index after projection.
|
||||||
|
- Unique conflict resolution is deterministic from merged CRDT facts.
|
||||||
|
- Merge replay does not create additional metadata churn.
|
||||||
|
- Splitting the same operations into different sync batches converges to the
|
||||||
|
same visible result.
|
||||||
|
- Unique projection does not emit app-visible CRDT conflict objects.
|
||||||
|
- Current `flag` projection does not hide rows or retarget foreign keys.
|
||||||
|
|
||||||
|
## Test Cases
|
||||||
|
|
||||||
|
### Current Flag Policy
|
||||||
|
|
||||||
|
1. Given two replicas concurrently insert different rows with the same unique
|
||||||
|
value, when they merge, then both rows remain visible, the deterministic
|
||||||
|
winner keeps the original value, and the loser receives a deterministic
|
||||||
|
conflict-free value.
|
||||||
|
|
||||||
|
2. Given an older insert conflicts with a newer update on another row, when they
|
||||||
|
merge, then the row with the older unique-field claim keeps the unique value,
|
||||||
|
regardless of the other row's insert time.
|
||||||
|
|
||||||
|
3. Given two rows concurrently update the same unique field to the same value,
|
||||||
|
when they merge, then both rows remain visible and only the deterministic
|
||||||
|
winner keeps the claimed value.
|
||||||
|
|
||||||
|
4. Given the same conflict arrives in one batch or several smaller batches, when
|
||||||
|
all operations have merged, then the same row keeps the unique value and the
|
||||||
|
same rows receive the same conflict-free values.
|
||||||
|
|
||||||
|
5. Given a conflicting unique value is a nullable unique column, when the loser
|
||||||
|
can be made conflict-free by setting the materialized value to null, then the
|
||||||
|
loser remains visible and the unique index remains valid.
|
||||||
|
|
||||||
|
6. Given a conflicting unique value is a UUID column that is not an FK column,
|
||||||
|
when the loser is released, then the materialized UUID is deterministic from
|
||||||
|
the table, column, original value, and loser row id.
|
||||||
|
|
||||||
|
### Deferred Merge And Overwrite Policies
|
||||||
|
|
||||||
|
7. Given overwrite is enabled for a unique index, when a loser has visible
|
||||||
|
restrict children, then FK projection retargets those children through the
|
||||||
|
row alias or blocks the projection deterministically; no visible orphan is
|
||||||
|
produced.
|
||||||
|
|
||||||
|
8. Given merge is enabled for a unique index, when loser rows contain newer
|
||||||
|
non-key field values, then field folding follows documented per-column
|
||||||
|
policies and every replica computes the same winner row.
|
||||||
|
|
||||||
|
9. Given a row participates in multiple unique conflict groups, when merge or
|
||||||
|
overwrite projection runs, then row aliases are deterministic and do not form
|
||||||
|
cycles.
|
||||||
|
|
||||||
|
10. Given FK repair changes a unique-indexed visible value, when combined unique
|
||||||
|
and FK projection runs, then the fixed-point result matches a full
|
||||||
|
recomputation oracle.
|
||||||
|
|
||||||
|
These deferred tests should not be implemented until the row-alias and combined
|
||||||
|
projection model exists.
|
||||||
|
|
||||||
|
## What These Tests Prove
|
||||||
|
|
||||||
|
- Current implementation safety: the `flag` policy preserves uniqueness without
|
||||||
|
row hiding or FK retargeting.
|
||||||
|
- Convergence: same CRDT facts produce the same visible unique values.
|
||||||
|
- Idempotence: repeated merges do not keep rewriting the same conflict.
|
||||||
|
- Scope control: future `merge` and `overwrite` behavior is documented but not
|
||||||
|
treated as supported until the projection algorithm can handle second-order FK
|
||||||
|
effects.
|
||||||
|
|
@ -2,9 +2,18 @@ part of 'recorder.dart';
|
||||||
|
|
||||||
typedef _MergeRowKey = (String, UuidValue);
|
typedef _MergeRowKey = (String, UuidValue);
|
||||||
typedef _MergeFieldKey = (String, UuidValue, String);
|
typedef _MergeFieldKey = (String, UuidValue, String);
|
||||||
|
typedef _MergeContext = ({
|
||||||
|
Map<_MergeRowKey, CrdtDataRow> rows,
|
||||||
|
Map<_MergeFieldKey, CrdtDataField> fields,
|
||||||
|
Map<_MergeFieldKey, Hlc> incomingFieldHlcs,
|
||||||
|
Map<_MergeRowKey, CrdtDataDeleted> tombstones,
|
||||||
|
});
|
||||||
|
|
||||||
/// Adds merge-specific behavior to [CrdtMutationRecorder].
|
/// Adds merge-specific behavior to [CrdtMutationRecorder].
|
||||||
extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
||||||
|
CrdtUniqueConflictResolver get _uniqueConflictResolver =>
|
||||||
|
CrdtUniqueConflictResolver(this);
|
||||||
|
|
||||||
/// Locks the current user row so merges can serialize with other work.
|
/// Locks the current user row so merges can serialize with other work.
|
||||||
Future<void> lockCurrentUser(Transaction transaction) async {
|
Future<void> lockCurrentUser(Transaction transaction) async {
|
||||||
final user = _getEffectiveUser(transaction);
|
final user = _getEffectiveUser(transaction);
|
||||||
|
|
@ -33,7 +42,7 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
||||||
{for (final change in operations) change.uuidNodeId},
|
{for (final change in operations) change.uuidNodeId},
|
||||||
transaction,
|
transaction,
|
||||||
);
|
);
|
||||||
final metadata = await _loadMergeMetadata(mergeSet, transaction);
|
final context = await _loadMergeContext(mergeSet, transaction);
|
||||||
|
|
||||||
for (final operation in operations) {
|
for (final operation in operations) {
|
||||||
if (!_isCrdtTrackedTableName(operation.tableName)) {
|
if (!_isCrdtTrackedTableName(operation.tableName)) {
|
||||||
|
|
@ -45,25 +54,21 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
||||||
await _applyMergeInsert(
|
await _applyMergeInsert(
|
||||||
insert,
|
insert,
|
||||||
remoteNodes,
|
remoteNodes,
|
||||||
metadata.rows,
|
context,
|
||||||
metadata.fields,
|
|
||||||
metadata.tombstones,
|
|
||||||
transaction,
|
transaction,
|
||||||
);
|
);
|
||||||
case final CrdtMergeUpdate update:
|
case final CrdtMergeUpdate update:
|
||||||
await _applyMergeUpdate(
|
await _applyMergeUpdate(
|
||||||
update,
|
update,
|
||||||
remoteNodes,
|
remoteNodes,
|
||||||
metadata.rows,
|
context,
|
||||||
metadata.fields,
|
|
||||||
transaction,
|
transaction,
|
||||||
);
|
);
|
||||||
case final CrdtMergeDelete delete:
|
case final CrdtMergeDelete delete:
|
||||||
await _applyMergeDelete(
|
await _applyMergeDelete(
|
||||||
delete,
|
delete,
|
||||||
remoteNodes,
|
remoteNodes,
|
||||||
metadata.rows,
|
context,
|
||||||
metadata.tombstones,
|
|
||||||
transaction,
|
transaction,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
@ -171,14 +176,7 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
Future<
|
Future<_MergeContext> _loadMergeContext(
|
||||||
({
|
|
||||||
Map<_MergeRowKey, CrdtDataRow> rows,
|
|
||||||
Map<_MergeFieldKey, CrdtDataField> fields,
|
|
||||||
Map<_MergeRowKey, CrdtDataDeleted> tombstones,
|
|
||||||
})
|
|
||||||
>
|
|
||||||
_loadMergeMetadata(
|
|
||||||
CrdtMergeSet mergeSet,
|
CrdtMergeSet mergeSet,
|
||||||
Transaction transaction,
|
Transaction transaction,
|
||||||
) async {
|
) async {
|
||||||
|
|
@ -188,6 +186,10 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
||||||
|
|
||||||
final rows = <_MergeRowKey, CrdtDataRow>{};
|
final rows = <_MergeRowKey, CrdtDataRow>{};
|
||||||
final fields = <_MergeFieldKey, CrdtDataField>{};
|
final fields = <_MergeFieldKey, CrdtDataField>{};
|
||||||
|
final incomingFieldHlcs = {
|
||||||
|
for (final update in mergeSet.whereType<CrdtMergeUpdate>())
|
||||||
|
(update.tableName, update.uuidRowId, update.columnName): update.hlc,
|
||||||
|
};
|
||||||
final tombstones = <_MergeRowKey, CrdtDataDeleted>{};
|
final tombstones = <_MergeRowKey, CrdtDataDeleted>{};
|
||||||
|
|
||||||
for (final MapEntry(key: tableName, value: rowIds) in rowIdsByTable.entries) {
|
for (final MapEntry(key: tableName, value: rowIds) in rowIdsByTable.entries) {
|
||||||
|
|
@ -245,6 +247,7 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
||||||
return (
|
return (
|
||||||
rows: rows,
|
rows: rows,
|
||||||
fields: fields,
|
fields: fields,
|
||||||
|
incomingFieldHlcs: incomingFieldHlcs,
|
||||||
tombstones: tombstones,
|
tombstones: tombstones,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
@ -252,16 +255,14 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
||||||
Future<void> _applyMergeInsert(
|
Future<void> _applyMergeInsert(
|
||||||
CrdtMergeInsert insert,
|
CrdtMergeInsert insert,
|
||||||
Map<UuidValue, CrdtNode> remoteNodes,
|
Map<UuidValue, CrdtNode> remoteNodes,
|
||||||
Map<_MergeRowKey, CrdtDataRow> rows,
|
_MergeContext context,
|
||||||
Map<_MergeFieldKey, CrdtDataField> fields,
|
|
||||||
Map<_MergeRowKey, CrdtDataDeleted> tombstones,
|
|
||||||
Transaction transaction,
|
Transaction transaction,
|
||||||
) async {
|
) async {
|
||||||
final rowKey = (insert.tableName, insert.uuidRowId);
|
final rowKey = (insert.tableName, insert.uuidRowId);
|
||||||
final incomingHlc = insert.hlc;
|
final incomingHlc = insert.hlc;
|
||||||
final remoteNode = _requireRemoteNode(remoteNodes, insert.uuidNodeId);
|
final remoteNode = _requireRemoteNode(remoteNodes, insert.uuidNodeId);
|
||||||
|
|
||||||
final currentRow = rows[rowKey];
|
final currentRow = context.rows[rowKey];
|
||||||
if (currentRow != null && incomingHlc <= currentRow.hlc) {
|
if (currentRow != null && incomingHlc <= currentRow.hlc) {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
@ -269,11 +270,18 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
||||||
final data = _sanitizeMergeRowData(insert.tableName, insert.databaseColumns);
|
final data = _sanitizeMergeRowData(insert.tableName, insert.databaseColumns);
|
||||||
|
|
||||||
if (currentRow == null) {
|
if (currentRow == null) {
|
||||||
rows[rowKey] = await _applyMergeInsertForMissingRow(
|
final resolvedInsert = await _uniqueConflictResolver._resolveForIncomingInsert(
|
||||||
|
insert,
|
||||||
|
data,
|
||||||
|
context,
|
||||||
|
transaction,
|
||||||
|
);
|
||||||
|
|
||||||
|
context.rows[rowKey] = await _applyMergeInsertForMissingRow(
|
||||||
insert,
|
insert,
|
||||||
remoteNode,
|
remoteNode,
|
||||||
incomingHlc,
|
incomingHlc,
|
||||||
data,
|
resolvedInsert.data,
|
||||||
transaction,
|
transaction,
|
||||||
);
|
);
|
||||||
} else {
|
} else {
|
||||||
|
|
@ -283,8 +291,7 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
||||||
remoteNode,
|
remoteNode,
|
||||||
incomingHlc,
|
incomingHlc,
|
||||||
data,
|
data,
|
||||||
fields,
|
context,
|
||||||
tombstones,
|
|
||||||
transaction,
|
transaction,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
@ -293,12 +300,11 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
||||||
Future<void> _applyMergeUpdate(
|
Future<void> _applyMergeUpdate(
|
||||||
CrdtMergeUpdate update,
|
CrdtMergeUpdate update,
|
||||||
Map<UuidValue, CrdtNode> remoteNodes,
|
Map<UuidValue, CrdtNode> remoteNodes,
|
||||||
Map<_MergeRowKey, CrdtDataRow> rows,
|
_MergeContext context,
|
||||||
Map<_MergeFieldKey, CrdtDataField> fields,
|
|
||||||
Transaction transaction,
|
Transaction transaction,
|
||||||
) async {
|
) async {
|
||||||
final rowKey = (update.tableName, update.uuidRowId);
|
final rowKey = (update.tableName, update.uuidRowId);
|
||||||
final row = rows[rowKey];
|
final row = context.rows[rowKey];
|
||||||
if (row == null) return;
|
if (row == null) return;
|
||||||
|
|
||||||
final shouldApply = await _shouldMergeFieldMetadataIfNewer(
|
final shouldApply = await _shouldMergeFieldMetadataIfNewer(
|
||||||
|
|
@ -308,15 +314,22 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
||||||
row: row,
|
row: row,
|
||||||
remoteNode: _requireRemoteNode(remoteNodes, update.uuidNodeId),
|
remoteNode: _requireRemoteNode(remoteNodes, update.uuidNodeId),
|
||||||
incomingHlc: update.hlc,
|
incomingHlc: update.hlc,
|
||||||
fields: fields,
|
fields: context.fields,
|
||||||
transaction: transaction,
|
transaction: transaction,
|
||||||
);
|
);
|
||||||
|
|
||||||
if (!shouldApply) return;
|
if (!shouldApply) return;
|
||||||
|
|
||||||
|
final resolvedUpdates = await _uniqueConflictResolver._resolveForIncomingUpdate(
|
||||||
|
update,
|
||||||
|
row,
|
||||||
|
context,
|
||||||
|
transaction,
|
||||||
|
);
|
||||||
await _updateDomainRows(
|
await _updateDomainRows(
|
||||||
update.tableName,
|
update.tableName,
|
||||||
{update.uuidRowId},
|
{update.uuidRowId},
|
||||||
{update.columnName: update.value},
|
resolvedUpdates,
|
||||||
transaction,
|
transaction,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
@ -324,21 +337,20 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
||||||
Future<void> _applyMergeDelete(
|
Future<void> _applyMergeDelete(
|
||||||
CrdtMergeDelete delete,
|
CrdtMergeDelete delete,
|
||||||
Map<UuidValue, CrdtNode> remoteNodes,
|
Map<UuidValue, CrdtNode> remoteNodes,
|
||||||
Map<_MergeRowKey, CrdtDataRow> rows,
|
_MergeContext context,
|
||||||
Map<_MergeRowKey, CrdtDataDeleted> tombstones,
|
|
||||||
Transaction transaction,
|
Transaction transaction,
|
||||||
) async {
|
) async {
|
||||||
final rowKey = (delete.tableName, delete.uuidRowId);
|
final rowKey = (delete.tableName, delete.uuidRowId);
|
||||||
final row = rows[rowKey];
|
final row = context.rows[rowKey];
|
||||||
if (row == null) return;
|
if (row == null) return;
|
||||||
|
|
||||||
final currentTombstone = tombstones[rowKey];
|
final currentTombstone = context.tombstones[rowKey];
|
||||||
final currentClFlag = currentTombstone?.clFlag ?? 1;
|
final currentClFlag = currentTombstone?.clFlag ?? 1;
|
||||||
final currentHlc = currentTombstone?.hlc ?? row.hlc;
|
final currentHlc = currentTombstone?.hlc ?? row.hlc;
|
||||||
if (delete.clFlag < currentClFlag) return;
|
if (delete.clFlag < currentClFlag) return;
|
||||||
if (delete.clFlag == currentClFlag && delete.hlc <= currentHlc) return;
|
if (delete.clFlag == currentClFlag && delete.hlc <= currentHlc) return;
|
||||||
|
|
||||||
tombstones[rowKey] = await _upsertMergeTombstone(
|
context.tombstones[rowKey] = await _upsertMergeTombstone(
|
||||||
row,
|
row,
|
||||||
_requireRemoteNode(remoteNodes, delete.uuidNodeId),
|
_requireRemoteNode(remoteNodes, delete.uuidNodeId),
|
||||||
delete.hlc,
|
delete.hlc,
|
||||||
|
|
@ -503,17 +515,18 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
||||||
transaction,
|
transaction,
|
||||||
);
|
);
|
||||||
|
|
||||||
final row = _requireMergeInsertTableRow(insert);
|
final tableRow = _requireMergeInsertTableRow(insert);
|
||||||
|
final patchedRow = _db.serializationManager.patchTableRow(tableRow, data);
|
||||||
try {
|
try {
|
||||||
await _db.updateRow(
|
await _db.updateRow(
|
||||||
row,
|
patchedRow,
|
||||||
columns: row.table.managedColumns
|
columns: patchedRow.table.managedColumns
|
||||||
.where((c) => data.containsKey(c.columnName))
|
.where((c) => data.containsKey(c.columnName))
|
||||||
.toList(),
|
.toList(),
|
||||||
transaction: transaction,
|
transaction: transaction,
|
||||||
);
|
);
|
||||||
} on DatabaseUpdateRowException {
|
} on DatabaseUpdateRowException {
|
||||||
await _db.insertRow(row, transaction: transaction);
|
await _db.insertRow(patchedRow, transaction: transaction);
|
||||||
}
|
}
|
||||||
|
|
||||||
return insertedCrdtRow;
|
return insertedCrdtRow;
|
||||||
|
|
@ -525,8 +538,7 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
||||||
CrdtNode remoteNode,
|
CrdtNode remoteNode,
|
||||||
Hlc incomingHlc,
|
Hlc incomingHlc,
|
||||||
Map<String, Object?> data,
|
Map<String, Object?> data,
|
||||||
Map<_MergeFieldKey, CrdtDataField> fields,
|
_MergeContext context,
|
||||||
Map<_MergeRowKey, CrdtDataDeleted> tombstones,
|
|
||||||
Transaction transaction,
|
Transaction transaction,
|
||||||
) async {
|
) async {
|
||||||
final (_, columnsByName) = _schema[insert.tableName]!;
|
final (_, columnsByName) = _schema[insert.tableName]!;
|
||||||
|
|
@ -542,7 +554,7 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
||||||
row: currentRow,
|
row: currentRow,
|
||||||
remoteNode: remoteNode,
|
remoteNode: remoteNode,
|
||||||
incomingHlc: incomingHlc,
|
incomingHlc: incomingHlc,
|
||||||
fields: fields,
|
fields: context.fields,
|
||||||
transaction: transaction,
|
transaction: transaction,
|
||||||
schemaColumn: schemaColumn,
|
schemaColumn: schemaColumn,
|
||||||
);
|
);
|
||||||
|
|
@ -551,20 +563,31 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
||||||
updatedValues[columnName] = data[columnName];
|
updatedValues[columnName] = data[columnName];
|
||||||
}
|
}
|
||||||
|
|
||||||
if (updatedValues.isNotEmpty) {
|
var updatesToApply = updatedValues;
|
||||||
|
if (updatesToApply.isNotEmpty) {
|
||||||
|
updatesToApply = await _uniqueConflictResolver._resolveForIncomingUpdates(
|
||||||
|
tableName: insert.tableName,
|
||||||
|
row: currentRow,
|
||||||
|
updates: updatesToApply,
|
||||||
|
context: context,
|
||||||
|
transaction: transaction,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (updatesToApply.isNotEmpty) {
|
||||||
await _updateDomainRows(
|
await _updateDomainRows(
|
||||||
insert.tableName,
|
insert.tableName,
|
||||||
{insert.uuidRowId},
|
{insert.uuidRowId},
|
||||||
updatedValues,
|
updatesToApply,
|
||||||
transaction,
|
transaction,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
final rowKey = (insert.tableName, insert.uuidRowId);
|
final rowKey = (insert.tableName, insert.uuidRowId);
|
||||||
final currentTombstone = tombstones[rowKey];
|
final currentTombstone = context.tombstones[rowKey];
|
||||||
if ((currentTombstone?.clFlag ?? 1) == 1 &&
|
if ((currentTombstone?.clFlag ?? 1) == 1 &&
|
||||||
incomingHlc > (currentTombstone?.hlc ?? currentRow.hlc)) {
|
incomingHlc > (currentTombstone?.hlc ?? currentRow.hlc)) {
|
||||||
tombstones[rowKey] = await _upsertMergeTombstone(
|
context.tombstones[rowKey] = await _upsertMergeTombstone(
|
||||||
currentRow,
|
currentRow,
|
||||||
remoteNode,
|
remoteNode,
|
||||||
incomingHlc,
|
incomingHlc,
|
||||||
|
|
@ -640,3 +663,16 @@ extension CrdtMergeRecorderExtension on CrdtMutationRecorder {
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
extension on DatabaseSerializationManager {
|
||||||
|
/// Patches a [TableRow] with the given data.
|
||||||
|
T patchTableRow<T extends TableRow>(T row, Map<String, Object?> data) {
|
||||||
|
final rowJson = Map<String, dynamic>.from(row.toJson() as Map);
|
||||||
|
for (final column in row.table.managedColumns) {
|
||||||
|
if (column.columnName != 'id' && data.containsKey(column.columnName)) {
|
||||||
|
rowJson[column.fieldName] = data[column.columnName];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return deserialize<T>(rowJson);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,379 @@
|
||||||
|
part of '../recorder.dart';
|
||||||
|
|
||||||
|
typedef _UniqueConflict = ({
|
||||||
|
CrdtDataRow row,
|
||||||
|
_UniqueIndexConflictRelease uniqueIndex,
|
||||||
|
});
|
||||||
|
|
||||||
|
/// Resolves unique constraint conflicts that appear while merging CRDT changes.
|
||||||
|
class CrdtUniqueConflictResolver {
|
||||||
|
/// Creates a resolver for the merge recorder.
|
||||||
|
CrdtUniqueConflictResolver(this._recorder);
|
||||||
|
|
||||||
|
final CrdtMutationRecorder _recorder;
|
||||||
|
|
||||||
|
/// Resolves unique conflicts for an incoming insert.
|
||||||
|
Future<({Map<String, Object?> data, bool changed})> _resolveForIncomingInsert(
|
||||||
|
CrdtMergeInsert insert,
|
||||||
|
Map<String, Object?> data,
|
||||||
|
_MergeContext context,
|
||||||
|
Transaction transaction,
|
||||||
|
) async {
|
||||||
|
final resolvedData = Map<String, Object?>.from(data);
|
||||||
|
var changed = false;
|
||||||
|
final conflicts = await _findVisibleUniqueConflicts(
|
||||||
|
tableName: insert.tableName,
|
||||||
|
rowId: insert.uuidRowId,
|
||||||
|
values: data,
|
||||||
|
transaction: transaction,
|
||||||
|
);
|
||||||
|
|
||||||
|
for (final conflict in conflicts) {
|
||||||
|
final incomingClaimHlc = _incomingInsertUniqueClaim(
|
||||||
|
insert: insert,
|
||||||
|
uniqueIndex: conflict.uniqueIndex,
|
||||||
|
context: context,
|
||||||
|
);
|
||||||
|
final conflictClaimHlc = await _uniqueIndexHlc(
|
||||||
|
tableName: insert.tableName,
|
||||||
|
row: conflict.row,
|
||||||
|
uniqueIndex: conflict.uniqueIndex,
|
||||||
|
fields: context.fields,
|
||||||
|
transaction: transaction,
|
||||||
|
);
|
||||||
|
|
||||||
|
if (incomingClaimHlc > conflictClaimHlc) {
|
||||||
|
resolvedData.addAll(
|
||||||
|
_uniqueConflictFreeValuesForData(
|
||||||
|
insert.tableName,
|
||||||
|
insert.uuidRowId,
|
||||||
|
resolvedData,
|
||||||
|
conflict.uniqueIndex,
|
||||||
|
),
|
||||||
|
);
|
||||||
|
changed = true;
|
||||||
|
} else {
|
||||||
|
await _releaseUniqueConflictForRow(
|
||||||
|
tableName: insert.tableName,
|
||||||
|
rowId: conflict.row.uuidRowId,
|
||||||
|
uniqueIndex: conflict.uniqueIndex,
|
||||||
|
transaction: transaction,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return (data: resolvedData, changed: changed);
|
||||||
|
}
|
||||||
|
|
||||||
|
Hlc _incomingInsertUniqueClaim({
|
||||||
|
required CrdtMergeInsert insert,
|
||||||
|
required _UniqueIndexConflictRelease uniqueIndex,
|
||||||
|
required _MergeContext context,
|
||||||
|
}) {
|
||||||
|
var uniqueHlc = insert.hlc;
|
||||||
|
for (final column in uniqueIndex.columns) {
|
||||||
|
uniqueHlc = uniqueHlc.maxBetween(
|
||||||
|
context.incomingFieldHlcs[(
|
||||||
|
insert.tableName,
|
||||||
|
insert.uuidRowId,
|
||||||
|
column.columnName,
|
||||||
|
)],
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return uniqueHlc;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Resolves unique conflicts for a single incoming field update.
|
||||||
|
Future<Map<String, Object?>> _resolveForIncomingUpdate(
|
||||||
|
CrdtMergeUpdate update,
|
||||||
|
CrdtDataRow row,
|
||||||
|
_MergeContext context,
|
||||||
|
Transaction transaction,
|
||||||
|
) {
|
||||||
|
return _resolveForIncomingUpdates(
|
||||||
|
tableName: update.tableName,
|
||||||
|
row: row,
|
||||||
|
updates: {update.columnName: update.value},
|
||||||
|
context: context,
|
||||||
|
transaction: transaction,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Resolves unique conflicts for one or more incoming field updates.
|
||||||
|
Future<Map<String, Object?>> _resolveForIncomingUpdates({
|
||||||
|
required String tableName,
|
||||||
|
required CrdtDataRow row,
|
||||||
|
required Map<String, Object?> updates,
|
||||||
|
required _MergeContext context,
|
||||||
|
required Transaction transaction,
|
||||||
|
}) async {
|
||||||
|
if (updates.keys.every(
|
||||||
|
(columnName) => !_isUniqueIndexedColumn(tableName, columnName),
|
||||||
|
)) {
|
||||||
|
return updates;
|
||||||
|
}
|
||||||
|
|
||||||
|
final tableDefinition = _recorder._tableDefinitionsByName[tableName];
|
||||||
|
if (tableDefinition == null) {
|
||||||
|
return updates;
|
||||||
|
}
|
||||||
|
|
||||||
|
final uniqueColumnNamesToRead = {
|
||||||
|
for (final uniqueIndex in _recorder._uniqueIndexesForTable(tableDefinition))
|
||||||
|
for (final column in uniqueIndex.columns) column.columnName,
|
||||||
|
};
|
||||||
|
if (uniqueColumnNamesToRead.isEmpty) {
|
||||||
|
return updates;
|
||||||
|
}
|
||||||
|
|
||||||
|
final currentValues = await _recorder._readDomainColumnValues(
|
||||||
|
tableName,
|
||||||
|
{row.uuidRowId},
|
||||||
|
uniqueColumnNamesToRead.toList(),
|
||||||
|
transaction,
|
||||||
|
);
|
||||||
|
final values = {
|
||||||
|
...?currentValues[row.uuidRowId],
|
||||||
|
...updates,
|
||||||
|
};
|
||||||
|
|
||||||
|
final conflicts = await _findVisibleUniqueConflicts(
|
||||||
|
tableName: tableName,
|
||||||
|
rowId: row.uuidRowId,
|
||||||
|
values: values,
|
||||||
|
transaction: transaction,
|
||||||
|
);
|
||||||
|
|
||||||
|
final resolvedUpdates = Map<String, Object?>.from(updates);
|
||||||
|
for (final conflict in conflicts) {
|
||||||
|
final rowClaimHlc = await _uniqueIndexHlc(
|
||||||
|
tableName: tableName,
|
||||||
|
row: row,
|
||||||
|
uniqueIndex: conflict.uniqueIndex,
|
||||||
|
fields: context.fields,
|
||||||
|
transaction: transaction,
|
||||||
|
);
|
||||||
|
final conflictClaimHlc = await _uniqueIndexHlc(
|
||||||
|
tableName: tableName,
|
||||||
|
row: conflict.row,
|
||||||
|
uniqueIndex: conflict.uniqueIndex,
|
||||||
|
fields: context.fields,
|
||||||
|
transaction: transaction,
|
||||||
|
);
|
||||||
|
|
||||||
|
if (rowClaimHlc > conflictClaimHlc) {
|
||||||
|
final releasedValues = _uniqueConflictFreeValuesForData(
|
||||||
|
tableName,
|
||||||
|
row.uuidRowId,
|
||||||
|
values,
|
||||||
|
conflict.uniqueIndex,
|
||||||
|
);
|
||||||
|
for (final columnName in conflict.uniqueIndex.columnNames) {
|
||||||
|
if (releasedValues.containsKey(columnName)) {
|
||||||
|
resolvedUpdates[columnName] = releasedValues[columnName];
|
||||||
|
values[columnName] = releasedValues[columnName];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
await _releaseUniqueConflictForRow(
|
||||||
|
tableName: tableName,
|
||||||
|
rowId: conflict.row.uuidRowId,
|
||||||
|
uniqueIndex: conflict.uniqueIndex,
|
||||||
|
transaction: transaction,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return resolvedUpdates;
|
||||||
|
}
|
||||||
|
|
||||||
|
Map<String, Object?> _uniqueConflictFreeValuesForData(
|
||||||
|
String tableName,
|
||||||
|
UuidValue rowId,
|
||||||
|
Map<String, Object?> data,
|
||||||
|
_UniqueIndexConflictRelease uniqueIndex,
|
||||||
|
) {
|
||||||
|
final tableDefinition = _recorder._tableDefinitionsByName[tableName];
|
||||||
|
if (tableDefinition == null) return data;
|
||||||
|
|
||||||
|
final released = Map<String, Object?>.from(data);
|
||||||
|
final values = {
|
||||||
|
for (final column in uniqueIndex.columns)
|
||||||
|
column.columnName: released[column.columnName],
|
||||||
|
};
|
||||||
|
if (values.values.any((value) => value == null)) return released;
|
||||||
|
|
||||||
|
for (final column in uniqueIndex.columns) {
|
||||||
|
released[column.columnName] = _recorder._conflictFreeValue(
|
||||||
|
column,
|
||||||
|
values[column.columnName],
|
||||||
|
tableName,
|
||||||
|
rowId,
|
||||||
|
'conflict',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return released;
|
||||||
|
}
|
||||||
|
|
||||||
|
Future<void> _releaseUniqueConflictForRow({
|
||||||
|
required String tableName,
|
||||||
|
required UuidValue rowId,
|
||||||
|
required _UniqueIndexConflictRelease uniqueIndex,
|
||||||
|
required Transaction transaction,
|
||||||
|
}) async {
|
||||||
|
final columnNames = uniqueIndex.columnNames.toList();
|
||||||
|
final valuesByRowId = await _recorder._readDomainColumnValues(
|
||||||
|
tableName,
|
||||||
|
{rowId},
|
||||||
|
columnNames,
|
||||||
|
transaction,
|
||||||
|
);
|
||||||
|
final values = valuesByRowId[rowId];
|
||||||
|
if (values == null || values.values.any((value) => value == null)) return;
|
||||||
|
|
||||||
|
final releasedValues = _uniqueConflictFreeValuesForData(
|
||||||
|
tableName,
|
||||||
|
rowId,
|
||||||
|
values,
|
||||||
|
uniqueIndex,
|
||||||
|
);
|
||||||
|
await _recorder._updateDomainRows(tableName, {rowId}, releasedValues, transaction);
|
||||||
|
}
|
||||||
|
|
||||||
|
Future<List<_UniqueConflict>> _findVisibleUniqueConflicts({
|
||||||
|
required String tableName,
|
||||||
|
required UuidValue rowId,
|
||||||
|
required Map<String, Object?> values,
|
||||||
|
required Transaction transaction,
|
||||||
|
}) async {
|
||||||
|
final tableDefinition = _recorder._tableDefinitionsByName[tableName];
|
||||||
|
if (tableDefinition == null) return const [];
|
||||||
|
|
||||||
|
final uniqueIndexes = _recorder._uniqueIndexesForTable(tableDefinition);
|
||||||
|
if (uniqueIndexes.isEmpty) return const [];
|
||||||
|
|
||||||
|
final uniqueIndexesByConflictId = <UuidValue, Set<_UniqueIndexConflictRelease>>{};
|
||||||
|
for (final uniqueIndex in uniqueIndexes) {
|
||||||
|
final columnValues = {
|
||||||
|
for (final column in uniqueIndex.columns)
|
||||||
|
if (values.containsKey(column.columnName))
|
||||||
|
column.columnName: values[column.columnName],
|
||||||
|
};
|
||||||
|
if (columnValues.length != uniqueIndex.columns.length) continue;
|
||||||
|
if (columnValues.values.any((value) => value == null)) continue;
|
||||||
|
|
||||||
|
final uniqueConflictIds = await _findVisibleRowsByUniqueValues(
|
||||||
|
tableName: tableName,
|
||||||
|
rowId: rowId,
|
||||||
|
values: columnValues,
|
||||||
|
transaction: transaction,
|
||||||
|
);
|
||||||
|
for (final conflictId in uniqueConflictIds) {
|
||||||
|
uniqueIndexesByConflictId.putIfAbsent(conflictId, () => {}).add(uniqueIndex);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (uniqueIndexesByConflictId.isEmpty) return const [];
|
||||||
|
final conflictRows = await _recorder._findCrdtRows(
|
||||||
|
tableName,
|
||||||
|
uniqueIndexesByConflictId.keys.toSet(),
|
||||||
|
transaction,
|
||||||
|
include: CrdtDataRow.include(
|
||||||
|
node: CrdtNode.include(),
|
||||||
|
deleted: CrdtDataDeleted.include(node: CrdtNode.include()),
|
||||||
|
),
|
||||||
|
);
|
||||||
|
return [
|
||||||
|
for (final conflictRow in conflictRows)
|
||||||
|
for (final uniqueIndex in uniqueIndexesByConflictId[conflictRow.uuidRowId]!)
|
||||||
|
(
|
||||||
|
row: conflictRow,
|
||||||
|
uniqueIndex: uniqueIndex,
|
||||||
|
),
|
||||||
|
];
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Returns the HLC for this row's current claim on [uniqueIndex].
|
||||||
|
///
|
||||||
|
/// Composite indexes use the newest field HLC among their columns; rows with
|
||||||
|
/// no field metadata fall back to the row insert HLC. Missing field metadata
|
||||||
|
/// is loaded lazily because conflicting rows may not be part of the preloaded
|
||||||
|
/// merge context.
|
||||||
|
Future<Hlc> _uniqueIndexHlc({
|
||||||
|
required String tableName,
|
||||||
|
required CrdtDataRow row,
|
||||||
|
required _UniqueIndexConflictRelease uniqueIndex,
|
||||||
|
required Map<_MergeFieldKey, CrdtDataField> fields,
|
||||||
|
required Transaction transaction,
|
||||||
|
}) async {
|
||||||
|
final (_, columnsByName) = _recorder._schema[tableName]!;
|
||||||
|
final uniqueColumnNames = {
|
||||||
|
for (final column in uniqueIndex.columns) column.columnName,
|
||||||
|
};
|
||||||
|
final missingColumnIds = <int>{};
|
||||||
|
final maxFieldHlc = <Hlc>[];
|
||||||
|
for (final columnName in uniqueColumnNames) {
|
||||||
|
final field = fields[(tableName, row.uuidRowId, columnName)];
|
||||||
|
if (field == null) {
|
||||||
|
final columnId = columnsByName[columnName]?.id;
|
||||||
|
if (columnId != null) missingColumnIds.add(columnId);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
maxFieldHlc.add(field.hlc);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (missingColumnIds.isNotEmpty && row.id != null) {
|
||||||
|
final loadedFields = await CrdtDataField.db.find(
|
||||||
|
_recorder._session,
|
||||||
|
where: (t) => t.rowId.equals(row.id) & t.columnId.inSet(missingColumnIds),
|
||||||
|
include: CrdtDataField.include(
|
||||||
|
column: CrdtSchemaColumn.include(),
|
||||||
|
node: CrdtNode.include(),
|
||||||
|
),
|
||||||
|
transaction: transaction,
|
||||||
|
);
|
||||||
|
|
||||||
|
for (final field in loadedFields) {
|
||||||
|
final columnName = field.column?.name;
|
||||||
|
if (columnName == null) continue;
|
||||||
|
fields[(tableName, row.uuidRowId, columnName)] = field;
|
||||||
|
maxFieldHlc.add(field.hlc);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (maxFieldHlc.isEmpty) return row.hlc;
|
||||||
|
return maxFieldHlc.reduce((left, right) => left.maxBetween(right));
|
||||||
|
}
|
||||||
|
|
||||||
|
Future<Set<UuidValue>> _findVisibleRowsByUniqueValues({
|
||||||
|
required String tableName,
|
||||||
|
required UuidValue rowId,
|
||||||
|
required Map<String, Object?> values,
|
||||||
|
required Transaction transaction,
|
||||||
|
}) async {
|
||||||
|
return _recorder._findVisibleDomainRowIdsWhere(
|
||||||
|
tableName: tableName,
|
||||||
|
predicates: [
|
||||||
|
for (final MapEntry(key: columnName, value: value) in values.entries)
|
||||||
|
'd."${_escapeIdentifier(columnName)}" = ${_sqlLiteral(value)}',
|
||||||
|
'd."id" <> ${_sqlLiteral(rowId)}',
|
||||||
|
],
|
||||||
|
transaction: transaction,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
bool _isUniqueIndexedColumn(String tableName, String columnName) {
|
||||||
|
final tableDefinition = _recorder._tableDefinitionsByName[tableName];
|
||||||
|
if (tableDefinition == null) return false;
|
||||||
|
return _recorder
|
||||||
|
._uniqueIndexesForTable(tableDefinition)
|
||||||
|
.any((i) => i.columns.any((c) => c.columnName == columnName));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
extension on _UniqueIndexConflictRelease {
|
||||||
|
Set<String> get columnNames => {
|
||||||
|
for (final column in columns) column.columnName,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
@ -12,6 +12,7 @@ import 'schema.dart';
|
||||||
import 'session.dart';
|
import 'session.dart';
|
||||||
|
|
||||||
part 'merge.dart';
|
part 'merge.dart';
|
||||||
|
part 'merge_utils/unique_resolver.dart';
|
||||||
|
|
||||||
typedef _ReferencingForeignKey = ({
|
typedef _ReferencingForeignKey = ({
|
||||||
String childTableName,
|
String childTableName,
|
||||||
|
|
@ -893,6 +894,7 @@ WHERE c."id" IN ($whereRowIds)
|
||||||
values[column.columnName],
|
values[column.columnName],
|
||||||
tableDefinition.name,
|
tableDefinition.name,
|
||||||
rowId,
|
rowId,
|
||||||
|
'deleted',
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -925,6 +927,18 @@ WHERE c."id" IN ($whereRowIds)
|
||||||
required String columnName,
|
required String columnName,
|
||||||
required Object? value,
|
required Object? value,
|
||||||
required Transaction transaction,
|
required Transaction transaction,
|
||||||
|
}) async {
|
||||||
|
return _findVisibleDomainRowIdsWhere(
|
||||||
|
tableName: tableName,
|
||||||
|
predicates: ['d."${_escapeIdentifier(columnName)}" = ${_sqlLiteral(value)}'],
|
||||||
|
transaction: transaction,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
Future<Set<UuidValue>> _findVisibleDomainRowIdsWhere({
|
||||||
|
required String tableName,
|
||||||
|
required List<String> predicates,
|
||||||
|
required Transaction transaction,
|
||||||
}) async {
|
}) async {
|
||||||
final (tableId, _) = _schema[tableName]!;
|
final (tableId, _) = _schema[tableName]!;
|
||||||
final userId = _getHlcManager(transaction).normalizedUserId;
|
final userId = _getHlcManager(transaction).normalizedUserId;
|
||||||
|
|
@ -936,7 +950,7 @@ LEFT JOIN "crdt_data_rows" r
|
||||||
ON r."userId" = $userId AND r."tblId" = $tableId AND r."uuidRowId" = d."id"
|
ON r."userId" = $userId AND r."tblId" = $tableId AND r."uuidRowId" = d."id"
|
||||||
LEFT JOIN "crdt_data_tombstone" tomb
|
LEFT JOIN "crdt_data_tombstone" tomb
|
||||||
ON tomb."rowId" = r."id"
|
ON tomb."rowId" = r."id"
|
||||||
WHERE d."${_escapeIdentifier(columnName)}" = ${_sqlLiteral(value)}
|
WHERE (${predicates.join(') AND (')})
|
||||||
AND (tomb."id" IS NULL OR tomb."clFlag" % 2 = 1)
|
AND (tomb."id" IS NULL OR tomb."clFlag" % 2 = 1)
|
||||||
''',
|
''',
|
||||||
transaction: transaction,
|
transaction: transaction,
|
||||||
|
|
@ -1017,21 +1031,20 @@ WHERE "id" IN (${_sqlLiteralList(rowIds)})
|
||||||
Object? value,
|
Object? value,
|
||||||
String tableName,
|
String tableName,
|
||||||
UuidValue conflictingId,
|
UuidValue conflictingId,
|
||||||
|
String releaseSuffix,
|
||||||
) {
|
) {
|
||||||
switch (column.kind) {
|
switch (column.kind) {
|
||||||
case _UniqueConflictReleaseKind.setNull:
|
case _UniqueConflictReleaseKind.setNull:
|
||||||
return null;
|
return null;
|
||||||
case _UniqueConflictReleaseKind.textSuffix:
|
case _UniqueConflictReleaseKind.textSuffix:
|
||||||
if (value is String) {
|
if (value is String) {
|
||||||
return '${value}__deleted__${conflictingId.uuid}';
|
return '${value}__${releaseSuffix}__${conflictingId.uuid}';
|
||||||
}
|
}
|
||||||
case _UniqueConflictReleaseKind.syntheticUuid:
|
case _UniqueConflictReleaseKind.syntheticUuid:
|
||||||
if (value != null) {
|
if (value != null) {
|
||||||
return _syntheticDeletedUuid(
|
return const Uuid().v5obj(
|
||||||
tableName,
|
Namespace.oid.value,
|
||||||
column.columnName,
|
'$tableName.${column.columnName}:${value}__${releaseSuffix}__$conflictingId',
|
||||||
UuidValueJsonExtension.fromJson(value),
|
|
||||||
conflictingId,
|
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -1140,20 +1153,6 @@ bool _isForeignKeyColumn(TableDefinition table, String columnName) {
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
UuidValue _syntheticDeletedUuid(
|
|
||||||
String tableName,
|
|
||||||
String columnName,
|
|
||||||
UuidValue value,
|
|
||||||
UuidValue conflictingId,
|
|
||||||
) {
|
|
||||||
return UuidValue.withValidation(
|
|
||||||
const Uuid().v5(
|
|
||||||
Namespace.oid.value,
|
|
||||||
'$tableName.$columnName:${value.uuid}:${conflictingId.uuid}',
|
|
||||||
),
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
String _sqlLiteral(Object? value) => ValueEncoder.instance.convert(value);
|
String _sqlLiteral(Object? value) => ValueEncoder.instance.convert(value);
|
||||||
|
|
||||||
String _sqlLiteralList(Iterable<Object?> values) =>
|
String _sqlLiteralList(Iterable<Object?> values) =>
|
||||||
|
|
|
||||||
|
|
@ -74,6 +74,10 @@ melos:
|
||||||
- packages/*
|
- packages/*
|
||||||
|
|
||||||
scripts:
|
scripts:
|
||||||
|
cleanup: |
|
||||||
|
find . -type d \( -name generated -o -name protocol \) -print0 |
|
||||||
|
xargs -0 -I@ find @ -mindepth 1 -maxdepth 1 ! -name analysis_options.yaml -print0 |
|
||||||
|
xargs -0 rm -rf
|
||||||
setup:
|
setup:
|
||||||
run: dart pub get && melos generate && melos create-migration
|
run: dart pub get && melos generate && melos create-migration
|
||||||
lint:
|
lint:
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,12 @@
|
||||||
|
class: UniqueComposite
|
||||||
|
table: unique_composite
|
||||||
|
database: all
|
||||||
|
fields:
|
||||||
|
id: UuidValue?, defaultPersist=random_v7
|
||||||
|
scope: String
|
||||||
|
value: String
|
||||||
|
|
||||||
|
indexes:
|
||||||
|
unique_composite_scope_value_idx:
|
||||||
|
fields: scope, value
|
||||||
|
unique: true
|
||||||
|
|
@ -215,6 +215,159 @@ void main() {
|
||||||
});
|
});
|
||||||
},
|
},
|
||||||
);
|
);
|
||||||
|
|
||||||
|
group(
|
||||||
|
'Given a remote update that loses a composite unique conflict, ',
|
||||||
|
() {
|
||||||
|
late UniqueComposite loser;
|
||||||
|
late CrdtMergeUpdate remoteUpdate;
|
||||||
|
|
||||||
|
setUp(() async {
|
||||||
|
await session.db.transactionForUser(testCrdtUserId, (tx) async {
|
||||||
|
await UniqueComposite.db.insertRow(
|
||||||
|
session,
|
||||||
|
UniqueComposite(
|
||||||
|
id: const Uuid().v7obj(),
|
||||||
|
scope: 'left-scope',
|
||||||
|
value: 'shared-value',
|
||||||
|
),
|
||||||
|
transaction: tx,
|
||||||
|
);
|
||||||
|
loser = await UniqueComposite.db.insertRow(
|
||||||
|
session,
|
||||||
|
UniqueComposite(
|
||||||
|
id: const Uuid().v7obj(),
|
||||||
|
scope: 'right-scope',
|
||||||
|
value: 'shared-value',
|
||||||
|
),
|
||||||
|
transaction: tx,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
final loserCrdtRow = await CrdtDataRow.db.findFirstRow(
|
||||||
|
session,
|
||||||
|
where: (t) => t.uuidRowId.equals(loser.id),
|
||||||
|
include: CrdtDataRow.include(node: CrdtNode.include()),
|
||||||
|
);
|
||||||
|
|
||||||
|
remoteUpdate = CrdtMergeUpdate(
|
||||||
|
tableName: UniqueComposite.t.tableName,
|
||||||
|
uuidRowId: loser.id!,
|
||||||
|
uuidNodeId: const Uuid().v7obj(),
|
||||||
|
hlcDatetime: loserCrdtRow!.hlc.datetime.advance(),
|
||||||
|
hlcCounter: loserCrdtRow.hlc.counter,
|
||||||
|
columnName: UniqueComposite.t.scope.columnName,
|
||||||
|
value: 'left-scope',
|
||||||
|
);
|
||||||
|
|
||||||
|
mergeSet = [remoteUpdate];
|
||||||
|
});
|
||||||
|
|
||||||
|
group('when merging, ', () {
|
||||||
|
setUp(() async {
|
||||||
|
await session.db.mergeChanges(
|
||||||
|
mergeSet,
|
||||||
|
userId: testCrdtUserId,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test(
|
||||||
|
'then the losing row receives conflict-free composite unique values.',
|
||||||
|
() async {
|
||||||
|
final row = await UniqueComposite.db.findById(session, loser.id!);
|
||||||
|
|
||||||
|
expect(row, isNotNull);
|
||||||
|
expect(row!.scope, 'left-scope__conflict__${loser.id!.uuid}');
|
||||||
|
expect(row.value, 'shared-value__conflict__${loser.id!.uuid}');
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
test(
|
||||||
|
'then the silently released unique column does not receive CRDT field metadata.',
|
||||||
|
() async {
|
||||||
|
final fields = await CrdtDataField.db.find(
|
||||||
|
session,
|
||||||
|
where: (t) => t.row.uuidRowId.equals(loser.id),
|
||||||
|
include: CrdtDataField.include(
|
||||||
|
column: CrdtSchemaColumn.include(),
|
||||||
|
node: CrdtNode.include(),
|
||||||
|
),
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(fields, hasLength(1));
|
||||||
|
expect(fields.first.column!.name, UniqueComposite.t.scope.columnName);
|
||||||
|
expect(fields.first.hlc, remoteUpdate.hlc);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
});
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
group(
|
||||||
|
'Given a remote update that loses a UUID unique conflict, ',
|
||||||
|
() {
|
||||||
|
final sharedValue = const Uuid().v7obj();
|
||||||
|
final originalLoserValue = const Uuid().v7obj();
|
||||||
|
late UniqueUuid loser;
|
||||||
|
|
||||||
|
setUp(() async {
|
||||||
|
await session.db.transactionForUser(testCrdtUserId, (tx) async {
|
||||||
|
await UniqueUuid.db.insertRow(
|
||||||
|
session,
|
||||||
|
UniqueUuid(id: const Uuid().v7obj(), value: sharedValue),
|
||||||
|
transaction: tx,
|
||||||
|
);
|
||||||
|
loser = await UniqueUuid.db.insertRow(
|
||||||
|
session,
|
||||||
|
UniqueUuid(id: const Uuid().v7obj(), value: originalLoserValue),
|
||||||
|
transaction: tx,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
final loserCrdtRow = await CrdtDataRow.db.findFirstRow(
|
||||||
|
session,
|
||||||
|
where: (t) => t.uuidRowId.equals(loser.id),
|
||||||
|
include: CrdtDataRow.include(node: CrdtNode.include()),
|
||||||
|
);
|
||||||
|
|
||||||
|
final remoteUpdate = CrdtMergeUpdate(
|
||||||
|
tableName: UniqueUuid.t.tableName,
|
||||||
|
uuidRowId: loser.id!,
|
||||||
|
uuidNodeId: const Uuid().v7obj(),
|
||||||
|
hlcDatetime: loserCrdtRow!.hlc.datetime.advance(),
|
||||||
|
hlcCounter: loserCrdtRow.hlc.counter,
|
||||||
|
columnName: UniqueUuid.t.value.columnName,
|
||||||
|
value: sharedValue,
|
||||||
|
);
|
||||||
|
|
||||||
|
mergeSet = [remoteUpdate];
|
||||||
|
});
|
||||||
|
|
||||||
|
group('when merging, ', () {
|
||||||
|
setUp(() async {
|
||||||
|
await session.db.mergeChanges(
|
||||||
|
mergeSet,
|
||||||
|
userId: testCrdtUserId,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test(
|
||||||
|
'then the losing row receives a deterministic conflict-free UUID unique value.',
|
||||||
|
() async {
|
||||||
|
final row = await UniqueUuid.db.findById(session, loser.id!);
|
||||||
|
final expectedConflictValue = const Uuid().v5obj(
|
||||||
|
Namespace.oid.value,
|
||||||
|
'${UniqueUuid.t.tableName}.${UniqueUuid.t.value.columnName}:'
|
||||||
|
'${sharedValue.uuid}__conflict__${loser.id!.uuid}',
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(row, isNotNull);
|
||||||
|
expect(row!.value, expectedConflictValue);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
});
|
||||||
|
},
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
extension on DateTime {
|
extension on DateTime {
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,301 @@
|
||||||
|
import 'package:serverpod/serverpod.dart';
|
||||||
|
import 'package:serverpod_offline_sync_server/serverpod_offline_sync_server.dart';
|
||||||
|
import 'package:serverpod_offline_sync_test_client/serverpod_offline_sync_test_client.dart'
|
||||||
|
as client;
|
||||||
|
import 'package:serverpod_offline_sync_test_server/src/generated/protocol.dart'
|
||||||
|
as server;
|
||||||
|
import 'package:test/test.dart';
|
||||||
|
|
||||||
|
import '../test_tools/client_session.dart';
|
||||||
|
import '../test_tools/serverpod_test_tools.dart';
|
||||||
|
|
||||||
|
void main() {
|
||||||
|
initTestClientSession(withPersistentUser: true);
|
||||||
|
|
||||||
|
final clientSyncTables = [client.Unique.t];
|
||||||
|
final serverSyncTables = [server.Unique.t];
|
||||||
|
|
||||||
|
late client.Client syncHttpClient;
|
||||||
|
late CrdtDatabaseSession serverSession;
|
||||||
|
late CrdtDatabaseSession firstClientSession;
|
||||||
|
late CrdtDatabaseSession secondClientSession;
|
||||||
|
|
||||||
|
withServerpod(
|
||||||
|
'Given a server and two client CRDT sessions for the same user that are initialized with the unique sync table',
|
||||||
|
rollbackDatabase: RollbackDatabase.disabled,
|
||||||
|
(sessionBuilder, _) {
|
||||||
|
final rawServerSession = sessionBuilder.build();
|
||||||
|
|
||||||
|
rawServerSession.serverpod
|
||||||
|
..initializeCrdtSync(
|
||||||
|
syncTables: serverSyncTables,
|
||||||
|
)
|
||||||
|
..authenticationHandler = (session, token) async => AuthenticationInfo(
|
||||||
|
testCrdtUserId.toString(),
|
||||||
|
<Scope>{},
|
||||||
|
authId: const Uuid().v4(),
|
||||||
|
);
|
||||||
|
|
||||||
|
setUp(() async {
|
||||||
|
syncHttpClient = client.Client(
|
||||||
|
'http://localhost:${rawServerSession.server.port}',
|
||||||
|
)..authKeyProvider = TestClientAuthKeyProvider();
|
||||||
|
|
||||||
|
firstClientSession = CrdtDatabaseSession.wraps(
|
||||||
|
testSession,
|
||||||
|
syncTables: clientSyncTables,
|
||||||
|
persistentUserId: testCrdtUserId,
|
||||||
|
);
|
||||||
|
await firstClientSession.db.initialize();
|
||||||
|
|
||||||
|
secondClientSession = CrdtDatabaseSession.wraps(
|
||||||
|
await createAdditionalTestSession(),
|
||||||
|
syncTables: clientSyncTables,
|
||||||
|
persistentUserId: testCrdtUserId,
|
||||||
|
);
|
||||||
|
await secondClientSession.db.initialize();
|
||||||
|
|
||||||
|
serverSession = CrdtDatabaseSession.wraps(
|
||||||
|
rawServerSession,
|
||||||
|
syncTables: serverSyncTables,
|
||||||
|
);
|
||||||
|
await serverSession.db.initialize();
|
||||||
|
});
|
||||||
|
|
||||||
|
tearDown(() async {
|
||||||
|
await serverSession.clearUserTables();
|
||||||
|
});
|
||||||
|
|
||||||
|
group('and an older server row conflicts with a newer incoming row,', () {
|
||||||
|
late client.Unique olderRow;
|
||||||
|
late client.Unique newerRow;
|
||||||
|
|
||||||
|
setUp(() async {
|
||||||
|
olderRow = await client.Unique.db.insertRow(
|
||||||
|
firstClientSession,
|
||||||
|
client.Unique(id: const Uuid().v7obj(), name: 'shared-name'),
|
||||||
|
);
|
||||||
|
|
||||||
|
await Future<void>.delayed(const Duration(milliseconds: 2));
|
||||||
|
newerRow = await client.Unique.db.insertRow(
|
||||||
|
secondClientSession,
|
||||||
|
client.Unique(id: const Uuid().v7obj(), name: 'shared-name'),
|
||||||
|
);
|
||||||
|
|
||||||
|
await syncHttpClient.crdt.syncOnce(firstClientSession);
|
||||||
|
});
|
||||||
|
|
||||||
|
group('when the incoming row synchronizes,', () {
|
||||||
|
setUp(() async {
|
||||||
|
await syncHttpClient.crdt.syncOnce(secondClientSession);
|
||||||
|
await syncHttpClient.crdt.syncOnce(firstClientSession);
|
||||||
|
});
|
||||||
|
|
||||||
|
test(
|
||||||
|
'then both rows remain visible and the older row keeps the unique value.',
|
||||||
|
() async {
|
||||||
|
await _expectVisibleUniqueRowsOnAllNodes(
|
||||||
|
firstClientSession: firstClientSession,
|
||||||
|
secondClientSession: secondClientSession,
|
||||||
|
serverSession: serverSession,
|
||||||
|
originalValueId: olderRow.id!,
|
||||||
|
conflictValueId: newerRow.id!,
|
||||||
|
);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
group('and a newer server row conflicts with an older incoming row,', () {
|
||||||
|
late client.Unique olderRow;
|
||||||
|
late client.Unique newerRow;
|
||||||
|
|
||||||
|
setUp(() async {
|
||||||
|
olderRow = await client.Unique.db.insertRow(
|
||||||
|
secondClientSession,
|
||||||
|
client.Unique(id: const Uuid().v7obj(), name: 'shared-name'),
|
||||||
|
);
|
||||||
|
|
||||||
|
await Future<void>.delayed(const Duration(milliseconds: 2));
|
||||||
|
newerRow = await client.Unique.db.insertRow(
|
||||||
|
firstClientSession,
|
||||||
|
client.Unique(id: const Uuid().v7obj(), name: 'shared-name'),
|
||||||
|
);
|
||||||
|
|
||||||
|
await syncHttpClient.crdt.syncOnce(firstClientSession);
|
||||||
|
});
|
||||||
|
|
||||||
|
group('when the incoming row synchronizes,', () {
|
||||||
|
setUp(() async {
|
||||||
|
await syncHttpClient.crdt.syncOnce(secondClientSession);
|
||||||
|
await syncHttpClient.crdt.syncOnce(firstClientSession);
|
||||||
|
});
|
||||||
|
|
||||||
|
test(
|
||||||
|
'then both rows remain visible and the older row keeps the unique value.',
|
||||||
|
() async {
|
||||||
|
await _expectVisibleUniqueRowsOnAllNodes(
|
||||||
|
firstClientSession: firstClientSession,
|
||||||
|
secondClientSession: secondClientSession,
|
||||||
|
serverSession: serverSession,
|
||||||
|
originalValueId: olderRow.id!,
|
||||||
|
conflictValueId: newerRow.id!,
|
||||||
|
);
|
||||||
|
},
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
group('and a newer server update conflicts with an older incoming update,', () {
|
||||||
|
late client.Unique olderUpdatedRow;
|
||||||
|
late client.Unique newerUpdatedRow;
|
||||||
|
|
||||||
|
setUp(() async {
|
||||||
|
olderUpdatedRow = await client.Unique.db.insertRow(
|
||||||
|
firstClientSession,
|
||||||
|
client.Unique(id: const Uuid().v7obj(), name: 'first-name'),
|
||||||
|
);
|
||||||
|
newerUpdatedRow = await client.Unique.db.insertRow(
|
||||||
|
firstClientSession,
|
||||||
|
client.Unique(id: const Uuid().v7obj(), name: 'second-name'),
|
||||||
|
);
|
||||||
|
|
||||||
|
// Ensures the row and update are visible on all nodes.
|
||||||
|
await syncHttpClient.crdt.syncOnce(firstClientSession);
|
||||||
|
await syncHttpClient.crdt.syncOnce(secondClientSession);
|
||||||
|
|
||||||
|
// The `firstClientSession` holds the older update.
|
||||||
|
await client.Unique.db.updateRow(
|
||||||
|
firstClientSession,
|
||||||
|
olderUpdatedRow.copyWith(name: 'shared-name'),
|
||||||
|
);
|
||||||
|
|
||||||
|
// The `secondClientSession` holds the newer update.
|
||||||
|
await Future<void>.delayed(const Duration(milliseconds: 2));
|
||||||
|
await client.Unique.db.updateRow(
|
||||||
|
secondClientSession,
|
||||||
|
newerUpdatedRow.copyWith(name: 'shared-name'),
|
||||||
|
);
|
||||||
|
|
||||||
|
// The `secondClientSession` syncs first, so the newer update
|
||||||
|
// reaches the server first.
|
||||||
|
await syncHttpClient.crdt.syncOnce(secondClientSession);
|
||||||
|
});
|
||||||
|
|
||||||
|
group('when the incoming update synchronizes,', () {
|
||||||
|
setUp(() async {
|
||||||
|
await syncHttpClient.crdt.syncOnce(firstClientSession);
|
||||||
|
await syncHttpClient.crdt.syncOnce(secondClientSession);
|
||||||
|
});
|
||||||
|
|
||||||
|
test(
|
||||||
|
'then both rows remain visible and the row with the older unique update keeps the unique value.',
|
||||||
|
() async {
|
||||||
|
await _expectVisibleUniqueRowsOnAllNodes(
|
||||||
|
firstClientSession: firstClientSession,
|
||||||
|
secondClientSession: secondClientSession,
|
||||||
|
serverSession: serverSession,
|
||||||
|
originalValueId: olderUpdatedRow.id!,
|
||||||
|
conflictValueId: newerUpdatedRow.id!,
|
||||||
|
);
|
||||||
|
},
|
||||||
|
timeout: const Timeout(Duration(seconds: 60)),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
group('and an older incoming insert conflicts with a newer server update,', () {
|
||||||
|
late client.Unique insertedRow;
|
||||||
|
late client.Unique updatedRow;
|
||||||
|
|
||||||
|
setUp(() async {
|
||||||
|
updatedRow = await client.Unique.db.insertRow(
|
||||||
|
firstClientSession,
|
||||||
|
client.Unique(id: const Uuid().v7obj(), name: 'first-name'),
|
||||||
|
);
|
||||||
|
|
||||||
|
await Future<void>.delayed(const Duration(milliseconds: 2));
|
||||||
|
insertedRow = await client.Unique.db.insertRow(
|
||||||
|
secondClientSession,
|
||||||
|
client.Unique(id: const Uuid().v7obj(), name: 'shared-name'),
|
||||||
|
);
|
||||||
|
|
||||||
|
await Future<void>.delayed(const Duration(milliseconds: 2));
|
||||||
|
await client.Unique.db.updateRow(
|
||||||
|
firstClientSession,
|
||||||
|
updatedRow.copyWith(name: 'shared-name'),
|
||||||
|
);
|
||||||
|
|
||||||
|
await syncHttpClient.crdt.syncOnce(firstClientSession);
|
||||||
|
});
|
||||||
|
|
||||||
|
group('when the incoming insert synchronizes,', () {
|
||||||
|
setUp(() async {
|
||||||
|
await syncHttpClient.crdt.syncOnce(secondClientSession);
|
||||||
|
await syncHttpClient.crdt.syncOnce(firstClientSession);
|
||||||
|
});
|
||||||
|
|
||||||
|
test(
|
||||||
|
'then both rows remain visible and the row with the older unique value keeps it on every node.',
|
||||||
|
() async {
|
||||||
|
await _expectVisibleUniqueRowsOnAllNodes(
|
||||||
|
firstClientSession: firstClientSession,
|
||||||
|
secondClientSession: secondClientSession,
|
||||||
|
serverSession: serverSession,
|
||||||
|
originalValueId: insertedRow.id!,
|
||||||
|
conflictValueId: updatedRow.id!,
|
||||||
|
);
|
||||||
|
},
|
||||||
|
timeout: const Timeout(Duration(seconds: 60)),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
},
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
Future<void> _expectVisibleUniqueRowsOnAllNodes({
|
||||||
|
required CrdtDatabaseSession firstClientSession,
|
||||||
|
required CrdtDatabaseSession secondClientSession,
|
||||||
|
required CrdtDatabaseSession serverSession,
|
||||||
|
required UuidValue originalValueId,
|
||||||
|
required UuidValue conflictValueId,
|
||||||
|
}) async {
|
||||||
|
final firstClientRows = await client.Unique.db.find(firstClientSession);
|
||||||
|
expect(firstClientRows, hasLength(2));
|
||||||
|
expect(
|
||||||
|
firstClientRows.singleWhere((row) => row.id == originalValueId).name,
|
||||||
|
'shared-name',
|
||||||
|
);
|
||||||
|
expect(
|
||||||
|
firstClientRows.singleWhere((row) => row.id == conflictValueId).name,
|
||||||
|
'shared-name__conflict__${conflictValueId.uuid}',
|
||||||
|
);
|
||||||
|
|
||||||
|
final secondClientRows = await client.Unique.db.find(secondClientSession);
|
||||||
|
expect(secondClientRows, hasLength(2));
|
||||||
|
expect(
|
||||||
|
secondClientRows.singleWhere((row) => row.id == originalValueId).name,
|
||||||
|
'shared-name',
|
||||||
|
);
|
||||||
|
expect(
|
||||||
|
secondClientRows.singleWhere((row) => row.id == conflictValueId).name,
|
||||||
|
'shared-name__conflict__${conflictValueId.uuid}',
|
||||||
|
);
|
||||||
|
|
||||||
|
final serverRows = await server.Unique.db.find(serverSession);
|
||||||
|
expect(serverRows, hasLength(2));
|
||||||
|
expect(
|
||||||
|
serverRows.singleWhere((row) => row.id == originalValueId).name,
|
||||||
|
'shared-name',
|
||||||
|
);
|
||||||
|
expect(
|
||||||
|
serverRows.singleWhere((row) => row.id == conflictValueId).name,
|
||||||
|
'shared-name__conflict__${conflictValueId.uuid}',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
class TestClientAuthKeyProvider implements ClientAuthKeyProvider {
|
||||||
|
@override
|
||||||
|
Future<String?> get authHeaderValue async => 'Bearer token';
|
||||||
|
}
|
||||||
|
|
@ -86,6 +86,7 @@ Future<void> _initialize() async {
|
||||||
Person.t,
|
Person.t,
|
||||||
Town.t,
|
Town.t,
|
||||||
Unique.t,
|
Unique.t,
|
||||||
|
UniqueComposite.t,
|
||||||
UniqueUuid.t,
|
UniqueUuid.t,
|
||||||
],
|
],
|
||||||
persistentUserId: _withPersistentUser ? _testCrdtUserId : null,
|
persistentUserId: _withPersistentUser ? _testCrdtUserId : null,
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue