fix(server): forget the IP addresses of sign-up attempts

Serverpod's rate limit for new anonymous accounts (30 per IP address and
hour) notes every attempt with the address it came from, and deletes
those notes only for email sign-in, which this instance doesn't offer. So
the IP address of everyone who ever started the app stayed in the
database, and in every backup, for good.

The server now deletes them hourly once the limit's hour is past, so an
address is kept for at most two hours. The rows already there go at the
first run after the deploy; backups keep them until they expire (about
three months).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMm7usAo6q9ZZRoPkWjoxi
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
Tizian 「ティツィアーン」 2026-10-10 05:37:00 +02:00
commit 0d03ef7973
Signed by: raisondetredev
GPG key ID: 07F6FCF405488B64
8 changed files with 131 additions and 9 deletions

View file

@ -161,7 +161,7 @@ A spaced-repetition learning app: studysets of flashcards, reviewed on an FSRS s
- **Explore and copies** (ADR 0005, stage 3, decided 2026-10-09): `ExploreEndpoint` (`requireLogin`: anonymous people explore and learn too) lists the latest release of each published studyset not withdrawn **by fit, never by popularity** (the person's roadmap): search over titles and authors (LIKE's wildcards in what is typed are only themselves), languages (a primary one takes its regional ones), kinds of content held, the newest release first, 30 to a page; the next page only when asked for (Show more, no endless scrolling). What Explore shows of a release is kept beside its repository (`studyset_release`'s title, description, languages, credits), so listing never reads git. `release` gives out the latest release's JSON (one `git cat-file --batch` run) and `file` each file by its hash. **The app makes the copy** (`ReleaseCopier`, `feature/explore/`), as an import: through the person's own repositories, the files among their own files (asset ids in the content swapped for theirs), the tags by name among their own tags, the studysets in their tree, each list in its order; everything is read first, so a release with an item in a schema this app doesn't know is refused whole (`ReleaseUnreadable`). The server never writes into a learner's space. Every item of a copy names where it came from (`published`: publication, item, release; `core/content/published_ref.dart`; card schema 9, note 3, step 3, path 2, PDF 2, studyset 5, the top studyset also with `location`), merged per side like the rest of a card, note or step, so a later release can be taken from where the copy last took one. **A later release is offered, never applied by itself** (`CopyUpdater`, `CopyUpdateBanner` on the copy's top studyset: "v4 is out: 12 changed, 5 new, 2 withdrawn", Update): the app compares it with the release the copy last took (`release(version:)`; releases never change, so the repository caches them) item by item. Cards, notes and steps take the release's content as a revision on the one where they last took a release (`takeRelease`, as `takeDictionaryUpdate`), merging per side: what only the release changed comes in, what the learner changed stays, a side both changed is a conflict settled in the editor. Studysets, paths and PDFs, kept whole, take each field of their payload the learner left as the old release had it; one they changed stays theirs (personal fields, such as a PDF's deadline, too). New items come in where they belong, unless the learner deleted that part; tags the release added come and those it took away go, the learner's own stay; files are fetched only when the person hasn't the same file already (`AssetRepository.idOf`). What the author withdrew stays the learner's, studied on, marked (`withdrawn` in its origin; the card browser's filter Withdrawn by the author); what the learner deleted stays deleted; progress is never touched. A copy of a studyset its author withdrew is offered nothing. **"Learn this here"** (ADR 0005, rule 8): `LearnerCountFutureCall`, daily, counts per published studyset the people of this instance with the `identified` scope who chose to be counted (`countLearning` in their settings, settings schema 8, off unless switched on in Settings › Privacy) and studied a copy of it (a studyset whose payload names it) on 3 or more days of the last 30, fresh starts left out; payloads are decoded in Dart, so one that isn't JSON counts for nothing rather than failing the count. Only `learnersShown` (rounded down: 5, 10, 20, 50, …; none below 5) ever leaves the server, in Explore and the author's Publish sheet. **A copy is published within its licence** (rule 10): the server looks up the original's (`PublishRefusal.forkForbidden` under "no derivatives"; `licenseMustStay` under a licence that keeps adaptations under it, `keepsAdaptations`, in every studyset of the copy), and the release keeps what it is based on (`basedOn`, shown in Explore); a person publishing a copy of their own is its author, and the copy of another instance's studyset goes with its location alone. The server never updates synced rows itself: the sync engine (v0.0.10) stores a ByteData the server updates through it as its JSON text, not its bytes (`_updateRowWithoutRecording`), unreported upstream so far. In the app, Explore is the old app's feed: cards with title, author, the start of the description, the latest release, what it holds, the licence and Add to library (its progress, then In your library, opening the copy); a summary sheet with all of it. - **Explore and copies** (ADR 0005, stage 3, decided 2026-10-09): `ExploreEndpoint` (`requireLogin`: anonymous people explore and learn too) lists the latest release of each published studyset not withdrawn **by fit, never by popularity** (the person's roadmap): search over titles and authors (LIKE's wildcards in what is typed are only themselves), languages (a primary one takes its regional ones), kinds of content held, the newest release first, 30 to a page; the next page only when asked for (Show more, no endless scrolling). What Explore shows of a release is kept beside its repository (`studyset_release`'s title, description, languages, credits), so listing never reads git. `release` gives out the latest release's JSON (one `git cat-file --batch` run) and `file` each file by its hash. **The app makes the copy** (`ReleaseCopier`, `feature/explore/`), as an import: through the person's own repositories, the files among their own files (asset ids in the content swapped for theirs), the tags by name among their own tags, the studysets in their tree, each list in its order; everything is read first, so a release with an item in a schema this app doesn't know is refused whole (`ReleaseUnreadable`). The server never writes into a learner's space. Every item of a copy names where it came from (`published`: publication, item, release; `core/content/published_ref.dart`; card schema 9, note 3, step 3, path 2, PDF 2, studyset 5, the top studyset also with `location`), merged per side like the rest of a card, note or step, so a later release can be taken from where the copy last took one. **A later release is offered, never applied by itself** (`CopyUpdater`, `CopyUpdateBanner` on the copy's top studyset: "v4 is out: 12 changed, 5 new, 2 withdrawn", Update): the app compares it with the release the copy last took (`release(version:)`; releases never change, so the repository caches them) item by item. Cards, notes and steps take the release's content as a revision on the one where they last took a release (`takeRelease`, as `takeDictionaryUpdate`), merging per side: what only the release changed comes in, what the learner changed stays, a side both changed is a conflict settled in the editor. Studysets, paths and PDFs, kept whole, take each field of their payload the learner left as the old release had it; one they changed stays theirs (personal fields, such as a PDF's deadline, too). New items come in where they belong, unless the learner deleted that part; tags the release added come and those it took away go, the learner's own stay; files are fetched only when the person hasn't the same file already (`AssetRepository.idOf`). What the author withdrew stays the learner's, studied on, marked (`withdrawn` in its origin; the card browser's filter Withdrawn by the author); what the learner deleted stays deleted; progress is never touched. A copy of a studyset its author withdrew is offered nothing. **"Learn this here"** (ADR 0005, rule 8): `LearnerCountFutureCall`, daily, counts per published studyset the people of this instance with the `identified` scope who chose to be counted (`countLearning` in their settings, settings schema 8, off unless switched on in Settings › Privacy) and studied a copy of it (a studyset whose payload names it) on 3 or more days of the last 30, fresh starts left out; payloads are decoded in Dart, so one that isn't JSON counts for nothing rather than failing the count. Only `learnersShown` (rounded down: 5, 10, 20, 50, …; none below 5) ever leaves the server, in Explore and the author's Publish sheet. **A copy is published within its licence** (rule 10): the server looks up the original's (`PublishRefusal.forkForbidden` under "no derivatives"; `licenseMustStay` under a licence that keeps adaptations under it, `keepsAdaptations`, in every studyset of the copy), and the release keeps what it is based on (`basedOn`, shown in Explore); a person publishing a copy of their own is its author, and the copy of another instance's studyset goes with its location alone. The server never updates synced rows itself: the sync engine (v0.0.10) stores a ByteData the server updates through it as its JSON text, not its bytes (`_updateRowWithoutRecording`), unreported upstream so far. In the app, Explore is the old app's feed: cards with title, author, the start of the description, the latest release, what it holds, the licence and Add to library (its progress, then In your library, opening the copy); a summary sheet with all of it.
- **Reports and suggestions** (ADR 0005, stage 4, rule 11): `feedback_thread` (a report or suggestion: `publicationId` and `kind` columns, everything else in its payload: what it is about by release item id and version, a report's category, the text, a suggestion's proposed payload and the files it names), `feedback_message` (replies) and `feedback_decision` (an author's closed, reopened, hidden, shown, accepted, declined; the latest of each pair stands) are synced tables each person writes in their own space; a thread or reply names its thread by id only, as it may be another person's. `gatherFeedback` (`lib/src/feedback/`) assembles them for `FeedbackEndpoint.threadsOf` (`requireLogin`: public) and `InboxEndpoint` (`identified`: the inbox, unseen counts, `markSeen`, `block`, `file`) and decides what counts as it reads (see the ADR), since a modified app could write anything. `feedback_block` and `feedback_seen` are server-only; `deletePerson` deletes both. A release's commit credits the writers of suggestions accepted since the last (`Co-authored-by`). In the app (`feature/feedback/`): the card's Report sheet offers "Send to the author too" on a card from a release; a copy's ⋮ menu reports the studyset and lists its threads, as Explore's summary does; the editors of cards (⋮), notes and steps (app bar) suggest to the author (refused under "no derivatives"); the Inbox is in the Profile tab, with a count on the tab (`InboxCountBloc`, refreshed at start and whenever the inbox is read or acted on; no timer); a thread's sheet shows a suggestion beside the author's own item, its replies, and the author's decisions. Accepting writes the proposed content as the author's own revision (`FeedbackRepository.accept`), the writer's files fetched by hash. - **Reports and suggestions** (ADR 0005, stage 4, rule 11): `feedback_thread` (a report or suggestion: `publicationId` and `kind` columns, everything else in its payload: what it is about by release item id and version, a report's category, the text, a suggestion's proposed payload and the files it names), `feedback_message` (replies) and `feedback_decision` (an author's closed, reopened, hidden, shown, accepted, declined; the latest of each pair stands) are synced tables each person writes in their own space; a thread or reply names its thread by id only, as it may be another person's. `gatherFeedback` (`lib/src/feedback/`) assembles them for `FeedbackEndpoint.threadsOf` (`requireLogin`: public) and `InboxEndpoint` (`identified`: the inbox, unseen counts, `markSeen`, `block`, `file`) and decides what counts as it reads (see the ADR), since a modified app could write anything. `feedback_block` and `feedback_seen` are server-only; `deletePerson` deletes both. A release's commit credits the writers of suggestions accepted since the last (`Co-authored-by`). In the app (`feature/feedback/`): the card's Report sheet offers "Send to the author too" on a card from a release; a copy's ⋮ menu reports the studyset and lists its threads, as Explore's summary does; the editors of cards (⋮), notes and steps (app bar) suggest to the author (refused under "no derivatives"); the Inbox is in the Profile tab, with a count on the tab (`InboxCountBloc`, refreshed at start and whenever the inbox is read or acted on; no timer); a thread's sheet shows a suggestion beside the author's own item, its replies, and the author's decisions. Accepting writes the proposed content as the author's own revision (`FeedbackRepository.accept`), the writer's files fetched by hash.
- **Deleting a person** goes through `deletePerson` (`lib/src/auth/account.dart`), in one transaction and in ADR 0001 rule 6's order: their sessions end first (a device with a valid session would re-upload everything), then their space goes with all its content through the foreign keys, then the person with their identities. `AccountEndpoint.delete` deletes the caller; the app also calls it when an anonymous account is given up (signed out of, or switched away from to GitHub), since nobody could ever sign into it again. - **Deleting a person** goes through `deletePerson` (`lib/src/auth/account.dart`), in one transaction and in ADR 0001 rule 6's order: their sessions end first (a device with a valid session would re-upload everything), then their space goes with all its content through the foreign keys, then the person with their identities. `AccountEndpoint.delete` deletes the caller; the app also calls it when an anonymous account is given up (signed out of, or switched away from to GitHub), since nobody could ever sign into it again.
- **Pruning**: `PruningFutureCall` deletes anonymous people unused for 90 days, daily, scheduled when the server starts. Scheduling a future call always adds an entry, so the server cancels the previous one by its identifier first. - **Pruning**: `PruningFutureCall` deletes anonymous people unused for 90 days, daily, scheduled when the server starts; and hourly the notes Serverpod's rate limit makes of anonymous sign-ups (`serverpod_auth_idp_rate_limited_request_attempt`, each with the IP address it came from) once the limit's hour is past (`forgetOldSignUpAttempts`): Serverpod deletes them only for email sign-in, so they would otherwise stay for good, and the privacy page says the instance keeps no IP addresses beyond that. Scheduling a future call always adds an entry, so the server cancels the previous one by its identifier first.
- **One person's data per device**: `PersonFolders` wipes every other person's folder when a person's data opens: their sessions ended on this device. Opening waits for the previous person's database to close. - **One person's data per device**: `PersonFolders` wipes every other person's folder when a person's data opens: their sessions ended on this device. Opening waits for the previous person's database to close.
- **Session logs are not persisted** (`persistentEnabled: false` in every run mode) for privacy. Don't re-enable them. - **Session logs are not persisted** (`persistentEnabled: false` in every run mode) for privacy. Don't re-enable them.
- **Secrets** live in `halkyon_learn_server/config/passwords.yaml` (git-ignored). Never change `serverSideSessionKeyHashPepper`: it would sign everyone out, and anonymous accounts can't sign back in. - **Secrets** live in `halkyon_learn_server/config/passwords.yaml` (git-ignored). Never change `serverSideSessionKeyHashPepper`: it would sign everyone out, and anonymous accounts can't sign back in.

View file

@ -34,6 +34,15 @@ Future<void> run(List<String> args) async {
.pruning .pruning
.pruneAnonymousPeople(); .pruneAnonymousPeople();
// The IP addresses of sign-up attempts, hourly, once the rate limit is
// done with them (decided 2026-10-10).
await pod.futureCalls.cancel(_attemptPruning);
await pod.futureCalls
.callRecurring(identifier: _attemptPruning)
.cron('27 * * * *')
.pruning
.forgetSignUpAttempts();
// So are the files nobody can reach any more (ADR 0004, rule 5). // So are the files nobody can reach any more (ADR 0004, rule 5).
await pod.futureCalls.cancel(_filePruning); await pod.futureCalls.cancel(_filePruning);
await pod.futureCalls await pod.futureCalls
@ -79,6 +88,9 @@ Future<void> _loadDictionaries(Serverpod pod) async {
/// The identifier of the daily pruning. /// The identifier of the daily pruning.
const _pruning = 'prune-abandoned-anonymous-people'; const _pruning = 'prune-abandoned-anonymous-people';
/// The identifier of the hourly forgetting of sign-up attempts.
const _attemptPruning = 'forget-sign-up-attempts';
/// The identifier of the daily pruning of files. /// The identifier of the daily pruning of files.
const _filePruning = 'prune-unreferenced-files'; const _filePruning = 'prune-unreferenced-files';

View file

@ -1,4 +1,5 @@
import 'package:halkyon_learn_server/src/asset/asset_endpoint.dart'; import 'package:halkyon_learn_server/src/asset/asset_endpoint.dart';
import 'package:halkyon_learn_server/src/auth/auth_config.dart';
import 'package:halkyon_learn_server/src/generated/protocol.dart'; import 'package:halkyon_learn_server/src/generated/protocol.dart';
import 'package:halkyon_learn_server/src/profile/profile_rules.dart'; import 'package:halkyon_learn_server/src/profile/profile_rules.dart';
import 'package:halkyon_learn_server/src/publish/release_repository.dart'; import 'package:halkyon_learn_server/src/publish/release_repository.dart';
@ -131,3 +132,22 @@ Future<int> pruneAbandonedAnonymousPeople(
} }
return abandoned.length; return abandoned.length;
} }
/// Deletes Serverpod's notes of sign-up attempts that are older than the
/// rate limit looks back ([anonymousSignUpLimit]) at [now], and returns how
/// many: each holds the IP address it came from, which the instance keeps no
/// longer than the limit needs it (decided 2026-10-10). Serverpod itself
/// deletes them only for email sign-in, which this instance doesn't offer.
Future<int> forgetOldSignUpAttempts(
Session session, {
required DateTime now,
}) async {
final cutoff = now.toUtc().subtract(anonymousSignUpLimit.timeframe);
// DELETE FROM "serverpod_auth_idp_rate_limited_request_attempt"
// WHERE "attemptedAt" < $1
final forgotten = await RateLimitedRequestAttempt.db.deleteWhere(
session,
where: (t) => t.attemptedAt < cutoff,
);
return forgotten.length;
}

View file

@ -16,6 +16,14 @@ final List<TokenManagerBuilder> tokenManagerBuilders = [
), ),
]; ];
/// How many new anonymous accounts one IP address may make, and in what
/// time. Serverpod notes each attempt with the address it came from, which
/// the instance forgets once the time is up (`forgetOldSignUpAttempts`).
const anonymousSignUpLimit = RateLimit(
maxAttempts: 30,
timeframe: Duration(hours: 1),
);
/// The ways people sign in on [pod]. /// The ways people sign in on [pod].
/// ///
/// Anonymous accounts are the trial mode for solo use. At most 30 new ones per /// Anonymous accounts are the trial mode for solo use. At most 30 new ones per
@ -26,11 +34,6 @@ final List<TokenManagerBuilder> tokenManagerBuilders = [
/// [gitHubSignInAvailable]); its helpers serve our own slim login /// [gitHubSignInAvailable]); its helpers serve our own slim login
/// (`github.dart`), and Serverpod's stock GitHub login endpoint isn't exposed. /// (`github.dart`), and Serverpod's stock GitHub login endpoint isn't exposed.
List<IdentityProviderBuilder> identityProviderBuilders(Serverpod pod) => [ List<IdentityProviderBuilder> identityProviderBuilders(Serverpod pod) => [
const AnonymousIdpConfig( const AnonymousIdpConfig(perIpAddressRateLimit: anonymousSignUpLimit),
perIpAddressRateLimit: RateLimit(
maxAttempts: 30,
timeframe: Duration(hours: 1),
),
),
if (gitHubSignInAvailable(pod)) GitHubIdpConfigFromPasswords(), if (gitHubSignInAvailable(pod)) GitHubIdpConfigFromPasswords(),
]; ];

View file

@ -2,8 +2,9 @@ import 'package:clock/clock.dart';
import 'package:halkyon_learn_server/src/auth/account.dart'; import 'package:halkyon_learn_server/src/auth/account.dart';
import 'package:serverpod/serverpod.dart'; import 'package:serverpod/serverpod.dart';
/// Deletes the anonymous people nobody can return to; the server schedules it /// Deletes the anonymous people nobody can return to, daily, and the IP
/// daily when it starts (`server.dart`). /// addresses of sign-up attempts the rate limit is done with, hourly; the
/// server schedules both when it starts (`server.dart`).
class PruningFutureCall extends FutureCall { class PruningFutureCall extends FutureCall {
/// Prunes one batch and logs how many people went. /// Prunes one batch and logs how many people went.
Future<void> pruneAnonymousPeople(Session session) async { Future<void> pruneAnonymousPeople(Session session) async {
@ -13,4 +14,11 @@ class PruningFutureCall extends FutureCall {
); );
session.log('Pruned $pruned abandoned anonymous people.'); session.log('Pruned $pruned abandoned anonymous people.');
} }
/// Forgets the sign-up attempts the rate limit is done with, with their IP
/// addresses, and logs how many.
Future<void> forgetSignUpAttempts(Session session) async {
final forgotten = await forgetOldSignUpAttempts(session, now: clock.now());
session.log('Forgot $forgotten sign-up attempts.');
}
} }

View file

@ -64,6 +64,8 @@ class FutureCalls extends _is.FutureCallDispatch<_FutureCallRef> {
'FilePruningPruneFilesFutureCall': FilePruningPruneFilesFutureCall(), 'FilePruningPruneFilesFutureCall': FilePruningPruneFilesFutureCall(),
'PruningPruneAnonymousPeopleFutureCall': 'PruningPruneAnonymousPeopleFutureCall':
PruningPruneAnonymousPeopleFutureCall(), PruningPruneAnonymousPeopleFutureCall(),
'PruningForgetSignUpAttemptsFutureCall':
PruningForgetSignUpAttemptsFutureCall(),
'DictionaryReloadReloadDictionariesFutureCall': 'DictionaryReloadReloadDictionariesFutureCall':
DictionaryReloadReloadDictionariesFutureCall(), DictionaryReloadReloadDictionariesFutureCall(),
'LearnerCountCountLearnersFutureCall': 'LearnerCountCountLearnersFutureCall':
@ -225,6 +227,13 @@ class _PruningFutureCallDispatcher {
null, null,
); );
} }
Future<void> forgetSignUpAttempts() {
return _invokeFutureCall(
'PruningForgetSignUpAttemptsFutureCall',
null,
);
}
} }
class _DictionaryReloadFutureCallDispatcher { class _DictionaryReloadFutureCallDispatcher {
@ -277,6 +286,19 @@ class PruningPruneAnonymousPeopleFutureCall extends _is.FutureCall
} }
} }
/// Forgets the sign-up attempts the rate limit is done with, with their IP
/// addresses, and logs how many.
class PruningForgetSignUpAttemptsFutureCall extends _is.FutureCall
implements _is.InvokableFutureCall {
@override
_ida.Future<void> invoke(
_is.Session session,
_is.SerializableModel? object,
) async {
await _igevg04y.PruningFutureCall().forgetSignUpAttempts(session);
}
}
/// Loads the dictionary files that changed since they were loaded. /// Loads the dictionary files that changed since they were loaded.
class DictionaryReloadReloadDictionariesFutureCall extends _is.FutureCall class DictionaryReloadReloadDictionariesFutureCall extends _is.FutureCall
implements _is.InvokableFutureCall { implements _is.InvokableFutureCall {

View file

@ -84,6 +84,48 @@ void main() {
}); });
}); });
withServerpod('Forgetting sign-up attempts', (sessionBuilder, _) {
late Session session;
final now = DateTime.now().toUtc();
setUp(() => session = sessionBuilder.build());
test('deletes those the rate limit is done with, with their IP '
'addresses, and keeps the rest', () async {
Future<UuidValue> attempt(Duration ago) async =>
(await RateLimitedRequestAttempt.db.insertRow(
session,
RateLimitedRequestAttempt(
domain: 'anonymous',
source: 'account_creation',
key: '203.0.113.7',
ipAddress: '203.0.113.7',
attemptedAt: now.subtract(ago),
),
)).id!;
final old = await attempt(const Duration(days: 3));
final done = await attempt(const Duration(minutes: 61));
final recent = await attempt(const Duration(minutes: 59));
// Others may have been left by earlier runs.
expect(
await forgetOldSignUpAttempts(session, now: now),
greaterThanOrEqualTo(2),
);
for (final forgotten in [old, done]) {
expect(
await RateLimitedRequestAttempt.db.findById(session, forgotten),
isNull,
);
}
expect(
await RateLimitedRequestAttempt.db.findById(session, recent),
isNotNull,
);
});
});
withServerpod('Pruning abandoned anonymous people', (sessionBuilder, _) { withServerpod('Pruning abandoned anonymous people', (sessionBuilder, _) {
late Session session; late Session session;
final now = DateTime.now().toUtc(); final now = DateTime.now().toUtc();

View file

@ -1700,6 +1700,21 @@ class _PruningFutureCall {
await _localUniqueSession.close(); await _localUniqueSession.close();
} }
} }
Future<void> forgetSignUpAttempts(
_ist.TestSessionBuilder sessionBuilder,
) async {
var _localUniqueSession =
(sessionBuilder as _ist.InternalTestSessionBuilder).internalBuild();
try {
await _iip9210p.PruningForgetSignUpAttemptsFutureCall().invoke(
_localUniqueSession,
null,
);
} finally {
await _localUniqueSession.close();
}
}
} }
class _DictionaryReloadFutureCall { class _DictionaryReloadFutureCall {