fix(deploy): the server starts once more after a deploy with a migration

After v0.13.0's deploy every request to learn.halkyon.eu took a minute or
two and syncs failed, until the server was restarted by hand. The
migration changed a synced table, so the new server, as it first used
the database, found the sync engine's registry of synced tables changed,
and then rebuilt every space's projections in every session until it
stopped (serverpod_offline_sync issue #150). The maintenance run that
applies the migrations doesn't spare it, as AGENTS.md wrongly said.

redeploy.sh now restarts the server once it answers whenever the deploy
brought a migration, and waits for it to answer again: started a second
time, it finds the registry as the first start left it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
Tizian 「ティツィアーン」 2026-10-13 22:45:57 +02:00
commit 9b5ea88a8a
Signed by: raisondetredev
GPG key ID: 07F6FCF405488B64
2 changed files with 15 additions and 2 deletions

View file

@ -36,7 +36,7 @@ Tests need no Docker. `config/test.yaml` sets `database.dataPath`, so Serverpod
Tooling traps: rules of `leancode_lint`'s analyzer plugin are ignored with its prefix (`// ignore: leancode_lint/missing_equatable_props`); `dart fix --apply` moves `flutter_driver` into the app's `dependencies` (as `any`) on every run, so restore it as a dev dependency (`sdk: flutter`) afterwards; never `dart format` generated code, only the hand-written paths, as `tool/check.sh` does (the server's `test/integration/test_tools/` is generated too, so format test files one by one, never the whole `test/` folder); and never rebase or check out other commits while `serverpod start` runs: its watcher regenerates code for each intermediate state, leaves stale generated files behind and hot-reloads them. Stop it first, or work in a separate worktree. Tooling traps: rules of `leancode_lint`'s analyzer plugin are ignored with its prefix (`// ignore: leancode_lint/missing_equatable_props`); `dart fix --apply` moves `flutter_driver` into the app's `dependencies` (as `any`) on every run, so restore it as a dev dependency (`sdk: flutter`) afterwards; never `dart format` generated code, only the hand-written paths, as `tool/check.sh` does (the server's `test/integration/test_tools/` is generated too, so format test files one by one, never the whole `test/` folder); and never rebase or check out other commits while `serverpod start` runs: its watcher regenerates code for each intermediate state, leaves stale generated files behind and hot-reloads them. Stop it first, or work in a separate worktree.
After a migration that adds or changes synced tables, restart `serverpod start` once it has applied it. The sync engine (checked with `serverpod_offline_sync` v0.0.8) reconciles its registry of synced tables when a server first uses the database, and if that changed the registry, every later session of that server rebuilds the projections of every space, one transaction each, until the server stops ([serverpod_offline_sync#150](https://github.com/marcelomendoncasoares/serverpod_offline_sync/issues/150), open). With the many people the tests leave in the development database, syncs then take seconds and the sync tests time out. The restarted server finds the registry as it left it. Deploys aren't affected: their maintenance run applies the migrations and reconciles before it exits (it reads the runtime settings through the engine's wrapper), so the server started after it finds nothing changed. Drop this note once #150 is fixed. After a migration that adds or changes synced tables, restart `serverpod start` once it has applied it. The sync engine (checked with `serverpod_offline_sync` v0.0.8) reconciles its registry of synced tables when a server first uses the database, and if that changed the registry, every later session of that server rebuilds the projections of every space, one transaction each, until the server stops ([serverpod_offline_sync#150](https://github.com/marcelomendoncasoares/serverpod_offline_sync/issues/150), open). With the many people the tests leave in the development database, syncs then take seconds and the sync tests time out. The restarted server finds the registry as it left it. Deploys are affected too (found 2026-10-13: after v0.13.0's deploy every request took a minute or two and syncs failed, as the new server, not the maintenance run, reconciled the registry), so `deploy/redeploy.sh` starts the server once more after a deploy that brought a migration. Drop this note, and that restart, once #150 is fixed.
Checklist after doing changes, in this order: Checklist after doing changes, in this order:

View file

@ -10,7 +10,8 @@
# - if every new commit is signed by a key in deploy/allowed-signers, as the deployed # - if every new commit is signed by a key in deploy/allowed-signers, as the deployed
# checkout lists them (so no commit can add its own key), # checkout lists them (so no commit can add its own key),
# dumps the database first (the last 3 dumps stay in ~/backups), applies the migrations # dumps the database first (the last 3 dumps stay in ~/backups), applies the migrations
# before switching to the new server, and fails unless that server answers afterwards. A failure leaves the checkout at the deployed commit, so # before switching to the new server, starts it once more after a migration (see below),
# and fails unless that server answers afterwards. A failure leaves the checkout at the deployed commit, so
# running the CI job again retries. By hand, with your own key: # running the CI job again retries. By hand, with your own key:
# #
# ssh learn-deploy halkyon-learn/deploy/redeploy.sh <commit> # ssh learn-deploy halkyon-learn/deploy/redeploy.sh <commit>
@ -77,6 +78,18 @@ main() {
docker compose up -d --force-recreate --no-deps caddy docker compose up -d --force-recreate --no-deps caddy
fi fi
wait_until_answering wait_until_answering
# After a migration the new server is started once more. The sync engine
# (serverpod_offline_sync v0.0.10) reconciles its registry of synced tables as a
# server first uses the database; when a migration changed a synced table, that
# changes the registry, and the server then rebuilds every space's projections in
# every session until it stops (issue #150): after v0.13.0's deploy every request
# took a minute or two, and syncs failed, until the server was restarted
# (2026-10-13). The maintenance run doesn't spare it. Started again, the server
# finds the registry as the first start left it.
if ! git diff --quiet "$deployed" "$commit" -- ../halkyon_learn_server/migrations; then
docker compose restart server
wait_until_answering
fi
trap - EXIT trap - EXIT
docker image prune -f >/dev/null docker image prune -f >/dev/null
echo "Deployed $commit." echo "Deployed $commit."