perf(export): zip an export with files on an isolate of its own

A studyset with many PDFs took a while to pack, and the window froze
meanwhile. The zip is now made on another isolate (ExportArchiver, an
isolate in the app and in place in widget tests, whose fake clock can't
wait for one), as Anki packages are read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMm7usAo6q9ZZRoPkWjoxi
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
Tizian 「ティツィアーン」 2026-10-10 02:00:30 +02:00
commit e04be6603d
Signed by: raisondetredev
GPG key ID: 07F6FCF405488B64
6 changed files with 105 additions and 20 deletions

View file

@ -85,7 +85,7 @@ A spaced-repetition learning app: studysets of flashcards, reviewed on an FSRS s
- **App-wide vs. per person.** `Dependencies` holds what outlives a person (the client, auth). Everything of the signed-in person (their local database, its sync, their repositories) is `PersonDependencies`, opened by `PersonScope` inside `SessionGate`'s keyed subtree (`Dependencies.openPerson`) and disposed with it; widgets reach it through `PersonScope.of(context)`. - **App-wide vs. per person.** `Dependencies` holds what outlives a person (the client, auth). Everything of the signed-in person (their local database, its sync, their repositories) is `PersonDependencies`, opened by `PersonScope` inside `SessionGate`'s keyed subtree (`Dependencies.openPerson`) and disposed with it; widgets reach it through `PersonScope.of(context)`.
- **The device's own settings** (appearance, language, citation style; `core/device_settings/`) belong to the device, not to a person: kept in `<app support>/device_settings.json`, read in `Dependencies.create()` so the first frame already has them, and provided by `DeviceSettingsScope` above `SessionGate`'s keyed subtree, which both its plain app and `PersonApp` read. What should follow the person to their other devices goes into their synced `PersonSettings` instead. - **The device's own settings** (appearance, language, citation style; `core/device_settings/`) belong to the device, not to a person: kept in `<app support>/device_settings.json`, read in `Dependencies.create()` so the first frame already has them, and provided by `DeviceSettingsScope` above `SessionGate`'s keyed subtree, which both its plain app and `PersonApp` read. What should follow the person to their other devices goes into their synced `PersonSettings` instead.
- **Every person has an app of their own** (`PersonApp`, `lib/src/app/`), which `SessionGate` builds inside the keyed subtree: a `MaterialApp.router` with its own `GoRouter`, and `PersonScope` around the router's navigator (`MaterialApp.builder`). So every screen and dialog is inside the person's scope and reads their repositories from `PersonScope.of(context)` wherever it is built, and a change of person disposes the router with every screen on it and starts again at home. Until someone is signed in, `SessionGate` shows a plain `MaterialApp`: the spinner, or the failure with a way to try again. - **Every person has an app of their own** (`PersonApp`, `lib/src/app/`), which `SessionGate` builds inside the keyed subtree: a `MaterialApp.router` with its own `GoRouter`, and `PersonScope` around the router's navigator (`MaterialApp.builder`). So every screen and dialog is inside the person's scope and reads their repositories from `PersonScope.of(context)` wherever it is built, and a change of person disposes the router with every screen on it and starts again at home. Until someone is signed in, `SessionGate` shows a plain `MaterialApp`: the spinner, or the failure with a way to try again.
- **Platform plugins sit behind small interfaces in `Dependencies`**, with fakes in tests: picking files (`FilePicker`, `file_selector`), saving files the app made, such as exports (`FileSaver`: `file_selector`'s dialog on the desktop, `file_picker`'s on Android and iOS, where `file_selector` has none), playing sounds (`SoundPlayer`, `audioplayers`), playing YouTube videos (`YoutubeVideos`, `youtube_player_iframe` where a web view exists, else the browser), reading PDFs (`PdfEngine`, pdfrx on PDFium, which `flutter test` can't load: widget tests use `FakePdfEngine`, whose PDF bytes say their pages and chapters), reading Anki packages on another isolate (`AnkiPackageReader`; a widget test's fake clock can't wait for one), and the Pomodoro timer's side of the system (`PomodoroPlatform`, `FakePomodoroPlatform` in tests). - **Platform plugins sit behind small interfaces in `Dependencies`**, with fakes in tests: picking files (`FilePicker`, `file_selector`), saving files the app made, such as exports (`FileSaver`: `file_selector`'s dialog on the desktop, `file_picker`'s on Android and iOS, where `file_selector` has none), playing sounds (`SoundPlayer`, `audioplayers`), playing YouTube videos (`YoutubeVideos`, `youtube_player_iframe` where a web view exists, else the browser), reading PDFs (`PdfEngine`, pdfrx on PDFium, which `flutter test` can't load: widget tests use `FakePdfEngine`, whose PDF bytes say their pages and chapters), reading Anki packages on another isolate (`AnkiPackageReader`; a widget test's fake clock can't wait for one), zipping an export with its files on another isolate (`ExportArchiver`, for the same reason), and the Pomodoro timer's side of the system (`PomodoroPlatform`, `FakePomodoroPlatform` in tests).
- **Material's `Card` widget clashes with the domain's `Card`**: files that need both import `material_ui` with `hide Card`. - **Material's `Card` widget clashes with the domain's `Card`**: files that need both import `material_ui` with `hide Card`.
- **State: `flutter_bloc`, uniformly.** Blocs fold repository streams into immutable states with value equality, are provided as close to their consumer as possible, and log through one global `BlocObserver`. Why: the data layer is stream-shaped, event → state machines test without widgets, and one pattern everywhere is itself the point. - **State: `flutter_bloc`, uniformly.** Blocs fold repository streams into immutable states with value equality, are provided as close to their consumer as possible, and log through one global `BlocObserver`. Why: the data layer is stream-shaped, event → state machines test without widgets, and one pattern everywhere is itself the point.
- **Value equality comes from `equatable`**, mixed in (`with Equatable`), for models and states alike; `leancode_lint`'s `missing_equatable_props` catches a field left out of `props`. - **Value equality comes from `equatable`**, mixed in (`with Equatable`), for models and states alike; `leancode_lint`'s `missing_equatable_props` catches a field left out of `props`.

View file

@ -33,6 +33,7 @@ import 'package:halkyon_learn_flutter/src/feature/explore/data/copy_updater.dart
import 'package:halkyon_learn_flutter/src/feature/explore/data/explore_repository.dart'; import 'package:halkyon_learn_flutter/src/feature/explore/data/explore_repository.dart';
import 'package:halkyon_learn_flutter/src/feature/explore/data/release_copier.dart'; import 'package:halkyon_learn_flutter/src/feature/explore/data/release_copier.dart';
import 'package:halkyon_learn_flutter/src/feature/explore/data/serverpod_explore_gateway.dart'; import 'package:halkyon_learn_flutter/src/feature/explore/data/serverpod_explore_gateway.dart';
import 'package:halkyon_learn_flutter/src/feature/export/data/export_archive.dart';
import 'package:halkyon_learn_flutter/src/feature/feedback/data/feedback_repository.dart'; import 'package:halkyon_learn_flutter/src/feature/feedback/data/feedback_repository.dart';
import 'package:halkyon_learn_flutter/src/feature/feedback/data/serverpod_feedback_gateway.dart'; import 'package:halkyon_learn_flutter/src/feature/feedback/data/serverpod_feedback_gateway.dart';
import 'package:halkyon_learn_flutter/src/feature/feedback/data/sync_feedback_gateway.dart'; import 'package:halkyon_learn_flutter/src/feature/feedback/data/sync_feedback_gateway.dart';
@ -122,6 +123,9 @@ class Dependencies({
/// Reads Anki packages to import. /// Reads Anki packages to import.
required final AnkiPackageReader ankiPackages, required final AnkiPackageReader ankiPackages,
/// Packs exports with files into a `.zip`.
required final ExportArchiver exportArchiver,
/// Keeps the device's own settings, such as its appearance. /// Keeps the device's own settings, such as its appearance.
required final DeviceSettingsStore deviceSettingsStore, required final DeviceSettingsStore deviceSettingsStore,
@ -185,6 +189,7 @@ class Dependencies({
videos: IframeYoutubeVideos(), videos: IframeYoutubeVideos(),
pdfEngine: PdfrxEngine(), pdfEngine: PdfrxEngine(),
ankiPackages: IsolateAnkiPackageReader(), ankiPackages: IsolateAnkiPackageReader(),
exportArchiver: IsolateExportArchiver(),
deviceSettingsStore: deviceSettingsStore, deviceSettingsStore: deviceSettingsStore,
deviceSettings: await deviceSettingsStore.read(), deviceSettings: await deviceSettingsStore.read(),
appVersion: appVersion, appVersion: appVersion,

View file

@ -0,0 +1,42 @@
import 'dart:isolate';
import 'dart:typed_data';
import 'package:archive/archive.dart';
import 'package:halkyon_learn_flutter/src/feature/export/data/studyset_markdown.dart';
/// Packs an export's files into a `.zip` (see [zipExport]).
abstract interface class ExportArchiver() {
/// [files] as a `.zip`, in their order.
Future<Uint8List> zip(List<ExportFile> files);
}
/// [ExportArchiver] on an isolate of its own: a studyset with many PDFs
/// takes a while to pack, and the app stays responsive meanwhile.
class IsolateExportArchiver() implements ExportArchiver {
@override
Future<Uint8List> zip(List<ExportFile> files) =>
Isolate.run(() => zipExport(files), debugName: 'Export archive');
}
/// [files] as a `.zip`, in their order: PDFs, images and sounds stored as
/// they are, everything else deflated.
Uint8List zipExport(List<ExportFile> files) {
final archive = Archive();
for (final file in files) {
archive.add(
ArchiveFile.bytes(file.path, file.bytes)
// PDFs, images and sounds are compressed already: deflating them
// again takes long and saves nothing.
..compression = _compressed.hasMatch(file.path)
? CompressionType.none
: null,
);
}
return ZipEncoder().encodeBytes(archive);
}
/// Files whose content is compressed already, by their extension.
final _compressed = RegExp(
r'\.(pdf|png|jpe?g|gif|webp|mp3|m4a|aac|ogg|opus|mp4|webm|zip)$',
caseSensitive: false,
);

View file

@ -1,6 +1,5 @@
import 'dart:convert'; import 'dart:convert';
import 'package:archive/archive.dart';
import 'package:flutter_bloc/flutter_bloc.dart'; import 'package:flutter_bloc/flutter_bloc.dart';
import 'package:halkyon_learn_flutter/l10n/app_localizations.dart'; import 'package:halkyon_learn_flutter/l10n/app_localizations.dart';
import 'package:halkyon_learn_flutter/src/core/dependencies/dependencies_scope.dart'; import 'package:halkyon_learn_flutter/src/core/dependencies/dependencies_scope.dart';
@ -20,6 +19,7 @@ Future<void> exportStudyset(BuildContext context, Studyset studyset) async {
final messenger = ScaffoldMessenger.of(context); final messenger = ScaffoldMessenger.of(context);
final person = PersonScope.of(context); final person = PersonScope.of(context);
final saver = DependenciesScope.of(context).fileSaver; final saver = DependenciesScope.of(context).fileSaver;
final archiver = DependenciesScope.of(context).exportArchiver;
final style = context.read<DeviceSettingsBloc>().state.citationStyle; final style = context.read<DeviceSettingsBloc>().state.citationStyle;
final exporter = StudysetMarkdownExporter( final exporter = StudysetMarkdownExporter(
person.studysets, person.studysets,
@ -56,20 +56,12 @@ Future<void> exportStudyset(BuildContext context, Studyset studyset) async {
mimeType: 'text/markdown', mimeType: 'text/markdown',
); );
} else { } else {
final archive = Archive()..add(ArchiveFile.bytes('$name.md', markdown));
for (final file in exported.files) {
archive.add(
ArchiveFile.bytes(file.path, file.bytes)
// PDFs, images and sounds are compressed already: deflating
// them again takes long and saves nothing.
..compression = _compressed.hasMatch(file.path)
? CompressionType.none
: null,
);
}
saved = await saver.save( saved = await saver.save(
name: '$name.zip', name: '$name.zip',
bytes: ZipEncoder().encodeBytes(archive), bytes: await archiver.zip([
(path: '$name.md', bytes: markdown),
...exported.files,
]),
mimeType: 'application/zip', mimeType: 'application/zip',
); );
} }
@ -128,9 +120,3 @@ String _fileStem(String title) {
.replaceAll(RegExp(r'\s+'), ' '); .replaceAll(RegExp(r'\s+'), ' ');
return stem.isEmpty || stem.startsWith('.') ? 'studyset' : stem; return stem.isEmpty || stem.startsWith('.') ? 'studyset' : stem;
} }
/// Files whose content is compressed already, by their extension.
final _compressed = RegExp(
r'\.(pdf|png|jpe?g|gif|webp|mp3|m4a|aac|ogg|opus|mp4|webm|zip)$',
caseSensitive: false,
);

View file

@ -0,0 +1,43 @@
import 'dart:convert';
import 'dart:typed_data';
import 'package:archive/archive.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:halkyon_learn_flutter/src/feature/export/data/export_archive.dart';
import 'package:halkyon_learn_flutter/src/feature/export/data/studyset_markdown.dart';
final markdown = Uint8List.fromList(utf8.encode('# Spanish\n' * 100));
final pdf = Uint8List.fromList(utf8.encode('%PDF-1.7 ' * 100));
final files = <ExportFile>[
(path: 'Spanish.md', bytes: markdown),
(path: 'files/pdfs/Grammar.PDF', bytes: pdf),
];
void main() {
test('packs the files in their order: what is compressed already as it '
'is, the rest deflated', () {
final zip = ZipDecoder().decodeBytes(zipExport(files));
expect(
[for (final file in zip.files) file.name],
[
'Spanish.md',
'files/pdfs/Grammar.PDF',
],
);
expect(zip.files[0].content, markdown);
expect(zip.files[0].compression, CompressionType.deflate);
expect(zip.files[1].content, pdf);
expect(zip.files[1].compression, CompressionType.none);
});
test('packs the same on an isolate of its own', () async {
final zip = ZipDecoder().decodeBytes(
await IsolateExportArchiver().zip(files),
);
expect(zip.files[0].content, markdown);
expect(zip.files[1].content, pdf);
});
}

View file

@ -30,6 +30,8 @@ import 'package:halkyon_learn_flutter/src/feature/explore/data/copy_updater.dart
import 'package:halkyon_learn_flutter/src/feature/explore/data/explore_gateway.dart'; import 'package:halkyon_learn_flutter/src/feature/explore/data/explore_gateway.dart';
import 'package:halkyon_learn_flutter/src/feature/explore/data/explore_repository.dart'; import 'package:halkyon_learn_flutter/src/feature/explore/data/explore_repository.dart';
import 'package:halkyon_learn_flutter/src/feature/explore/data/release_copier.dart'; import 'package:halkyon_learn_flutter/src/feature/explore/data/release_copier.dart';
import 'package:halkyon_learn_flutter/src/feature/export/data/export_archive.dart';
import 'package:halkyon_learn_flutter/src/feature/export/data/studyset_markdown.dart';
import 'package:halkyon_learn_flutter/src/feature/feedback/data/feedback_gateway.dart'; import 'package:halkyon_learn_flutter/src/feature/feedback/data/feedback_gateway.dart';
import 'package:halkyon_learn_flutter/src/feature/feedback/data/feedback_repository.dart'; import 'package:halkyon_learn_flutter/src/feature/feedback/data/feedback_repository.dart';
import 'package:halkyon_learn_flutter/src/feature/geo_map/data/geo_map_repository.dart'; import 'package:halkyon_learn_flutter/src/feature/geo_map/data/geo_map_repository.dart';
@ -154,6 +156,7 @@ Dependencies testDependencies({
videos: videos ?? FakeYoutubeVideos(), videos: videos ?? FakeYoutubeVideos(),
pdfEngine: pdfEngine ?? FakePdfEngine(), pdfEngine: pdfEngine ?? FakePdfEngine(),
ankiPackages: InlineAnkiPackageReader(), ankiPackages: InlineAnkiPackageReader(),
exportArchiver: InlineExportArchiver(),
deviceSettingsStore: deviceSettingsStore:
deviceSettingsStore ?? MemoryDeviceSettingsStore(deviceSettings), deviceSettingsStore ?? MemoryDeviceSettingsStore(deviceSettings),
deviceSettings: deviceSettings, deviceSettings: deviceSettings,
@ -296,3 +299,9 @@ class InlineAnkiPackageReader() implements AnkiPackageReader {
Future<AnkiPackage> read(Uint8List bytes) async => Future<AnkiPackage> read(Uint8List bytes) async =>
readAnkiPackage(bytes, temp: Directory.systemTemp); readAnkiPackage(bytes, temp: Directory.systemTemp);
} }
/// Packs exports where it is asked to, as [InlineAnkiPackageReader] reads.
class InlineExportArchiver() implements ExportArchiver {
@override
Future<Uint8List> zip(List<ExportFile> files) async => zipExport(files);
}