perf(export): zip an export with files on an isolate of its own
A studyset with many PDFs took a while to pack, and the window froze meanwhile. The zip is now made on another isolate (ExportArchiver, an isolate in the app and in place in widget tests, whose fake clock can't wait for one), as Anki packages are read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MMm7usAo6q9ZZRoPkWjoxi Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
This commit is contained in:
parent
c7486a5b1e
commit
e04be6603d
6 changed files with 105 additions and 20 deletions
|
|
@ -85,7 +85,7 @@ A spaced-repetition learning app: studysets of flashcards, reviewed on an FSRS s
|
||||||
- **App-wide vs. per person.** `Dependencies` holds what outlives a person (the client, auth). Everything of the signed-in person (their local database, its sync, their repositories) is `PersonDependencies`, opened by `PersonScope` inside `SessionGate`'s keyed subtree (`Dependencies.openPerson`) and disposed with it; widgets reach it through `PersonScope.of(context)`.
|
- **App-wide vs. per person.** `Dependencies` holds what outlives a person (the client, auth). Everything of the signed-in person (their local database, its sync, their repositories) is `PersonDependencies`, opened by `PersonScope` inside `SessionGate`'s keyed subtree (`Dependencies.openPerson`) and disposed with it; widgets reach it through `PersonScope.of(context)`.
|
||||||
- **The device's own settings** (appearance, language, citation style; `core/device_settings/`) belong to the device, not to a person: kept in `<app support>/device_settings.json`, read in `Dependencies.create()` so the first frame already has them, and provided by `DeviceSettingsScope` above `SessionGate`'s keyed subtree, which both its plain app and `PersonApp` read. What should follow the person to their other devices goes into their synced `PersonSettings` instead.
|
- **The device's own settings** (appearance, language, citation style; `core/device_settings/`) belong to the device, not to a person: kept in `<app support>/device_settings.json`, read in `Dependencies.create()` so the first frame already has them, and provided by `DeviceSettingsScope` above `SessionGate`'s keyed subtree, which both its plain app and `PersonApp` read. What should follow the person to their other devices goes into their synced `PersonSettings` instead.
|
||||||
- **Every person has an app of their own** (`PersonApp`, `lib/src/app/`), which `SessionGate` builds inside the keyed subtree: a `MaterialApp.router` with its own `GoRouter`, and `PersonScope` around the router's navigator (`MaterialApp.builder`). So every screen and dialog is inside the person's scope and reads their repositories from `PersonScope.of(context)` wherever it is built, and a change of person disposes the router with every screen on it and starts again at home. Until someone is signed in, `SessionGate` shows a plain `MaterialApp`: the spinner, or the failure with a way to try again.
|
- **Every person has an app of their own** (`PersonApp`, `lib/src/app/`), which `SessionGate` builds inside the keyed subtree: a `MaterialApp.router` with its own `GoRouter`, and `PersonScope` around the router's navigator (`MaterialApp.builder`). So every screen and dialog is inside the person's scope and reads their repositories from `PersonScope.of(context)` wherever it is built, and a change of person disposes the router with every screen on it and starts again at home. Until someone is signed in, `SessionGate` shows a plain `MaterialApp`: the spinner, or the failure with a way to try again.
|
||||||
- **Platform plugins sit behind small interfaces in `Dependencies`**, with fakes in tests: picking files (`FilePicker`, `file_selector`), saving files the app made, such as exports (`FileSaver`: `file_selector`'s dialog on the desktop, `file_picker`'s on Android and iOS, where `file_selector` has none), playing sounds (`SoundPlayer`, `audioplayers`), playing YouTube videos (`YoutubeVideos`, `youtube_player_iframe` where a web view exists, else the browser), reading PDFs (`PdfEngine`, pdfrx on PDFium, which `flutter test` can't load: widget tests use `FakePdfEngine`, whose PDF bytes say their pages and chapters), reading Anki packages on another isolate (`AnkiPackageReader`; a widget test's fake clock can't wait for one), and the Pomodoro timer's side of the system (`PomodoroPlatform`, `FakePomodoroPlatform` in tests).
|
- **Platform plugins sit behind small interfaces in `Dependencies`**, with fakes in tests: picking files (`FilePicker`, `file_selector`), saving files the app made, such as exports (`FileSaver`: `file_selector`'s dialog on the desktop, `file_picker`'s on Android and iOS, where `file_selector` has none), playing sounds (`SoundPlayer`, `audioplayers`), playing YouTube videos (`YoutubeVideos`, `youtube_player_iframe` where a web view exists, else the browser), reading PDFs (`PdfEngine`, pdfrx on PDFium, which `flutter test` can't load: widget tests use `FakePdfEngine`, whose PDF bytes say their pages and chapters), reading Anki packages on another isolate (`AnkiPackageReader`; a widget test's fake clock can't wait for one), zipping an export with its files on another isolate (`ExportArchiver`, for the same reason), and the Pomodoro timer's side of the system (`PomodoroPlatform`, `FakePomodoroPlatform` in tests).
|
||||||
- **Material's `Card` widget clashes with the domain's `Card`**: files that need both import `material_ui` with `hide Card`.
|
- **Material's `Card` widget clashes with the domain's `Card`**: files that need both import `material_ui` with `hide Card`.
|
||||||
- **State: `flutter_bloc`, uniformly.** Blocs fold repository streams into immutable states with value equality, are provided as close to their consumer as possible, and log through one global `BlocObserver`. Why: the data layer is stream-shaped, event → state machines test without widgets, and one pattern everywhere is itself the point.
|
- **State: `flutter_bloc`, uniformly.** Blocs fold repository streams into immutable states with value equality, are provided as close to their consumer as possible, and log through one global `BlocObserver`. Why: the data layer is stream-shaped, event → state machines test without widgets, and one pattern everywhere is itself the point.
|
||||||
- **Value equality comes from `equatable`**, mixed in (`with Equatable`), for models and states alike; `leancode_lint`'s `missing_equatable_props` catches a field left out of `props`.
|
- **Value equality comes from `equatable`**, mixed in (`with Equatable`), for models and states alike; `leancode_lint`'s `missing_equatable_props` catches a field left out of `props`.
|
||||||
|
|
|
||||||
|
|
@ -33,6 +33,7 @@ import 'package:halkyon_learn_flutter/src/feature/explore/data/copy_updater.dart
|
||||||
import 'package:halkyon_learn_flutter/src/feature/explore/data/explore_repository.dart';
|
import 'package:halkyon_learn_flutter/src/feature/explore/data/explore_repository.dart';
|
||||||
import 'package:halkyon_learn_flutter/src/feature/explore/data/release_copier.dart';
|
import 'package:halkyon_learn_flutter/src/feature/explore/data/release_copier.dart';
|
||||||
import 'package:halkyon_learn_flutter/src/feature/explore/data/serverpod_explore_gateway.dart';
|
import 'package:halkyon_learn_flutter/src/feature/explore/data/serverpod_explore_gateway.dart';
|
||||||
|
import 'package:halkyon_learn_flutter/src/feature/export/data/export_archive.dart';
|
||||||
import 'package:halkyon_learn_flutter/src/feature/feedback/data/feedback_repository.dart';
|
import 'package:halkyon_learn_flutter/src/feature/feedback/data/feedback_repository.dart';
|
||||||
import 'package:halkyon_learn_flutter/src/feature/feedback/data/serverpod_feedback_gateway.dart';
|
import 'package:halkyon_learn_flutter/src/feature/feedback/data/serverpod_feedback_gateway.dart';
|
||||||
import 'package:halkyon_learn_flutter/src/feature/feedback/data/sync_feedback_gateway.dart';
|
import 'package:halkyon_learn_flutter/src/feature/feedback/data/sync_feedback_gateway.dart';
|
||||||
|
|
@ -122,6 +123,9 @@ class Dependencies({
|
||||||
/// Reads Anki packages to import.
|
/// Reads Anki packages to import.
|
||||||
required final AnkiPackageReader ankiPackages,
|
required final AnkiPackageReader ankiPackages,
|
||||||
|
|
||||||
|
/// Packs exports with files into a `.zip`.
|
||||||
|
required final ExportArchiver exportArchiver,
|
||||||
|
|
||||||
/// Keeps the device's own settings, such as its appearance.
|
/// Keeps the device's own settings, such as its appearance.
|
||||||
required final DeviceSettingsStore deviceSettingsStore,
|
required final DeviceSettingsStore deviceSettingsStore,
|
||||||
|
|
||||||
|
|
@ -185,6 +189,7 @@ class Dependencies({
|
||||||
videos: IframeYoutubeVideos(),
|
videos: IframeYoutubeVideos(),
|
||||||
pdfEngine: PdfrxEngine(),
|
pdfEngine: PdfrxEngine(),
|
||||||
ankiPackages: IsolateAnkiPackageReader(),
|
ankiPackages: IsolateAnkiPackageReader(),
|
||||||
|
exportArchiver: IsolateExportArchiver(),
|
||||||
deviceSettingsStore: deviceSettingsStore,
|
deviceSettingsStore: deviceSettingsStore,
|
||||||
deviceSettings: await deviceSettingsStore.read(),
|
deviceSettings: await deviceSettingsStore.read(),
|
||||||
appVersion: appVersion,
|
appVersion: appVersion,
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,42 @@
|
||||||
|
import 'dart:isolate';
|
||||||
|
import 'dart:typed_data';
|
||||||
|
|
||||||
|
import 'package:archive/archive.dart';
|
||||||
|
import 'package:halkyon_learn_flutter/src/feature/export/data/studyset_markdown.dart';
|
||||||
|
|
||||||
|
/// Packs an export's files into a `.zip` (see [zipExport]).
|
||||||
|
abstract interface class ExportArchiver() {
|
||||||
|
/// [files] as a `.zip`, in their order.
|
||||||
|
Future<Uint8List> zip(List<ExportFile> files);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// [ExportArchiver] on an isolate of its own: a studyset with many PDFs
|
||||||
|
/// takes a while to pack, and the app stays responsive meanwhile.
|
||||||
|
class IsolateExportArchiver() implements ExportArchiver {
|
||||||
|
@override
|
||||||
|
Future<Uint8List> zip(List<ExportFile> files) =>
|
||||||
|
Isolate.run(() => zipExport(files), debugName: 'Export archive');
|
||||||
|
}
|
||||||
|
|
||||||
|
/// [files] as a `.zip`, in their order: PDFs, images and sounds stored as
|
||||||
|
/// they are, everything else deflated.
|
||||||
|
Uint8List zipExport(List<ExportFile> files) {
|
||||||
|
final archive = Archive();
|
||||||
|
for (final file in files) {
|
||||||
|
archive.add(
|
||||||
|
ArchiveFile.bytes(file.path, file.bytes)
|
||||||
|
// PDFs, images and sounds are compressed already: deflating them
|
||||||
|
// again takes long and saves nothing.
|
||||||
|
..compression = _compressed.hasMatch(file.path)
|
||||||
|
? CompressionType.none
|
||||||
|
: null,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return ZipEncoder().encodeBytes(archive);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Files whose content is compressed already, by their extension.
|
||||||
|
final _compressed = RegExp(
|
||||||
|
r'\.(pdf|png|jpe?g|gif|webp|mp3|m4a|aac|ogg|opus|mp4|webm|zip)$',
|
||||||
|
caseSensitive: false,
|
||||||
|
);
|
||||||
|
|
@ -1,6 +1,5 @@
|
||||||
import 'dart:convert';
|
import 'dart:convert';
|
||||||
|
|
||||||
import 'package:archive/archive.dart';
|
|
||||||
import 'package:flutter_bloc/flutter_bloc.dart';
|
import 'package:flutter_bloc/flutter_bloc.dart';
|
||||||
import 'package:halkyon_learn_flutter/l10n/app_localizations.dart';
|
import 'package:halkyon_learn_flutter/l10n/app_localizations.dart';
|
||||||
import 'package:halkyon_learn_flutter/src/core/dependencies/dependencies_scope.dart';
|
import 'package:halkyon_learn_flutter/src/core/dependencies/dependencies_scope.dart';
|
||||||
|
|
@ -20,6 +19,7 @@ Future<void> exportStudyset(BuildContext context, Studyset studyset) async {
|
||||||
final messenger = ScaffoldMessenger.of(context);
|
final messenger = ScaffoldMessenger.of(context);
|
||||||
final person = PersonScope.of(context);
|
final person = PersonScope.of(context);
|
||||||
final saver = DependenciesScope.of(context).fileSaver;
|
final saver = DependenciesScope.of(context).fileSaver;
|
||||||
|
final archiver = DependenciesScope.of(context).exportArchiver;
|
||||||
final style = context.read<DeviceSettingsBloc>().state.citationStyle;
|
final style = context.read<DeviceSettingsBloc>().state.citationStyle;
|
||||||
final exporter = StudysetMarkdownExporter(
|
final exporter = StudysetMarkdownExporter(
|
||||||
person.studysets,
|
person.studysets,
|
||||||
|
|
@ -56,20 +56,12 @@ Future<void> exportStudyset(BuildContext context, Studyset studyset) async {
|
||||||
mimeType: 'text/markdown',
|
mimeType: 'text/markdown',
|
||||||
);
|
);
|
||||||
} else {
|
} else {
|
||||||
final archive = Archive()..add(ArchiveFile.bytes('$name.md', markdown));
|
|
||||||
for (final file in exported.files) {
|
|
||||||
archive.add(
|
|
||||||
ArchiveFile.bytes(file.path, file.bytes)
|
|
||||||
// PDFs, images and sounds are compressed already: deflating
|
|
||||||
// them again takes long and saves nothing.
|
|
||||||
..compression = _compressed.hasMatch(file.path)
|
|
||||||
? CompressionType.none
|
|
||||||
: null,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
saved = await saver.save(
|
saved = await saver.save(
|
||||||
name: '$name.zip',
|
name: '$name.zip',
|
||||||
bytes: ZipEncoder().encodeBytes(archive),
|
bytes: await archiver.zip([
|
||||||
|
(path: '$name.md', bytes: markdown),
|
||||||
|
...exported.files,
|
||||||
|
]),
|
||||||
mimeType: 'application/zip',
|
mimeType: 'application/zip',
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
@ -128,9 +120,3 @@ String _fileStem(String title) {
|
||||||
.replaceAll(RegExp(r'\s+'), ' ');
|
.replaceAll(RegExp(r'\s+'), ' ');
|
||||||
return stem.isEmpty || stem.startsWith('.') ? 'studyset' : stem;
|
return stem.isEmpty || stem.startsWith('.') ? 'studyset' : stem;
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Files whose content is compressed already, by their extension.
|
|
||||||
final _compressed = RegExp(
|
|
||||||
r'\.(pdf|png|jpe?g|gif|webp|mp3|m4a|aac|ogg|opus|mp4|webm|zip)$',
|
|
||||||
caseSensitive: false,
|
|
||||||
);
|
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,43 @@
|
||||||
|
import 'dart:convert';
|
||||||
|
import 'dart:typed_data';
|
||||||
|
|
||||||
|
import 'package:archive/archive.dart';
|
||||||
|
import 'package:flutter_test/flutter_test.dart';
|
||||||
|
import 'package:halkyon_learn_flutter/src/feature/export/data/export_archive.dart';
|
||||||
|
import 'package:halkyon_learn_flutter/src/feature/export/data/studyset_markdown.dart';
|
||||||
|
|
||||||
|
final markdown = Uint8List.fromList(utf8.encode('# Spanish\n' * 100));
|
||||||
|
final pdf = Uint8List.fromList(utf8.encode('%PDF-1.7 ' * 100));
|
||||||
|
|
||||||
|
final files = <ExportFile>[
|
||||||
|
(path: 'Spanish.md', bytes: markdown),
|
||||||
|
(path: 'files/pdfs/Grammar.PDF', bytes: pdf),
|
||||||
|
];
|
||||||
|
|
||||||
|
void main() {
|
||||||
|
test('packs the files in their order: what is compressed already as it '
|
||||||
|
'is, the rest deflated', () {
|
||||||
|
final zip = ZipDecoder().decodeBytes(zipExport(files));
|
||||||
|
|
||||||
|
expect(
|
||||||
|
[for (final file in zip.files) file.name],
|
||||||
|
[
|
||||||
|
'Spanish.md',
|
||||||
|
'files/pdfs/Grammar.PDF',
|
||||||
|
],
|
||||||
|
);
|
||||||
|
expect(zip.files[0].content, markdown);
|
||||||
|
expect(zip.files[0].compression, CompressionType.deflate);
|
||||||
|
expect(zip.files[1].content, pdf);
|
||||||
|
expect(zip.files[1].compression, CompressionType.none);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('packs the same on an isolate of its own', () async {
|
||||||
|
final zip = ZipDecoder().decodeBytes(
|
||||||
|
await IsolateExportArchiver().zip(files),
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(zip.files[0].content, markdown);
|
||||||
|
expect(zip.files[1].content, pdf);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
@ -30,6 +30,8 @@ import 'package:halkyon_learn_flutter/src/feature/explore/data/copy_updater.dart
|
||||||
import 'package:halkyon_learn_flutter/src/feature/explore/data/explore_gateway.dart';
|
import 'package:halkyon_learn_flutter/src/feature/explore/data/explore_gateway.dart';
|
||||||
import 'package:halkyon_learn_flutter/src/feature/explore/data/explore_repository.dart';
|
import 'package:halkyon_learn_flutter/src/feature/explore/data/explore_repository.dart';
|
||||||
import 'package:halkyon_learn_flutter/src/feature/explore/data/release_copier.dart';
|
import 'package:halkyon_learn_flutter/src/feature/explore/data/release_copier.dart';
|
||||||
|
import 'package:halkyon_learn_flutter/src/feature/export/data/export_archive.dart';
|
||||||
|
import 'package:halkyon_learn_flutter/src/feature/export/data/studyset_markdown.dart';
|
||||||
import 'package:halkyon_learn_flutter/src/feature/feedback/data/feedback_gateway.dart';
|
import 'package:halkyon_learn_flutter/src/feature/feedback/data/feedback_gateway.dart';
|
||||||
import 'package:halkyon_learn_flutter/src/feature/feedback/data/feedback_repository.dart';
|
import 'package:halkyon_learn_flutter/src/feature/feedback/data/feedback_repository.dart';
|
||||||
import 'package:halkyon_learn_flutter/src/feature/geo_map/data/geo_map_repository.dart';
|
import 'package:halkyon_learn_flutter/src/feature/geo_map/data/geo_map_repository.dart';
|
||||||
|
|
@ -154,6 +156,7 @@ Dependencies testDependencies({
|
||||||
videos: videos ?? FakeYoutubeVideos(),
|
videos: videos ?? FakeYoutubeVideos(),
|
||||||
pdfEngine: pdfEngine ?? FakePdfEngine(),
|
pdfEngine: pdfEngine ?? FakePdfEngine(),
|
||||||
ankiPackages: InlineAnkiPackageReader(),
|
ankiPackages: InlineAnkiPackageReader(),
|
||||||
|
exportArchiver: InlineExportArchiver(),
|
||||||
deviceSettingsStore:
|
deviceSettingsStore:
|
||||||
deviceSettingsStore ?? MemoryDeviceSettingsStore(deviceSettings),
|
deviceSettingsStore ?? MemoryDeviceSettingsStore(deviceSettings),
|
||||||
deviceSettings: deviceSettings,
|
deviceSettings: deviceSettings,
|
||||||
|
|
@ -296,3 +299,9 @@ class InlineAnkiPackageReader() implements AnkiPackageReader {
|
||||||
Future<AnkiPackage> read(Uint8List bytes) async =>
|
Future<AnkiPackage> read(Uint8List bytes) async =>
|
||||||
readAnkiPackage(bytes, temp: Directory.systemTemp);
|
readAnkiPackage(bytes, temp: Directory.systemTemp);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Packs exports where it is asked to, as [InlineAnkiPackageReader] reads.
|
||||||
|
class InlineExportArchiver() implements ExportArchiver {
|
||||||
|
@override
|
||||||
|
Future<Uint8List> zip(List<ExportFile> files) async => zipExport(files);
|
||||||
|
}
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue