halkyon/tool/anki/make_fixtures.py

129 lines
4.7 KiB
Python
Raw Normal View History

Import Anki decks with their files and history Files in content: the person's images and sounds as asset rows synced like all content, their files kept by hash on each device, uploaded after each sync, downloaded on demand and checked against their hash (core/asset/AssetRepository). Rich text names them as asset:<id> URLs, in the editor's image block and an audio block of ours; any other URL is never fetched. Sounds play through audioplayers, one at a time. Content holding only an image or a sound isn't blank, and lists show it as such. The Anki import ("Import from Anki" under Add material): .apkg and .colpkg packages in all three formats Anki has written, read on an isolate of their own. Modern packages compress everything with zstd in frames that don't record their size, which pure_zstd v1.0.0 refuses; our fork decodes them. Fields become rich text by a converter of our own (lines, lists, formatting, images, sounds, cloze deletions hidden on the front). One card per note, the files first, studysets as chosen (flatten, sub-studysets, or tags by subdeck), the notes' tags, and, if wanted, the review history, which FSRS replays: the first template's reviews to the card's schedule, a reversed note type's second to the reverse one. Imported cards say where they came from in their first revision (card schema 3); the dashboard doesn't count them as written. The test fixtures are exported by Anki itself (tool/anki/make_fixtures.py). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-02 04:52:02 +02:00
#!/usr/bin/env python3
"""Builds the Anki packages the app's import tests read, with Anki itself.
The `anki` Python package is Anki's own backend, so the packages are exactly
what Anki exports: one collection, exported in the latest format, in the
legacy format ("Support older Anki versions"), and repacked the way Anki
before 2.1.50 wrote packages (only `collection.anki2`, no `meta`).
The collection holds, on purpose:
- a Basic note with bold text, a line break, an image, and two tags, one of
them hierarchical (deck "Japanese");
- a "Basic (and reversed card)" note with a sound, both its cards answered,
one of them first wrong (deck "Japanese::Animals");
- a Cloze note with two deletions, never reviewed;
- a Basic note with a list and HTML entities ("Japanese::Animals");
- a manual reschedule in the review log (no answer).
Run from the repository root, in a virtual environment with Anki's package:
python3 -m venv /tmp/anki && /tmp/anki/bin/pip install anki==26.9.3
/tmp/anki/bin/python tool/anki/make_fixtures.py
It rewrites halkyon_learn_flutter/test/feature/anki/fixtures/*.apkg. Ids and
times differ on every run; the tests don't depend on them.
"""
import io
import os
import tempfile
import zipfile
from pathlib import Path
from anki.collection import Collection, ExportAnkiPackageOptions
from anki.decks import DeckId
OUT = Path("halkyon_learn_flutter/test/feature/anki/fixtures")
Fix what the first Android run found The flow ran on a Pixel 9a and its emulator (2026-10-02); what it found: - The composer stayed on the card just written after "New flashcard", and on the original after "Duplicate": leaving a card changed the state before the new one was in it, so the screen took the new card for gone. It now waits for the card it asked for. - Bottom sheets reached under the status bar with the keyboard open (useSafeArea). - The field being written stayed behind the keyboard: the editor scrolls only within itself, so the card face now keeps it in view when it opens and when the keyboard rises, as a text field does. - A tap on read-only rich text put a caret in it and opened the keyboard on phones, and in the composer never opened the field. RichContentView is display only now: no selection, taps go to what surrounds it, a sound's button still plays (decided 2026-10-02). - The card types broke their names on a phone; they are choice chips that wrap (decided 2026-10-02). - The week overview marked days before the person's first activity as missed; they show nothing now. An import starts the activity too, without counting as studying (decided 2026-10-02). - A composer preview cut short gave no sign of it; it fades out over its last 32 dp, only when cut short (decided 2026-10-02). - "Isn't on this device yet" showed for any image or sound that failed; a file that is here but can't be decoded now says so, and offers no retry. - The signed-in account card was narrower than the anonymous one. - The composer's title said "Create flashcards" also when opened on a stored card; it says "Edit flashcards in …" then (decided 2026-10-02). - The Anki fixtures carry a valid PNG and a silent MP3, so they also serve to try the import by hand (decided 2026-10-02); regenerated with Anki 26.9.3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-02 19:52:39 +02:00
# Valid files, so the packages also serve to try the import by hand: a 1x1
# PNG (one orange pixel), and 0.26 s of silence as an MP3 (ten MPEG-1 Layer
# III frames, 128 kbit/s, 44.1 kHz, mono, whose zeroed side information
# decodes to silence).
Import Anki decks with their files and history Files in content: the person's images and sounds as asset rows synced like all content, their files kept by hash on each device, uploaded after each sync, downloaded on demand and checked against their hash (core/asset/AssetRepository). Rich text names them as asset:<id> URLs, in the editor's image block and an audio block of ours; any other URL is never fetched. Sounds play through audioplayers, one at a time. Content holding only an image or a sound isn't blank, and lists show it as such. The Anki import ("Import from Anki" under Add material): .apkg and .colpkg packages in all three formats Anki has written, read on an isolate of their own. Modern packages compress everything with zstd in frames that don't record their size, which pure_zstd v1.0.0 refuses; our fork decodes them. Fields become rich text by a converter of our own (lines, lists, formatting, images, sounds, cloze deletions hidden on the front). One card per note, the files first, studysets as chosen (flatten, sub-studysets, or tags by subdeck), the notes' tags, and, if wanted, the review history, which FSRS replays: the first template's reviews to the card's schedule, a reversed note type's second to the reverse one. Imported cards say where they came from in their first revision (card schema 3); the dashboard doesn't count them as written. The test fixtures are exported by Anki itself (tool/anki/make_fixtures.py). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-02 04:52:02 +02:00
PNG = bytes.fromhex(
Fix what the first Android run found The flow ran on a Pixel 9a and its emulator (2026-10-02); what it found: - The composer stayed on the card just written after "New flashcard", and on the original after "Duplicate": leaving a card changed the state before the new one was in it, so the screen took the new card for gone. It now waits for the card it asked for. - Bottom sheets reached under the status bar with the keyboard open (useSafeArea). - The field being written stayed behind the keyboard: the editor scrolls only within itself, so the card face now keeps it in view when it opens and when the keyboard rises, as a text field does. - A tap on read-only rich text put a caret in it and opened the keyboard on phones, and in the composer never opened the field. RichContentView is display only now: no selection, taps go to what surrounds it, a sound's button still plays (decided 2026-10-02). - The card types broke their names on a phone; they are choice chips that wrap (decided 2026-10-02). - The week overview marked days before the person's first activity as missed; they show nothing now. An import starts the activity too, without counting as studying (decided 2026-10-02). - A composer preview cut short gave no sign of it; it fades out over its last 32 dp, only when cut short (decided 2026-10-02). - "Isn't on this device yet" showed for any image or sound that failed; a file that is here but can't be decoded now says so, and offers no retry. - The signed-in account card was narrower than the anonymous one. - The composer's title said "Create flashcards" also when opened on a stored card; it says "Edit flashcards in …" then (decided 2026-10-02). - The Anki fixtures carry a valid PNG and a silent MP3, so they also serve to try the import by hand (decided 2026-10-02); regenerated with Anki 26.9.3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-02 19:52:39 +02:00
"89504e470d0a1a0a0000000d49484452000000010000000108060000001f15c489"
"0000000d49444154789c63f8dfc0f01f000680027f104c1be10000000049454e44ae426082"
Import Anki decks with their files and history Files in content: the person's images and sounds as asset rows synced like all content, their files kept by hash on each device, uploaded after each sync, downloaded on demand and checked against their hash (core/asset/AssetRepository). Rich text names them as asset:<id> URLs, in the editor's image block and an audio block of ours; any other URL is never fetched. Sounds play through audioplayers, one at a time. Content holding only an image or a sound isn't blank, and lists show it as such. The Anki import ("Import from Anki" under Add material): .apkg and .colpkg packages in all three formats Anki has written, read on an isolate of their own. Modern packages compress everything with zstd in frames that don't record their size, which pure_zstd v1.0.0 refuses; our fork decodes them. Fields become rich text by a converter of our own (lines, lists, formatting, images, sounds, cloze deletions hidden on the front). One card per note, the files first, studysets as chosen (flatten, sub-studysets, or tags by subdeck), the notes' tags, and, if wanted, the review history, which FSRS replays: the first template's reviews to the card's schedule, a reversed note type's second to the reverse one. Imported cards say where they came from in their first revision (card schema 3); the dashboard doesn't count them as written. The test fixtures are exported by Anki itself (tool/anki/make_fixtures.py). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-02 04:52:02 +02:00
)
Fix what the first Android run found The flow ran on a Pixel 9a and its emulator (2026-10-02); what it found: - The composer stayed on the card just written after "New flashcard", and on the original after "Duplicate": leaving a card changed the state before the new one was in it, so the screen took the new card for gone. It now waits for the card it asked for. - Bottom sheets reached under the status bar with the keyboard open (useSafeArea). - The field being written stayed behind the keyboard: the editor scrolls only within itself, so the card face now keeps it in view when it opens and when the keyboard rises, as a text field does. - A tap on read-only rich text put a caret in it and opened the keyboard on phones, and in the composer never opened the field. RichContentView is display only now: no selection, taps go to what surrounds it, a sound's button still plays (decided 2026-10-02). - The card types broke their names on a phone; they are choice chips that wrap (decided 2026-10-02). - The week overview marked days before the person's first activity as missed; they show nothing now. An import starts the activity too, without counting as studying (decided 2026-10-02). - A composer preview cut short gave no sign of it; it fades out over its last 32 dp, only when cut short (decided 2026-10-02). - "Isn't on this device yet" showed for any image or sound that failed; a file that is here but can't be decoded now says so, and offers no retry. - The signed-in account card was narrower than the anonymous one. - The composer's title said "Create flashcards" also when opened on a stored card; it says "Edit flashcards in …" then (decided 2026-10-02). - The Anki fixtures carry a valid PNG and a silent MP3, so they also serve to try the import by hand (decided 2026-10-02); regenerated with Anki 26.9.3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-02 19:52:39 +02:00
MP3 = (bytes.fromhex("fffb90c4") + bytes(413)) * 10
Import Anki decks with their files and history Files in content: the person's images and sounds as asset rows synced like all content, their files kept by hash on each device, uploaded after each sync, downloaded on demand and checked against their hash (core/asset/AssetRepository). Rich text names them as asset:<id> URLs, in the editor's image block and an audio block of ours; any other URL is never fetched. Sounds play through audioplayers, one at a time. Content holding only an image or a sound isn't blank, and lists show it as such. The Anki import ("Import from Anki" under Add material): .apkg and .colpkg packages in all three formats Anki has written, read on an isolate of their own. Modern packages compress everything with zstd in frames that don't record their size, which pure_zstd v1.0.0 refuses; our fork decodes them. Fields become rich text by a converter of our own (lines, lists, formatting, images, sounds, cloze deletions hidden on the front). One card per note, the files first, studysets as chosen (flatten, sub-studysets, or tags by subdeck), the notes' tags, and, if wanted, the review history, which FSRS replays: the first template's reviews to the card's schedule, a reversed note type's second to the reverse one. Imported cards say where they came from in their first revision (card schema 3); the dashboard doesn't count them as written. The test fixtures are exported by Anki itself (tool/anki/make_fixtures.py). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>
2026-10-02 04:52:02 +02:00
AGAIN, GOOD, EASY = 0, 2, 3
def main():
OUT.mkdir(parents=True, exist_ok=True)
with tempfile.TemporaryDirectory() as work:
col = Collection(os.path.join(work, "collection.anki2"))
col.media.write_data("cat.png", PNG)
col.media.write_data("neko.mp3", MP3)
japanese = col.decks.id("Japanese")
animals = col.decks.id("Japanese::Animals")
def add(model, deck, fields, tags):
note = col.new_note(col.models.by_name(model))
for i, field in enumerate(fields):
note.fields[i] = field
note.tags = tags
col.add_note(note, DeckId(deck))
return note
def answer(card_id, rating):
card = col.get_card(card_id)
card.start_timer()
states = col._backend.get_scheduling_states(card.id)
col.sched.answer_card(
col.sched.build_answer(card=card, states=states, rating=rating)
)
cat = add(
"Basic",
japanese,
["<b>猫</b>", 'cat<br>a pet<img src="cat.png">'],
["animal", "JLPT::N5"],
)
dog = add(
"Basic (and reversed card)",
animals,
["犬 [sound:neko.mp3]", "dog"],
["animal"],
)
add("Cloze", japanese, ["{{c1::Tokyo}} is the capital of {{c2::Japan::country}}", "extra"], [])
add(
"Basic",
animals,
["<ul><li>one</li><li>two</li></ul>", "&lt;tag&gt; &amp; more"],
[],
)
answer(cat.card_ids()[0], EASY)
forward, reverse = sorted(dog.card_ids(), key=lambda c: col.get_card(c).ord)
answer(forward, AGAIN)
answer(forward, GOOD)
answer(reverse, EASY)
col.sched.set_due_date([cat.card_ids()[0]], "5")
for legacy, name in [(False, "anki_latest.apkg"), (True, "anki_legacy.apkg")]:
col.export_anki_package(
out_path=str(OUT / name),
options=ExportAnkiPackageOptions(
with_scheduling=True,
with_deck_configs=False,
with_media=True,
legacy=legacy,
),
limit=None,
)
col.close()
# Before 2.1.50: the collection as collection.anki2, no meta, no stub.
with zipfile.ZipFile(OUT / "anki_legacy.apkg") as legacy:
out = io.BytesIO()
with zipfile.ZipFile(out, "w", zipfile.ZIP_DEFLATED) as old:
for info in legacy.infolist():
if info.filename in ("meta", "collection.anki2"):
continue
name = "collection.anki2" if info.filename == "collection.anki21" else info.filename
old.writestr(name, legacy.read(info.filename))
(OUT / "anki_before_2_1_50.apkg").write_bytes(out.getvalue())
if __name__ == "__main__":
main()