Tizian 「ティツィアーン」
57aa532069
Streaming encoders leave Frame_Content_Size out unless told the size up front, and their readers often can't know it either: every zstd frame in an Anki package (its collection, its media list, each media file) is written that way. Such a frame needed the exact size passed as expectedSize, and anything else was refused. Now it gets the most its blocks can hold, Block_Maximum_Size (128 KiB) each, found by walking the block headers alone, and the output is trimmed to what the blocks held. The output is still allocated once. expectedSize keeps its exact check when given, and maxOutputSize bounds the allocation as before. Checked against libzstd (python-zstandard) on Anki 26.09.3's exports: a small package's four frames and a 30 MB collection decode byte for byte the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
38 lines
1.7 KiB
Markdown
38 lines
1.7 KiB
Markdown
# Changelog
|
|
|
|
## Unreleased
|
|
|
|
- Decodes a frame that declares no content size without `expectedSize`, as
|
|
streaming encoders write them (Anki's packages, for one): the output is
|
|
allocated at the most the frame's blocks can hold, 128 KiB each, found from
|
|
their headers, and trimmed to what they held. `expectedSize` still checks
|
|
the size exactly when given.
|
|
|
|
## 1.0.0
|
|
|
|
First release.
|
|
|
|
A zstd decompressor in plain Dart. Decompression only, native platforms only,
|
|
one frame per call.
|
|
|
|
- Decodes RFC 8878 frames: raw, RLE and compressed blocks; raw, RLE, compressed
|
|
and treeless literals in one-stream and four-stream form; Huffman weights in
|
|
both the direct nibble and FSE-coded forms; predefined, RLE, FSE and repeat
|
|
sequence tables for all three symbols; repeated offsets including the
|
|
no-literals shift; multi-block frames reusing an earlier block's tables.
|
|
- Verifies the content checksum when a frame carries one, with XXH64 in
|
|
`lib/src/xxhash.dart`.
|
|
- Refuses dictionaries, several frames in one buffer, and any input left over
|
|
after a frame.
|
|
- Bounds what a frame header can make it allocate. `ZstdDecoder` takes a
|
|
`maxOutputSize`, 64 MiB by default, and a caller passing `expectedSize` gets
|
|
an exact check against the frame's own declared size.
|
|
- Every failure is a `ZstdException`.
|
|
|
|
Correctness is measured against libzstd through pyzstd rather than against
|
|
expectations written by hand: 2,615 blocks of real elevation data across 32
|
|
containers, plus a committed corpus of 23 cases covering the format branches
|
|
that real data never reaches. 131 tests.
|
|
|
|
Roughly four times slower than native zstd. On a Pixel 10 Pro XL, profile AOT,
|
|
a 128 KiB block decodes in 0.414 ms, which is 302 MB/s.
|