halkyon-days/pure-zstd
Watch
1
0
Fork
You've already forked pure-zstd
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
pure-zstd/pubspec.yaml

25 lines
610 B
YAML
Raw Normal View History

Add a pure-Dart zstd decoder, as a benchmark A feasibility spike, standalone so it can move to its own repository. Not wired into the app and not a dependency of it. Decompression only, and only what our containers need: no dictionaries, no multi-frame, no streaming, and the content checksum is parsed but never verified. Native only, since the bit reader carries 56 bits in an int. It is correct on real data. The committed fixture decodes to the samples the generator wrote, and tool/oracle.py drives a byte-for-byte comparison against pyzstd over 32 containers and 2,615 blocks from both tiers, every thinned column count from 3601 to 361, and 88 S to 80 N. All identical. The numbers, AOT on this box, 131,072 bytes a block: 0.66 ms median against pyzstd's 0.175, so about 200 MB/s against 750. Across three tiles the range is 0.66 to 0.94 ms, and a 113 block sweep is 67 to 105 ms. The cost is the sequence loop and nothing else: a terrain block at level 19 carries around 1,700 literals and 14,400 sequences, so optimising the literal path bought nothing.
2026-09-21 21:37:56 -06:00
name: pure_zstd
Add the metadata pub.dev scores a package on description, repository, issue_tracker and topics, plus a changelog, which between them are most of a package's discoverability. The repository URLs are a guess at the new home and want checking before the first publish. publish_to stays none so an accidental publish from inside this checkout cannot succeed. Remove it in the new repository. Version 0.1.0 rather than 0.0.1: a published version can be retracted but never deleted, so the first one should mean something.
2026-09-21 23:45:05 -06:00
description: >-
A zstd decompressor written in plain Dart. Decodes RFC 8878 frames with no
native library and no platform channel, verified against libzstd.
version: 0.1.0
Bring the decoder over from the project it grew in Eighteen commits of history, grafted onto the scaffold rather than over it. It leaves behind everything that only meant something inside that project: paths under one machine's mount, references to a transcoder that is not here, and a benchmark defaulting to a tile nobody else has. One fixture is real terrain and stays, because it is the only small input that exercises the direct-nibble Huffman weight form. Its licence requires attribution to travel with it, so the README's acknowledgements and the corpus manifest both carry the Copernicus notice now. Everything else committed here is synthetic or was written by the zstd CLI, including the .hgtz container, whose name looks like a tile and whose samples are a function of their position.
2026-09-22 19:25:08 -06:00
repository: https://github.com/tagadvance/pure-zstd
issue_tracker: https://github.com/tagadvance/pure-zstd/issues
Add the metadata pub.dev scores a package on description, repository, issue_tracker and topics, plus a changelog, which between them are most of a package's discoverability. The repository URLs are a guess at the new home and want checking before the first publish. publish_to stays none so an accidental publish from inside this checkout cannot succeed. Remove it in the new repository. Version 0.1.0 rather than 0.0.1: a published version can be retracted but never deleted, so the first one should mean something.
2026-09-21 23:45:05 -06:00
Bring the decoder over from the project it grew in Eighteen commits of history, grafted onto the scaffold rather than over it. It leaves behind everything that only meant something inside that project: paths under one machine's mount, references to a transcoder that is not here, and a benchmark defaulting to a tile nobody else has. One fixture is real terrain and stays, because it is the only small input that exercises the direct-nibble Huffman weight form. Its licence requires attribution to travel with it, so the README's acknowledgements and the corpus manifest both carry the Copernicus notice now. Everything else committed here is synthetic or was written by the zstd CLI, including the .hgtz container, whose name looks like a tile and whose samples are a function of their position.
2026-09-22 19:25:08 -06:00
# Remove this line to publish. A published version can be retracted but never
# deleted, so it stays until the first release is deliberate.
Add a pure-Dart zstd decoder, as a benchmark A feasibility spike, standalone so it can move to its own repository. Not wired into the app and not a dependency of it. Decompression only, and only what our containers need: no dictionaries, no multi-frame, no streaming, and the content checksum is parsed but never verified. Native only, since the bit reader carries 56 bits in an int. It is correct on real data. The committed fixture decodes to the samples the generator wrote, and tool/oracle.py drives a byte-for-byte comparison against pyzstd over 32 containers and 2,615 blocks from both tiers, every thinned column count from 3601 to 361, and 88 S to 80 N. All identical. The numbers, AOT on this box, 131,072 bytes a block: 0.66 ms median against pyzstd's 0.175, so about 200 MB/s against 750. Across three tiles the range is 0.66 to 0.94 ms, and a 113 block sweep is 67 to 105 ms. The cost is the sequence loop and nothing else: a terrain block at level 19 carries around 1,700 literals and 14,400 sequences, so optimising the literal path bought nothing.
2026-09-21 21:37:56 -06:00
publish_to: none
Add the metadata pub.dev scores a package on description, repository, issue_tracker and topics, plus a changelog, which between them are most of a package's discoverability. The repository URLs are a guess at the new home and want checking before the first publish. publish_to stays none so an accidental publish from inside this checkout cannot succeed. Remove it in the new repository. Version 0.1.0 rather than 0.0.1: a published version can be retracted but never deleted, so the first one should mean something.
2026-09-21 23:45:05 -06:00
topics:
- zstd
- zstandard
- compression
- decompression
Add a pure-Dart zstd decoder, as a benchmark A feasibility spike, standalone so it can move to its own repository. Not wired into the app and not a dependency of it. Decompression only, and only what our containers need: no dictionaries, no multi-frame, no streaming, and the content checksum is parsed but never verified. Native only, since the bit reader carries 56 bits in an int. It is correct on real data. The committed fixture decodes to the samples the generator wrote, and tool/oracle.py drives a byte-for-byte comparison against pyzstd over 32 containers and 2,615 blocks from both tiers, every thinned column count from 3601 to 361, and 88 S to 80 N. All identical. The numbers, AOT on this box, 131,072 bytes a block: 0.66 ms median against pyzstd's 0.175, so about 200 MB/s against 750. Across three tiles the range is 0.66 to 0.94 ms, and a 113 block sweep is 67 to 105 ms. The cost is the sequence loop and nothing else: a terrain block at level 19 carries around 1,700 literals and 14,400 sequences, so optimising the literal path bought nothing.
2026-09-21 21:37:56 -06:00
environment:
sdk: ^3.8.0
dev_dependencies:
lints: ^6.0.0
test: ^1.25.0
Reference in a new issue Copy permalink