Tufa Labs' tufazip has compressed enwik9, the standard 1 GB Wikipedia test file, down to 105,388,353 bytes without losing a bit. That's the total for version 2, including the decompressor, on Matt Mahoney's Large Text Compression Benchmark, which added the entry in an update dated November 19. Tufa Labs says it's the best result on the benchmark from a compressor that uses a neural network as its only predictor. The entries above it, by its account, combine a transformer with hand-built models.
Why it matters: compression and language modelling are the same problem. A model that predicts the next piece of text well can encode it in fewer bits. Tufazip is a clean test of that idea: no hand-tuned context mixing, just one transformer that learns while it compresses. It also shows what that costs.
How it works
According to the Tufa Labs write-up by Tommy He:
- Model: a 12-layer, decoder-only transformer with 163M parameters (RoPE, SwiGLU, RMSNorm), a 4K word vocabulary and a 192-token context.
- Online learning: the encoder and decoder each predict, code and update in lockstep. Because both sides learn the same weights from the same data, the weights learned along the way "never need to be transferred."
- Warm start (v2): instead of starting from scratch, v2 starts from a quantized offline prior: a 34M-parameter model stored in 4.33 MB and expanded into the 12-layer network.
- Replay: revisiting earlier text cut the payload from 107.22 MB to 105.92 MB in one experiment, at the cost of more compute on both sides.
The numbers
| Size (bytes) | |
|---|---|
| tufazip v2: compressed enwik9 | 100,958,324 |
| tufazip v2: decompressor | 4,430,029 |
| tufazip v2: total | 105,388,353 |
| tufazip 0.1.0 (v1): total | 105,979,069 |
| DNA-4: total (next entry below) | 105,554,360 |
| nncp v3.2: total | 107,261,318 |
| cmix v21: total | 108,244,767 |
v2 figures and neighbours: Large Text Compression Benchmark table. v1 figures: tufazip GitHub README.
The warm start saved 590,716 bytes over v1, as Glitchwire notes (our check of the two totals agrees). Glitchwire puts v2 at 0.843 bits per byte. Tufa Labs' post gives v1 as 0.848.
The cost: a week on two B200s
This isn't a compressor you'll run on a laptop. The GitHub repo requires Linux and exactly two Nvidia B200 GPUs (CUDA 13.0), and lists the test machine as an Nvidia DGX B200. Its submission table for v1 gives about 647,595 seconds to compress and 630,136 seconds to decompress, roughly 7.5 and 7.3 days, with peak memory of 68,525 MiB and 73,748 MiB respectively. The benchmark's v2 row lists the same two times and a 73,748 memory figure. Digit and Glitchwire give v2 slightly different timings (about 631,548 s and 645,748 s). Either way, it's about a week in each direction.
Where it actually ranks
The coverage disagrees here, so read headlines carefully:
- Tufa Labs' post says it "took #2" on the benchmark, dated to the time of its September 7 submission.
- Glitchwire says v2 "takes second place" behind fx2-cmix-transformer.
- In the benchmark's main table as we read it, tufazip v2 is the 13th row, and fx2-cmix-transformer (96,996,198 bytes) isn't first. ex-girlfriend 2.0 leads at 88,983,124. explainx says 16th.
What holds up across sources is the narrower claim: it's the best entry whose sole predictor is a neural network. Digit says the same thing, noting that tufazip "cannot claim the top position."
Limits Tufa Labs names
The write-up says the system is tuned to one fixed file, and acknowledges a Goodhart problem: optimising for enwik9 isn't the same as building a general compressor. Bigger didn't help in the settings it tested, either. A 267M-parameter model did worse on enwik9 than a 188M one.
There's also a gap in what's public. The GitHub repo, MIT-licensed with a single commit when we checked, documents a deterministic decoder for the v1 artifact (105,924,360 bytes), not the v2 result.
Who should care
If you work on LLM efficiency, tufazip is a useful data point: one transformer, learning online, now beats long-standing neural compressors like NNCP on a fixed text. If you need practical compression, nothing here changes your tools. The week of B200 time to decode a gigabyte is the point, not a footnote. Watch for whether Tufa Labs publishes the v2 artifact and code, and whether independent reruns reproduce the 105,388,353-byte figure.
Sources
- Tufa Labs — tufazip: Learning to Compress a Billion Bytes of Wikipedia tufalabs.ai
- Matt Mahoney — Large Text Compression Benchmark mattmahoney.net
- GitHub — Tufalabs/tufazip github.com
- Digit — Tufazip explained: The transformer that squeezes 1GB of Wikipedia into 105MB digit.in
- Glitchwire — Tufazip Compresses 1 GB of Wikipedia to 105 MB Using Only a Transformer glitchwire.com
- explainx — tufazip: 1 GB of Wikipedia to 105.39 MB, Lossless explainx.ai