XZ Utils VS LZ4

You're having a breakdown. Relax. I'm just trying to have a conversation with you. Nothing personal to get emotional. Here are some resources. For tar, for gz, for xz, for webp, for 7z, for zstd, etc. But, some file extensions were developed for POSIX-like system and how do you expect them to work on windows 10 natively? Thankfully, the community has made it work even tho. It's like me crying over "can you run... Source: over 2 years ago

Thank you for the links but the https://tukaani.org/xz/ link essentially sent me back to python's lmza module and a slightly different lzmaffi library both of which don't seem to have options for reading the dictionary. I do understand this is low-level stuff coded in C/C++ but I wondered if any one made the abstraction to read this data also. Maybe I'll need to make a "script" to read this data... 😑️ this is... Source: almost 3 years ago

These libraries are themselves wrappers of low-level libraries, primarily for performance considerations (as they are almost always coded in C). If you really want to dig into the structure of those formats, you should rather investigate the possibilities of these low-level libraries. Like https://tukaani.org/xz/ for xz/lzma, or https://zlib.net/ for lzw. Source: almost 3 years ago

LZ4[1] intentionally compresses worse than Deflate—where Deflate is LZ77 with Huffman, LZ4 is just LZ77. It’s “we’re saturating Gigabit Ethernet and have some compute left over” compression, not “we need to squeeze our backups as much as possible” compression. If filtering+LZ4 does better than Deflate, then you’ve chosen the filter well. Curiously, for some kinds of binary serialization a good filter in... - Source: Hacker News / over 1 year ago

It doesn't destroy performance for the simple reason that nowadays memory access has higher latency than pure compute. If you need to use compute to produce some data to be stored in memory, your overall throughput could very well be faster than without compression. There have been a large amount of innovation on fast compression in recent years. Traditional compression tools like gzip or xz are geared towards... - Source: Hacker News / over 1 year ago

Just looked it up [1] > LZ4 is lossless compression algorithm, providing compression speed > 500 MB/s per core (>0.15 Bytes/cycle). It features an extremely fast decoder, with speed in multiple GB/s per core (~1 Byte/cycle). [1]: https://lz4.github.io/lz4/. - Source: Hacker News / over 1 year ago

Planet: weekly As you can see the planet file has the LZ4 extension, this is a compression-algorithm which a very good speed/size ratio and reduces the size with about 20-25 GB. It. Source: almost 2 years ago

I found this interesting dataLZ4 HC (r101) -9) increases transfer speed by about 22%, while only losing a 10% performance on decompression. That's a 12% performance increase overall, not bad! Source: about 2 years ago