5 ms·

Good luck trying to send a file compressed with Zstd or Brotli to people and have them know what to do with them. I’d certainly have to google to figure out how

by mikl 5y ago

Good luck trying to send a file compressed with Zstd or Brotli to people and have them know what to do with them. I’d certainly have to google to figure out how to open one of those. And I’m an engineer and aware that those formats exist.

deleted 5y ago
[deleted]
- deleted 5y ago
  [deleted]
lifthrasiir 5y ago
Zstandard and Brotli are also not an archiving format, which the layperson normally expects to be a "compression" format, so the comparison is a bit off. But that doesn't really matter because there is 7-zip which is an archiving format AND compression format AND competitive to RAR AND open-source.
- ramraj07 5y ago
  But slower.
  - lifthrasiir 5y ago
    It depends on which time you measure. 7zip's LZMA2 is, like most LZ algorithms, asymmetric that it is quite fast to decompress no matter how much you spend on the compression. By comparison RAR uses PPM in the highest setting, which is more or less symmetric so the compression and decompression takes about the same amount of time. When the target ratio is roughly same and high enough, LZMA2 is indeed slower than (RAR's implementation of) PPM for the compression but much faster for the decompression.
    - account42 5y ago
      > It depends on which time you measure. 7zip's LZMA2 is, like most LZ algorithms, asymmetric that it is quite fast to decompress no matter how much you spend on the compression. In fact, better compression with LZ formats often means faster decompression (at least compared to other settings of the same format) as there is less compressed data to pass through the expensive entropy coder (which itself has symmetric cost for encoding and decoding).
    - ComputerGuru 5y ago
      There is no world in which LZMA2 decompression can be considered “quite fast.” It’s decompression speed bears no resemblance to that of other LZ algorithms like LZ4 (which is definitely fast). https://www.opencpu.org/posts/brotli-benchmarks/ https://www.opencpu.org/posts/brotli-benchmarks/ Distributions have stopped using xz/lzma in favor of slightly inferior (size-wise) alternatives due to the time and memory decompression requirements of lzma.
- mikl 5y ago
  Yeah, you’d need to use something like *nix `tar` to do the archiving part. I think newer versions of tar already have support for Zstd.
  - account42 5y ago
    > I think newer versions of tar already have support for Zstd. You don't really need to have support for a particular compression fromat in the tar command since the tar format does not support compression at all and instead the output of tar is passed through a compressor such as gzip or xz that has its own encapsulation format. All the support in tar does is add switch to start that (de)compressor for you and, for GNU tar, to detect it based on the file extensions.
  - 2ion 5y ago
    a tar.something is just a tar archive passed through a compression filter like zstd or gzip. Anyway, tar needs metadata extensions in order to allow random access; most tarballs you'll find don't. So you need to read 5TB of data to find the file at point 5TB in an archive of size 10TB. Which is why as a general purpose archiving format, tar needs to be applied to use cases where it works. It gets old really quick if you don't have random access and random access requirements, plus, remediated by SSDs these days for a bit, it's really time consuming.
em500 5y ago
You won't be able to do that with RAR files either. The only archive/compression format that you can send anyone is ZIP, since both Windows and macOS can open them ootb.
- wongarsu 5y ago
  I wouldn't send a RAR to my grandparents, but if I had to send pictures for a school reunion I would expect everyone to either know how to open a RAR or to be able to figure it out. Worst case they will google rar and will find WinRAR as the first result. For zstd or brotli I can't even name a GUI program able to open them outside a somewhat obscure 7zip fork.
  - nix23 5y ago
    Why would you try to compress pictures....in your case zip is as good as everything else (and less problematic to open)
    - lifthrasiir 5y ago
      Or if your friends all have WinZip and can tolerate .zipx (which is a file extension for later and thus less interoperable version of ZIP), it does support JPEG recompression so it can be actually better than RAR.
      - anthk 5y ago
        Any lossy compression format it's crap.
        lifthrasiir 5y ago
        No, JPEG recompression I meant is not lossy here; it is essentially taking the internal JPEG data structure and losslessly compressing them in more modern techniques. If you want there is also an open source algorithm [1] developed and used by Dropbox. [1] https://github.com/dropbox/lepton https://github.com/dropbox/lepton

by mikl 5y ago

deleted 5y ago

[deleted]

deleted 5y ago
[deleted]

lifthrasiir 5y ago

Zstandard and Brotli are also not an archiving format, which the layperson normally expects to be a "compression" format, so the comparison is a bit off. But that doesn't really matter because there is 7-zip which is an archiving format AND compression format AND competitive to RAR AND open-source.

ramraj07 5y ago
But slower.
- lifthrasiir 5y ago
  It depends on which time you measure. 7zip's LZMA2 is, like most LZ algorithms, asymmetric that it is quite fast to decompress no matter how much you spend on the compression. By comparison RAR uses PPM in the highest setting, which is more or less symmetric so the compression and decompression takes about the same amount of time. When the target ratio is roughly same and high enough, LZMA2 is indeed slower than (RAR's implementation of) PPM for the compression but much faster for the decompression.
  - account42 5y ago
    > It depends on which time you measure. 7zip's LZMA2 is, like most LZ algorithms, asymmetric that it is quite fast to decompress no matter how much you spend on the compression. In fact, better compression with LZ formats often means faster decompression (at least compared to other settings of the same format) as there is less compressed data to pass through the expensive entropy coder (which itself has symmetric cost for encoding and decoding).
  - ComputerGuru 5y ago
    There is no world in which LZMA2 decompression can be considered “quite fast.” It’s decompression speed bears no resemblance to that of other LZ algorithms like LZ4 (which is definitely fast). https://www.opencpu.org/posts/brotli-benchmarks/ https://www.opencpu.org/posts/brotli-benchmarks/ Distributions have stopped using xz/lzma in favor of slightly inferior (size-wise) alternatives due to the time and memory decompression requirements of lzma.
mikl 5y ago
Yeah, you’d need to use something like *nix `tar` to do the archiving part. I think newer versions of tar already have support for Zstd.
- account42 5y ago
  > I think newer versions of tar already have support for Zstd. You don't really need to have support for a particular compression fromat in the tar command since the tar format does not support compression at all and instead the output of tar is passed through a compressor such as gzip or xz that has its own encapsulation format. All the support in tar does is add switch to start that (de)compressor for you and, for GNU tar, to detect it based on the file extensions.
- 2ion 5y ago
  a tar.something is just a tar archive passed through a compression filter like zstd or gzip. Anyway, tar needs metadata extensions in order to allow random access; most tarballs you'll find don't. So you need to read 5TB of data to find the file at point 5TB in an archive of size 10TB. Which is why as a general purpose archiving format, tar needs to be applied to use cases where it works. It gets old really quick if you don't have random access and random access requirements, plus, remediated by SSDs these days for a bit, it's really time consuming.

em500 5y ago

You won't be able to do that with RAR files either. The only archive/compression format that you can send anyone is ZIP, since both Windows and macOS can open them ootb.

wongarsu 5y ago
I wouldn't send a RAR to my grandparents, but if I had to send pictures for a school reunion I would expect everyone to either know how to open a RAR or to be able to figure it out. Worst case they will google rar and will find WinRAR as the first result. For zstd or brotli I can't even name a GUI program able to open them outside a somewhat obscure 7zip fork.
- nix23 5y ago
  Why would you try to compress pictures....in your case zip is as good as everything else (and less problematic to open)
  - lifthrasiir 5y ago
    Or if your friends all have WinZip and can tolerate .zipx (which is a file extension for later and thus less interoperable version of ZIP), it does support JPEG recompression so it can be actually better than RAR.
    - anthk 5y ago
      Any lossy compression format it's crap.
      - lifthrasiir 5y ago
        No, JPEG recompression I meant is not lossy here; it is essentially taking the internal JPEG data structure and losslessly compressing them in more modern techniques. If you want there is also an open source algorithm [1] developed and used by Dropbox. [1] https://github.com/dropbox/lepton https://github.com/dropbox/lepton