6 ms·

New Solution Can Cut Video File Size By Half – Without Losing Quality

82 points by jackau 14y ago

lucian1900 14y ago
I'm a bit sceptical of such glorious claims from a company that apparently just made a UI for a libjpeg.
- pornel 14y ago
  AFAIK JPEGmini has much better psychovisual optimization than libjpeg, so I think their claim that they can compress JPEG better within limits of the format is fair. The way you encode lossy formats makes a lot of difference. For example the same x264 encoder gives dramatically worse results when aiming for optimal PSNR rather than using its psychovisual optimizer: • http://x264.nl/developers/Dark_Shikari/imagecoding/x264.png http://x264.nl/developers/Dark_Shikari/imagecoding/x264.png • http://x264.nl/developers/Dark_Shikari/imagecoding/x264_psnr.png http://x264.nl/developers/Dark_Shikari/imagecoding/x264_psnr... http://x264dev.multimedia.cx/archives/541 http://x264dev.multimedia.cx/archives/541
  - ZeroGravitas 14y ago
    Is it not a bit ironic to be linking to x264 developers to support the thesis that a new H.264 encoder can be twice as effective in compressing H.264 than the competition? Either they've done something magical and can outcompress x264 to that degree (i.e. in the ballpark of HEVC, yet without breaking compatability), or they're comparing against a terrible H.264 encoder and they've just wrapped x264 in a proprietary shell of BS (and/or they're liars?).
    - jfb 14y ago
      It's undoubtedly bullshit.
batterseapower 14y ago
Previously on HN: http://news.ycombinator.com/item?id=2940505 http://news.ycombinator.com/item?id=2940505
- jackau 14y ago
  It is true that it's the same company, however, the link you're referring to is about image compression and not video compression.
- runn1ng 14y ago
  So, in the case of jpegs, their compression is not lossless, they just claim the losses are not that perceptually visible. Apart from the strong PR claims, this is actually not that bad, no? If you have two lossy compressions, but one is perceptually better, you will rather use the perceptually better one, no?
  - niggler 14y ago
    I see this in the same way that apple pitches the retina display (where supposedly you wouldn't be able to more finely resolve pixels from some standard distance).
LatvjuAvs 14y ago
Of course, until you compare 1by1 pixel, you see no difference. But when you do, yeah, I understand why size drops is that big.
midvar 14y ago
Especially since this states H.264 compliance, I'd be willing to say this can't possibly live up to it's claims. Video encoders are notorious for their bloated claims and faulty comparisons. After a quick perusal of their site I see they compare themselves only to web-m, and use generalized %'s to claim differences between products and processes. They say they've optimized for human perceptual quality. Perhaps they have. But did they really do a better job than the open source H.264 alternative (x264) used by everyone from YouTube and Hulu, to Facebook? One thats been in development for _years_ now and beats just about everything thrown at it in a fair comparison? I think not. I'd love to be wrong though. When they do eventually come out with comparisons, I hope they have read this first : http://x264dev.multimedia.cx/archives/472 http://x264dev.multimedia.cx/archives/472
judofyr 14y ago
What encoder did they compare it with?
mistercow 14y ago
>the compression method mimics the human eye and removes elements that would not have been processed by the human eye in the first place. That's just a marketing hype decsription of how all lossy compression works.
- Hari_Seldon 14y ago
  Agreed, reminds me of mp3. >Elements of the recordings that are easily perceived are represented with exacting precision, while other parts that are not very audible can be represented less accurately. Meanwhile, inaudible information can be discarded altogether. Fraunhofer Institute
- Xcelerate 14y ago
  I wrote a JPEG encoder once, and if I remember correctly you can change the quantization tables and the Huffman tables. My encoder just used the default tables in the standard, but it wouldn't surprise me to optimize these using a psychovisual fitness function.
  - mistercow 14y ago
    Yes, but the default tables in the standard do mimic the human eye's limitations. That's what increasing the quant factor on higher frequencies is.
josephlord 14y ago
Incremental improvements in H.264 are still possible but not a halving of bitrate in the general case. The required rate depends very much on the content of the video and the size at which it will be viewed. Any encoder manufacturer can find some great looking and easy to encode images that can be massively reduced in data rate without much visible impact. However in tougher scenarios they will be unlikely to beat the existing market leaders by more than 10% or so under similar constraints (latentcy/CPU/multi-pass possible) under the review of experts. Tough scenarios will be when there is lots of noise and non-uniform movement. Pop concerts can be very challenging with crowd movements, flashing lights and low general light levels leading to noise.
- Tloewald 14y ago
  The more detailed claim is 5-6Mbps streams can be reduced to 3-4, so it's more like a 25% improvement methinks. Their first claimed success was with JPEGmini — this is a lot more credible since JPEG is a pretty obsolete format that simply continues because it's so well supported. JPEG2000 delivers the purported benefits of JPEGmini for real -- it's just poorly supported. I suspect all they're really doing is fine-tuning compression settings based on differencing input and output using a decent perceptual algorithm. I uploaded a photo straight out of a Nikon V1 onto JPEGmini and got a 2.2x reduction. I could do better than that by dialing down quality until I saw a difference. JPEG fine is way overkill for most purposes, and that's their market.
huhtenberg 14y ago
For the lazy - http://jpegmini.com http://jpegmini.com -- I grabbed 4th pair of their demo image [0], the high quality image is about 4.8M, the JpegMini'd version - 1.3M. Then I took the high quality image and started bumping up standard JPEG compression level until I reduced the file size to 1.3M. That mapped to 17/100 setting, with 1 being highest quality, 100 - highest compression). Then I made an arithmetic per-pixel diff between the original high quality image and two 1.3M versions. Cropped them to 2000 x 2000 and saved as PNGs (cropping is to fit the imgur file size limits). Here they are - http://imgur.com/1FOwXhq http://imgur.com/1FOwXhq http://imgur.com/3efRooj http://imgur.com/3efRooj Now, have a look and tell me that JPEGMini hasn't got something exciting going on. -- [0] http://media.jpegmini.com/homepageImages/20121203/hipnshoot.zip http://media.jpegmini.com/homepageImages/20121203/hipnshoot....
- Xcelerate 14y ago
  Haha, I only kind of skimmed your post then downloaded that file. I opened the images and was like "I don't know how high-quality this is. The images look exactly the same to me." And then I saw the file sizes and had the ah-ha moment.
- deadairspace 14y ago
  I don't see much of a difference. Is that the point? Are you saying that JPEGMini are just compressing to ~83% quality?
  - huhtenberg 14y ago
    No, there is a noticeable difference. JPEGMini diff shows less color fringing (fewer "colored" pixels) and it also appears to better preserve edges and outlines.
- ricardobeat 14y ago
  JPEGmini better preserves color information, but it doesn't seem to be that much better.
- conroe64 14y ago
kierank 14y ago
They use x264 on near-default settings.
lutusp 14y ago
Let them put up a long, complex video -- an action movie, for example -- using H.264, and their own method, for side-by-side comparison, both with regard to quality and file size. Let them offer something better than hand-waving press releases. I say this because claims like this tend to be very source-dependent. Change the source, change the claim.
- Tuna-Fish 14y ago
  They appear to be using modern x264. This does outperform most (all?) commercial H.264 encoders to the degree they claim, so their claims can have merit. However, this also means that if you want better encoding, you can just grab x264 and do it yourself.
  - lutusp 14y ago
    > They appear to be using modern x264. This does outperform most (all?) commercial H.264 encoders to the degree they claim, so their claims can have merit. Yes, but it also means they're not entitled to call it a "patent pending" method.
gmartres 14y ago
From beamrvideo.com, Clip1_mini.mov contains: videomini ver. 90.15 options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=umh subme=8 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 ip_ratio=1.41 aq=1:1.00 And Clip2_mini.mov contains: videomini ver. 89.15 options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=umh subme=8 psy=0 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 ip_ratio=1.40 aq=1:1.00 For comparaison, here's what a clip encoded with x264 contains: x264 - core 125 r2200 999b753 - H.264/MPEG-4 AVC codec - Copyleft 2003-2012 - http://www.videolan.org/x264.html http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=5 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=2 keyint=240 keyint_min=23 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=2pass mbtree=1 bitrate=373 ratetol=1.0 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 cplxblur=20.0 qblur=0.5 ip_ratio=1.40 aq=1:1.00 So their encoder is based on x264 (without credit). The settings are slightly different between the clips, so maybe they have some magic thing to detect which settings to use. In any case, it'd be interesting to reencode the original clip using plain x264, once using the same options they used and once using some sane x264 preset.
- Daiz 14y ago
  >In any case, it'd be interesting to reencode the original clip using plain x264, once using the same options they used and once using some sane x264 preset. I'm up for that. It'll take a while to download all the source videos, but I'll report back with results and videos when I'm done. I already did a test encode with x264 --preset veryslow --tune film --level 4.0 --crf 21 on the Bourne Identity clip, and the result is within the same realm than the Beamr video with ~58% the filesize. Which is not surprising at all, considering Beamr's settings. EDIT: Testing with nearly identical settings (x264 --bframes 0 -- subme 8 --no-psy --no-mbtree) using the latest x264 gives me about 40% the bitrate of the Beamr example for the Bourne Identity clip, which unsurprisingly looks notably worse. So maybe Beamr's "magical" encoding technology is all about fiddling with x264's CRF to make it double the bitrate at a given value compared to vanilla x264?
Daiz 14y ago
As others have noted, Beamr's "magical" technology simply seems to be using x264, the state-of-the-art in H.264 encoders, which also happens to be free and open source software. Looking at their site, beamrvideo.com, the bitrates they list for the originals are what you'll typically find on Blu-ray discs, not on online streams. Anyone who knows a thing or two about video encoding and x264 will know that you can re-encode these to a much smaller size while retaining most of the visual quality. As such, claiming that they can reduce the bitrate to half with no notable loss in quality is like claiming that you can reduce the bit depth and sampling rate of a 24-bit 192 kHz audio file to 16-bit 48 kHz with no notable loss in quality - it's certainly true, but you hardly needed the 24-bit 192 kHz quality to begin with. They don't seem to have done anything new or substantial by themselves, and as such, this thing is basically nothing but pure marketing bullshit. If you want to do high quality H.264 encodes, just use x264 directly.
- deleted 14y ago
  [deleted]
ZeroGravitas 14y ago
So, since it's seem to be fairly well established that they're just using x264, anyone got any idea how they plan on making money from this, er, product/service they're offering?
beagle3 14y ago
Just using x264 with veryslow and/or two pass setting usually cuts file sizes by half compared to e.g. an IP or phone camera output. They could have some magic there, but you can get similar results at home without having to pay anyone. You just have to read a little about how to run x264. x264 is awesome. super awesome, even.

82 points by jackau 14y ago

lucian1900 14y ago

I'm a bit sceptical of such glorious claims from a company that apparently just made a UI for a libjpeg.

pornel 14y ago
AFAIK JPEGmini has much better psychovisual optimization than libjpeg, so I think their claim that they can compress JPEG better within limits of the format is fair. The way you encode lossy formats makes a lot of difference. For example the same x264 encoder gives dramatically worse results when aiming for optimal PSNR rather than using its psychovisual optimizer: • http://x264.nl/developers/Dark_Shikari/imagecoding/x264.png http://x264.nl/developers/Dark_Shikari/imagecoding/x264.png • http://x264.nl/developers/Dark_Shikari/imagecoding/x264_psnr.png http://x264.nl/developers/Dark_Shikari/imagecoding/x264_psnr... http://x264dev.multimedia.cx/archives/541 http://x264dev.multimedia.cx/archives/541
- ZeroGravitas 14y ago
  Is it not a bit ironic to be linking to x264 developers to support the thesis that a new H.264 encoder can be twice as effective in compressing H.264 than the competition? Either they've done something magical and can outcompress x264 to that degree (i.e. in the ballpark of HEVC, yet without breaking compatability), or they're comparing against a terrible H.264 encoder and they've just wrapped x264 in a proprietary shell of BS (and/or they're liars?).
  - jfb 14y ago
    It's undoubtedly bullshit.

batterseapower 14y ago

Previously on HN: http://news.ycombinator.com/item?id=2940505 http://news.ycombinator.com/item?id=2940505

jackau 14y ago
It is true that it's the same company, however, the link you're referring to is about image compression and not video compression.
runn1ng 14y ago
So, in the case of jpegs, their compression is not lossless, they just claim the losses are not that perceptually visible. Apart from the strong PR claims, this is actually not that bad, no? If you have two lossy compressions, but one is perceptually better, you will rather use the perceptually better one, no?
- niggler 14y ago
  I see this in the same way that apple pitches the retina display (where supposedly you wouldn't be able to more finely resolve pixels from some standard distance).

LatvjuAvs 14y ago

Of course, until you compare 1by1 pixel, you see no difference. But when you do, yeah, I understand why size drops is that big.

midvar 14y ago

Especially since this states H.264 compliance, I'd be willing to say this can't possibly live up to it's claims. Video encoders are notorious for their bloated claims and faulty comparisons. After a quick perusal of their site I see they compare themselves only to web-m, and use generalized %'s to claim differences between products and processes. They say they've optimized for human perceptual quality. Perhaps they have. But did they really do a better job than the open source H.264 alternative (x264) used by everyone from YouTube and Hulu, to Facebook? One thats been in development for _years_ now and beats just about everything thrown at it in a fair comparison? I think not. I'd love to be wrong though. When they do eventually come out with comparisons, I hope they have read this first : http://x264dev.multimedia.cx/archives/472 http://x264dev.multimedia.cx/archives/472

judofyr 14y ago

What encoder did they compare it with?

mistercow 14y ago

>the compression method mimics the human eye and removes elements that would not have been processed by the human eye in the first place. That's just a marketing hype decsription of how all lossy compression works.

Hari_Seldon 14y ago
Agreed, reminds me of mp3. >Elements of the recordings that are easily perceived are represented with exacting precision, while other parts that are not very audible can be represented less accurately. Meanwhile, inaudible information can be discarded altogether. Fraunhofer Institute
Xcelerate 14y ago
I wrote a JPEG encoder once, and if I remember correctly you can change the quantization tables and the Huffman tables. My encoder just used the default tables in the standard, but it wouldn't surprise me to optimize these using a psychovisual fitness function.
- mistercow 14y ago
  Yes, but the default tables in the standard do mimic the human eye's limitations. That's what increasing the quant factor on higher frequencies is.

josephlord 14y ago

Incremental improvements in H.264 are still possible but not a halving of bitrate in the general case. The required rate depends very much on the content of the video and the size at which it will be viewed. Any encoder manufacturer can find some great looking and easy to encode images that can be massively reduced in data rate without much visible impact. However in tougher scenarios they will be unlikely to beat the existing market leaders by more than 10% or so under similar constraints (latentcy/CPU/multi-pass possible) under the review of experts. Tough scenarios will be when there is lots of noise and non-uniform movement. Pop concerts can be very challenging with crowd movements, flashing lights and low general light levels leading to noise.

Tloewald 14y ago
The more detailed claim is 5-6Mbps streams can be reduced to 3-4, so it's more like a 25% improvement methinks. Their first claimed success was with JPEGmini — this is a lot more credible since JPEG is a pretty obsolete format that simply continues because it's so well supported. JPEG2000 delivers the purported benefits of JPEGmini for real -- it's just poorly supported. I suspect all they're really doing is fine-tuning compression settings based on differencing input and output using a decent perceptual algorithm. I uploaded a photo straight out of a Nikon V1 onto JPEGmini and got a 2.2x reduction. I could do better than that by dialing down quality until I saw a difference. JPEG fine is way overkill for most purposes, and that's their market.

huhtenberg 14y ago

For the lazy - http://jpegmini.com http://jpegmini.com -- I grabbed 4th pair of their demo image [0], the high quality image is about 4.8M, the JpegMini'd version - 1.3M. Then I took the high quality image and started bumping up standard JPEG compression level until I reduced the file size to 1.3M. That mapped to 17/100 setting, with 1 being highest quality, 100 - highest compression). Then I made an arithmetic per-pixel diff between the original high quality image and two 1.3M versions. Cropped them to 2000 x 2000 and saved as PNGs (cropping is to fit the imgur file size limits). Here they are - http://imgur.com/1FOwXhq http://imgur.com/1FOwXhq http://imgur.com/3efRooj http://imgur.com/3efRooj Now, have a look and tell me that JPEGMini hasn't got something exciting going on. -- [0] http://media.jpegmini.com/homepageImages/20121203/hipnshoot.zip http://media.jpegmini.com/homepageImages/20121203/hipnshoot....

Xcelerate 14y ago
Haha, I only kind of skimmed your post then downloaded that file. I opened the images and was like "I don't know how high-quality this is. The images look exactly the same to me." And then I saw the file sizes and had the ah-ha moment.
deadairspace 14y ago
I don't see much of a difference. Is that the point? Are you saying that JPEGMini are just compressing to ~83% quality?
- huhtenberg 14y ago
  No, there is a noticeable difference. JPEGMini diff shows less color fringing (fewer "colored" pixels) and it also appears to better preserve edges and outlines.
ricardobeat 14y ago
JPEGmini better preserves color information, but it doesn't seem to be that much better.
conroe64 14y ago

kierank 14y ago

They use x264 on near-default settings.

lutusp 14y ago

Let them put up a long, complex video -- an action movie, for example -- using H.264, and their own method, for side-by-side comparison, both with regard to quality and file size. Let them offer something better than hand-waving press releases. I say this because claims like this tend to be very source-dependent. Change the source, change the claim.

Tuna-Fish 14y ago
They appear to be using modern x264. This does outperform most (all?) commercial H.264 encoders to the degree they claim, so their claims can have merit. However, this also means that if you want better encoding, you can just grab x264 and do it yourself.
- lutusp 14y ago
  > They appear to be using modern x264. This does outperform most (all?) commercial H.264 encoders to the degree they claim, so their claims can have merit. Yes, but it also means they're not entitled to call it a "patent pending" method.

gmartres 14y ago

From beamrvideo.com, Clip1_mini.mov contains: videomini ver. 90.15 options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=umh subme=8 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 ip_ratio=1.41 aq=1:1.00 And Clip2_mini.mov contains: videomini ver. 89.15 options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=umh subme=8 psy=0 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 ip_ratio=1.40 aq=1:1.00 For comparaison, here's what a clip encoded with x264 contains: x264 - core 125 r2200 999b753 - H.264/MPEG-4 AVC codec - Copyleft 2003-2012 - http://www.videolan.org/x264.html http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=5 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=2 keyint=240 keyint_min=23 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=2pass mbtree=1 bitrate=373 ratetol=1.0 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 cplxblur=20.0 qblur=0.5 ip_ratio=1.40 aq=1:1.00 So their encoder is based on x264 (without credit). The settings are slightly different between the clips, so maybe they have some magic thing to detect which settings to use. In any case, it'd be interesting to reencode the original clip using plain x264, once using the same options they used and once using some sane x264 preset.

Daiz 14y ago
>In any case, it'd be interesting to reencode the original clip using plain x264, once using the same options they used and once using some sane x264 preset. I'm up for that. It'll take a while to download all the source videos, but I'll report back with results and videos when I'm done. I already did a test encode with x264 --preset veryslow --tune film --level 4.0 --crf 21 on the Bourne Identity clip, and the result is within the same realm than the Beamr video with ~58% the filesize. Which is not surprising at all, considering Beamr's settings. EDIT: Testing with nearly identical settings (x264 --bframes 0 -- subme 8 --no-psy --no-mbtree) using the latest x264 gives me about 40% the bitrate of the Beamr example for the Bourne Identity clip, which unsurprisingly looks notably worse. So maybe Beamr's "magical" encoding technology is all about fiddling with x264's CRF to make it double the bitrate at a given value compared to vanilla x264?

Daiz 14y ago

As others have noted, Beamr's "magical" technology simply seems to be using x264, the state-of-the-art in H.264 encoders, which also happens to be free and open source software. Looking at their site, beamrvideo.com, the bitrates they list for the originals are what you'll typically find on Blu-ray discs, not on online streams. Anyone who knows a thing or two about video encoding and x264 will know that you can re-encode these to a much smaller size while retaining most of the visual quality. As such, claiming that they can reduce the bitrate to half with no notable loss in quality is like claiming that you can reduce the bit depth and sampling rate of a 24-bit 192 kHz audio file to 16-bit 48 kHz with no notable loss in quality - it's certainly true, but you hardly needed the 24-bit 192 kHz quality to begin with. They don't seem to have done anything new or substantial by themselves, and as such, this thing is basically nothing but pure marketing bullshit. If you want to do high quality H.264 encodes, just use x264 directly.

deleted 14y ago
[deleted]

ZeroGravitas 14y ago

So, since it's seem to be fairly well established that they're just using x264, anyone got any idea how they plan on making money from this, er, product/service they're offering?

beagle3 14y ago

Just using x264 with veryslow and/or two pass setting usually cuts file sizes by half compared to e.g. an IP or phone camera output. They could have some magic there, but you can get similar results at home without having to pay anyone. You just have to read a little about how to run x264. x264 is awesome. super awesome, even.