5 ms·

Wavelets were supposed to be a big thing like 15-20 years ago ... what happened? Do any mainstream modern codecs use wavelets? I’m not aware of any Edit: looks

Wavelets were supposed to be a big thing like 15-20 years ago ... what happened? Do any mainstream modern codecs use wavelets? I’m not aware of any Edit: looks like the Dirac video codec is based on wavelets. Good to know all of that research wasn’t for nothing!

AceJohnny2 7y ago
I'm partly speaking out of my ass from second-hand experience 10 years ago but... Wavelet-based codecs turned out to be much harder to implement in hardware that fourier-transform-based ones (for which we have FFT!). That disincentivized development, which is why you only have Dirac (an open-source, non-HW-industry-backed effort) that uses it.
lp251 7y ago
JPEG2000, I believe.
- zyxzevn 7y ago
  Blocked by Patent-bullshit
  - userbinator 7y ago
    I believe either the patents have expired or the basic functionality was always royalty-free; but what really killed JPEG2000 was the immense complexity and increased processing power required. It still has applications in some niche areas, and probably most people's contact with it will be in the form of images embedded in PDFs (many of archive.org's ebooks use JPEG2000), where the noticeably slower rendering of pages is a big turn-off.
    - markus92 7y ago
      Also digital cinema is pretty much all JPEG2000 (every frame is individually compressed).
PorterDuff 7y ago
ImMix Cube products maybe? My memory fades.
goalieca 7y ago
Wavelets are the basis of modern video codecs. People also use them to watermark videos. I never found much advanced application in image processing which was my field of research.
- pavlov 7y ago
  There's no wavelet-based video codec in widespread use (unless one counts DCP, a file format used for physical deliveries of movies to theaters, where the picture track is encoded as JPEG2000 frames).
  - goalieca 7y ago
    H.264 is pretty widespread.
    - ksec 7y ago
      H.264 is not a wavelet-based video codec.
ad404b8a372f2b9 7y ago
I attended a talk by Stephane Mallat a few months back about the work his lab is doing on wavelet scattering transforms. It seems they're making great progress in developing deep neural networks based on them that don't need to be learned. It's not as hyped as mainstream deep learning methods but I think it holds a lot of promise since it cuts down on learning time, is mostly unsupervised, gives you control over the network's features and is a more theoretically principled way to build networks than just defining the loss and crossing your fingers while it's optimizing as we're doing right now (bit of an hyperbole). I could go on, it's absolutely fascinating. More info (in english & french): https://edouardoyallon.github.io/thesis.pdf https://edouardoyallon.github.io/thesis.pdf
- ska 7y ago
  I'm not familiar with this particular research direction. On the more general area Mallat has a good and approachable (not very technical) book on the area: "A Wavelet Tour of Signal Processing". Worth a look for anyone intrigued by wavelets.
- heyitsguay 7y ago
  My old research group was working on stuff relating to scattering network theory and application. The tl;dr is it's got nice math but isn't practically useful for much. The network construction also isn't as similar to, e.g., convnets as some of the scattering literature suggested.
rasz 7y ago
DjVu ebook format uses wavelet compression. From my experience its painfully slow.
vanderZwan 7y ago
Not a codec, but still image-related: one of the most powerful modules in Darktable is the equalizer, which is based on wavelets[0][1]. [0] http://www.darktable.org/2011/11/darktable-and-research/ http://www.darktable.org/2011/11/darktable-and-research/ [1] https://www.roberthutton.net/archives/203 https://www.roberthutton.net/archives/203
jhbadger 7y ago
There was even a whole pop-sci book -- Hubbard's "The World According to Wavelets" (1996) that was a bit like Mandelbrot's "The Fractal Geometry of Nature" in that it predicted an amazing future for the technique and claimed that science would be revolutionized by it. But it is nice to see that wavelets, like fractals, do have their uses even if the revolution never came.
zrav 7y ago
According to the x264 developers (can't find the reference anymore, sorry), wavelet based lossy compression is good for optimizing for PSNR, not for perceptual image/audio quality, being prone to loss of detail. x264 beats Dirac and modern image compression formats like HEIF (based on HEVC) easily beat JPEG2000. Modern codecs usually use transforms derived from the classic DCT and DST.
ksec 7y ago
The problem is that the economical model of Wavelet, or TCO of Wavelet codec doesn't do well. And like everything in out world it is only the cost and values on offers that matters. The total energy ( CPU cycles, or hardware implementation ) it requires to encode and decode far outweighs the benefits it offers. Not to mention the current block based still has many tricks. VVC ( H.266 ) seems to be doing well. ( Patents issues aside )
- TooSmugToFail 7y ago
  Do you happen to know what's the encoding time for a DWT based encoder vs H.264? Assuming the same hardware, of course... Are we talking about roughly the same encoding time, or the DWT is slower by an order of magnitude?

AceJohnny2 7y ago

I'm partly speaking out of my ass from second-hand experience 10 years ago but... Wavelet-based codecs turned out to be much harder to implement in hardware that fourier-transform-based ones (for which we have FFT!). That disincentivized development, which is why you only have Dirac (an open-source, non-HW-industry-backed effort) that uses it.

lp251 7y ago

JPEG2000, I believe.

zyxzevn 7y ago
Blocked by Patent-bullshit
- userbinator 7y ago
  I believe either the patents have expired or the basic functionality was always royalty-free; but what really killed JPEG2000 was the immense complexity and increased processing power required. It still has applications in some niche areas, and probably most people's contact with it will be in the form of images embedded in PDFs (many of archive.org's ebooks use JPEG2000), where the noticeably slower rendering of pages is a big turn-off.
  - markus92 7y ago
    Also digital cinema is pretty much all JPEG2000 (every frame is individually compressed).

PorterDuff 7y ago

ImMix Cube products maybe? My memory fades.

goalieca 7y ago

Wavelets are the basis of modern video codecs. People also use them to watermark videos. I never found much advanced application in image processing which was my field of research.

pavlov 7y ago
There's no wavelet-based video codec in widespread use (unless one counts DCP, a file format used for physical deliveries of movies to theaters, where the picture track is encoded as JPEG2000 frames).
- goalieca 7y ago
  H.264 is pretty widespread.
  - ksec 7y ago
    H.264 is not a wavelet-based video codec.

ad404b8a372f2b9 7y ago

I attended a talk by Stephane Mallat a few months back about the work his lab is doing on wavelet scattering transforms. It seems they're making great progress in developing deep neural networks based on them that don't need to be learned. It's not as hyped as mainstream deep learning methods but I think it holds a lot of promise since it cuts down on learning time, is mostly unsupervised, gives you control over the network's features and is a more theoretically principled way to build networks than just defining the loss and crossing your fingers while it's optimizing as we're doing right now (bit of an hyperbole). I could go on, it's absolutely fascinating. More info (in english & french): https://edouardoyallon.github.io/thesis.pdf https://edouardoyallon.github.io/thesis.pdf

ska 7y ago
I'm not familiar with this particular research direction. On the more general area Mallat has a good and approachable (not very technical) book on the area: "A Wavelet Tour of Signal Processing". Worth a look for anyone intrigued by wavelets.
heyitsguay 7y ago
My old research group was working on stuff relating to scattering network theory and application. The tl;dr is it's got nice math but isn't practically useful for much. The network construction also isn't as similar to, e.g., convnets as some of the scattering literature suggested.

rasz 7y ago

DjVu ebook format uses wavelet compression. From my experience its painfully slow.

vanderZwan 7y ago

Not a codec, but still image-related: one of the most powerful modules in Darktable is the equalizer, which is based on wavelets[0][1]. [0] http://www.darktable.org/2011/11/darktable-and-research/ http://www.darktable.org/2011/11/darktable-and-research/ [1] https://www.roberthutton.net/archives/203 https://www.roberthutton.net/archives/203

jhbadger 7y ago

There was even a whole pop-sci book -- Hubbard's "The World According to Wavelets" (1996) that was a bit like Mandelbrot's "The Fractal Geometry of Nature" in that it predicted an amazing future for the technique and claimed that science would be revolutionized by it. But it is nice to see that wavelets, like fractals, do have their uses even if the revolution never came.

zrav 7y ago

According to the x264 developers (can't find the reference anymore, sorry), wavelet based lossy compression is good for optimizing for PSNR, not for perceptual image/audio quality, being prone to loss of detail. x264 beats Dirac and modern image compression formats like HEIF (based on HEVC) easily beat JPEG2000. Modern codecs usually use transforms derived from the classic DCT and DST.

ksec 7y ago

The problem is that the economical model of Wavelet, or TCO of Wavelet codec doesn't do well. And like everything in out world it is only the cost and values on offers that matters. The total energy ( CPU cycles, or hardware implementation ) it requires to encode and decode far outweighs the benefits it offers. Not to mention the current block based still has many tricks. VVC ( H.266 ) seems to be doing well. ( Patents issues aside )

TooSmugToFail 7y ago
Do you happen to know what's the encoding time for a DWT based encoder vs H.264? Assuming the same hardware, of course... Are we talking about roughly the same encoding time, or the DWT is slower by an order of magnitude?