4 ms·

This glosses over the best part: the quantized DCT is where the magic happens. Basically the 8x8 block is converted in to one of a handful (64) different possib

This glosses over the best part: the quantized DCT is where the magic happens. Basically the 8x8 block is converted in to one of a handful (64) different possible choices, they look kinda like electron orbitals, modulated by coefficients. I didn't understand JPEG until I saw this matrix of choices and then a light went on and I understand why the DCT was necessary. Basically the entire image is reduced down to a finite collection of pre-computed shapes. https://image.slidesharecdn.com/jpeg-dct-131006054709-phpapp01/95/jpeg-dct-12-638.jpg?cb=1381039084 https://image.slidesharecdn.com/jpeg-dct-131006054709-phpapp...

jarenmf 5y ago
That's interesting, I'm a bit confused, in this case high frequencies are not removed they are just quantized like all other frequencies. How is this reconciled with the assumption that high frequencies carry little information?
- tinus_hn 5y ago
  Where is the contradiction? As far as the theory goes, the transform using the DCT is lossless apart from rounding errors. The quantization step just rounds more roughly. Which is where the lossy compression happens. Basically you’re not removing the high frequencies because they carry little information, you remove the information they carry. If you just remove the high frequency you end up with a blurred, smooth picture that doesn’t look like the original. But if you quantize the strength you get a pretty similar looking picture, apart from the degenerate case of sharp edges in a drawing.
  - magicalhippo 5y ago
    > If you just remove the high frequency you end up with a blurred, smooth picture that doesn’t look like the original. Well it looks like a blurred, smoothed picture. This can be exploited during decoding if you want a smaller size than the original. For example if you only decode the 2x2 lower-frequency components of a 2000x1000 image, you get a nice downsampled 500x250 image "for free". For example when making thumbnails this can be very useful, as for larger images you can often use just 2x2 or 1x1, dramatically cutting down the work needed for the final resample.
    - tinus_hn 5y ago
      True, similarly I’ve always wondered whether DJ mixing tools directly decode the mp3s at the pitch they are going to play. No idea if that is practical though.
- NavinF 5y ago
  The coefficients get quantized (rounded to the nearest integer) after dividing by 2^n. That way, each of those numbers occupy n fewer bits on disk. You wanna use a larger n for the high-frequency coefficients since they’re less important. (Except when that assumption breaks, which is fairly often: JPEG compressed screenshots of text is unreadable because text is exclusively composed of sharp high-frequency edges) Oh and you can figure out what camera/software took a photo by looking at the value of n for each frequency. Everyone does it slightly differently to achieve different compression ratios.
tomashubelbauer 5y ago
I agree, I had the same click happen when I saw this set of shapes in a YouTube video on how JPEG works: https://www.youtube.com/watch?v=Kv1Hiv3ox8I https://www.youtube.com/watch?v=Kv1Hiv3ox8I.
- SavantIdiot 5y ago
  This video is really well done. I worked on optimizing the Xing video player for Pentium MMX ages ago (almost 30 years?!), and had to figure this out by reading the same one book on the topic over and over again. I never quite got the details, but this video would have saved me weeks of study. Pre-internet days sucked for learning complex topics.
Gravityloss 5y ago
This is a good way to get an insight on almost all transformations.
mistercow 5y ago
The way you’ve phrased this is not correct. Those aren’t “choices”, but basis functions. Each 8x8 block is broken into a sum of all of those basis functions, with each basis function multiplied by a coefficient. The output of the DCT is that list of coefficients. The quantization part is where the magic happens, but it’s not about choosing between those patterns. It’s about “snapping” the coefficients to fixed steps. Basically, the human visual cortex is bad at noticing subtle differences at higher frequencies (the bottom right area of your linked image). You can’t easily tell if the coefficient for that bottom right pattern is, say 100 or 103. So JPEG stores those coefficients with lower precision, which requires fewer bytes after the entropy coding step.
- SavantIdiot 5y ago
  Thanks, now I understand it even better!
- jarenmf 5y ago
  Essentially higher frequencies are represented using fewer number of shades and many of them will go to zero. Not only due to human cortex but also that there are not much information at high frequency region most images.
  - mistercow 5y ago
    True, although there will be more in images with a lot of fine, sharp detail, which is part of why those images will tend to look particularly bad when heavily compressed. An image of small text, for example, will have a ton of ringing artifacts around the edges, which gives it that “deep fried” look.

jarenmf 5y ago

That's interesting, I'm a bit confused, in this case high frequencies are not removed they are just quantized like all other frequencies. How is this reconciled with the assumption that high frequencies carry little information?

tinus_hn 5y ago
Where is the contradiction? As far as the theory goes, the transform using the DCT is lossless apart from rounding errors. The quantization step just rounds more roughly. Which is where the lossy compression happens. Basically you’re not removing the high frequencies because they carry little information, you remove the information they carry. If you just remove the high frequency you end up with a blurred, smooth picture that doesn’t look like the original. But if you quantize the strength you get a pretty similar looking picture, apart from the degenerate case of sharp edges in a drawing.
- magicalhippo 5y ago
  > If you just remove the high frequency you end up with a blurred, smooth picture that doesn’t look like the original. Well it looks like a blurred, smoothed picture. This can be exploited during decoding if you want a smaller size than the original. For example if you only decode the 2x2 lower-frequency components of a 2000x1000 image, you get a nice downsampled 500x250 image "for free". For example when making thumbnails this can be very useful, as for larger images you can often use just 2x2 or 1x1, dramatically cutting down the work needed for the final resample.
  - tinus_hn 5y ago
    True, similarly I’ve always wondered whether DJ mixing tools directly decode the mp3s at the pitch they are going to play. No idea if that is practical though.
NavinF 5y ago
The coefficients get quantized (rounded to the nearest integer) after dividing by 2^n. That way, each of those numbers occupy n fewer bits on disk. You wanna use a larger n for the high-frequency coefficients since they’re less important. (Except when that assumption breaks, which is fairly often: JPEG compressed screenshots of text is unreadable because text is exclusively composed of sharp high-frequency edges) Oh and you can figure out what camera/software took a photo by looking at the value of n for each frequency. Everyone does it slightly differently to achieve different compression ratios.

tomashubelbauer 5y ago

I agree, I had the same click happen when I saw this set of shapes in a YouTube video on how JPEG works: https://www.youtube.com/watch?v=Kv1Hiv3ox8I https://www.youtube.com/watch?v=Kv1Hiv3ox8I.

SavantIdiot 5y ago
This video is really well done. I worked on optimizing the Xing video player for Pentium MMX ages ago (almost 30 years?!), and had to figure this out by reading the same one book on the topic over and over again. I never quite got the details, but this video would have saved me weeks of study. Pre-internet days sucked for learning complex topics.

Gravityloss 5y ago

This is a good way to get an insight on almost all transformations.

mistercow 5y ago

The way you’ve phrased this is not correct. Those aren’t “choices”, but basis functions. Each 8x8 block is broken into a sum of all of those basis functions, with each basis function multiplied by a coefficient. The output of the DCT is that list of coefficients. The quantization part is where the magic happens, but it’s not about choosing between those patterns. It’s about “snapping” the coefficients to fixed steps. Basically, the human visual cortex is bad at noticing subtle differences at higher frequencies (the bottom right area of your linked image). You can’t easily tell if the coefficient for that bottom right pattern is, say 100 or 103. So JPEG stores those coefficients with lower precision, which requires fewer bytes after the entropy coding step.

SavantIdiot 5y ago
Thanks, now I understand it even better!
jarenmf 5y ago
Essentially higher frequencies are represented using fewer number of shades and many of them will go to zero. Not only due to human cortex but also that there are not much information at high frequency region most images.
- mistercow 5y ago
  True, although there will be more in images with a lot of fine, sharp detail, which is part of why those images will tend to look particularly bad when heavily compressed. An image of small text, for example, will have a ton of ringing artifacts around the edges, which gives it that “deep fried” look.