7 ms·

DCT is already lossy [1], so the statements around 8 megapixels are completely pointless, and worst of all, its 1990 lossy technology. Wavelet transformations c

DCT is already lossy [1], so the statements around 8 megapixels are completely pointless, and worst of all, its 1990 lossy technology. Wavelet transformations completely destroy any DCT. That said, if their emphasis is on producing pictures with minimal time delta at highest resolution, algorithms used for still pictures are out of place. Video compression algorithms still use DCT and wavelets, but they do so only after they have reduced redundancies between series of pictures, a process that tends to work significantly better than anything you can get out of these lossy transformations when you want to preserve quality. Of course, eliminating redundancy in a series of pictures might have tipped them off to the fact that the image sensor isn't actually producing fresh pictures at the rate they want. 1: as used in JPEG. The transformation itself is perfectly invertible, assuming infinite precision arithmetic.

jpap 13y ago
You are right on the loss: it's purposefully introduced as a quantization step after performing the DCT, and before losslessly compressing the resulting coefficients with Huffman and encoding to the final JPEG bitstream. Despite all of that, JPEG has now become computationally tractable. I remember the days where it took tens of seconds to encode a JPEG on a commodity machine. Now, with the help of SIMD, we can encode a high quality image in msec on a mobile device. Fortunately you can choose the quantization matrix that determines the amount of loss. Even if you were to choose a unitary matrix, no human, not even superman with his laser eyes, can "detect" the quantization noise. For SnappyCam, I chose to invest in JPEG a little more because it's a ubiquitous standard for still image compression.... and with the right hardware and algorithms, quite tractable. I'll consider adding a JPEG "quality setting" so you can choose the amount of loss introduced... sounds like a great idea to me. The idea behind SnappyCam was also to code each picture independently, and not rely on motion prediction or video codecs. If you try and pull a single frame from a HD video you might be disappointed: they compress the YUV dynamic range (studio swing) and it looks washed out, even if you land on an i-frame. Lastly, as far as I can tell, the image sensor is yielding complete scans with each frame. I'd hazard a guess to say that any motion prediction or frame deltas might actually slow the whole chain down.
- revelation 13y ago
  It's just bizarre that you would be doing the complete JPEG process at the instant you get the image from the sensor. As you note, there are a plethora of steps that JPEG performs, from color space conversion, to DCT transformation (essentially a gigantic matrix multiplication), Huffman coding, quantization, arithmetic coding and encoding as JPEG bitstream. The only reason would be that you are pressed for memory or bandwidth, but certainly you have the resources to store one full frame and produce deltas, or just apply part of the JPEG chain, enough to remedy memory pressure. You can always encode it to an actual JPEG after the process. And yes, pulling single frames from a completely encoded video isn't helpful, because they can get away with more compression. But there are very sophisticated algorithms for eliminating the redundancy between frames, which would have been my first avenue in attempting to do something like this.
  - jpap 13y ago
    Oh, I actually do both---see the other thread on the topic. I buffer as much as possible, while also encoding on any other cores that are available. There's a reason why a "simple camera app" can total some 80 kLOC. :)
  - gruseom 13y ago
    which would have been my first avenue in attempting to do something like this. Have you attempted to do something like this? Because he not only has attempted, he's done it. Therefore I think you should stop talking down to him ("completely pointless", "it's just bizarre", "my first avenue"). It comes across as wanting to prove how smart you are instead of seeking to learn from someone who has done incredible work and—lucky for us—is bursting at the seams with enthusiasm to share it. Oh and congratulations jpap on what's looking like the most successful and technically solidest HN launch in quite some time! I hope your hard work pays off.
    - eclipxe 13y ago
      Thank you for saying what I'm sure a lot of us were thinking.
jasonwatkinspdx 13y ago
> Wavelet transformations completely destroy any DCT. It's not quite that simple. They have different strengths and weaknesses, so you can't say one is categorically better than the other. DCT has uniform frequency resolution, which sounds desirable but isn't once you start quantizing due to ringing, etc. See slide 25 or so of [1]. Wavelets overcome this problem by adapting the resolution in opposite ways at the extremes. This works fantastic for medium signal rates, but can have severe low passing at low rates. For low rate coding DCT with the bells and whistles (deblocking, etc) will typically win. [1]: http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro...

jpap 13y ago

You are right on the loss: it's purposefully introduced as a quantization step after performing the DCT, and before losslessly compressing the resulting coefficients with Huffman and encoding to the final JPEG bitstream. Despite all of that, JPEG has now become computationally tractable. I remember the days where it took tens of seconds to encode a JPEG on a commodity machine. Now, with the help of SIMD, we can encode a high quality image in msec on a mobile device. Fortunately you can choose the quantization matrix that determines the amount of loss. Even if you were to choose a unitary matrix, no human, not even superman with his laser eyes, can "detect" the quantization noise. For SnappyCam, I chose to invest in JPEG a little more because it's a ubiquitous standard for still image compression.... and with the right hardware and algorithms, quite tractable. I'll consider adding a JPEG "quality setting" so you can choose the amount of loss introduced... sounds like a great idea to me. The idea behind SnappyCam was also to code each picture independently, and not rely on motion prediction or video codecs. If you try and pull a single frame from a HD video you might be disappointed: they compress the YUV dynamic range (studio swing) and it looks washed out, even if you land on an i-frame. Lastly, as far as I can tell, the image sensor is yielding complete scans with each frame. I'd hazard a guess to say that any motion prediction or frame deltas might actually slow the whole chain down.

revelation 13y ago
It's just bizarre that you would be doing the complete JPEG process at the instant you get the image from the sensor. As you note, there are a plethora of steps that JPEG performs, from color space conversion, to DCT transformation (essentially a gigantic matrix multiplication), Huffman coding, quantization, arithmetic coding and encoding as JPEG bitstream. The only reason would be that you are pressed for memory or bandwidth, but certainly you have the resources to store one full frame and produce deltas, or just apply part of the JPEG chain, enough to remedy memory pressure. You can always encode it to an actual JPEG after the process. And yes, pulling single frames from a completely encoded video isn't helpful, because they can get away with more compression. But there are very sophisticated algorithms for eliminating the redundancy between frames, which would have been my first avenue in attempting to do something like this.
- jpap 13y ago
  Oh, I actually do both---see the other thread on the topic. I buffer as much as possible, while also encoding on any other cores that are available. There's a reason why a "simple camera app" can total some 80 kLOC. :)
- gruseom 13y ago
  which would have been my first avenue in attempting to do something like this. Have you attempted to do something like this? Because he not only has attempted, he's done it. Therefore I think you should stop talking down to him ("completely pointless", "it's just bizarre", "my first avenue"). It comes across as wanting to prove how smart you are instead of seeking to learn from someone who has done incredible work and—lucky for us—is bursting at the seams with enthusiasm to share it. Oh and congratulations jpap on what's looking like the most successful and technically solidest HN launch in quite some time! I hope your hard work pays off.
  - eclipxe 13y ago
    Thank you for saying what I'm sure a lot of us were thinking.

jasonwatkinspdx 13y ago

> Wavelet transformations completely destroy any DCT. It's not quite that simple. They have different strengths and weaknesses, so you can't say one is categorically better than the other. DCT has uniform frequency resolution, which sounds desirable but isn't once you start quantizing due to ringing, etc. See slide 25 or so of [1]. Wavelets overcome this problem by adapting the resolution in opposite ways at the extremes. This works fantastic for medium signal rates, but can have severe low passing at low rates. For low rate coding DCT with the bells and whistles (deblocking, etc) will typically win. [1]: http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro_to_video1.pdf http://people.xiph.org/~tterribe/pubs/lca2012/auckland/intro...