9 ms·

JPEG Committee releases a call for evidence for image compression based on AI

113 points by jonsneyers 7y ago

The_rationalist 7y ago
I wonder if this is a reaction to the issue I opened :) https://github.com/google/brunsli/issues/60 https://github.com/google/brunsli/issues/60
PaulHoule 7y ago
I wouldn't be surprised if you could make an AI that knows something about what scenes look like, and something about what JPEG artifacts look like that and could clean up existing JPEG images.
- The_rationalist 7y ago
  Isn't that task called image denoising? : https://paperswithcode.com/task/image-denoising https://paperswithcode.com/task/image-denoising
  - ska 7y ago
    Artifact removal and denoising are only sort-of-kind-of the same thing. At a high enough level, you are using a model of some sort to estimate (or guesstimate) changes in the image but the models for "X type of noise from the sensor" and "artifact due to block size choice" or "artifact due to aliasing in the RF domain" or whatever are quite different.
- derefr 7y ago
  The JPEG artifact “look” comes from the fact that JPEG data is essentially just a matrix of point-samples (like any other raster format) which happen to carry a gradient and angle, rather than a solid color. So, at a low sampling resolution, you see weird discontinuities where one sample gradient (gradel?) sharply transitions to another sample gradient, with a discontinuity in either color or angle. An AI that “fixes” JPEG artifacts would just be a regular upscaling/supersampling AI—just one trained on JPEG sample-matrices instead of BMP sample-matrices, so that it can take advantage of the extra information each JPEG sample-point carries.
  - skunkworker 7y ago
    My understanding of lossy compression like JPEG may be limited, but wouldn't this ML network be guessing the eigenvectors to put back into a matrix that were removed because they weren't dominant enough?
    - gugagore 7y ago
      Unless I'm missing a connection you've made, no. You might have been exposed to the singular-value decomposition to compress a matrix, and representing an image as a matrix e.g. [1]. JPEG encoding is about transforming the signal into a frequency domain (think FFTs) where it is less detrimental to throw away information. If there is a connection to what you're saying, I expect you should try to make some statement about the eigenvalues of the DCT matrix [2] [1] http://www.math.utah.edu/~goller/F15_M2270/BradyMathews_SVDImage.pdf http://www.math.utah.edu/~goller/F15_M2270/BradyMathews_SVDI... [2] https://math.stackexchange.com/questions/962533/why-does-the-discrete-cosine-transform-as-matrix-multiplication-work-this-way https://math.stackexchange.com/questions/962533/why-does-the...
      - skunkworker 7y ago
        You're right, it was just an early linear algebra thing in college using SVD to do basic image compression.
      - cvwright 7y ago
        I interpreted the GP as saying that the AI could try to infer what the DCT coefficients had been prior to quantization. I don't know enough about JPEG or about AI to know whether that would be doable.
    - 0-_-0 7y ago
      Removed or just quantized, yes.
- rahidz 7y ago
  Get ESRGAN and a model that's trained on JPG compression and you got it https://upscale.wiki/wiki/Model_Database https://upscale.wiki/wiki/Model_Database has a nice list, and I use IEU to make things even easier: https://github.com/ptrsuder/IEU.Winforms https://github.com/ptrsuder/IEU.Winforms
Ajedi32 7y ago
I find the idea of AI-based image compression rather fascinating. In theory it seems like a great fit for AI; "perceived quality" is exactly the sort of fuzzy metric AI tends to be good at optimizing for, and a compression algorithm possessing an understanding of high-level visual features like shapes, lighting, etc seems like it would have a lot of potential. Taken to the extreme, a sufficiently advanced AI compression system could theoretically end up storing representations of high-level features like faces in the compressed data; which could lead to some really bizarre looking compression artifacts when the system breaks down.
- derefr 7y ago
  > Taken to the extreme, a sufficiently advanced AI compression system could theoretically end up storing representations of high-level features like faces in the compressed data Or, you could have a compression system created specifically for photographic images (as opposed to a “general compressor” like LZ77), where these high-level features are part of the (de)compressor itself. Now, lossy domain-specific compressors are pretty common; one good example is audio “voice codecs” for telecom, in which every snippet of audio is encoded as a phoneme-like symbol, such that noise comes out the other end as a gobbledygook that sounds a lot like words. But much more interesting to me is a lossless domain-specific compressor, which I’m not aware of any examples of. (Please tell me if about some if you know them!) In such a compressor, the input would be “rebased” against the model, such that the output looks like an instruction stream ala “start with [this synthesized template with these parameters]; and then add point-modifications {A, B, C, ...} to get from there back to the original input.” So inputs that were more like something it “understood” would compress well (i.e. just be described by a template) while inputs that were less-well-understood wouldn’t get much from the procedural-generation phase, and would end up being specified entirely as point-corrections, and so not really compress at all.
  - imurray 7y ago
    A relevant NeurIPS 2019 paper for lossless compression (for e.g., medical imaging): https://papers.nips.cc/paper/9383-integer-discrete-flows-and-lossless-compression https://papers.nips.cc/paper/9383-integer-discrete-flows-and...
leeoniya 7y ago
sure, what can go wrong? https://www.zdnet.com/article/xerox-scanners-alter-numbers-in-scanned-documents/ https://www.zdnet.com/article/xerox-scanners-alter-numbers-i...
- basilgohar 7y ago
  That still baffles me to this day. --Edit-- I think there are differences between the case of using this for compression and how Xerox implemented (which they claim was due to compression). It can be argued that most advanced codecs of today already handle compression via very complex and advanced heuristics already comparable to what a lot of people would confuse for AI and/or ML today, so I think it's not as spooky as it may sound at first. The Xerox implementation actually changed values on the page from one to another innocuously. I would hope that the AI being talked about is not "This 6 looks more like it should be an 8, so I'm going to put one there upon decode" and more like, "this block or unit of the image has these characteristics, and theses methods work very well to reproduce these characteristics when next to these other blocks..." etc. But that's the best way I can phrase it.
- fowl2 7y ago
  thought experiment: if this was our only way of compressing we could lose the ability to represent things "too far" from whatever the AI understands.
- m463 7y ago
  How funny if it happened with video compression. Movies could accidentally "transpose characters" too - meaning Mark Walhberg instead of Matt Damon
ksec 7y ago
JPEG XL The JPEG XL Image Coding System (ISO/IEC 18181) has produced an open source reference implementation available on the JPEG XL Gitlab repository. The software is available under Apache 2, which includes a royalty-free patent grant. Speed tests indicate the multithreaded encoder and decoder outperforms libjpeg-turbo. Independent subjective and objective evaluation experiments have indicated competitive performance with commonly used image coding solutions while offering new functionalities such as lossless transcoding from legacy JPEG format to JPEG XL. The standardization process has reached the Draft International Standard stage. That to me is the biggest news. I remember reading a presentation where it had even better quality at medium bitrate than AVIF. Since JPEG does not belong to Google or Apple, hopefully it would be a format easier to swallow from both parties.
- wmf 7y ago
  Half the technology in JPEG XL is from Google but hopefully people won't get petty about that. A bigger problem is that various companies just did a lot of work to adopt HEIF so they're not going to be eager to churn onto JPEG XL. Generally people wait 5-10 years before they're open to upgrading.
  - eln1 7y ago
    Also got this ANS coding: https://en.wikipedia.org/wiki/Asymmetric_numeral_systems https://en.wikipedia.org/wiki/Asymmetric_numeral_systems
  - ksec 7y ago
    >Half the technology in JPEG XL is from Google Yes but it is submitted as an open standard to an Organisation that doesn't run or belong to Google. Is HEIF used anywhere other than on Apple devices?
    - edw 7y ago
      Canon, Android.
      - 7y ago
RcouF1uZ4gsC 7y ago
I think we already have the tools to do this, but there might be issues with fidelity. First: Use AI to generate the text description of an image. http://homepages.inf.ed.ac.uk/keller/publications/jair16.pdf http://homepages.inf.ed.ac.uk/keller/publications/jair16.pdf https://www.captionbot.ai/ https://www.captionbot.ai/ Second: Use AI to generate an image from the text description. https://news.developer.nvidia.com/ai-model-can-generate-images-from-natural-language-descriptions/ https://news.developer.nvidia.com/ai-model-can-generate-imag...
- mehh 7y ago
  What do you mean by us AI? Do you mean a Neural Network or something else, I find the use of the broad term AI a bit weird, there is lots of stuff that can fit under the AI category.
carapace 7y ago
See also "Hutter Prize" http://www.hutter1.net/prize/index.htm http://www.hutter1.net/prize/index.htm https://en.wikipedia.org/wiki/Hutter_Prize https://en.wikipedia.org/wiki/Hutter_Prize
chubot 7y ago
Where are the Weissman scores?
rcarmo 7y ago
I was instantly reminded of this “pineapple”: https://twitter.com/cassmarketos/status/1229473344480673792?s=21 https://twitter.com/cassmarketos/status/1229473344480673792?... Of course, an actual working compression algorithm is quite likely. I recall that recently someone was upscaling and colorizing ancient movie reels to modern standards, and there is plenty of low-hanging fruit in fine-tuning parameters to suit specific images, so I actually look forward to this. I just hope they don’t name it JPEG2020, JPEG2000 was a bit of a dud.
- BubRoss 7y ago
  Jpeg2000 was a dud because of licensing and patents, not having a year in the name.
  - userbinator 7y ago
    ...and performance (or lack thereof) --- AFAIK the baseline J2K was always implementable freely (similar to the situation with JPEG; arithmetic compression was specified but largely unimplemented due to patents, while baseline JPEG was supposed to be free), but the abysmal performance relative to JPEG for a minor increase in compression efficiency, along with the surprising amount of complexity in the spec, made it a poor choice. There are lots of scanned ebooks on archive.org that use J2K compression, and the sluggishness when turning pages is extremely noticeable compared to others which use normal JPEG.
- sgdread 7y ago
  DeOldify [1] by Jason Antic got really good results with old photos and movies colorization. [1] https://github.com/jantic/DeOldify https://github.com/jantic/DeOldify
- hnews_account_1 7y ago
  I don't know how recolorization algos work, but a month or so ago, I saw this old 1900s image of a new York street that was colorized on reddit. Everyone on the thread was commenting about how there was so much violet hue in every single person's dress in the photo. Iirc that recolorization was done by a neural net too. Then I realised what must've happened. There were tons of open air carts on the street selling eggplant. My guess is that the NN was either shown that or recognized that part of the image and colored almost everything else it saw using that as its base point. Not the best technique.
keenmaster 7y ago
Does anyone have an estimate of how small an AI-compressed image will be compared to an image compressed using traditional compression algorithms? On average of course, and controlling for image quality.
- 0-_-0 7y ago
  As an example from 29 Feb 2016, for lossless compression you can go from 8 to 3 bits per color according to Table 5 here: https://arxiv.org/abs/1601.06759 https://arxiv.org/abs/1601.06759
rhacker 7y ago
I know this is more relevant for movies, but it would be rather cool to have a movie compressed for a year of computation, and deliverable in about 100MB, then let it decompress for a few hours on your computer back to its original 9GB size.
- 0-_-0 7y ago
  How big is the market for that vs streaming 9 GB on demand?
SrslyJosh 7y ago
> Fake news, copyright violations, media forensics, privacy and security are emerging challenges in digital media. JPEG has determined that blockchain and distributed ledger technologies (DLT) have great potential as a technology component to address these challenges in transparent and trustable media transactions. You've gotta be kidding me.
jiggawatts 7y ago
I've been in the C#, Java, and Rust world for so long that every time I come across C/C++ library source it cracks me up that an image compressor has to have functions to redefine basic concepts like memory allocation, assertions, and threading. Reminds me of a "lightweight" zip compression library that I recently used that was literally 98% overhead and 2% actual compression code. It had its own definitions of everything: integers, integer properties, memory allocation, streams (as in sequences of memory blocks, not even real I/O!), error handling, progress notifications, etc... Pages and pages of conditional macros and compiler-specific hacks. Just to call a single a function that took an array and returned an array. Ridiculous. https://gitlab.com/wg1/jpeg-xl/-/blob/master/include/jpegxl/memory_manager.h https://gitlab.com/wg1/jpeg-xl/-/blob/master/include/jpegxl/... https://gitlab.com/wg1/jpeg-xl/-/blob/master/include/jpegxl/parallel_runner.h https://gitlab.com/wg1/jpeg-xl/-/blob/master/include/jpegxl/... https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/compiler_specific.h https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/compil... https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/os_specific.h https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/os_spe... https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/arch_specific.h https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/arch_s... That last one is trying to work out the NUMA topology in a portable way... good luck with that! That's not something I expected from a JPEG library. Like memory management, they'd actually be better off requesting thread-management function pointers from the hosting application. Otherwise, this would not be a "good citizen" in constrained environments such as embedded in a web server or database engine. Random examples of silliness from just this one file: They ignore the first 2 cores on 3+ core systems: https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/os_specific.cc#L266 https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/os_spe... They'll be limited max 64 threads (Half the threads on the larger single-socket AMD EPYC 2 CPUs!): https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/os_specific.cc#L211 https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/os_spe... because they're not using the Processor Group Aware API: https://docs.microsoft.com/en-us/windows/win32/procthread/processor-groups https://docs.microsoft.com/en-us/windows/win32/procthread/pr... Sure, 64-core monsters are still somewhat rare today, but 16-core/32-thread desktops CPUs are now affordable and the TSMC 5nm process will likely result in 96-128 core monstrosities this year. That's 384-512 threads for a dual-socket server or workstation! If you're making an image codec library "for the next decade", a little future-proofing might go a long way. Look at it this way: Image decoding throughput is one of the main performance issues I have right now with Lightroom and I work with "only" 36 Mpx images. There are 80 Mpx cameras just around the corner... For example, there was a blog article a couple of years ago basically saying that the major Amdahl's law limitation to decoding formats like JPEG is the Huffman bitstream decoding, which is Step #1 in the process. The author pointed out that simply restarting the Huffman bitstream a few times for a typical PNG or JPEG file would increase the size less than 1%, but would allow truly parallel decoding for every step. For all the apparent effort to make JPEG-XL parallelised by default, it seems odd that there doesn't appear to be any mention of parallelising the ANS bitstream decoding...
jonsneyers 7y ago
Highlights: - Call for Evidence launched for AI-based image codecs - Call for Evidence launched for point cloud codecs - Next-gen image codec JPEG XL reaches Draft International Standard stage - Light-weight JPEG XS codec is looking at compression of camera raws
bloak 7y ago
Apparently they're "responsible for the popular JPEG, JPEG 2000, JPEG XR, JPSearch, JPEG XT and more recently, the JPEG XS, JPEG Systems, JPEG Pleno and JPEG XL families of imaging standards". Some of those may be more popular than others: https://xkcd.com/2254/ https://xkcd.com/2254/
- jonsneyers 7y ago
  JPEG 2000 is actually quite popular, for example any movie you see in a cinema is encoded using JPEG 2000 – they don't use video codecs for digital cinema, but encode each frame separately, to avoid video compression artifacts. Also in medical imaging, nearly everything is using JPEG 2000. Apple products have supported it for a long time. JPEG XR is also known as "Windows Media Photo" and "HD Photo" and has been supported in the Microsoft ecosystem since Windows Vista and Internet Explorer 9. JPEG XS, XL and Pleno are of course much newer and will still have to be battle-tested. But yes, none of the newer JPEG standards have reached the popularity of the first JPEG codec, which is by far the most popular image format ever.
  - todotask 7y ago
    JPEG XL repo seems to have adopt Google Brunsli.
  - PaulHoule 7y ago
    The FBI uses JPEG 2000 for fingerprints.
  - Ajedi32 7y ago
    Personally I'm really excited about the potential of JPEG XS for wireless VR. Currently most of the high quality wireless setups require expensive, high-bandwidth solutions like WiGig in order to avoid the latency penalty of compressing the video stream. If JPEG XS can get those streams small enough to fit into a regular 5 GHz Wi-Fi connection, that would be a _huge_ deal for the affordability of wireless VR setups.
  - clouddrover 7y ago
    An advantage JPEG XL has over previous efforts is that it’s royalty free and that JPEGs can be losslessly converted to JPEG XL (and back again) and still get about a 20% file size reduction. Being able to convert to JPEG XL with no quality loss will help adoption.

113 points by jonsneyers 7y ago

I wonder if this is a reaction to the issue I opened :) https://github.com/google/brunsli/issues/60 https://github.com/google/brunsli/issues/60

PaulHoule 7y ago

I wouldn't be surprised if you could make an AI that knows something about what scenes look like, and something about what JPEG artifacts look like that and could clean up existing JPEG images.

The_rationalist 7y ago
Isn't that task called image denoising? : https://paperswithcode.com/task/image-denoising https://paperswithcode.com/task/image-denoising
- ska 7y ago
  Artifact removal and denoising are only sort-of-kind-of the same thing. At a high enough level, you are using a model of some sort to estimate (or guesstimate) changes in the image but the models for "X type of noise from the sensor" and "artifact due to block size choice" or "artifact due to aliasing in the RF domain" or whatever are quite different.
derefr 7y ago
The JPEG artifact “look” comes from the fact that JPEG data is essentially just a matrix of point-samples (like any other raster format) which happen to carry a gradient and angle, rather than a solid color. So, at a low sampling resolution, you see weird discontinuities where one sample gradient (gradel?) sharply transitions to another sample gradient, with a discontinuity in either color or angle. An AI that “fixes” JPEG artifacts would just be a regular upscaling/supersampling AI—just one trained on JPEG sample-matrices instead of BMP sample-matrices, so that it can take advantage of the extra information each JPEG sample-point carries.
- skunkworker 7y ago
  My understanding of lossy compression like JPEG may be limited, but wouldn't this ML network be guessing the eigenvectors to put back into a matrix that were removed because they weren't dominant enough?
  - gugagore 7y ago
    Unless I'm missing a connection you've made, no. You might have been exposed to the singular-value decomposition to compress a matrix, and representing an image as a matrix e.g. [1]. JPEG encoding is about transforming the signal into a frequency domain (think FFTs) where it is less detrimental to throw away information. If there is a connection to what you're saying, I expect you should try to make some statement about the eigenvalues of the DCT matrix [2] [1] http://www.math.utah.edu/~goller/F15_M2270/BradyMathews_SVDImage.pdf http://www.math.utah.edu/~goller/F15_M2270/BradyMathews_SVDI... [2] https://math.stackexchange.com/questions/962533/why-does-the-discrete-cosine-transform-as-matrix-multiplication-work-this-way https://math.stackexchange.com/questions/962533/why-does-the...
    - skunkworker 7y ago
      You're right, it was just an early linear algebra thing in college using SVD to do basic image compression.
    - cvwright 7y ago
      I interpreted the GP as saying that the AI could try to infer what the DCT coefficients had been prior to quantization. I don't know enough about JPEG or about AI to know whether that would be doable.
  - 0-_-0 7y ago
    Removed or just quantized, yes.
rahidz 7y ago
Get ESRGAN and a model that's trained on JPG compression and you got it https://upscale.wiki/wiki/Model_Database https://upscale.wiki/wiki/Model_Database has a nice list, and I use IEU to make things even easier: https://github.com/ptrsuder/IEU.Winforms https://github.com/ptrsuder/IEU.Winforms

Ajedi32 7y ago

I find the idea of AI-based image compression rather fascinating. In theory it seems like a great fit for AI; "perceived quality" is exactly the sort of fuzzy metric AI tends to be good at optimizing for, and a compression algorithm possessing an understanding of high-level visual features like shapes, lighting, etc seems like it would have a lot of potential. Taken to the extreme, a sufficiently advanced AI compression system could theoretically end up storing representations of high-level features like faces in the compressed data; which could lead to some really bizarre looking compression artifacts when the system breaks down.

derefr 7y ago
> Taken to the extreme, a sufficiently advanced AI compression system could theoretically end up storing representations of high-level features like faces in the compressed data Or, you could have a compression system created specifically for photographic images (as opposed to a “general compressor” like LZ77), where these high-level features are part of the (de)compressor itself. Now, lossy domain-specific compressors are pretty common; one good example is audio “voice codecs” for telecom, in which every snippet of audio is encoded as a phoneme-like symbol, such that noise comes out the other end as a gobbledygook that sounds a lot like words. But much more interesting to me is a lossless domain-specific compressor, which I’m not aware of any examples of. (Please tell me if about some if you know them!) In such a compressor, the input would be “rebased” against the model, such that the output looks like an instruction stream ala “start with [this synthesized template with these parameters]; and then add point-modifications {A, B, C, ...} to get from there back to the original input.” So inputs that were more like something it “understood” would compress well (i.e. just be described by a template) while inputs that were less-well-understood wouldn’t get much from the procedural-generation phase, and would end up being specified entirely as point-corrections, and so not really compress at all.
- imurray 7y ago
  A relevant NeurIPS 2019 paper for lossless compression (for e.g., medical imaging): https://papers.nips.cc/paper/9383-integer-discrete-flows-and-lossless-compression https://papers.nips.cc/paper/9383-integer-discrete-flows-and...

leeoniya 7y ago

sure, what can go wrong? https://www.zdnet.com/article/xerox-scanners-alter-numbers-in-scanned-documents/ https://www.zdnet.com/article/xerox-scanners-alter-numbers-i...

basilgohar 7y ago
That still baffles me to this day. --Edit-- I think there are differences between the case of using this for compression and how Xerox implemented (which they claim was due to compression). It can be argued that most advanced codecs of today already handle compression via very complex and advanced heuristics already comparable to what a lot of people would confuse for AI and/or ML today, so I think it's not as spooky as it may sound at first. The Xerox implementation actually changed values on the page from one to another innocuously. I would hope that the AI being talked about is not "This 6 looks more like it should be an 8, so I'm going to put one there upon decode" and more like, "this block or unit of the image has these characteristics, and theses methods work very well to reproduce these characteristics when next to these other blocks..." etc. But that's the best way I can phrase it.
fowl2 7y ago
thought experiment: if this was our only way of compressing we could lose the ability to represent things "too far" from whatever the AI understands.
m463 7y ago
How funny if it happened with video compression. Movies could accidentally "transpose characters" too - meaning Mark Walhberg instead of Matt Damon

ksec 7y ago

JPEG XL The JPEG XL Image Coding System (ISO/IEC 18181) has produced an open source reference implementation available on the JPEG XL Gitlab repository. The software is available under Apache 2, which includes a royalty-free patent grant. Speed tests indicate the multithreaded encoder and decoder outperforms libjpeg-turbo. Independent subjective and objective evaluation experiments have indicated competitive performance with commonly used image coding solutions while offering new functionalities such as lossless transcoding from legacy JPEG format to JPEG XL. The standardization process has reached the Draft International Standard stage. That to me is the biggest news. I remember reading a presentation where it had even better quality at medium bitrate than AVIF. Since JPEG does not belong to Google or Apple, hopefully it would be a format easier to swallow from both parties.

wmf 7y ago
Half the technology in JPEG XL is from Google but hopefully people won't get petty about that. A bigger problem is that various companies just did a lot of work to adopt HEIF so they're not going to be eager to churn onto JPEG XL. Generally people wait 5-10 years before they're open to upgrading.
- eln1 7y ago
  Also got this ANS coding: https://en.wikipedia.org/wiki/Asymmetric_numeral_systems https://en.wikipedia.org/wiki/Asymmetric_numeral_systems
- ksec 7y ago
  >Half the technology in JPEG XL is from Google Yes but it is submitted as an open standard to an Organisation that doesn't run or belong to Google. Is HEIF used anywhere other than on Apple devices?
  - edw 7y ago
    Canon, Android.
    - 7y ago

RcouF1uZ4gsC 7y ago

I think we already have the tools to do this, but there might be issues with fidelity. First: Use AI to generate the text description of an image. http://homepages.inf.ed.ac.uk/keller/publications/jair16.pdf http://homepages.inf.ed.ac.uk/keller/publications/jair16.pdf https://www.captionbot.ai/ https://www.captionbot.ai/ Second: Use AI to generate an image from the text description. https://news.developer.nvidia.com/ai-model-can-generate-images-from-natural-language-descriptions/ https://news.developer.nvidia.com/ai-model-can-generate-imag...

mehh 7y ago
What do you mean by us AI? Do you mean a Neural Network or something else, I find the use of the broad term AI a bit weird, there is lots of stuff that can fit under the AI category.

carapace 7y ago

See also "Hutter Prize" http://www.hutter1.net/prize/index.htm http://www.hutter1.net/prize/index.htm https://en.wikipedia.org/wiki/Hutter_Prize https://en.wikipedia.org/wiki/Hutter_Prize

chubot 7y ago

Where are the Weissman scores?

rcarmo 7y ago

I was instantly reminded of this “pineapple”: https://twitter.com/cassmarketos/status/1229473344480673792?s=21 https://twitter.com/cassmarketos/status/1229473344480673792?... Of course, an actual working compression algorithm is quite likely. I recall that recently someone was upscaling and colorizing ancient movie reels to modern standards, and there is plenty of low-hanging fruit in fine-tuning parameters to suit specific images, so I actually look forward to this. I just hope they don’t name it JPEG2020, JPEG2000 was a bit of a dud.

BubRoss 7y ago
Jpeg2000 was a dud because of licensing and patents, not having a year in the name.
- userbinator 7y ago
  ...and performance (or lack thereof) --- AFAIK the baseline J2K was always implementable freely (similar to the situation with JPEG; arithmetic compression was specified but largely unimplemented due to patents, while baseline JPEG was supposed to be free), but the abysmal performance relative to JPEG for a minor increase in compression efficiency, along with the surprising amount of complexity in the spec, made it a poor choice. There are lots of scanned ebooks on archive.org that use J2K compression, and the sluggishness when turning pages is extremely noticeable compared to others which use normal JPEG.
sgdread 7y ago
DeOldify [1] by Jason Antic got really good results with old photos and movies colorization. [1] https://github.com/jantic/DeOldify https://github.com/jantic/DeOldify
hnews_account_1 7y ago
I don't know how recolorization algos work, but a month or so ago, I saw this old 1900s image of a new York street that was colorized on reddit. Everyone on the thread was commenting about how there was so much violet hue in every single person's dress in the photo. Iirc that recolorization was done by a neural net too. Then I realised what must've happened. There were tons of open air carts on the street selling eggplant. My guess is that the NN was either shown that or recognized that part of the image and colored almost everything else it saw using that as its base point. Not the best technique.

keenmaster 7y ago

Does anyone have an estimate of how small an AI-compressed image will be compared to an image compressed using traditional compression algorithms? On average of course, and controlling for image quality.

0-_-0 7y ago
As an example from 29 Feb 2016, for lossless compression you can go from 8 to 3 bits per color according to Table 5 here: https://arxiv.org/abs/1601.06759 https://arxiv.org/abs/1601.06759

rhacker 7y ago

I know this is more relevant for movies, but it would be rather cool to have a movie compressed for a year of computation, and deliverable in about 100MB, then let it decompress for a few hours on your computer back to its original 9GB size.

0-_-0 7y ago
How big is the market for that vs streaming 9 GB on demand?

SrslyJosh 7y ago

> Fake news, copyright violations, media forensics, privacy and security are emerging challenges in digital media. JPEG has determined that blockchain and distributed ledger technologies (DLT) have great potential as a technology component to address these challenges in transparent and trustable media transactions. You've gotta be kidding me.

jiggawatts 7y ago

I've been in the C#, Java, and Rust world for so long that every time I come across C/C++ library source it cracks me up that an image compressor has to have functions to redefine basic concepts like memory allocation, assertions, and threading. Reminds me of a "lightweight" zip compression library that I recently used that was literally 98% overhead and 2% actual compression code. It had its own definitions of everything: integers, integer properties, memory allocation, streams (as in sequences of memory blocks, not even real I/O!), error handling, progress notifications, etc... Pages and pages of conditional macros and compiler-specific hacks. Just to call a single a function that took an array and returned an array. Ridiculous. https://gitlab.com/wg1/jpeg-xl/-/blob/master/include/jpegxl/memory_manager.h https://gitlab.com/wg1/jpeg-xl/-/blob/master/include/jpegxl/... https://gitlab.com/wg1/jpeg-xl/-/blob/master/include/jpegxl/parallel_runner.h https://gitlab.com/wg1/jpeg-xl/-/blob/master/include/jpegxl/... https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/compiler_specific.h https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/compil... https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/os_specific.h https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/os_spe... https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/arch_specific.h https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/arch_s... That last one is trying to work out the NUMA topology in a portable way... good luck with that! That's not something I expected from a JPEG library. Like memory management, they'd actually be better off requesting thread-management function pointers from the hosting application. Otherwise, this would not be a "good citizen" in constrained environments such as embedded in a web server or database engine. Random examples of silliness from just this one file: They ignore the first 2 cores on 3+ core systems: https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/os_specific.cc#L266 https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/os_spe... They'll be limited max 64 threads (Half the threads on the larger single-socket AMD EPYC 2 CPUs!): https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/os_specific.cc#L211 https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/os_spe... because they're not using the Processor Group Aware API: https://docs.microsoft.com/en-us/windows/win32/procthread/processor-groups https://docs.microsoft.com/en-us/windows/win32/procthread/pr... Sure, 64-core monsters are still somewhat rare today, but 16-core/32-thread desktops CPUs are now affordable and the TSMC 5nm process will likely result in 96-128 core monstrosities this year. That's 384-512 threads for a dual-socket server or workstation! If you're making an image codec library "for the next decade", a little future-proofing might go a long way. Look at it this way: Image decoding throughput is one of the main performance issues I have right now with Lightroom and I work with "only" 36 Mpx images. There are 80 Mpx cameras just around the corner... For example, there was a blog article a couple of years ago basically saying that the major Amdahl's law limitation to decoding formats like JPEG is the Huffman bitstream decoding, which is Step #1 in the process. The author pointed out that simply restarting the Huffman bitstream a few times for a typical PNG or JPEG file would increase the size less than 1%, but would allow truly parallel decoding for every step. For all the apparent effort to make JPEG-XL parallelised by default, it seems odd that there doesn't appear to be any mention of parallelising the ANS bitstream decoding...

jonsneyers 7y ago

Highlights: - Call for Evidence launched for AI-based image codecs - Call for Evidence launched for point cloud codecs - Next-gen image codec JPEG XL reaches Draft International Standard stage - Light-weight JPEG XS codec is looking at compression of camera raws

bloak 7y ago

Apparently they're "responsible for the popular JPEG, JPEG 2000, JPEG XR, JPSearch, JPEG XT and more recently, the JPEG XS, JPEG Systems, JPEG Pleno and JPEG XL families of imaging standards". Some of those may be more popular than others: https://xkcd.com/2254/ https://xkcd.com/2254/

jonsneyers 7y ago
JPEG 2000 is actually quite popular, for example any movie you see in a cinema is encoded using JPEG 2000 – they don't use video codecs for digital cinema, but encode each frame separately, to avoid video compression artifacts. Also in medical imaging, nearly everything is using JPEG 2000. Apple products have supported it for a long time. JPEG XR is also known as "Windows Media Photo" and "HD Photo" and has been supported in the Microsoft ecosystem since Windows Vista and Internet Explorer 9. JPEG XS, XL and Pleno are of course much newer and will still have to be battle-tested. But yes, none of the newer JPEG standards have reached the popularity of the first JPEG codec, which is by far the most popular image format ever.
- todotask 7y ago
  JPEG XL repo seems to have adopt Google Brunsli.
- PaulHoule 7y ago
  The FBI uses JPEG 2000 for fingerprints.
- Ajedi32 7y ago
  Personally I'm really excited about the potential of JPEG XS for wireless VR. Currently most of the high quality wireless setups require expensive, high-bandwidth solutions like WiGig in order to avoid the latency penalty of compressing the video stream. If JPEG XS can get those streams small enough to fit into a regular 5 GHz Wi-Fi connection, that would be a _huge_ deal for the affordability of wireless VR setups.
- clouddrover 7y ago
  An advantage JPEG XL has over previous efforts is that it’s royalty free and that JPEGs can be losslessly converted to JPEG XL (and back again) and still get about a 20% file size reduction. Being able to convert to JPEG XL with no quality loss will help adoption.