16 ms·

With Google's RAISR, images can be up to 75% smaller without losing detail

107 points by newman314 10y ago

SQL2219 10y ago
I wonder how it compares to this: https://tinyjpg.com/ https://tinyjpg.com/
- explorigin 10y ago
  The cool thing is that it's not a compression algorithm so you could use both. TinyJPG to compress the smaller-sized image, RAISR to upsample the smaller image. If you've ever played with GIMP (or Photoshop I suppose), RAISR looks like unsharp mask with an neural network controlling the inputs.
  - derefr 10y ago
    Now create a psychophysical compressor that optimizes for the weighted combination of smallest compressed size and most-faithful RAISR restoration. I wonder if there's a guaranteed-optimal way to do that, given whatever RAISR is using as an algorithm; some logical inverse to the particular way it adds information. Maybe truncated wavelet compression?
jerf 10y ago
This article and the linked blog post from Google [1] suggest that this has already been deployed. How does it work in the field? For instance, I can imagine it being a JS library that can be downloaded and cached for client-side use, or it could be deployed as an Android module that the Android browser can use. But I don't know where to go looking for this because I don't use Google+, have no photos on it, and wouldn't be sure whether or not I was seeing it. [1]: https://www.blog.google/products/google-plus/saving-you-bandwidth-through-machine-learning/ https://www.blog.google/products/google-plus/saving-you-band...
- gwern 10y ago
  >RAISR produces results that are comparable to or better than the currently available super-resolution methods, and does so roughly 10 to 100 times faster, allowing it to be run on a typical mobile device in real-time....For example, in addition to improving digital “pinch to zoom” on your phone, one could capture, save, or transmit images at lower resolution and super-resolve on demand without any visible degradation in quality, all while utilizing less of mobile data and storage plans. > In our proposed approach, the run-time is more than one to two orders of magnitude faster than the best competing methods currently available, while producing results comparable or better than state-of-the-art. From the blog post & article, I think the idea is that you ship a small CNN (a few MB model size, which will accordingly run in realtime on mobile GPUs) to the Android device, and then when browsing, it downloads a small thumbnail (cheap on bandwidth, quick to download), and upscale it as necessary locally on the device.
  - jacobolus 10y ago
    In general they’d get better bandwidth savings for the same quality and decode effort by shipping the higher-resolution image with more aggressive image compression (using http://bellard.org/bpg/ http://bellard.org/bpg/ or something, not JPEG, and ideally running the decode in hardware). Seems like a dubious method overall. The main impression I get is “woo! neural nets can do everything! let’s use them everywhere!” I don’t believe their claim that this is “comparable or better than the state of the art” for image upscaling without much better evidence than they provide in their paper.
wodenokoto 10y ago
The article makes it sound like it is some sort of machine learning algorithm, which makes me wonder: How is the upscaling done client side?
- sushisource 10y ago
  It's very much within the realm of possibility to have a pre-trained neural net on the client, which then simply asks it questions. For example, swiftkey does this already with its keyboard prediction. Prisma does it with image styling.
  - gkbrk 10y ago
    I actually think Prisma sends the image to their servers for processing, their app size is way too small to contain that many trained networks. Also it doesn't work without an internet connection if I'm correct.
    - HisGraceTheDuck 10y ago
      That used to be the case but recent updates do the processing on the device now. I don't know how they managed it but they did.
  - wodenokoto 10y ago
    It is possible, no doubt. You also have image decoder on the client and getting support for a new format is difficult because it is the browser vendors job to distribute these. So how do they practically get the client to decode / upscale? JavaScript?
tornadoboy55 10y ago
The biggest about this is not being able to 'compress' images to 25%, but that you can now take (some) images that we're previously only low-res, and upscale them!
- CamelCaseName 10y ago
  One day we'll be pointing at pictures yelling "Enhance!", and not in jest!
  - fulafel 10y ago
    Good idea for a dystopian CSI. "This AI invented image detail shows you are guilty"
    - patates 10y ago
      Well then it means statistically there's a good chance that what produced the end low res image is incriminating. And of course I'm not serious. Although on a serious note, does anyone know if and how statistical probability is accepted as evidence in a court of law?
      - tornadoboy55 10y ago
        Yeah, it is, that's called circumstantial evidence. You do need direct evidence to validate the circumstantial evidence though..
        patates 10y ago
        It even has a name, didn't expect that. Thank you.
    - tekoyaki 10y ago
      New episode of Black Mirror or maybe some comedy sketch. "Your face is too generic that the AI drew you as all of our suspects."
ZenoArrow 10y ago
I'd be interested to see if similar techniques could be used for scaling up pixel art. Tools to upscale pixel art do exist already, but the results are a bit hit and miss. Could be a good stress test for the machine learning algorithms too.
- andai 10y ago
  That sounds interesting. Do you have any links? I must not understand what you mean by scaling up, I though the best way to scale up pixel art was to disable interpolation, so each pixel just becomes a perfectly sharp square?
  - ZenoArrow 10y ago
    >"That sounds interesting. Do you have any links?" Plenty to take a look at here: https://en.wikipedia.org/wiki/Pixel_art_scaling_algorithms https://en.wikipedia.org/wiki/Pixel_art_scaling_algorithms
    - jacobolus 10y ago
      Existing pixel-art-specific upscaling methods are going to be better than this, for pixel art. But if they wanted they could probably try to train their neural network for pixel art specifically, similar to https://github.com/nagadomi/waifu2x https://github.com/nagadomi/waifu2x
EclipseAnatomy 10y ago
Works kind of like PerfectResize for PS then, except this downscales or I might be completely wrong..
- andai 10y ago
  I believe the down-scaling is done the old fashioned way. The up-scaling uses a neural network to guess what the original image was. See waifu2x [0][1]. The new thing here is that they're using it to save bandwidth, and running it on the user side. [0] https://raw.githubusercontent.com/nagadomi/waifu2x/master/images/slide.png https://raw.githubusercontent.com/nagadomi/waifu2x/master/im... [1] http://waifu2x.udp.jp/ http://waifu2x.udp.jp/
tinco 10y ago
The title is a bit misleading. Detail was lost, that is the law. Raisr makes up detail based on some learned likelihood of that detail being there. You could add any image to Raisr and it will make up more detail, you could let it loop over the same picture over and over and it will probably add infinite detail, or up to some fixed point. What's cool is that for many pictures, it's good enough at making up detail that the result is credible.
- tyingq 10y ago
  Seems similar to image inpainting...educated guesses that can be stunningly accurate, or stunningly bad :)
  - kartan 10y ago
    Yes. And as such it has obvious artistic value, as images look better. But I can see that people can misuse them CSI style. You can add more detail to an image, but it is mainly an artistic interpretation that is not the real original image.
- jacobolus 10y ago
  From TFA, “the quality gain is clear to see”, and then they show a picture where pixels have been upsized into weird painterly blotches, with essentially the same level of detail as the original. This doesn’t seem to produce results which are obviously better than the state of the art from 20 years ago. (Or if so, this article certainly doesn’t demonstrate it.) So is the novelty supposed to be that it’s less computationally expensive? Here’s a site I just found in a quick web search with some comparisons of other upscaling tools: http://www.americaswonderlands.com/digital_photo_interpolation.htm http://www.americaswonderlands.com/digital_photo_interpolati...
  - walrus01 10y ago
    The blotches in the example on the right resemble what happens when you run an ordinary raster image through some freely available raster-to-vector image processing tools.
acqq 10y ago
From Google blog: https://www.blog.google/products/google-plus/saving-you-bandwidth-through-machine-learning/ https://www.blog.google/products/google-plus/saving-you-band... "To help everyone be able to see the beautiful photos that photographers share to Google+ in their full glory" "we’ve only begun to roll this out for high-resolution images when they appear in the streams of a subset of Android devices." Anybody has an idea which subset?
- tw04 10y ago
  I'm fairly confident you can assume they're referring to pixel devices.
  - Vexs 10y ago
    And it's probably something like 20% of them.
- lern_too_spel 10y ago
  Probably random. https://support.google.com/googleplay/android-developer/answer/6346149?hl=en https://support.google.com/googleplay/android-developer/answ...
tedunangst 10y ago
If you click enough links, you may eventually find your way to the paper: https://arxiv.org/abs/1606.01299 https://arxiv.org/abs/1606.01299
- itcrowd 10y ago
  Thanks. I'd like to make some comments on the paper - it is very readable, even for non-ML folk - first author was an intern at Google. Cool for him to have a project like this handed to him. - paper is quite long and I don't think it would ever be published in a reputable journal. The content/length ratio is too low, for one - another reason I think it would not get accepted is the lack of novelty. What is being done here is a basic interpolation scheme (which explains the "10-100X speedup over SotA) but with weights determined by a lookup table. The table itself is learned beforehand by analyzing a large number of other images.
  - almostarockstar 10y ago
    I havn't time to read through it, but based on your comments, I could see it being accepted to a conference, which is where most CS publications come from (in europe anyway).
chubs 10y ago
I wonder if this could be used to upscale, say, a 1080p movie frame-by-frame to 4k? Could be fantastic if so. Imagine if it could be built into an FPGA and done in real-time by high-end TVs. Very impressive!
- deleted 10y ago
  [deleted]
- jackvalentine 10y ago
  I thought that 4k devices often had the ability to do this kind of up-scaling already?
- andai 10y ago
  The trick is that using a simple frame-by-frame approach means the details filled in only depend on the frame, not on the details which were filled in to the previous frame, so you get crazy jitter. Deepart.io now has a video mode [0] that takes this into account. [0] https://deepart.io/video/ https://deepart.io/video/
- Const-me 10y ago
  A friend of mine did what you want. Contact him if you want to use the technology, dunno how much he charges for licensing. http://www.infognition.com/VideoEnhancer/ http://www.infognition.com/VideoEnhancer/
- spyder 10y ago
  For videos it's possible to get even better results using pixel information from adjacent frames: https://www.youtube.com/watch?v=5kaI_qr0pFA https://www.youtube.com/watch?v=5kaI_qr0pFA https://people.csail.mit.edu/celiu/CVPR2011/default.html https://people.csail.mit.edu/celiu/CVPR2011/default.html
ingenter 10y ago
Here's a list of various image interpolation techniques: - http://www.ifp.illinois.edu/~jyang29/papers/chap1.pdf http://www.ifp.illinois.edu/~jyang29/papers/chap1.pdf * Image super-resolution: Historical overview and future challenges - http://www.robots.ox.ac.uk/~vgg/publications/papers/pickup08.pdf http://www.robots.ox.ac.uk/~vgg/publications/papers/pickup08... * Machine Learning in Multi-frame Image Super-resolution - http://www.cs.huji.ac.il/~peleg/papers/icpr90-SuperResolutionSequences.pdf http://www.cs.huji.ac.il/~peleg/papers/icpr90-SuperResolutio... - http://www.wisdom.weizmann.ac.il/~vision/SingleImageSR.html http://www.wisdom.weizmann.ac.il/~vision/SingleImageSR.html - http://chiranjivi.tripod.com/EDITut.html http://chiranjivi.tripod.com/EDITut.html - http://www.tecnick.com/pagefiles/appunti/iNEDI_tesi_Nicola_Asuni.pdf http://www.tecnick.com/pagefiles/appunti/iNEDI_tesi_Nicola_A... - http://www.eurasip.org/Proceedings/Eusipco/Eusipco2009/contents/papers/1569192778.pdf http://www.eurasip.org/Proceedings/Eusipco/Eusipco2009/conte... - http://bengal.missouri.edu/~kes25c/ http://bengal.missouri.edu/~kes25c/ * http://bengal.missouri.edu/~kes25c/nnedi3.zip http://bengal.missouri.edu/~kes25c/nnedi3.zip * http://forum.doom9.org/showthread.php?t=147695 http://forum.doom9.org/showthread.php?t=147695 - http://arxiv.org/pdf/1501.00092v2.pdf http://arxiv.org/pdf/1501.00092v2.pdf * http://waifu2x.udp.jp/ http://waifu2x.udp.jp/ * https://github.com/nagadomi/waifu2x https://github.com/nagadomi/waifu2x * http://waifu2x-avisynth.sunnyone.org/ http://waifu2x-avisynth.sunnyone.org/ * https://github.com/sunnyone/Waifu2xAvisynth https://github.com/sunnyone/Waifu2xAvisynth - http://i-programmer.info/news/192-photography-a-imaging/10100-no-comment-super-resolution.html http://i-programmer.info/news/192-photography-a-imaging/1010... * https://github.com/david-gpu/srez https://github.com/david-gpu/srez - http://arxiv.org/pdf/1609.04802v1.pdf http://arxiv.org/pdf/1609.04802v1.pdf - https://github.com/alexjc/neural-enhance https://github.com/alexjc/neural-enhance
- dr_zoidberg 10y ago
  Great list, leaving a comment here to "bookmark" it.
nailer 10y ago
This exactly what Magic Pony (acquired by Twitter last year) does. https://www.inverse.com/article/17200-twitter-buys-london-video-company-magic-pony-to-improve-its-live-video https://www.inverse.com/article/17200-twitter-buys-london-vi... There's an example of them finding a low res image of Barack Obama, and filling in the missing detail via ML to make a compelling high resolution image that still looks like Barack Obama: https://fsmedia.imgix.net/25/36/be/9d/fd04/4c51/9f79/ef675b0143e3/magic-ponys-machine-learning-of-an-image-of-barack-obama.png https://fsmedia.imgix.net/25/36/be/9d/fd04/4c51/9f79/ef675b0... (Upvotes for anyone who a) replies with a better link and b) is Zehan [if b) then HI ZEHAN!])
deadcast 10y ago
Does anyone know if there's something like this available now to experiment with? I've been looking for a new way to ultra compress images and never heard of these techniques before. Looks like Google hasn't released any tools yet to the public, if they ever will.
- randyrand 10y ago
  For storage purposes: If you want, you could just downsample your images right now, and just wait around for this or similar to be released. I dont recommend it.
mschwaig 10y ago
Inventing details out of thin air to make a low resolution image look more detailed feels like cheating to me.
nurettin 10y ago
I think this is what futurama's "better resolution than real life" TV sets are going to look like.
jgalt212 10y ago
How many of these compression plays are driven by a real market need and how many are driven by Pied Piper envy?

107 points by newman314 10y ago

SQL2219 10y ago

I wonder how it compares to this: https://tinyjpg.com/ https://tinyjpg.com/

explorigin 10y ago
The cool thing is that it's not a compression algorithm so you could use both. TinyJPG to compress the smaller-sized image, RAISR to upsample the smaller image. If you've ever played with GIMP (or Photoshop I suppose), RAISR looks like unsharp mask with an neural network controlling the inputs.
- derefr 10y ago
  Now create a psychophysical compressor that optimizes for the weighted combination of smallest compressed size and most-faithful RAISR restoration. I wonder if there's a guaranteed-optimal way to do that, given whatever RAISR is using as an algorithm; some logical inverse to the particular way it adds information. Maybe truncated wavelet compression?

jerf 10y ago

This article and the linked blog post from Google [1] suggest that this has already been deployed. How does it work in the field? For instance, I can imagine it being a JS library that can be downloaded and cached for client-side use, or it could be deployed as an Android module that the Android browser can use. But I don't know where to go looking for this because I don't use Google+, have no photos on it, and wouldn't be sure whether or not I was seeing it. [1]: https://www.blog.google/products/google-plus/saving-you-bandwidth-through-machine-learning/ https://www.blog.google/products/google-plus/saving-you-band...

gwern 10y ago
>RAISR produces results that are comparable to or better than the currently available super-resolution methods, and does so roughly 10 to 100 times faster, allowing it to be run on a typical mobile device in real-time....For example, in addition to improving digital “pinch to zoom” on your phone, one could capture, save, or transmit images at lower resolution and super-resolve on demand without any visible degradation in quality, all while utilizing less of mobile data and storage plans. > In our proposed approach, the run-time is more than one to two orders of magnitude faster than the best competing methods currently available, while producing results comparable or better than state-of-the-art. From the blog post & article, I think the idea is that you ship a small CNN (a few MB model size, which will accordingly run in realtime on mobile GPUs) to the Android device, and then when browsing, it downloads a small thumbnail (cheap on bandwidth, quick to download), and upscale it as necessary locally on the device.
- jacobolus 10y ago
  In general they’d get better bandwidth savings for the same quality and decode effort by shipping the higher-resolution image with more aggressive image compression (using http://bellard.org/bpg/ http://bellard.org/bpg/ or something, not JPEG, and ideally running the decode in hardware). Seems like a dubious method overall. The main impression I get is “woo! neural nets can do everything! let’s use them everywhere!” I don’t believe their claim that this is “comparable or better than the state of the art” for image upscaling without much better evidence than they provide in their paper.

wodenokoto 10y ago

The article makes it sound like it is some sort of machine learning algorithm, which makes me wonder: How is the upscaling done client side?

sushisource 10y ago
It's very much within the realm of possibility to have a pre-trained neural net on the client, which then simply asks it questions. For example, swiftkey does this already with its keyboard prediction. Prisma does it with image styling.
- gkbrk 10y ago
  I actually think Prisma sends the image to their servers for processing, their app size is way too small to contain that many trained networks. Also it doesn't work without an internet connection if I'm correct.
  - HisGraceTheDuck 10y ago
    That used to be the case but recent updates do the processing on the device now. I don't know how they managed it but they did.
- wodenokoto 10y ago
  It is possible, no doubt. You also have image decoder on the client and getting support for a new format is difficult because it is the browser vendors job to distribute these. So how do they practically get the client to decode / upscale? JavaScript?

tornadoboy55 10y ago

The biggest about this is not being able to 'compress' images to 25%, but that you can now take (some) images that we're previously only low-res, and upscale them!

CamelCaseName 10y ago
One day we'll be pointing at pictures yelling "Enhance!", and not in jest!
- fulafel 10y ago
  Good idea for a dystopian CSI. "This AI invented image detail shows you are guilty"
  - patates 10y ago
    Well then it means statistically there's a good chance that what produced the end low res image is incriminating. And of course I'm not serious. Although on a serious note, does anyone know if and how statistical probability is accepted as evidence in a court of law?
    - tornadoboy55 10y ago
      Yeah, it is, that's called circumstantial evidence. You do need direct evidence to validate the circumstantial evidence though..
      - patates 10y ago
        It even has a name, didn't expect that. Thank you.
  - tekoyaki 10y ago
    New episode of Black Mirror or maybe some comedy sketch. "Your face is too generic that the AI drew you as all of our suspects."

ZenoArrow 10y ago

I'd be interested to see if similar techniques could be used for scaling up pixel art. Tools to upscale pixel art do exist already, but the results are a bit hit and miss. Could be a good stress test for the machine learning algorithms too.

andai 10y ago
That sounds interesting. Do you have any links? I must not understand what you mean by scaling up, I though the best way to scale up pixel art was to disable interpolation, so each pixel just becomes a perfectly sharp square?
- ZenoArrow 10y ago
  >"That sounds interesting. Do you have any links?" Plenty to take a look at here: https://en.wikipedia.org/wiki/Pixel_art_scaling_algorithms https://en.wikipedia.org/wiki/Pixel_art_scaling_algorithms
  - jacobolus 10y ago
    Existing pixel-art-specific upscaling methods are going to be better than this, for pixel art. But if they wanted they could probably try to train their neural network for pixel art specifically, similar to https://github.com/nagadomi/waifu2x https://github.com/nagadomi/waifu2x

EclipseAnatomy 10y ago

Works kind of like PerfectResize for PS then, except this downscales or I might be completely wrong..

andai 10y ago
I believe the down-scaling is done the old fashioned way. The up-scaling uses a neural network to guess what the original image was. See waifu2x [0][1]. The new thing here is that they're using it to save bandwidth, and running it on the user side. [0] https://raw.githubusercontent.com/nagadomi/waifu2x/master/images/slide.png https://raw.githubusercontent.com/nagadomi/waifu2x/master/im... [1] http://waifu2x.udp.jp/ http://waifu2x.udp.jp/

tinco 10y ago

The title is a bit misleading. Detail was lost, that is the law. Raisr makes up detail based on some learned likelihood of that detail being there. You could add any image to Raisr and it will make up more detail, you could let it loop over the same picture over and over and it will probably add infinite detail, or up to some fixed point. What's cool is that for many pictures, it's good enough at making up detail that the result is credible.

tyingq 10y ago
Seems similar to image inpainting...educated guesses that can be stunningly accurate, or stunningly bad :)
- kartan 10y ago
  Yes. And as such it has obvious artistic value, as images look better. But I can see that people can misuse them CSI style. You can add more detail to an image, but it is mainly an artistic interpretation that is not the real original image.
jacobolus 10y ago
From TFA, “the quality gain is clear to see”, and then they show a picture where pixels have been upsized into weird painterly blotches, with essentially the same level of detail as the original. This doesn’t seem to produce results which are obviously better than the state of the art from 20 years ago. (Or if so, this article certainly doesn’t demonstrate it.) So is the novelty supposed to be that it’s less computationally expensive? Here’s a site I just found in a quick web search with some comparisons of other upscaling tools: http://www.americaswonderlands.com/digital_photo_interpolation.htm http://www.americaswonderlands.com/digital_photo_interpolati...
- walrus01 10y ago
  The blotches in the example on the right resemble what happens when you run an ordinary raster image through some freely available raster-to-vector image processing tools.

acqq 10y ago

From Google blog: https://www.blog.google/products/google-plus/saving-you-bandwidth-through-machine-learning/ https://www.blog.google/products/google-plus/saving-you-band... "To help everyone be able to see the beautiful photos that photographers share to Google+ in their full glory" "we’ve only begun to roll this out for high-resolution images when they appear in the streams of a subset of Android devices." Anybody has an idea which subset?

tw04 10y ago
I'm fairly confident you can assume they're referring to pixel devices.
- Vexs 10y ago
  And it's probably something like 20% of them.
lern_too_spel 10y ago
Probably random. https://support.google.com/googleplay/android-developer/answer/6346149?hl=en https://support.google.com/googleplay/android-developer/answ...

tedunangst 10y ago

If you click enough links, you may eventually find your way to the paper: https://arxiv.org/abs/1606.01299 https://arxiv.org/abs/1606.01299

itcrowd 10y ago
Thanks. I'd like to make some comments on the paper - it is very readable, even for non-ML folk - first author was an intern at Google. Cool for him to have a project like this handed to him. - paper is quite long and I don't think it would ever be published in a reputable journal. The content/length ratio is too low, for one - another reason I think it would not get accepted is the lack of novelty. What is being done here is a basic interpolation scheme (which explains the "10-100X speedup over SotA) but with weights determined by a lookup table. The table itself is learned beforehand by analyzing a large number of other images.
- almostarockstar 10y ago
  I havn't time to read through it, but based on your comments, I could see it being accepted to a conference, which is where most CS publications come from (in europe anyway).

chubs 10y ago

I wonder if this could be used to upscale, say, a 1080p movie frame-by-frame to 4k? Could be fantastic if so. Imagine if it could be built into an FPGA and done in real-time by high-end TVs. Very impressive!

deleted 10y ago
[deleted]
jackvalentine 10y ago
I thought that 4k devices often had the ability to do this kind of up-scaling already?
andai 10y ago
The trick is that using a simple frame-by-frame approach means the details filled in only depend on the frame, not on the details which were filled in to the previous frame, so you get crazy jitter. Deepart.io now has a video mode [0] that takes this into account. [0] https://deepart.io/video/ https://deepart.io/video/
Const-me 10y ago
A friend of mine did what you want. Contact him if you want to use the technology, dunno how much he charges for licensing. http://www.infognition.com/VideoEnhancer/ http://www.infognition.com/VideoEnhancer/
spyder 10y ago
For videos it's possible to get even better results using pixel information from adjacent frames: https://www.youtube.com/watch?v=5kaI_qr0pFA https://www.youtube.com/watch?v=5kaI_qr0pFA https://people.csail.mit.edu/celiu/CVPR2011/default.html https://people.csail.mit.edu/celiu/CVPR2011/default.html

ingenter 10y ago

Here's a list of various image interpolation techniques: - http://www.ifp.illinois.edu/~jyang29/papers/chap1.pdf http://www.ifp.illinois.edu/~jyang29/papers/chap1.pdf * Image super-resolution: Historical overview and future challenges - http://www.robots.ox.ac.uk/~vgg/publications/papers/pickup08.pdf http://www.robots.ox.ac.uk/~vgg/publications/papers/pickup08... * Machine Learning in Multi-frame Image Super-resolution - http://www.cs.huji.ac.il/~peleg/papers/icpr90-SuperResolutionSequences.pdf http://www.cs.huji.ac.il/~peleg/papers/icpr90-SuperResolutio... - http://www.wisdom.weizmann.ac.il/~vision/SingleImageSR.html http://www.wisdom.weizmann.ac.il/~vision/SingleImageSR.html - http://chiranjivi.tripod.com/EDITut.html http://chiranjivi.tripod.com/EDITut.html - http://www.tecnick.com/pagefiles/appunti/iNEDI_tesi_Nicola_Asuni.pdf http://www.tecnick.com/pagefiles/appunti/iNEDI_tesi_Nicola_A... - http://www.eurasip.org/Proceedings/Eusipco/Eusipco2009/contents/papers/1569192778.pdf http://www.eurasip.org/Proceedings/Eusipco/Eusipco2009/conte... - http://bengal.missouri.edu/~kes25c/ http://bengal.missouri.edu/~kes25c/ * http://bengal.missouri.edu/~kes25c/nnedi3.zip http://bengal.missouri.edu/~kes25c/nnedi3.zip * http://forum.doom9.org/showthread.php?t=147695 http://forum.doom9.org/showthread.php?t=147695 - http://arxiv.org/pdf/1501.00092v2.pdf http://arxiv.org/pdf/1501.00092v2.pdf * http://waifu2x.udp.jp/ http://waifu2x.udp.jp/ * https://github.com/nagadomi/waifu2x https://github.com/nagadomi/waifu2x * http://waifu2x-avisynth.sunnyone.org/ http://waifu2x-avisynth.sunnyone.org/ * https://github.com/sunnyone/Waifu2xAvisynth https://github.com/sunnyone/Waifu2xAvisynth - http://i-programmer.info/news/192-photography-a-imaging/10100-no-comment-super-resolution.html http://i-programmer.info/news/192-photography-a-imaging/1010... * https://github.com/david-gpu/srez https://github.com/david-gpu/srez - http://arxiv.org/pdf/1609.04802v1.pdf http://arxiv.org/pdf/1609.04802v1.pdf - https://github.com/alexjc/neural-enhance https://github.com/alexjc/neural-enhance

dr_zoidberg 10y ago
Great list, leaving a comment here to "bookmark" it.

nailer 10y ago

This exactly what Magic Pony (acquired by Twitter last year) does. https://www.inverse.com/article/17200-twitter-buys-london-video-company-magic-pony-to-improve-its-live-video https://www.inverse.com/article/17200-twitter-buys-london-vi... There's an example of them finding a low res image of Barack Obama, and filling in the missing detail via ML to make a compelling high resolution image that still looks like Barack Obama: https://fsmedia.imgix.net/25/36/be/9d/fd04/4c51/9f79/ef675b0143e3/magic-ponys-machine-learning-of-an-image-of-barack-obama.png https://fsmedia.imgix.net/25/36/be/9d/fd04/4c51/9f79/ef675b0... (Upvotes for anyone who a) replies with a better link and b) is Zehan [if b) then HI ZEHAN!])

deadcast 10y ago

Does anyone know if there's something like this available now to experiment with? I've been looking for a new way to ultra compress images and never heard of these techniques before. Looks like Google hasn't released any tools yet to the public, if they ever will.

randyrand 10y ago
For storage purposes: If you want, you could just downsample your images right now, and just wait around for this or similar to be released. I dont recommend it.

mschwaig 10y ago

Inventing details out of thin air to make a low resolution image look more detailed feels like cheating to me.

nurettin 10y ago

I think this is what futurama's "better resolution than real life" TV sets are going to look like.

jgalt212 10y ago

How many of these compression plays are driven by a real market need and how many are driven by Pied Piper envy?