5 ms·

Very exciting, I hope this will help open-source hardware-accelerated image upscaling. To get an idea of what image upscaling can do, check out this subreddit

Very exciting, I hope this will help open-source hardware-accelerated image upscaling. To get an idea of what image upscaling can do, check out this subreddit dedicated to artificially increasing the resolution of old video game assets to give them a new life: https://www.reddit.com/r/GameUpscale/ https://www.reddit.com/r/GameUpscale/ Here's eg some personal experiments with the game Baldur's Gate: https://www.reddit.com/r/baldursgate/comments/aemu8u/upscaling_bg2s_assets_using_machine_learning/ https://www.reddit.com/r/baldursgate/comments/aemu8u/upscali... - Image in: https://i.imgur.com/xMbqIeo.jpeg https://i.imgur.com/xMbqIeo.jpeg - Image out, without human intervention: https://i.imgur.com/nC2o1vM.jpeg https://i.imgur.com/nC2o1vM.jpeg The community is using tools such as ESRGAN for that (https://github.com/xinntao/ESRGAN https://github.com/xinntao/ESRGAN), which leverages GPU acceleration as it uses PyTorch. So far, this upsampling ability was only used in commercial games: https://en.wikipedia.org/wiki/Deep_learning_super_sampling https://en.wikipedia.org/wiki/Deep_learning_super_sampling

kllrnohj 5y ago
> I hope this will enable open-source hardware-accelerated image upscaling That's already a thing (especially with AMDs FSR already being open source), but regardless keep in mind that DLSS 2.0+ requires engine support because it also uses things like motion vectors. Retrofitting this into old games is possible, but likely non-trivial, and would require the game to be adjusted it's not something the driver can just do. > However, recent NVIDIA GPUs contain dedicated silicon to perform this type of upscaling, which I'm really hoping will make things even faster! There's no dedicated silicon for DLSS. It's "just" the tensor cores, which are programmable and is something PyTorch already has access to. CUDA support for those have been around for a while after all: https://developer.nvidia.com/blog/programming-tensor-cores-cuda-9/ https://developer.nvidia.com/blog/programming-tensor-cores-c...
- MasterScrat 5y ago
  > There's no dedicated silicon for DLSS. You're right, corrected.
petermcneeley 5y ago
When does "upsampling" cross into being a derivative work?
zozbot234 5y ago
You don't need deep learning for state-of-the-art image/texture upscaling. The GIMP or Imagemagick with a Lanczos-like method give you that basically out of the box, and the underlying method could easily be optimized for GPU use. AIUI, even AMD's FSR does not use deep learning, and the results are very good quality.
GuB-42 5y ago
> Very exciting, I hope this will help open-source hardware-accelerated image upscaling. It is also a legal minefield. Nvidia is certainly going to look closely at whatever may contain the leaked code. They are unlikely to do much if you just copy their techniques (unless they are patented, you need to watch that out too), but on community projects, you have to make sure that none of your contributors copy the code itself. ReactOS (essentially a Windows reimplementation) explicitly tells people who had access to the Microsoft source code not to work on the relevant parts of the project. Probably for that reason.
- google234123 5y ago
  ReactOS is pretty well know to use leaked window code as a guide. But who cares about an unstable windows 7 implementation anyway so msft doesn’t care.
jjav 5y ago
> Very exciting, I hope this will help open-source hardware-accelerated image upscaling. Be aware that anyone who works in projects conceivably related to the source code in question needs to be very careful to never see it. Otherwise it becomes a legal minefield where future code can be accused to have been stolen (even if it very clearly never was).

kllrnohj 5y ago

> I hope this will enable open-source hardware-accelerated image upscaling That's already a thing (especially with AMDs FSR already being open source), but regardless keep in mind that DLSS 2.0+ requires engine support because it also uses things like motion vectors. Retrofitting this into old games is possible, but likely non-trivial, and would require the game to be adjusted it's not something the driver can just do. > However, recent NVIDIA GPUs contain dedicated silicon to perform this type of upscaling, which I'm really hoping will make things even faster! There's no dedicated silicon for DLSS. It's "just" the tensor cores, which are programmable and is something PyTorch already has access to. CUDA support for those have been around for a while after all: https://developer.nvidia.com/blog/programming-tensor-cores-cuda-9/ https://developer.nvidia.com/blog/programming-tensor-cores-c...

MasterScrat 5y ago
> There's no dedicated silicon for DLSS. You're right, corrected.

petermcneeley 5y ago

When does "upsampling" cross into being a derivative work?

zozbot234 5y ago

You don't need deep learning for state-of-the-art image/texture upscaling. The GIMP or Imagemagick with a Lanczos-like method give you that basically out of the box, and the underlying method could easily be optimized for GPU use. AIUI, even AMD's FSR does not use deep learning, and the results are very good quality.

GuB-42 5y ago

> Very exciting, I hope this will help open-source hardware-accelerated image upscaling. It is also a legal minefield. Nvidia is certainly going to look closely at whatever may contain the leaked code. They are unlikely to do much if you just copy their techniques (unless they are patented, you need to watch that out too), but on community projects, you have to make sure that none of your contributors copy the code itself. ReactOS (essentially a Windows reimplementation) explicitly tells people who had access to the Microsoft source code not to work on the relevant parts of the project. Probably for that reason.

google234123 5y ago
ReactOS is pretty well know to use leaked window code as a guide. But who cares about an unstable windows 7 implementation anyway so msft doesn’t care.

jjav 5y ago

> Very exciting, I hope this will help open-source hardware-accelerated image upscaling. Be aware that anyone who works in projects conceivably related to the source code in question needs to be very careful to never see it. Otherwise it becomes a legal minefield where future code can be accused to have been stolen (even if it very clearly never was).