4 ms·

Does anyone know if it will ever be possible to use this technology on demand to upscale local (not streamed) video files?

by spuz 4y ago

ubercow13 4y ago
There are already NN based shaders you can use to upscale video locally such as in madVR and with mpv and FSRCNXX. I'm not sure how that 'AI upscaling' is related to the algorithms used for this.
brucethemoose2 4y ago
You can already use 2D networks, and they work pretty well, but there actually is a dearth of usable temporal video upscalers even though there are like 100+ papers on the subject.
belltaco 4y ago
You can already play many local files in formats like mp4 directly in Edge or Chrome to get Nvidia VSR working, probably works with the new Edge VSR too. If you want a media player that does this and have a 30 or 40 series Nvidia card, you can use the below custom renderer being actively developed, which uses Nvidia VSR tech with a custom renderer for Media Player Classic. https://github.com/emoose/VideoRenderer https://github.com/emoose/VideoRenderer People have also been using MadVR to upscale videos which uses a similar renderer but I think it doesn't have GPU limitations.
- spuz 4y ago
  I should have been more specific and said that I want to upscale videos and then save them, not just view them. Thanks for the suggestions though.
  - vosper 4y ago
    I think that's what Topaz AI does? https://www.topazlabs.com/ https://www.topazlabs.com/
Havoc 4y ago
I've seen AI upscaled stuff on the pirate seas lately so the tech already exists & is in active use for non-streamed. I don't know the details unfortunately
rfwhyte 4y ago
https://www.topazlabs.com/topaz-video-ai https://www.topazlabs.com/topaz-video-ai
brucethemoose2 4y ago
And to be more specific, for non-realtime transcoding you want to use ESRGAN in VapourSynth with models from upscale.wiki. You generally want a denoiser in front of the AI model and a temporal softener behind it. This is quite a rabbit hole, and I can point to more specific filter setups if you want. For realtime, mpv shaders and mpc-be extensions linked above are good. TBH I havent done a subjective quality shootout between these in awhile. Smart TVs also have AI ASICs that can do a surprisingly good job. That + vapoursynth transcodes is what I lazily rely on these days.
- addandsubtract 4y ago
  I'd be interested in more info about the non-realtime upscaling. I've used ESRGAN before, but never a temporal softener. Can you share some best practices?

Does anyone know if it will ever be possible to use this technology on demand to upscale local (not streamed) video files?

by spuz 4y ago

Does anyone know if it will ever be possible to use this technology on demand to upscale local (not streamed) video files?

ubercow13 4y ago

There are already NN based shaders you can use to upscale video locally such as in madVR and with mpv and FSRCNXX. I'm not sure how that 'AI upscaling' is related to the algorithms used for this.

brucethemoose2 4y ago

You can already use 2D networks, and they work pretty well, but there actually is a dearth of usable temporal video upscalers even though there are like 100+ papers on the subject.

belltaco 4y ago

You can already play many local files in formats like mp4 directly in Edge or Chrome to get Nvidia VSR working, probably works with the new Edge VSR too. If you want a media player that does this and have a 30 or 40 series Nvidia card, you can use the below custom renderer being actively developed, which uses Nvidia VSR tech with a custom renderer for Media Player Classic. https://github.com/emoose/VideoRenderer https://github.com/emoose/VideoRenderer People have also been using MadVR to upscale videos which uses a similar renderer but I think it doesn't have GPU limitations.

spuz 4y ago
I should have been more specific and said that I want to upscale videos and then save them, not just view them. Thanks for the suggestions though.
- vosper 4y ago
  I think that's what Topaz AI does? https://www.topazlabs.com/ https://www.topazlabs.com/

Havoc 4y ago

I've seen AI upscaled stuff on the pirate seas lately so the tech already exists & is in active use for non-streamed. I don't know the details unfortunately

rfwhyte 4y ago

https://www.topazlabs.com/topaz-video-ai https://www.topazlabs.com/topaz-video-ai

brucethemoose2 4y ago

And to be more specific, for non-realtime transcoding you want to use ESRGAN in VapourSynth with models from upscale.wiki. You generally want a denoiser in front of the AI model and a temporal softener behind it. This is quite a rabbit hole, and I can point to more specific filter setups if you want. For realtime, mpv shaders and mpc-be extensions linked above are good. TBH I havent done a subjective quality shootout between these in awhile. Smart TVs also have AI ASICs that can do a surprisingly good job. That + vapoursynth transcodes is what I lazily rely on these days.

addandsubtract 4y ago
I'd be interested in more info about the non-realtime upscaling. I've used ESRGAN before, but never a temporal softener. Can you share some best practices?