5 ms·

This sounds like the same approach as Gigapixel AI from Topaz Labs. I haven't tried Gigapixel but I have used Topaz' Video Enhance AI, which is phenomenal. I'

This sounds like the same approach as Gigapixel AI from Topaz Labs. I haven't tried Gigapixel but I have used Topaz' Video Enhance AI, which is phenomenal. I've been using it to upscale old TV shows which never got an HD remaster, to UHD. Right now it's running through the first episode of Firefly, converting from 540p to 2160p (540p as the bluray rip was basically upscaled to 1080p from its original production, so I converted to 540p first in Handbrake with zero noticeable loss in quality since I used a near-lossless compression factor.. this provides better upscaling): https://i.imgur.com/hcRYM5n.jpg https://i.imgur.com/hcRYM5n.jpg When it's done I'll run it through Flowframes for framerate interpolation. Then maybe another pass in Handbrake to figure out an optimal size for the end file. Then I'll run through the rest of the season using the same settings I tested with this first episode.

tokamak-teapot 6y ago
It looks like the character changed into a silk version of his shirt with no chest pocket.
- heroprotagonist 6y ago
  Yeah, I'm not sure I have the settings quite right on this one yet. There are several AI models to choose from to get an optimal result, some of which have configurable parameters to control for this. I noticed with this model that really fine lines will have a tendency to get smoothed out a little. There's a similar model which should pull a little more detail but typically this one seems to work best. It's less noticeable once the video is in motion, compared to a still image. I also probably removed too much grain from this, hence the more 'silky' look. It's nice for skin but less so for textures.
parhamn 6y ago
That’s cool, thanks for sharing. How long does a conversion like that usually take?
- PaulBGD_ 6y ago
  In my personal experience with Topaz and a 3060 ti, usually 4-5 frames per second. Although it depends on the input and output resolutions.
liuliu 6y ago
Videos can use inter-frame information to help infer sub-pixel details. This post about "Super Resolution" is interesting because it starts with RAW format (which contains information about camera sensor arrangements), hence, the machine-learned model should not only memorize artificial details (what hair should be look like, what a tree-leaf should be look like etc, and use that to "hallucinate" a higher-resolution details, I liked to call this "hallucination" for that reason), but also relationship between complex interference of different sensors in their corresponding arrangements. You can read more about RAW format and why exposing RAW format for photography is exciting (on everyday's camera, i.e. your phone) from this post: https://blog.halide.cam/understanding-proraw-4eed556d4c54 https://blog.halide.cam/understanding-proraw-4eed556d4c54
zokier 6y ago
I might be old and grumpy, but I prefer the left image to the right one? The right one is watercolory and overly smoothed, like a cheap beautify photofilter
- heroprotagonist 6y ago
  You can adjust the grain and denoise settings to change that, or switch AI models. Probably some other settings I'm not expert with as well. This was just what I had running at the moment, which was a first pass on the first episode. I agree on the smoothness, it takes a little tweaking to get right.
deleted 6y ago
[deleted]

tokamak-teapot 6y ago

It looks like the character changed into a silk version of his shirt with no chest pocket.

heroprotagonist 6y ago
Yeah, I'm not sure I have the settings quite right on this one yet. There are several AI models to choose from to get an optimal result, some of which have configurable parameters to control for this. I noticed with this model that really fine lines will have a tendency to get smoothed out a little. There's a similar model which should pull a little more detail but typically this one seems to work best. It's less noticeable once the video is in motion, compared to a still image. I also probably removed too much grain from this, hence the more 'silky' look. It's nice for skin but less so for textures.

parhamn 6y ago

That’s cool, thanks for sharing. How long does a conversion like that usually take?

PaulBGD_ 6y ago
In my personal experience with Topaz and a 3060 ti, usually 4-5 frames per second. Although it depends on the input and output resolutions.

liuliu 6y ago

Videos can use inter-frame information to help infer sub-pixel details. This post about "Super Resolution" is interesting because it starts with RAW format (which contains information about camera sensor arrangements), hence, the machine-learned model should not only memorize artificial details (what hair should be look like, what a tree-leaf should be look like etc, and use that to "hallucinate" a higher-resolution details, I liked to call this "hallucination" for that reason), but also relationship between complex interference of different sensors in their corresponding arrangements. You can read more about RAW format and why exposing RAW format for photography is exciting (on everyday's camera, i.e. your phone) from this post: https://blog.halide.cam/understanding-proraw-4eed556d4c54 https://blog.halide.cam/understanding-proraw-4eed556d4c54

zokier 6y ago

I might be old and grumpy, but I prefer the left image to the right one? The right one is watercolory and overly smoothed, like a cheap beautify photofilter

heroprotagonist 6y ago
You can adjust the grain and denoise settings to change that, or switch AI models. Probably some other settings I'm not expert with as well. This was just what I had running at the moment, which was a first pass on the first episode. I agree on the smoothness, it takes a little tweaking to get right.

deleted 6y ago

[deleted]