6 ms·

I'm not sure I understand how the author compares the quality in the preprint. In the chart, it says to compare "perceptual quality", but the axis is only mark

I'm not sure I understand how the author compares the quality in the preprint. In the chart, it says to compare "perceptual quality", but the axis is only marked with "blurry" and "less blurry". Sharpness is not the only thing about the (perceptual or not) quality. I can tell that Anime4K's result is indeed very sharp, but the quality of the edges/lines are very unnatural even for the examples author provided. I personally would prefer a slightly blurry lines with less "oily effect". Also, I didn't see any comparison with ground truth, i.e. having a high-resolution image first, resize it down, use the proposed algorithms (among existing ones) to upscale it back, and then compare the upscaled results with the original image. I understand it may be hard to find enough examples of 4k animes, but we can do so with 1080p -> 480p -> 1080p etc. (I am not familiar with this domain, do similar researches normally do this or not in their analysis?)

janjanson 7y ago
I'm in the same boat. There are different metrics to judge the quality of an upscale (peak signal to noise ratio comes to mind), but it's obvious they are limited in that they can't capture perceptual quality very closely. While it's obvious the filter is much sharper than even NGU sharp, it also seems to come with a weird gradient effect and some artifacting. Another thing I find is that sharper filters like NGU sharp don't upscale as well as other upscaling options on frames with edges that aren't supposed to be sharp, probably because in some sense they try to hard ink in parts that aren't supposed to be so. This can happen either because the source is composited to be blurry for artistic effect, or because the source is low quality. I admittedly have not tried Anime4K, but I imagine Anime4K will have a similar effect. Of course, in the end, it's an entirely subjective thing. Personally I hold off using on using NGU sharp and use NGU Anti-Alias instead for the above reasons. EDIT: this is addressed in the readme: I think the results are worse! -Surely some people like sharper edges, some like softer ones. Do try it yourself on a few anime before reaching a definite conclusion. People tend to prefer sharper edges. Also, seeing the comparisons on a 1080p screen is not representative of the final results on a 4K screen, the pixel density and sharpness of the final image is simply not comparable. EDIT: I just tried this filter on a 4k monitor. To honest I don't think is very good. To me it reminds me of the bad parts of sharpeners turned up to the max. All the edges turn into a weird, sometimes jagged, smear, and originally blurry but detailed backgrounds just become a weird mess. I really don't think even people who like sharpness will prefer this filter for general viewing, and I find the chart given in the preprint (https://raw.githubusercontent.com/bloc97/Anime4K/master/results/Graph.png https://raw.githubusercontent.com/bloc97/Anime4K/master/resu...) extremely dubious.
asdfasgasdgasdg 7y ago
Edges seem pretty reasonable to me in the 1:1 image. I'd dramatically prefer to watch shows with the sharpness of the post-processed image. This is a taste thing, and different folks will have different strokes.
Nadya 7y ago
There is no ground truth because AFAIK there are no native 4k anime produced yet. There are _very_ few produced at 1080p. Most 1080p that are released by studios are just an upscaled 720p master. Fansubbers will sometimes release their own upscaled 720p when the studio upscale was done very poorly. To my knowledge, not much has changed since 2017 where only a single anime (Clockwork Planet) was produced in 1080p. The only two studios I can name offhand that I know have done 1080p masters are KyoAni and JC Staff. [0] 2017 reference: https://www.reddit.com/r/anime/comments/65wqeu/spring_2017_anime_resolution_chart/ https://www.reddit.com/r/anime/comments/65wqeu/spring_2017_a...
- ihuman 7y ago
  Your Name and the 2 Gundam Thunderbolt movies have 4K BDs, but I don't know if they are upscaled or native 4K. There's also a Gundam F91 and Space Adventure Cobra movie 4K remaster, but those were animated with cells, so I don't know how useful those would be for testing an upscaler.
  - fireattack 7y ago
    IIRC the consensus is Your Name is not native 4k. Not sure about the others.
- fireattack 7y ago
  Yeah I knew. But he can do it with 1080p/480p, or 720p/480p.
- egypturnash 7y ago
  A reasonable substitute for ground truth might be to get a single drawing done in an anime style with the appropriate pixel dimensions. There are many artists out there who can produce such a thing for a modest fee.
  - resoluteteeth 7y ago
    This would apparently cause other problems: > Interesting enough, waifu2x performed very poorly on anime. A plausible explaination is that the network was simply not trained to upscale these types of images. Usually anime style art have sharper lines and contain much more small details/textures compared to anime. The distribution of images used to train waifu2x must have been mostly art images from sites like DevianArt/Danbooru/Pixiv, and not anime.

janjanson 7y ago

I'm in the same boat. There are different metrics to judge the quality of an upscale (peak signal to noise ratio comes to mind), but it's obvious they are limited in that they can't capture perceptual quality very closely. While it's obvious the filter is much sharper than even NGU sharp, it also seems to come with a weird gradient effect and some artifacting. Another thing I find is that sharper filters like NGU sharp don't upscale as well as other upscaling options on frames with edges that aren't supposed to be sharp, probably because in some sense they try to hard ink in parts that aren't supposed to be so. This can happen either because the source is composited to be blurry for artistic effect, or because the source is low quality. I admittedly have not tried Anime4K, but I imagine Anime4K will have a similar effect. Of course, in the end, it's an entirely subjective thing. Personally I hold off using on using NGU sharp and use NGU Anti-Alias instead for the above reasons. EDIT: this is addressed in the readme: I think the results are worse! -Surely some people like sharper edges, some like softer ones. Do try it yourself on a few anime before reaching a definite conclusion. People tend to prefer sharper edges. Also, seeing the comparisons on a 1080p screen is not representative of the final results on a 4K screen, the pixel density and sharpness of the final image is simply not comparable. EDIT: I just tried this filter on a 4k monitor. To honest I don't think is very good. To me it reminds me of the bad parts of sharpeners turned up to the max. All the edges turn into a weird, sometimes jagged, smear, and originally blurry but detailed backgrounds just become a weird mess. I really don't think even people who like sharpness will prefer this filter for general viewing, and I find the chart given in the preprint (https://raw.githubusercontent.com/bloc97/Anime4K/master/results/Graph.png https://raw.githubusercontent.com/bloc97/Anime4K/master/resu...) extremely dubious.

asdfasgasdgasdg 7y ago

Edges seem pretty reasonable to me in the 1:1 image. I'd dramatically prefer to watch shows with the sharpness of the post-processed image. This is a taste thing, and different folks will have different strokes.

Nadya 7y ago

There is no ground truth because AFAIK there are no native 4k anime produced yet. There are _very_ few produced at 1080p. Most 1080p that are released by studios are just an upscaled 720p master. Fansubbers will sometimes release their own upscaled 720p when the studio upscale was done very poorly. To my knowledge, not much has changed since 2017 where only a single anime (Clockwork Planet) was produced in 1080p. The only two studios I can name offhand that I know have done 1080p masters are KyoAni and JC Staff. [0] 2017 reference: https://www.reddit.com/r/anime/comments/65wqeu/spring_2017_anime_resolution_chart/ https://www.reddit.com/r/anime/comments/65wqeu/spring_2017_a...

ihuman 7y ago
Your Name and the 2 Gundam Thunderbolt movies have 4K BDs, but I don't know if they are upscaled or native 4K. There's also a Gundam F91 and Space Adventure Cobra movie 4K remaster, but those were animated with cells, so I don't know how useful those would be for testing an upscaler.
- fireattack 7y ago
  IIRC the consensus is Your Name is not native 4k. Not sure about the others.
fireattack 7y ago
Yeah I knew. But he can do it with 1080p/480p, or 720p/480p.
egypturnash 7y ago
A reasonable substitute for ground truth might be to get a single drawing done in an anime style with the appropriate pixel dimensions. There are many artists out there who can produce such a thing for a modest fee.
- resoluteteeth 7y ago
  This would apparently cause other problems: > Interesting enough, waifu2x performed very poorly on anime. A plausible explaination is that the network was simply not trained to upscale these types of images. Usually anime style art have sharper lines and contain much more small details/textures compared to anime. The distribution of images used to train waifu2x must have been mostly art images from sites like DevianArt/Danbooru/Pixiv, and not anime.