4 ms·

I think I'm missing something. I get the Sketch-to-Photo synthesis that this is based on. It's really weird, neat stuff. But as a layman, I'm having trouble see

I think I'm missing something. I get the Sketch-to-Photo synthesis that this is based on. It's really weird, neat stuff. But as a layman, I'm having trouble seeing the difference between the result of this anime-to-sketch synthesis and what I'd expect to get out of a simple edge detection. Is the difference that it's more clever about which details to ignore?

greatgoat420 5y ago
I was actually going to ask if someone had done a comparison with edge detection.
- loosetypes 5y ago
  Also not familiar but presumably a temporal aspect weighs in, so whether something is a meaningful edge isn’t strictly dependent only on the content within a specific frame?
  - edge17 5y ago
    My high level guess is, you would do some linear cost minimization in deciding what are edges over time when doing things the traditional way. The neural network can handle nonlinearities for the optimization so you can get better results for some set of inputs (in this case, some class of anime images)
mattigames 5y ago
Not actually, here is what most common edge detection methods produce (last one is Anime2Sketch): https://i.imgur.com/nt0D1ef.png https://i.imgur.com/nt0D1ef.png
- Chris2048 5y ago
  link appears to be broken?
  - mattigames 5y ago
    It's working for me, here it is on a different hosting site if that helps https://i.ibb.co/pn5d01f/nt0-D1ef-d.webp https://i.ibb.co/pn5d01f/nt0-D1ef-d.webp
    - Chris2048 5y ago
      Nope, getting a timeout for that too.
      - mattigames 5y ago
        It must be your connection then (or the image format), last try: here it is again in Google Drive: https://drive.google.com/file/d/1rMagXGgth-pAXH2RQ-Ltj59jpgb9C3_b/view?usp=sharing https://drive.google.com/file/d/1rMagXGgth-pAXH2RQ-Ltj59jpgb...
- CobrastanJorji 5y ago
  Okay, that's pretty significant. Neat, thanks!
- gregoriol 5y ago
  Your edge detection examples are really basic stuff: there are way more advanced studies and examples. Search for example for "Coherent Line Drawing" by Henry Kang (image from the research: https://d3i71xaburhd42.cloudfront.net/92b2ec5ef4f58c4206fc5cee9c5f0e9435870c69/2-Figure2-1.png https://d3i71xaburhd42.cloudfront.net/92b2ec5ef4f58c4206fc5c...) or for "XDoG: An eXtended difference-of-Gaussians ..." by Holger Winnemoller, Jan Eric Kyprianidis, ... (https://ars.els-cdn.com/content/image/1-s2.0-S009784931200043X-gr2.jpg https://ars.els-cdn.com/content/image/1-s2.0-S00978493120004...)
fsloth 5y ago
I only dabble in graphics - but generallly simple edge detection needs really uniform tonality and no textures in the input to work well. Look for example in the more "sketchy" examples how the linework that "looks right" is extracted from a quite noisy input. Also, in the top example where there are houses, the contrast difference that gets extracted to linework is lower than in the character areas. So, the flat shaded images with explicit black outlines - yes, it's likely there is much difference with edge detection. But when the image has lots of different contrasts and tonalities this looks much more impressive.

greatgoat420 5y ago

I was actually going to ask if someone had done a comparison with edge detection.

loosetypes 5y ago
Also not familiar but presumably a temporal aspect weighs in, so whether something is a meaningful edge isn’t strictly dependent only on the content within a specific frame?
- edge17 5y ago
  My high level guess is, you would do some linear cost minimization in deciding what are edges over time when doing things the traditional way. The neural network can handle nonlinearities for the optimization so you can get better results for some set of inputs (in this case, some class of anime images)

mattigames 5y ago

Not actually, here is what most common edge detection methods produce (last one is Anime2Sketch): https://i.imgur.com/nt0D1ef.png https://i.imgur.com/nt0D1ef.png

Chris2048 5y ago
link appears to be broken?
- mattigames 5y ago
  It's working for me, here it is on a different hosting site if that helps https://i.ibb.co/pn5d01f/nt0-D1ef-d.webp https://i.ibb.co/pn5d01f/nt0-D1ef-d.webp
  - Chris2048 5y ago
    Nope, getting a timeout for that too.
    - mattigames 5y ago
      It must be your connection then (or the image format), last try: here it is again in Google Drive: https://drive.google.com/file/d/1rMagXGgth-pAXH2RQ-Ltj59jpgb9C3_b/view?usp=sharing https://drive.google.com/file/d/1rMagXGgth-pAXH2RQ-Ltj59jpgb...
CobrastanJorji 5y ago
Okay, that's pretty significant. Neat, thanks!
gregoriol 5y ago
Your edge detection examples are really basic stuff: there are way more advanced studies and examples. Search for example for "Coherent Line Drawing" by Henry Kang (image from the research: https://d3i71xaburhd42.cloudfront.net/92b2ec5ef4f58c4206fc5cee9c5f0e9435870c69/2-Figure2-1.png https://d3i71xaburhd42.cloudfront.net/92b2ec5ef4f58c4206fc5c...) or for "XDoG: An eXtended difference-of-Gaussians ..." by Holger Winnemoller, Jan Eric Kyprianidis, ... (https://ars.els-cdn.com/content/image/1-s2.0-S009784931200043X-gr2.jpg https://ars.els-cdn.com/content/image/1-s2.0-S00978493120004...)

fsloth 5y ago

I only dabble in graphics - but generallly simple edge detection needs really uniform tonality and no textures in the input to work well. Look for example in the more "sketchy" examples how the linework that "looks right" is extracted from a quite noisy input. Also, in the top example where there are houses, the contrast difference that gets extracted to linework is lower than in the character areas. So, the flat shaded images with explicit black outlines - yes, it's likely there is much difference with edge detection. But when the image has lots of different contrasts and tonalities this looks much more impressive.