7 ms·

Diffusion Curves: A Vector Representation for Smooth-Shaded Images

110 points by currywurst 14y ago

currywurst 14y ago
Found this in a discussion on SVG 2. The automatic generation of the curves from images is the killer feature !
Mordor 14y ago
Could this be used in image/video compression?
- agumonkey 14y ago
  I'm not sure if it can be compressed in the time domain though.
  - yk 14y ago
    Usually vector based formats compress very well in the time domain, since you only need to store the transformations. But without testing the code, I would be surprised if it is fast enough for something like 50 frames.
    - agumonkey 14y ago
      But to have 'organic' images you'd need to have a lot of small curves that would almost be like storing vector derivatives for pixels.
    - corysama 14y ago
      Convert the (x,y) curves to surfaces in the (x,y,time) domain and you should get a representation that is very fast to evaluate. With fast forward differencing, animating the curves would only need 6 adds per control point. If you are fine with "A Scanner Darkly"-style video it would work great! BTW: Vector videos have been used to squeeze full-screen cinematics into a Super Nintendo: http://fabiensanglard.net/anotherWorld_code_review/index.php http://fabiensanglard.net/anotherWorld_code_review/index.php
  - mistercow 14y ago
    It doesn't look like it performs very well in the time domain even without compression. If you pay attention to the very brief time when they talk about supporting animation, there's very clear "leaking" effect as the gradients shift. Maybe that could be corrected with very tightly controlled parameters, but tightly controlled parameters = more data + less margin for error.
- Geee 14y ago
  I think this might be useful for 'very-low bit-rate' video quality, so you would get somewhat more pleasant gradients instead of blocks. It would be kind of amazing to do the smooth interpolation in time-domain too, so the resulting video would be frame-rate independent.
cs702 14y ago
This is significant: it allows one programmatically to convert any raster image into a full-color vector drawing. (Think converting JPEGs into similar-looking but much smaller SVG-like files with one click.) Designers would LOVE to have this functionality built into applications like Inkscape and Adobe Illustrator. -- Edit: changed "SVGs" to "SVG-like files" per vlasta2's comment (thanks vlasta2!). Also changed "equal-looking" to "similar-looking," because the automatically-generated color gradients may not precisely match the original.
- vlasta2 14y ago
  Well, it certainly is impressive, but if I understood the video correctly, it actually requires different graphic primitives than SVG offers, so it would require a new format.
  - ygra 14y ago
    There is work underway in standardising this for SVG 2. But given the pace at which the standard process moves along it will still take some years :-)
- colanderman 14y ago
  I'm not sure that the resulting vector files will necessarily be "smaller". JPEG uses a decent compression algorithm, so its size is already pretty close to its actual information content (I'd wager within a factor of 2). A while back I did some experimenting with compressing line graphics (handwritten music notation specifically) as both CCITTv4 (a bi-level raster compression algorithm) and SVG/PS via potrace (a tracer program), and the file sizes were similar (if anything favoring CCITTv4).
  - cs702 14y ago
    colanderman: I'm almost certain that the resulting vector files will be much smaller, because the algorithm substitutes subtle and complex color variations with much simpler color gradients that are only approximately similar and therefore have much less actual information content. PS. Note that the resulting vector files are not just smaller, but the drawing elements contained in them are by definition resolution-independent, so all curves, diagonal lines, and gradients will render beautifully and without artifacts at all image sizes. PPS. Also, note that music notation has no color gradients.
domlebo70 14y ago
This seems amazingly simple (on face value). Anyone else want to have a go at doing up a prototype as a weekend project?
terhechte 14y ago
The link to the source code doesn't work. Did anyone figure out what the correct URL is? I'd love to play around with this. Edit: Nevermind, I found it out myself: http://www.henrykorol.net/DiffusionCurves.rar http://www.henrykorol.net/DiffusionCurves.rar
k2xl 14y ago
Vectorizing bitmaps has been done for a long time. Even Flash even had a built in feature for this (see "trace bitmap") (I think even Flash 5 had this feature). The significance of this is that it introduces a new algorithm for doing it. The core of the algorithm is diffusion filters. What would be VERY interesting is to see the vector size difference for photorealistic pictures using Diffusion curves vs a typical linear or gradient picture.
gjm11 14y ago
Very impressive! But ... Is it just me or is it just really inappropriate for a video demonstration accompanying an academic paper like this to begin with an unrelated 20-second commercial advertisement? Can INRIA really not host the demonstration somewhere that doesn't attach advertisements to their videos?
mistercow 14y ago
Damn, something exactly like this is one of the problems I've been playing with in my head for years, but never got very far on. Turns out it's been solved for four years. Very, very cool work.
crazygringo 14y ago
This seems very cool, but I'd really love to see a better comparison with gradient meshes. Because gradient meshes seem far more "intuitive" and exact, exactly matching the artist's "intentions". Whereas diffusion curves look very cool, but produce great complexity of gradients based on very simple curves -- almost too complex/unpredictable, I'd be worried, if I were an artist. Of course, this seems easier to deal with than gradient meshes, at least how they exist today. I'd love to hear from artists who have tried both. I also wonder what other languages/concepts might be developed in the future towards this same end!
- egypturnash 14y ago
  I'm an artist who's been using Illustrator as her main medium for about a decade. My reaction to this video is summed up in one word: WANT. Gradient meshes are a royal pain in the ass. I never use them; if I want complex, smooth transitions, I currently do it by drawing overlapping blurred shapes. Meshes are overcomplicated IMHO.
- unconed 14y ago
  The problem with gradient meshes is that they are 2D grids that run all the way through the shape. Thus you can't add control points on one side of your mesh without adding a bunch of useless ones across the whole length or width. Each mesh segment has to be a 4-sided bezier patch. If you try to gradient mesh a curved shape, the mesh lines run at very awkward angles, it's often better to start straight and bend it only after subdivision. Diffusion curves solve all of these problems, and seem to have no obvious downsides. I don't see how they would match the artist's intention any less, quite the opposite. You can alter any small part of the image without compromising the overall structure, and you can control spline tension through the blur radius. The only thing I would say is missing is the ability to skew the blur to be more intense on one side of the diffusion curve, thus creating asymmetrical gradients. In tools like Illustrator gradient meshes are flattened to images anyway when exported to PDF. So if a technique like this was implemented, it wouldn't change the workflow at all. In fact, I expect to see this in Illustrator in the next two versions.
  - andybak 14y ago
egypturnash 14y ago
The moment the video was over, I was hitting up Adobe's bug/feature request form. Hell yeah.
tdrd 14y ago
This paper is from 2008. Where is this being used?
sebmarkbage 14y ago
Here's an implementation in WebGL http://blog.calyptus.eu/seb/2010/11/webgl-not-just-for-3d/ http://blog.calyptus.eu/seb/2010/11/webgl-not-just-for-3d/

110 points by currywurst 14y ago

currywurst 14y ago

Found this in a discussion on SVG 2. The automatic generation of the curves from images is the killer feature !

Mordor 14y ago

Could this be used in image/video compression?

agumonkey 14y ago
I'm not sure if it can be compressed in the time domain though.
- yk 14y ago
  Usually vector based formats compress very well in the time domain, since you only need to store the transformations. But without testing the code, I would be surprised if it is fast enough for something like 50 frames.
  - agumonkey 14y ago
    But to have 'organic' images you'd need to have a lot of small curves that would almost be like storing vector derivatives for pixels.
  - corysama 14y ago
    Convert the (x,y) curves to surfaces in the (x,y,time) domain and you should get a representation that is very fast to evaluate. With fast forward differencing, animating the curves would only need 6 adds per control point. If you are fine with "A Scanner Darkly"-style video it would work great! BTW: Vector videos have been used to squeeze full-screen cinematics into a Super Nintendo: http://fabiensanglard.net/anotherWorld_code_review/index.php http://fabiensanglard.net/anotherWorld_code_review/index.php
- mistercow 14y ago
  It doesn't look like it performs very well in the time domain even without compression. If you pay attention to the very brief time when they talk about supporting animation, there's very clear "leaking" effect as the gradients shift. Maybe that could be corrected with very tightly controlled parameters, but tightly controlled parameters = more data + less margin for error.
Geee 14y ago
I think this might be useful for 'very-low bit-rate' video quality, so you would get somewhat more pleasant gradients instead of blocks. It would be kind of amazing to do the smooth interpolation in time-domain too, so the resulting video would be frame-rate independent.

cs702 14y ago

This is significant: it allows one programmatically to convert any raster image into a full-color vector drawing. (Think converting JPEGs into similar-looking but much smaller SVG-like files with one click.) Designers would LOVE to have this functionality built into applications like Inkscape and Adobe Illustrator. -- Edit: changed "SVGs" to "SVG-like files" per vlasta2's comment (thanks vlasta2!). Also changed "equal-looking" to "similar-looking," because the automatically-generated color gradients may not precisely match the original.

vlasta2 14y ago
Well, it certainly is impressive, but if I understood the video correctly, it actually requires different graphic primitives than SVG offers, so it would require a new format.
- ygra 14y ago
  There is work underway in standardising this for SVG 2. But given the pace at which the standard process moves along it will still take some years :-)
colanderman 14y ago
I'm not sure that the resulting vector files will necessarily be "smaller". JPEG uses a decent compression algorithm, so its size is already pretty close to its actual information content (I'd wager within a factor of 2). A while back I did some experimenting with compressing line graphics (handwritten music notation specifically) as both CCITTv4 (a bi-level raster compression algorithm) and SVG/PS via potrace (a tracer program), and the file sizes were similar (if anything favoring CCITTv4).
- cs702 14y ago
  colanderman: I'm almost certain that the resulting vector files will be much smaller, because the algorithm substitutes subtle and complex color variations with much simpler color gradients that are only approximately similar and therefore have much less actual information content. PS. Note that the resulting vector files are not just smaller, but the drawing elements contained in them are by definition resolution-independent, so all curves, diagonal lines, and gradients will render beautifully and without artifacts at all image sizes. PPS. Also, note that music notation has no color gradients.

domlebo70 14y ago

This seems amazingly simple (on face value). Anyone else want to have a go at doing up a prototype as a weekend project?

terhechte 14y ago

The link to the source code doesn't work. Did anyone figure out what the correct URL is? I'd love to play around with this. Edit: Nevermind, I found it out myself: http://www.henrykorol.net/DiffusionCurves.rar http://www.henrykorol.net/DiffusionCurves.rar

k2xl 14y ago

Vectorizing bitmaps has been done for a long time. Even Flash even had a built in feature for this (see "trace bitmap") (I think even Flash 5 had this feature). The significance of this is that it introduces a new algorithm for doing it. The core of the algorithm is diffusion filters. What would be VERY interesting is to see the vector size difference for photorealistic pictures using Diffusion curves vs a typical linear or gradient picture.

gjm11 14y ago

Very impressive! But ... Is it just me or is it just really inappropriate for a video demonstration accompanying an academic paper like this to begin with an unrelated 20-second commercial advertisement? Can INRIA really not host the demonstration somewhere that doesn't attach advertisements to their videos?

mistercow 14y ago

Damn, something exactly like this is one of the problems I've been playing with in my head for years, but never got very far on. Turns out it's been solved for four years. Very, very cool work.

crazygringo 14y ago

This seems very cool, but I'd really love to see a better comparison with gradient meshes. Because gradient meshes seem far more "intuitive" and exact, exactly matching the artist's "intentions". Whereas diffusion curves look very cool, but produce great complexity of gradients based on very simple curves -- almost too complex/unpredictable, I'd be worried, if I were an artist. Of course, this seems easier to deal with than gradient meshes, at least how they exist today. I'd love to hear from artists who have tried both. I also wonder what other languages/concepts might be developed in the future towards this same end!

egypturnash 14y ago
I'm an artist who's been using Illustrator as her main medium for about a decade. My reaction to this video is summed up in one word: WANT. Gradient meshes are a royal pain in the ass. I never use them; if I want complex, smooth transitions, I currently do it by drawing overlapping blurred shapes. Meshes are overcomplicated IMHO.
unconed 14y ago
The problem with gradient meshes is that they are 2D grids that run all the way through the shape. Thus you can't add control points on one side of your mesh without adding a bunch of useless ones across the whole length or width. Each mesh segment has to be a 4-sided bezier patch. If you try to gradient mesh a curved shape, the mesh lines run at very awkward angles, it's often better to start straight and bend it only after subdivision. Diffusion curves solve all of these problems, and seem to have no obvious downsides. I don't see how they would match the artist's intention any less, quite the opposite. You can alter any small part of the image without compromising the overall structure, and you can control spline tension through the blur radius. The only thing I would say is missing is the ability to skew the blur to be more intense on one side of the diffusion curve, thus creating asymmetrical gradients. In tools like Illustrator gradient meshes are flattened to images anyway when exported to PDF. So if a technique like this was implemented, it wouldn't change the workflow at all. In fact, I expect to see this in Illustrator in the next two versions.
- andybak 14y ago

egypturnash 14y ago

The moment the video was over, I was hitting up Adobe's bug/feature request form. Hell yeah.

tdrd 14y ago

This paper is from 2008. Where is this being used?

sebmarkbage 14y ago

Here's an implementation in WebGL http://blog.calyptus.eu/seb/2010/11/webgl-not-just-for-3d/ http://blog.calyptus.eu/seb/2010/11/webgl-not-just-for-3d/