6 ms·

Geometrize: Turn Images into Geometric Primitives

157 points by adrian_mrd 6y ago

kevlar1818 6y ago
Shoutout to Michael Fogleman's primitive[1] for likely inspiring this work. [1]: https://github.com/fogleman/primitive https://github.com/fogleman/primitive
- kc0bfv 6y ago
  Indeed, you seem to be correct. It gets a call-out down at the bottom of the page.
- usgroup 6y ago
  Looks like it’s a recursive minimisation. I wonder what an optimal minimisation would produce.
- JKCalhoun 6y ago
  Just purchased, thanks.
- dahart 6y ago
  > The algorithm tries to find the single most optimal shape that can be drawn to minimize the error between the target image and the drawn image. It repeats this process, adding one shape at a time. Around 50 to 200 shapes are needed to reach a result that is recognizable yet artistic and abstract. So one way to think about this is it’s a compression algorithm with aesthetic artifacts at high compression rates. I wonder if you could use a multi-dimensional Hough-like transform on the primitive of choice to avoid the hill climbing step. You can do this easily with lines, but ellipses and triangles will be more difficult I’m sure. Sounds like a fun Christmas break programming project!
- isoprophlex 6y ago
  This uses hill climbing to minimize RMS error, I wonder if gradient descent with some momentum would be faster? Thanks for sharing!
renewiltord 6y ago
Lovely effect. Thank you for sharing.
suyash 6y ago
Very cool project, I would love to use it as a way to teach computer graphics to students. Is there more explanation somewhere besides the brief description ?
- CyberDildonics 6y ago
  How would you use this to teach computer graphics to students?
  - suyash 6y ago
    one concept can be around how images are rendered from primitive shapes, showing that using this demo step by step.
    - CyberDildonics 6y ago
      Nothing is actually rendered this way. Polygonal manifolds aren't overlapped to create shading in 3D space. This isn't how 3D, compression, or image manipulation works, it is purely artistic.
      - suyash 6y ago
        I see, well I'm going to research this more. Even if it is not a literal explanation conceptually this demo can serve a learning aid.
        CyberDildonics 6y ago
        How?
istorical 6y ago
How would one most easily construct a pipeline or script to either split apart an mp4 into images, process them all with this tool, and then recombine into a video, or else modify this tool to take video as input? Anyone have any ideas / suggestions for a path to try that might require the least installation and scripting? And can anyone tell whether it's deterministic or not as far as whether the same input generates the same output each time?
- dawnerd 6y ago
  Ffmpeg can: https://trac.ffmpeg.org/wiki/Create%20a%20thumbnail%20image%20every%20X%20seconds%20of%20the%20video https://trac.ffmpeg.org/wiki/Create%20a%20thumbnail%20image%... https://superuser.com/a/624573 https://superuser.com/a/624573 You’ll just need to figure out the middle bit, shouldn’t be too hard though.
- thih9 6y ago
  Regarding actual input and output, I assume that’s relatively easy with ffmpeg. As for results, the challenge might be keeping the illusion of movement from frame to frame. It’s likely small changes to the input image result in largely different positioning of the shapes. When watched as a movie, this wouldn’t result in a fluid motion.
  - boxfire 6y ago
    With the way this optimizes it would produce crap, even if using the same seed frame to frame. One very interesting prospect is in taking the motion vectors out of the video and applying them to overlapping primitives in the shape data form as a first step to apply when fitting the shapes, that is to do this greedy hill climb in tandem with mpeg motion decoding. Maybe it wouldn't be too hard but it's not something I've got the motivation to try out and tweak right now
    - boxfire 6y ago
      And more on this, maybe seeding the next frame to favor the motion vectors mutation of the shape before this libraries random seed mutation
dawnerd 6y ago
I’ve been meaning to give something like this a shot to assist in lazy loading large images. Generate a low complexity svg that is hopefully pretty small and just fade in the full image once loaded.
- klysm 6y ago
  You should check out interlaced PNG and friends
- munawwar 6y ago
  I tried this recently: https://gist.github.com/Munawwar/6ac51e33e901d89750ee61319d064aa5 https://gist.github.com/Munawwar/6ac51e33e901d89750ee61319d0... The challenge is making it look pretty and at same time have a small svg output and at the same time not taking forever to generate (it's perf gets exponentialy slow). Took some time to tweak it to the settings I used. It looks nicer than blurhash.. however blurhash is way smaller and might be more practical alternative
  - dawnerd 6y ago
    Oh that’s very nice, I’ll give it a shot. I did try blurhash but it was just a bit too abstract? For large images. Think desktop full width.
- atulvi 6y ago
  Also checkout https://blurha.sh/ https://blurha.sh/ if you want a minimal palceholder image for the web
chris_wot 6y ago
Is there a library that goes with this?
- Someone 6y ago
  Open the page, read the first 25 words, and come back when you still have questions.
pantelisk 6y ago
Where have I seen this before? Oh yes https://alteredqualia.com/visualization/evolve/ https://alteredqualia.com/visualization/evolve/ (from 2008) (From the person that wrote the webgl renderer for three.js) Press start and you can see the algorithm at work, and how each shape is added randomly, score is computed, then the process repeats again
- semi-extrinsic 6y ago
  Interesting also to see here how computing power (and algos?) have improved tremendously. I assume the benchmarks are from ~ 2008. Getting to the 94% fitness levels takes 5 minutes on my phone - listed as >120 minutes on presumably a beefy workstation on this page.
wizzwizz4 6y ago
This, but in 3D, could be incredibly useful for modelling.
- carapace 6y ago
  You just rang a faint bell in my mind... There is such a project IIRC... if I can remember it I'll post a link...
memco 6y ago
The most interesting one I saw was this: https://www.geometrize.co.uk/assets/images/examples/building_geometrized.png https://www.geometrize.co.uk/assets/images/examples/building.... It is a building in the full res, but when the algorithm is applied it looks like a mountain. Every other picture I saw was, more or less, as recognizable as the original.
hnarn 6y ago
Honest question: If you take a copyrighted photograph and convert it to geometric shapes, how “blocky” does the picture have to be before it’s no longer a copyrighted work of the original artist? I mean, if it’s almost identical with a huge amount of shapes, I can understand that it’s reasonable to look at it as the same piece of work, but on the other hand if you make it blocky enough that it looks like an interpretation of the original image as a new work in its own right, what ownership does the original author have? None, some, complete?
- grawprog 6y ago
  I would imagine there's probably some precedent for this. You've been able to totally distort pictures beyond recognition with Photoshop filters for a long time. It seems like something that's probably come up before.
- robgough 6y ago
  Your question reminded me of this[1] horrifying story, which deals with that exact issue [1]: https://waxy.org/2011/06/kind_of_screwed/ https://waxy.org/2011/06/kind_of_screwed/
  - hug 6y ago
    And, on the other hand, is Richard Prince. https://petapixel.com/2013/04/26/appeals-court-overturns-previous-ruling-rules-fair-use-in-richard-prince-case/ https://petapixel.com/2013/04/26/appeals-court-overturns-pre...
- pdkl95 6y ago
  While copyright law's interaction with new technologies can be hard to predict, I strongly suspect that the level of "blocky-ness" doesn't matter. If an image was created from another image created by a different artist, the new image is as derivative work[1]. In general[2], as long as the new work is a transformative[3] difference from the original, a new copyright is created, although royalties might need to be paid to the author of the original work. [1] https://en.wikipedia.org/wiki/Derivative_work https://en.wikipedia.org/wiki/Derivative_work [2] see a real lawyer for actual legal advice [3] https://en.wikipedia.org/wiki/Transformation_%28law%29 https://en.wikipedia.org/wiki/Transformation_%28law%29
alceufc 6y ago
There was a challenge at stack overflow to encode an image in a tweet (140 characters at the time): https://stackoverflow.com/questions/891643/twitter-image-encoding-challenge https://stackoverflow.com/questions/891643/twitter-image-enc... Some of the solutions took a similar approach that used geometric primitives.
feiss 6y ago
Best example of this algorithm is Frou Frou’s “the dumbing down of love” music video, from back in 2002! Crap quality: https://youtu.be/N4IhGlNTJEg https://youtu.be/N4IhGlNTJEg I’ve always puzzled of how they morphed the shapes...
- feiss 6y ago
  Better version from the director’s vimeo: https://vimeo.com/44001799 https://vimeo.com/44001799
runawaybottle 6y ago
Would be a good use case for Chat/conferencing avatars. If this can be done in real time (even with a delay), fade transitions could be a nice alternative to video too.

157 points by adrian_mrd 6y ago

Shoutout to Michael Fogleman's primitive[1] for likely inspiring this work. [1]: https://github.com/fogleman/primitive https://github.com/fogleman/primitive

kc0bfv 6y ago
Indeed, you seem to be correct. It gets a call-out down at the bottom of the page.
usgroup 6y ago
Looks like it’s a recursive minimisation. I wonder what an optimal minimisation would produce.
JKCalhoun 6y ago
Just purchased, thanks.
dahart 6y ago
> The algorithm tries to find the single most optimal shape that can be drawn to minimize the error between the target image and the drawn image. It repeats this process, adding one shape at a time. Around 50 to 200 shapes are needed to reach a result that is recognizable yet artistic and abstract. So one way to think about this is it’s a compression algorithm with aesthetic artifacts at high compression rates. I wonder if you could use a multi-dimensional Hough-like transform on the primitive of choice to avoid the hill climbing step. You can do this easily with lines, but ellipses and triangles will be more difficult I’m sure. Sounds like a fun Christmas break programming project!
isoprophlex 6y ago
This uses hill climbing to minimize RMS error, I wonder if gradient descent with some momentum would be faster? Thanks for sharing!

renewiltord 6y ago

Lovely effect. Thank you for sharing.

suyash 6y ago

Very cool project, I would love to use it as a way to teach computer graphics to students. Is there more explanation somewhere besides the brief description ?

CyberDildonics 6y ago
How would you use this to teach computer graphics to students?
- suyash 6y ago
  one concept can be around how images are rendered from primitive shapes, showing that using this demo step by step.
  - CyberDildonics 6y ago
    Nothing is actually rendered this way. Polygonal manifolds aren't overlapped to create shading in 3D space. This isn't how 3D, compression, or image manipulation works, it is purely artistic.
    - suyash 6y ago
      I see, well I'm going to research this more. Even if it is not a literal explanation conceptually this demo can serve a learning aid.
      - CyberDildonics 6y ago
        How?

istorical 6y ago

How would one most easily construct a pipeline or script to either split apart an mp4 into images, process them all with this tool, and then recombine into a video, or else modify this tool to take video as input? Anyone have any ideas / suggestions for a path to try that might require the least installation and scripting? And can anyone tell whether it's deterministic or not as far as whether the same input generates the same output each time?

dawnerd 6y ago
Ffmpeg can: https://trac.ffmpeg.org/wiki/Create%20a%20thumbnail%20image%20every%20X%20seconds%20of%20the%20video https://trac.ffmpeg.org/wiki/Create%20a%20thumbnail%20image%... https://superuser.com/a/624573 https://superuser.com/a/624573 You’ll just need to figure out the middle bit, shouldn’t be too hard though.
thih9 6y ago
Regarding actual input and output, I assume that’s relatively easy with ffmpeg. As for results, the challenge might be keeping the illusion of movement from frame to frame. It’s likely small changes to the input image result in largely different positioning of the shapes. When watched as a movie, this wouldn’t result in a fluid motion.
- boxfire 6y ago
  With the way this optimizes it would produce crap, even if using the same seed frame to frame. One very interesting prospect is in taking the motion vectors out of the video and applying them to overlapping primitives in the shape data form as a first step to apply when fitting the shapes, that is to do this greedy hill climb in tandem with mpeg motion decoding. Maybe it wouldn't be too hard but it's not something I've got the motivation to try out and tweak right now
  - boxfire 6y ago
    And more on this, maybe seeding the next frame to favor the motion vectors mutation of the shape before this libraries random seed mutation

dawnerd 6y ago

I’ve been meaning to give something like this a shot to assist in lazy loading large images. Generate a low complexity svg that is hopefully pretty small and just fade in the full image once loaded.

klysm 6y ago
You should check out interlaced PNG and friends
munawwar 6y ago
I tried this recently: https://gist.github.com/Munawwar/6ac51e33e901d89750ee61319d064aa5 https://gist.github.com/Munawwar/6ac51e33e901d89750ee61319d0... The challenge is making it look pretty and at same time have a small svg output and at the same time not taking forever to generate (it's perf gets exponentialy slow). Took some time to tweak it to the settings I used. It looks nicer than blurhash.. however blurhash is way smaller and might be more practical alternative
- dawnerd 6y ago
  Oh that’s very nice, I’ll give it a shot. I did try blurhash but it was just a bit too abstract? For large images. Think desktop full width.
atulvi 6y ago
Also checkout https://blurha.sh/ https://blurha.sh/ if you want a minimal palceholder image for the web

chris_wot 6y ago

Is there a library that goes with this?

Someone 6y ago
Open the page, read the first 25 words, and come back when you still have questions.

pantelisk 6y ago

Where have I seen this before? Oh yes https://alteredqualia.com/visualization/evolve/ https://alteredqualia.com/visualization/evolve/ (from 2008) (From the person that wrote the webgl renderer for three.js) Press start and you can see the algorithm at work, and how each shape is added randomly, score is computed, then the process repeats again

semi-extrinsic 6y ago
Interesting also to see here how computing power (and algos?) have improved tremendously. I assume the benchmarks are from ~ 2008. Getting to the 94% fitness levels takes 5 minutes on my phone - listed as >120 minutes on presumably a beefy workstation on this page.

wizzwizz4 6y ago

This, but in 3D, could be incredibly useful for modelling.

carapace 6y ago
You just rang a faint bell in my mind... There is such a project IIRC... if I can remember it I'll post a link...

memco 6y ago

The most interesting one I saw was this: https://www.geometrize.co.uk/assets/images/examples/building_geometrized.png https://www.geometrize.co.uk/assets/images/examples/building.... It is a building in the full res, but when the algorithm is applied it looks like a mountain. Every other picture I saw was, more or less, as recognizable as the original.

hnarn 6y ago

Honest question: If you take a copyrighted photograph and convert it to geometric shapes, how “blocky” does the picture have to be before it’s no longer a copyrighted work of the original artist? I mean, if it’s almost identical with a huge amount of shapes, I can understand that it’s reasonable to look at it as the same piece of work, but on the other hand if you make it blocky enough that it looks like an interpretation of the original image as a new work in its own right, what ownership does the original author have? None, some, complete?

grawprog 6y ago
I would imagine there's probably some precedent for this. You've been able to totally distort pictures beyond recognition with Photoshop filters for a long time. It seems like something that's probably come up before.
robgough 6y ago
Your question reminded me of this[1] horrifying story, which deals with that exact issue [1]: https://waxy.org/2011/06/kind_of_screwed/ https://waxy.org/2011/06/kind_of_screwed/
- hug 6y ago
  And, on the other hand, is Richard Prince. https://petapixel.com/2013/04/26/appeals-court-overturns-previous-ruling-rules-fair-use-in-richard-prince-case/ https://petapixel.com/2013/04/26/appeals-court-overturns-pre...
pdkl95 6y ago
While copyright law's interaction with new technologies can be hard to predict, I strongly suspect that the level of "blocky-ness" doesn't matter. If an image was created from another image created by a different artist, the new image is as derivative work[1]. In general[2], as long as the new work is a transformative[3] difference from the original, a new copyright is created, although royalties might need to be paid to the author of the original work. [1] https://en.wikipedia.org/wiki/Derivative_work https://en.wikipedia.org/wiki/Derivative_work [2] see a real lawyer for actual legal advice [3] https://en.wikipedia.org/wiki/Transformation_%28law%29 https://en.wikipedia.org/wiki/Transformation_%28law%29

alceufc 6y ago

There was a challenge at stack overflow to encode an image in a tweet (140 characters at the time): https://stackoverflow.com/questions/891643/twitter-image-encoding-challenge https://stackoverflow.com/questions/891643/twitter-image-enc... Some of the solutions took a similar approach that used geometric primitives.

feiss 6y ago

Best example of this algorithm is Frou Frou’s “the dumbing down of love” music video, from back in 2002! Crap quality: https://youtu.be/N4IhGlNTJEg https://youtu.be/N4IhGlNTJEg I’ve always puzzled of how they morphed the shapes...

feiss 6y ago
Better version from the director’s vimeo: https://vimeo.com/44001799 https://vimeo.com/44001799

runawaybottle 6y ago

Would be a good use case for Chat/conferencing avatars. If this can be done in real time (even with a delay), fade transitions could be a nice alternative to video too.