4 ms·

Maybe Facebook's Segment Anything could replace the first clustering step?

by devit 3y ago

spaceprison 3y ago
I had a similar idea the other day after fighting with inkscape tracing! The problem with auto tracing is lack of content awareness so it's just shapes and colors leading to strange objects that require lots of tinkering. I'm going to try it: Use segment anything to get object masks, Trace each object separately and combine from there!
- larodi 3y ago
  It may be much slower, though segment-anything works quite okay from my tests. But still - great work, cant wait to use it in my next design. :D
- ttul 3y ago
  This would make for a nice ComfyUI custom node.
- westurner 3y ago
  > Comparing to Potrace which only accept binarized inputs (Black & White pixmap), VTracer has an image processing pipeline which can handle colored high resolution scans. tl;dr: Potrace uses a O(n^2) fitting algorithm, whereas vtracer is entirely O(n). What is the Big-O of the algorithm with Segment Anything or other segmentation approaches? Potrace: https://en.wikipedia.org/wiki/Potrace https://en.wikipedia.org/wiki/Potrace The Ctrl-L to Simplify Inkscape feature attempts to describe the same path with fewer points/bezier curves. Could this approach also help with 3d digitization?
  - westurner 3y ago
    TIL about https://github.com/fogleman/primitive https://github.com/fogleman/primitive from "Comparison of raster-to-vector conversion software" https://en.wikipedia.org/wiki/Comparison_of_raster-to-vector_conversion_software https://en.wikipedia.org/wiki/Comparison_of_raster-to-vector... which does already list vtracer (2020) visioncortex/vtracer: https://github.com/visioncortex/vtracer https://github.com/visioncortex/vtracer Vector graphics https://en.wikipedia.org/wiki/Vector_graphics https://en.wikipedia.org/wiki/Vector_graphics Rotoscoping: https://en.wikipedia.org/wiki/Rotoscoping https://en.wikipedia.org/wiki/Rotoscoping Sprite (computer graphics) https://en.wikipedia.org/wiki/Sprite_(computer_graphics) https://en.wikipedia.org/wiki/Sprite_(computer_graphics) E.g. pygame-web can do SVG sprites; so that you don't have to do pixel art and sprite scaling just works. 2.5D: https://en.wikipedia.org/wiki/2.5D https://en.wikipedia.org/wiki/2.5D 3D scanning: https://en.wikipedia.org/wiki/3D_scanning https://en.wikipedia.org/wiki/3D_scanning "Why Cities: Skylines 2 performs poorly" (2023) ... No AutoLOD Level of Depth https://news.ycombinator.com/item?id=38160089 https://news.ycombinator.com/item?id=38160089 Wavetale is a 3D game with extensive and visually impressive vector graphics.

Maybe Facebook's Segment Anything could replace the first clustering step?

by devit 3y ago

Maybe Facebook's Segment Anything could replace the first clustering step?

I had a similar idea the other day after fighting with inkscape tracing! The problem with auto tracing is lack of content awareness so it's just shapes and colors leading to strange objects that require lots of tinkering. I'm going to try it: Use segment anything to get object masks, Trace each object separately and combine from there!

larodi 3y ago
It may be much slower, though segment-anything works quite okay from my tests. But still - great work, cant wait to use it in my next design. :D
ttul 3y ago
This would make for a nice ComfyUI custom node.
westurner 3y ago
> Comparing to Potrace which only accept binarized inputs (Black & White pixmap), VTracer has an image processing pipeline which can handle colored high resolution scans. tl;dr: Potrace uses a O(n^2) fitting algorithm, whereas vtracer is entirely O(n). What is the Big-O of the algorithm with Segment Anything or other segmentation approaches? Potrace: https://en.wikipedia.org/wiki/Potrace https://en.wikipedia.org/wiki/Potrace The Ctrl-L to Simplify Inkscape feature attempts to describe the same path with fewer points/bezier curves. Could this approach also help with 3d digitization?
- westurner 3y ago
  TIL about https://github.com/fogleman/primitive https://github.com/fogleman/primitive from "Comparison of raster-to-vector conversion software" https://en.wikipedia.org/wiki/Comparison_of_raster-to-vector_conversion_software https://en.wikipedia.org/wiki/Comparison_of_raster-to-vector... which does already list vtracer (2020) visioncortex/vtracer: https://github.com/visioncortex/vtracer https://github.com/visioncortex/vtracer Vector graphics https://en.wikipedia.org/wiki/Vector_graphics https://en.wikipedia.org/wiki/Vector_graphics Rotoscoping: https://en.wikipedia.org/wiki/Rotoscoping https://en.wikipedia.org/wiki/Rotoscoping Sprite (computer graphics) https://en.wikipedia.org/wiki/Sprite_(computer_graphics) https://en.wikipedia.org/wiki/Sprite_(computer_graphics) E.g. pygame-web can do SVG sprites; so that you don't have to do pixel art and sprite scaling just works. 2.5D: https://en.wikipedia.org/wiki/2.5D https://en.wikipedia.org/wiki/2.5D 3D scanning: https://en.wikipedia.org/wiki/3D_scanning https://en.wikipedia.org/wiki/3D_scanning "Why Cities: Skylines 2 performs poorly" (2023) ... No AutoLOD Level of Depth https://news.ycombinator.com/item?id=38160089 https://news.ycombinator.com/item?id=38160089 Wavetale is a 3D game with extensive and visually impressive vector graphics.