5 ms·

AI Sculpting

90 points by mmastrac 4y ago

sklargh 4y ago
CAD or rapid part development seems like a natural application of this kind of tool.
cheschire 4y ago
"By studying the visual results along the process, our role as a designer became one of a curator." And this is why I'm uninterested in ML-based art generation. I've never celebrated the life of the person who sets up an exhibit at MOMA. ML is great for hotel or office art. It fills a void, but it's like eating hot dogs instead of a top-end steak. You have no idea what's in it, it's just empty calories, and there's no personality of a chef involved.
- lazzlazzlazz 4y ago
  Your thinking — that the curator isn't important, and the artist is the one worthy of celebration — makes sense when the bottleneck is in creating and not curating. But we are entering a new period where the bottleneck will be curation. Young people will celebrate brilliant curators and other kinds of creative meta artists. It is important to understand the reasons for why we value things a given way, instead of what you are doing which is just extending the status quo thinking uncritically.
  - cheschire 4y ago
    When fast food made access to warm savory flavors cheap and near-instant, did sit-down restaurants become an archaic concept? I feel insulted by your assertion that I'm thinking uncritically.
  - sublinear 4y ago
    How does one curate an uninspired infinity of junk? Is generated art really much of an improvement over watching a binary counter tick away in the hopes it lands on an interesting picture? In art, the process is just as important as the result... maybe more so.
    - hypertele-Xii 4y ago
      You can totally create something impressive by combining the best junk, though. Found art sculptures are already a thing.
    - injidup 4y ago
      DJ culture ist almost exactly that. Curation over creation.
  - majormajor 4y ago
    Arguably someone like Anna Wintour is already your celebrated brilliant curator. Or Criterion. If you want to stretch it, many photographers - the scene was already there, they just chose the framing. We already entered the era of "more art than you know what to do with" with recorded mass media and (especially) the internet.
  - monkpit 4y ago
    Why can’t an AI be the curator too?
    - lazzlazzlazz 4y ago
      It will be! No doubt.
- moritonal 4y ago
  At the basic level the artist chooses what to include and what to exclude from their base? Similar surely to the curator?
cwkoss 4y ago
It doesn't feel like the AI is making meaningful decisions in this, so I'm a bit underwhelmed. It seems like they've created a new sculptural style based on a system of constraints that yields an interesting new style, but I'm left wondering if all of those constraints are imposed by the humans: feels like the AI is being used as a tool to render the ideas of the humans rather than making interesting decisions itself. I tried to get chatGPT to write OpenSCAD a couple weeks ago. Made a good outline and structure for the code, but the spatial relationships were all wrong (somewhat unsurprisingly) I think taking a collection of 2d images and sculpting a plausible rigged 3d model is the direction of AI I'm more interested in. We have photogrammetry already, but what if the subjects are 'pokemon': and every image of them is from a different angle in a different pose - some features may be squishy, some rigid. The ability to take a character, extract a model and poses and propose additional poses would be incredible: such a tool could let a kid who learns how to draw create a character and drop it into a video game, and use it.
- knicholes 4y ago
  So this, then? https://www.youtube.com/watch?v=shy51E-MU8Y https://www.youtube.com/watch?v=shy51E-MU8Y (not rigging, but maybe you're talking about adding Mixamo to it?)
  - joewhatkins 4y ago
    From my understanding the model in the linked video only stylizes existing meshes from text. There’s plenty of papers that have tried text -> 3D model generation using photogrammetric-esque methods similar to what the parent comment suggested - the two minute papers video on one example is here. https://youtu.be/L3G0dx1Q0R8 https://youtu.be/L3G0dx1Q0R8 Outside of cherry-picked examples this style of model tends to suffer from what people are calling the “Janus Problem” - the easiest way for it to satisfy the loss is to simply make the object look like the input prompt from as many angles as possible. So if you enter “a rubber duck”, it tends to generate yellow blobs with multiple head-like appendages sprouting off from it. Google’s paper that tried this approach using Imagen as the text->image generator had great results, but they might be cherry-picked. Someone replicated it with stable diffusion as the text->image backend - still major Janus problems.
aaron695 4y ago
code_biologist 4y ago
My criticism is painfully characteristic of HN, but to be blunt: the results of this are ugly. I'm a recreational CAD user and recently I've tried to add more decorative flourish to my designs using ZBrush as a final processing step. Some recent examples: I've tried to add decorative moulding style carvings into a bracket. I've tried to fashion the ends of a workshop hook into rainforest vines. I completely suck to be clear. My dream workflow for AI sculpting is an img2img/StableDiffusion style interface where I can select portions of my model to alter and iterate on the prompt, within something like ZBrush. "The moulding texture is good, but I'd like a more Greek inspired style", "Can you make the bark texture on this wood deeper?", "Can you turn this neutral face into an angry grimace?" A simpler version that would still be really awesome is being able to generate texture brushes like [1][2] to easily sculpt with via prompts. "tight, densely packed neat feathers" or "the texture of a blanket with large, loose yarn knitting". [1] https://youtu.be/VeC6WEZzbTM?t=26 https://youtu.be/VeC6WEZzbTM?t=26 (ornamental trim demo) [2] https://environmentdesign.gumroad.com/l/mDbSP https://environmentdesign.gumroad.com/l/mDbSP (rock textures) (no affiliation with either)
- andybak 4y ago
  > the results of this are ugly. "I found the results to be ugly". I personally found some of the results to be non-ugly.
  - frereubu 4y ago
    I honestly think that most arguments people have about art would go away if they stopped using the verb "to be", i.e. "that artwork is...", and rephrased it as a personal opinion, which is almost always what it is. It generally leads to much more interesting conversations about art too.
    - andybak 4y ago
      You've accidentally rediscovered English Prime: https://en.wikipedia.org/wiki/E-Prime https://en.wikipedia.org/wiki/E-Prime I think it's a fascinating idea - I sometimes phrase it this way: any sentence that contains "is" is at least partially a lie.
fernly 4y ago
The basic technique, a gouging or scooping away of the material to leave a remnant outlined by negative spherical spaces, produces a result that to my brain is not merely ugly, as another commentor wrote, but actively, strongly unpleasant, repellent, shudderingly cringe-worthy. I could not physically remain in the same room with the human forms shown. But that's just me.
- eigenvalue 4y ago
  Sounds from your description like it’s adjacent to Trypophobia, a well documented phenomenon: https://en.m.wikipedia.org/wiki/Trypophobia https://en.m.wikipedia.org/wiki/Trypophobia
  - fernly 4y ago
    Oh, thank you! I always wondered if it had a name.
hndamien 4y ago
For comparison, here is the modern day Michelangelo. https://jago.art/en/jago/ https://jago.art/en/jago/
deleted 4y ago
[deleted]
krick 4y ago
This is quite boring generative art experiment that has nothing to do with AI (and by AI I mean ML, since calling ML "ML" is sooo 2021 and just lame nowadays). But the expectations I got from the title were more interesting to me that the actual post. It's actually kinda surprising I didn't see openAI-grade 3D model generator. Anyway, it's a matter of time now. And I think it is way more exciting than 2D. Stable diffusion is amazing as a technology, but I haven't seen a useful application so far, since it's only appropriate to use if I don't really care about what I want to get, and for a real artist "almost what I want" is only marginally more helpful than a sketch, and sketches are not that time consuming. But making pretty rough 3D model is time consuming, and since you need a lot of 3D models for an AAA game or a movie, often nobody even cares about the details anyway. This could shrink budgets by several orders of magnitude.
- aaronblohowiak 4y ago
  In terms of time and difficulty, is modeling the thing? I thought animating > rigging > modeling > texturing, but I haven’t worked i games in like 15 years or something
- monkpit 4y ago
  It seems like this is just the target 3D mesh +/- some error from the tool radius.
ausbah 4y ago
interesting technically and what I would call successful in at least the agent has learned something which can be hard for any RL project
BHSPitMonkey 4y ago
Not sure I'm sold on the work itself, but the graphics and animations on this page are really nicely done.

90 points by mmastrac 4y ago

CAD or rapid part development seems like a natural application of this kind of tool.

"By studying the visual results along the process, our role as a designer became one of a curator." And this is why I'm uninterested in ML-based art generation. I've never celebrated the life of the person who sets up an exhibit at MOMA. ML is great for hotel or office art. It fills a void, but it's like eating hot dogs instead of a top-end steak. You have no idea what's in it, it's just empty calories, and there's no personality of a chef involved.

lazzlazzlazz 4y ago
Your thinking — that the curator isn't important, and the artist is the one worthy of celebration — makes sense when the bottleneck is in creating and not curating. But we are entering a new period where the bottleneck will be curation. Young people will celebrate brilliant curators and other kinds of creative meta artists. It is important to understand the reasons for why we value things a given way, instead of what you are doing which is just extending the status quo thinking uncritically.
- cheschire 4y ago
  When fast food made access to warm savory flavors cheap and near-instant, did sit-down restaurants become an archaic concept? I feel insulted by your assertion that I'm thinking uncritically.
- sublinear 4y ago
  How does one curate an uninspired infinity of junk? Is generated art really much of an improvement over watching a binary counter tick away in the hopes it lands on an interesting picture? In art, the process is just as important as the result... maybe more so.
  - hypertele-Xii 4y ago
    You can totally create something impressive by combining the best junk, though. Found art sculptures are already a thing.
  - injidup 4y ago
    DJ culture ist almost exactly that. Curation over creation.
- majormajor 4y ago
  Arguably someone like Anna Wintour is already your celebrated brilliant curator. Or Criterion. If you want to stretch it, many photographers - the scene was already there, they just chose the framing. We already entered the era of "more art than you know what to do with" with recorded mass media and (especially) the internet.
- monkpit 4y ago
  Why can’t an AI be the curator too?
  - lazzlazzlazz 4y ago
    It will be! No doubt.
moritonal 4y ago
At the basic level the artist chooses what to include and what to exclude from their base? Similar surely to the curator?

cwkoss 4y ago

It doesn't feel like the AI is making meaningful decisions in this, so I'm a bit underwhelmed. It seems like they've created a new sculptural style based on a system of constraints that yields an interesting new style, but I'm left wondering if all of those constraints are imposed by the humans: feels like the AI is being used as a tool to render the ideas of the humans rather than making interesting decisions itself. I tried to get chatGPT to write OpenSCAD a couple weeks ago. Made a good outline and structure for the code, but the spatial relationships were all wrong (somewhat unsurprisingly) I think taking a collection of 2d images and sculpting a plausible rigged 3d model is the direction of AI I'm more interested in. We have photogrammetry already, but what if the subjects are 'pokemon': and every image of them is from a different angle in a different pose - some features may be squishy, some rigid. The ability to take a character, extract a model and poses and propose additional poses would be incredible: such a tool could let a kid who learns how to draw create a character and drop it into a video game, and use it.

knicholes 4y ago
So this, then? https://www.youtube.com/watch?v=shy51E-MU8Y https://www.youtube.com/watch?v=shy51E-MU8Y (not rigging, but maybe you're talking about adding Mixamo to it?)
- joewhatkins 4y ago
  From my understanding the model in the linked video only stylizes existing meshes from text. There’s plenty of papers that have tried text -> 3D model generation using photogrammetric-esque methods similar to what the parent comment suggested - the two minute papers video on one example is here. https://youtu.be/L3G0dx1Q0R8 https://youtu.be/L3G0dx1Q0R8 Outside of cherry-picked examples this style of model tends to suffer from what people are calling the “Janus Problem” - the easiest way for it to satisfy the loss is to simply make the object look like the input prompt from as many angles as possible. So if you enter “a rubber duck”, it tends to generate yellow blobs with multiple head-like appendages sprouting off from it. Google’s paper that tried this approach using Imagen as the text->image generator had great results, but they might be cherry-picked. Someone replicated it with stable diffusion as the text->image backend - still major Janus problems.

aaron695 4y ago

code_biologist 4y ago

My criticism is painfully characteristic of HN, but to be blunt: the results of this are ugly. I'm a recreational CAD user and recently I've tried to add more decorative flourish to my designs using ZBrush as a final processing step. Some recent examples: I've tried to add decorative moulding style carvings into a bracket. I've tried to fashion the ends of a workshop hook into rainforest vines. I completely suck to be clear. My dream workflow for AI sculpting is an img2img/StableDiffusion style interface where I can select portions of my model to alter and iterate on the prompt, within something like ZBrush. "The moulding texture is good, but I'd like a more Greek inspired style", "Can you make the bark texture on this wood deeper?", "Can you turn this neutral face into an angry grimace?" A simpler version that would still be really awesome is being able to generate texture brushes like [1][2] to easily sculpt with via prompts. "tight, densely packed neat feathers" or "the texture of a blanket with large, loose yarn knitting". [1] https://youtu.be/VeC6WEZzbTM?t=26 https://youtu.be/VeC6WEZzbTM?t=26 (ornamental trim demo) [2] https://environmentdesign.gumroad.com/l/mDbSP https://environmentdesign.gumroad.com/l/mDbSP (rock textures) (no affiliation with either)

andybak 4y ago
> the results of this are ugly. "I found the results to be ugly". I personally found some of the results to be non-ugly.
- frereubu 4y ago
  I honestly think that most arguments people have about art would go away if they stopped using the verb "to be", i.e. "that artwork is...", and rephrased it as a personal opinion, which is almost always what it is. It generally leads to much more interesting conversations about art too.
  - andybak 4y ago
    You've accidentally rediscovered English Prime: https://en.wikipedia.org/wiki/E-Prime https://en.wikipedia.org/wiki/E-Prime I think it's a fascinating idea - I sometimes phrase it this way: any sentence that contains "is" is at least partially a lie.

fernly 4y ago

The basic technique, a gouging or scooping away of the material to leave a remnant outlined by negative spherical spaces, produces a result that to my brain is not merely ugly, as another commentor wrote, but actively, strongly unpleasant, repellent, shudderingly cringe-worthy. I could not physically remain in the same room with the human forms shown. But that's just me.

eigenvalue 4y ago
Sounds from your description like it’s adjacent to Trypophobia, a well documented phenomenon: https://en.m.wikipedia.org/wiki/Trypophobia https://en.m.wikipedia.org/wiki/Trypophobia
- fernly 4y ago
  Oh, thank you! I always wondered if it had a name.

hndamien 4y ago

For comparison, here is the modern day Michelangelo. https://jago.art/en/jago/ https://jago.art/en/jago/

deleted 4y ago

[deleted]

krick 4y ago

This is quite boring generative art experiment that has nothing to do with AI (and by AI I mean ML, since calling ML "ML" is sooo 2021 and just lame nowadays). But the expectations I got from the title were more interesting to me that the actual post. It's actually kinda surprising I didn't see openAI-grade 3D model generator. Anyway, it's a matter of time now. And I think it is way more exciting than 2D. Stable diffusion is amazing as a technology, but I haven't seen a useful application so far, since it's only appropriate to use if I don't really care about what I want to get, and for a real artist "almost what I want" is only marginally more helpful than a sketch, and sketches are not that time consuming. But making pretty rough 3D model is time consuming, and since you need a lot of 3D models for an AAA game or a movie, often nobody even cares about the details anyway. This could shrink budgets by several orders of magnitude.

aaronblohowiak 4y ago
In terms of time and difficulty, is modeling the thing? I thought animating > rigging > modeling > texturing, but I haven’t worked i games in like 15 years or something
monkpit 4y ago
It seems like this is just the target 3D mesh +/- some error from the tool radius.

ausbah 4y ago

interesting technically and what I would call successful in at least the agent has learned something which can be hard for any RL project

BHSPitMonkey 4y ago

Not sure I'm sold on the work itself, but the graphics and animations on this page are really nicely done.