5 ms·
What is the input? Is it converting a text query like "chair" to a mesh? edit: Seems like mesh completion is the main input-output method, not just a neat feat
by sram1337 3y ago
What is the input? Is it converting a text query like "chair" to a mesh?
edit: Seems like mesh completion is the main input-output method, not just a neat feature.
- deleted 3y ago[deleted]
- CamperBob2 3y agoThat's what I was wondering. From the diagram it looks like the input is other chair meshes, which makes it somewhat less interesting.
- tayo42 3y agoReally the hardest thing with art is details and usually seperates good from bad. So if you can sketch what you want roughly without skill and have the details generated, that's extremely useful. And image to image with the existing diffusion models is useful and popular.
- nullptr_deref 3y agoI have no idea about your background when I am commenting here. But these are my two cents. NO. Details are mostly like icing on top of the cake. Sure, good details make good art but it is not always the case. True and beautiful art requires form + shape. What you are saying is something visually appealing. So, the reason why diffusion models feel so bland is because they are good with details but do not have precise forms and shape. Nowadays they are getting better, however, it still remains an issue. Form + shape > details is something they teach in Art 101.
- treyd 3y agoThere's also examples of tables, lamps, couches, etc in the video.
- all2 3y agoYou prompt this LLM using 3D meshes for it to complete, in the same manner you use language to prompt language specific LLMs.
- owenpalmer 3y agoThat's what it seems like. Although this is not an LLM. > Inspired by recent advances in powerful large language models, we adopt a sequence-based approach to autoregressively generate triangle meshes as sequences of triangles. It's only inspired by LLMs
- deleted 3y ago[deleted]
- adw 3y agoThis is sort of a distinction without a difference. It's an autoregressive sequence model; the distinction is how you're encoding data into (and out of) a sequence of tokens. LLMs are autoregressive sequence models where the "role" of the graph convolutional encoder here is filled by a BPE tokenizer (also a learned model, just a much simpler one than the model used here). That this works implies that you can probably port this idea to other domains by designing clever codecs which map their feature space into discrete token sequences, similarly. (Everything is feature engineering if you squint hard enough.)
- ShamelessC 3y agoThe only difference is the label, really. The underlying transformer architecture and the approach of using a codebook is identical to a large language model. The same approach was also used originally for image generation in DALL-E 1.
- anentropic 3y agoYeah it's hard to tell. It looks like the input is itself a 3D mesh? So the model is doing "shape completion" (e.g. they show generating a chair from just some legs)... or possibly generating "variations" when the input shape is more complete? But I guess it's a starting point... maybe you could use another model that does worse quality text-to-mesh as the input and get something more crisp and coherent from this one.