12 ms·
Hunyuan3D 2.0 – High-Resolution 3D Assets Generation
- artemonster 2y ago[flagged]
- pella 2y agoOuch; License: EUROPEAN UNION, UNITED KINGDOM AND SOUTH KOREA TENCENT HUNYUAN 3D 2.0 COMMUNITY LICENSE AGREEMENT Tencent Hunyuan 3D 2.0 Release Date: January 21, 2025 THIS LICENSE AGREEMENT DOES NOT APPLY IN THE EUROPEAN UNION, UNITED KINGDOM AND SOUTH KOREA AND IS EXPRESSLY LIMITED TO THE TERRITORY, AS DEFINED BELOW. https://github.com/Tencent/Hunyuan3D-2?tab=License-1-ov-file https://github.com/Tencent/Hunyuan3D-2?tab=License-1-ov-file
- gruez 2y agoIs this tied to EU regulations around AI models?
- deleted 2y ago[deleted]
- EMIRELADERO 2y agoI assume it's safe to ignore as model weights aren't copyrightable, probably.
- slt2021 2y agoyou dont know what kind of backdoors are hidden in the model weights
- EMIRELADERO 2y agoI mean... you can just firewall it?
- slt2021 2y agoyou dont know which prompt activates the backdoor, how can you firewall it if you run the model in production?
- foolfoolz 2y ago3d asset generation is a use case that for most doesn’t need to run in production
- regularfry 2y ago"In production" in this case is a stand-in for "in any environment with access to sensitive stuff" which might just include GPUs, if what the attacker wanted was crypto processing grunt. Besides, if you're providing 3D asset generation as a service (which I can imagine most deployments of this sort of thing will be, at least for now) then it absolutely is running in production. The purpose of that production environment is entirely to run asset generation.
- dkjaudyeqooe 2y agoSimply sanatieze the model outputs, which is the only thing that would escape running it in complete isolation.
- LiamPowell 2y agoCan you elaborate on how any sort of backdoor could be hidden in the model weights? It's a technical possibility to hide something in the code, but that would be a bit silly since there's not that much of it here. It's not technically possible to hide a backdoor in a set of numbers that are solely used as the operands to trivial mathematical operations, so I'm very curious about what sort of hidden backdoor you think is here.
- swframe2 2y agoWhen you run their demo locally, there are two places that trigger a warning that the code loads the weights unsafely. To learn more about this issue, search "pytorch model load safety issues" on Google.
- LiamPowell 2y agoI'm sure there's warnings about possibly loading code alongside the weights since they distribute the weights as pickled data, but: 1. It's trivial to go look at what's being loaded. 2. Any code that's in the distributed pickled data is not weights. The GP suggests that there are backdoors hidden in the weights which is nonsensical unless the code runs eval on the weights or something similar, which would make anyone looking at the code immediately realise it was doing something dodgy. As an example of (1), here's all the GLOBALs in the pickled code: GLOBAL 'collections OrderedDict' GLOBAL 'torch HalfStorage' GLOBAL 'torch._utils _rebuild_tensor_v2' None of these could be used for anything malicious as far as I know.
- suraci 2y agoYour concern is reasonable. According to DOD, Tencent - which published this model - is a Chinese military company https://www.bbc.com/news/articles/c9q78wn9g8zo https://www.bbc.com/news/articles/c9q78wn9g8zo
- dkjaudyeqooe 2y agoI'm trying to think of what kind of adversarial 3D model the weights could produce. Perhaps a 3D goatse?
- sebzim4500 2y agoInteresting. One of the diagrams suggests that the mesh is generated from the marching cubes algorithm but the geometry of the meshes shown above are clearly not generated in this way.
- GrantMoyer 2y agoTo me, the bird mesh actually does look like marching cubes output. Note the abundance of almost square triangle pairs on the front and sides. Also note that marching cubes doesn't nescessarily create stairstep-like artifacts; it can generate a smooth looking mesh given signed distance field input by slightly adjusting the locations of vertices based on the relative magnitude of the field at the surrounding lattice points.
- wumeow 2y agoThe meshes generated by the huggingface demo definitely look like the product of marching cubes.
- TinkersW 2y agoIf they are using MC, does that mean they are actually generating SDFs? If so it would be nice if you could output the SDF rather than the triangle mesh.
- MikeTheRocker 2y agoGenerative AI is going to drive the marginal cost of building 3D interactive content to zero. Unironically this will unlock the metaverse, cringe as that may sound. I'm more bullish than ever on AR/VR.
- deleted 2y ago[deleted]
- jsheard 2y agoI can only speak for myself, but a Metaverse consisting of infinite procedural slop sounds about as appealing as reading infinite LLM generated books, that is, not at all. "Cost to zero" implies drinking directly from the AI firehose with no human in the loop (those cost money) and entertainment produced in that manner is still dire, even in the relatively mature field of pure text generation.
- deleted 2y ago[deleted]
- echelon 2y agoYou're too old and jaded [1]. It's for kids inventing infinite worlds to role play and adventure. They're going to have a blast. [1] Not meant as an insult. Working professionals don't have time for this stuff.
- wizzwizz4 2y agoObject permanence and a communications channel is enough for this. Give children (who get along with each other) a pile of sticks and leave them alone for half an hour, and there's half a chance their game will ignore the sticks. Most children wouldn't want to have their play mediated by the computer in the way you describe, because the ergonomics are so poor.
- jsheard 2y agoI'm reminded of that guy who bought an AI enabled toy for his daughter and got increasingly exasperated as she kept turning it off and treating it as a normal toy. https://xcancel.com/altryne/status/1872090523420229780 https://xcancel.com/altryne/status/1872090523420229780
- denkmoon 2y agoFor the AI un-initiated; is this something you could feasibly run at home? eg on a 4090? (How can I tell how "big" the model is from the github or huggingface page?)
- sorenjan 2y agoThe hunyuan3d-dit-v2-0 model is 4.93 GB. ComfyUI is on their roadmap, might be best to wait for that, although it doesn't look complicated to use in their example code. https://huggingface.co/tencent/Hunyuan3D-2/tree/main/hunyuan3d-dit-v2-0 https://huggingface.co/tencent/Hunyuan3D-2/tree/main/hunyuan...
- swframe2 2y agoI tried using Hunyuan3D-2 on a 4090 GPU. The Windows install encountered build errors, but it worked better on WSL Ubuntu. I first tried it with CUDA 11.3 but got a build error. Switching to CUDA 12.4 worked better. I ran it with their demo image but it reported that the mesh was too big. I removed the mesh size check and it ran fine on the 4090. It is a bit slow on my i9 14k with 128G of memory. (I previously tried the stability 3d models: https://stability.ai/stable-3d https://stability.ai/stable-3d and this seems similar in quality and speed)
- denkmoon 2y agoCool, thanks. I'm kinda interested so hearing it at least runs on a 4090 means I might give it a go one weekend.
- godelski 2y agoAs with any generative model, trust but verify. Try it yourself. Frankly, as a generative researcher myself, there's a lot of reason to not trust what you see in papers and pages. They link a Huggingface page (great sign!): https://huggingface.co/spaces/tencent/Hunyuan3D-2 https://huggingface.co/spaces/tencent/Hunyuan3D-2 I tried to replicate the objects they show on their project page (https://3d-models.hunyuan.tencent.com/ https://3d-models.hunyuan.tencent.com/). The full prompts exist but are truncated so you can just inspect the element and grab the text. Here's what I got Leaf PNG: https://0x0.st/8HDL.png GLB: https://0x0.st/8HD9.glb Guitar PNG: https://0x0.st/8HDf.png other view: https://0x0.st/8HDO.png GLB: https://0x0.st/8HDV.glb Google Translate of Guitar: Prompt: A brown guitar is centered against a white background, creating a realistic photography style. This photo captures the culture of the instrument and conveys a tranquil atmosphere. PNG: https://0x0.st/8HDt.png and https://0x0.st/8HDv.png Note: Weird thing on top of guitar. But at least this time the strings aren't fusing into sound hole. I haven't tested my own prompts or the google translation of the Chinese prompts because I'm getting an over usage error (I'll edit comment if I get them). That said, these look pretty good. The paper and page images definitely look better, but these aren't like Stable Diffusion 1 paper vs Stable Diffusion 1 reality. But these are long and detailed prompts. Lots of prompt engineering. That should raise some suspicion. Real world has higher variance and let's get an idea how hard it is to use. So let's try some simpler things :) Prompt: A guitar PNG: https://0x0.st/8HDg.png Note: Not bad! Definitely overfit but does that matter here? A bit too thick for a electric guitar but too thin for acoustic. Prompt: A Monstera leaf PNG: https://0x0.st/8HD6.png https://0x0.st/8HDl.png https://0x0.st/8HDU.png Note: A bit wonkier. I picked this because it looked like the leaf in the example but this one is doing some odd things. It's definitely a leaf and monstera like but a bit of a mutant. Prompt: Mario from Super Mario Bros PNG: https://0x0.st/8Hkq.png Note: Now I'm VERY suspicious.... Prompt: Luigi from Super Mario Bros PNG: https://0x0.st/8Hkc.png https://0x0.st/8HkT.png https://0x0.st/8HkA.png Note: Highly overfit[0]. This is what I suspected. Luigi isn't just tall Mario. Where is the tie coming from? The suspender buttons are all messed up. Really went uncanny valley here. So this suggests we're really brittle. Prompt: Peach from Super Mario Bros PNG: https://0x0.st/8Hku.png https://0x0.st/8HkM.png Note: I'm fucking dying over here this is so funny. It's just a peach with a cute face hahahahaha Prompt: Toad from Super Mario Bros PNG: https://0x0.st/8Hke.png https://0x0.st/8Hk_.png https://0x0.st/8HkL.png Note: Lord have mercy on this toad, I think it is a mutated Squirtle. Paper can be found here (the arxiv badge on the page leads to a pdf in the repo, which github is slow to render those): https://arxiv.org/abs/2411.02293 https://arxiv.org/abs/2411.02293 (If you want to share images like I did all I'm doing is `curl -F'file=@foobar.png' https://0x0.st https://0x0.st`) [0] Overfit is a weird thing now. Maybe it doesn't generalize well, but sometimes that's not a problem. I think this is one of the bigger lessons we've learned with recent ML models. My viewpoint is "Sometimes you want a database with a human language interface. Sometimes you want to generalize". So we have to be more context driven here. But certainly there are a lot of things we should be careful about when we're talking about generation. These things are trained on A LOT of data. If you're more "database-like" then certainly there's potential legal ramifications... Edit: For context, by "look pretty good" I mean in comparison to other works I've seen. I think it is likely a ways from being useful in production. I'm not sure how much human labor would be required to fix the issues.
- geuis 2y agoQuestion related to 3D mesh models in general: has any significant work been done on models oriented towards photogrammetry? Case in point, I have a series of photos (48) that capture a small statue. The photos are high quality, the object was on a rotating platform. Lighting is consistent. The background is solid black. These normally are ideal variables for photogrammetry but none of the various common applications and websites do a very good job creating a mesh out of it that isn't super low poly and/or full of holes. I've been casually scanning huggingface for relevant models to try out but haven't really found anything.
- tzumby 2y agoI’m not an expert, only dabbled in photogrammetry, but it seems to me that the crux of that problem is identifying common pixels across images in order to sort of triangulate a point in the 3D space. It doesn’t sound like something an LLM would be good at.
- jocaal 2y agoRecently, a lot of development in this area has been in gaussian splatting and from what I have seen, the new methods are super effective. https://en.wikipedia.org/wiki/Gaussian_splatting https://en.wikipedia.org/wiki/Gaussian_splatting https://www.youtube.com/watch?v=6dPBaV6M9u4 https://www.youtube.com/watch?v=6dPBaV6M9u4
- geuis 2y agoYeah some very impressive stuff with splats going on. But I haven't seen much about going from splats to high quality 3D meshes. I've tried one or two with pretty poor results.
- jiggawatts 2y agoThere have been a few papers published on the topic, but it's "early days". Expect a lot of progress over the next couple of years.
- 2y ago
- xgkickt 2y agoAny user-generated content system suffers from what we call “the penis problem”.
- otikik 2y agoWow they really need to work on that first splash image[1]. All the assets there look hideous. [1] https://github.com/Tencent/Hunyuan3D-2/blob/main/assets/images/teaser.jpg https://github.com/Tencent/Hunyuan3D-2/blob/main/assets/imag...
- FrozenSynapse 2y agothe assets look good as a starting point, they may even be used as background elements
- _s_a_m_ 2y agoHas the word "advanced", gotta be good
- nunodonato 2y agoimagine something like this but geared towards 3d printing functional objects.