shivampkumar

4 ms·

1.
by shivampkumar 5mo ago
Agreed...I've been adding some like mtlgemm, mtldiffrast from other contributors already
2.
by shivampkumar 5mo ago
Not currently - TRELLIS.2 is single-image input only AFAIK
3.
by shivampkumar 5mo ago
In theory yes - the pipeline already does this to some extent with its low_vram mode, offloading models to CPU between stages. The challenge at 16GB is that even a single 1.3B sub-model at fp32 plus activations can push past what's ava
4.
by shivampkumar 5mo ago
The gather-scatter sparse conv should be fairly generic. Any model using 3x3x3 or 5x5x5 sparse convolutions on voxel grids could use it directly. The main thing that's TRELLIS-specific is the neighbor cache key format, but that's
5.
by shivampkumar 6mo ago
that makes so much sense...I am exploring if I can find someone who has done this well...If not I'll try to do it myself.
6.
by shivampkumar 6mo ago
The model needed about 15GB at peak during generation - the 4B model loads multiple sub-models (1.3B each for shape and texture flow). 8GB won't be enough, but both 24GB and 32GB both should be fine.
7.
by shivampkumar 6mo ago
added! will add more, maybe even a GIF
8.
by shivampkumar 6mo ago
i was able to get it in 3.5 mins from a single image on my 24gb m4 pro macbook I'm still working on this to try to replicate nvdiffrast better. Found an open source port, might look it tonight
9.
by shivampkumar 6mo ago
thanks!
10.
by shivampkumar 6mo ago
I mean I can see that it's niche. Did not expect so many upvotes, but ig it's less niche than I tought If you're not working with 3D on Apple Silicon this isn't relevant to you. For the subset of people who are, running
11.
by shivampkumar 6mo ago
I thought it was cool and then I found the open issue mentioned above, that convinced me its def something more people want. It IS significantly slower, about 3.5 minutes on my MacBook vs seconds on an H100. That's partly the pure-PyTo
12.
by shivampkumar 6mo ago
IMO TRELLIS.2 is slightly different case from the HF models scenario. It depends on five compiled CUDA-only extensions -- flex_gemm for sparse convolution, flash_attn, o_voxel for CUDA hashmap ops, cumesh for mesh processing, and nvdiffrast
13.
by shivampkumar 6mo ago
You're right, thanks for flagging this, let me run something and push images
14.
by shivampkumar 6mo ago
Hey, thanks for sharing this. I'm sure TRELLIS.2 definitely has room to improve, especially on texturing. From what I've seen personally, and community benchmarks, it does fair on geometry and visual fidelity among open-source opt
15.
Show HN: Run TRELLIS.2 Image-to-3D generation natively on Apple Silicon(github.com)
202 points by shivampkumar 6mo ago|40 comments