3 ms·
> But with 25 fps @ 2K it's not really usable yet Yet is the key word. Let's nor forget the context here. This is a __research__ paper, and not one by a big te
by godelski 3y ago
> But with 25 fps @ 2K it's not really usable yet
Yet is the key word. Let's nor forget the context here. This is a __research__ paper, and not one by a big tech company. Yes, a big university, but still. I'm absolutely certain you could get this to at least 30fps if not 45. They're doing this on a 3090, so that gives some context about how much money that lab has. If they had a bunch then they'd be using A100s or H100s and at least a node. What's actually really impressive is that they are training their model on a single 3090 in 15hrs. Most certainly you can scale this model, and then teach an even smaller distilled version that will be much faster. If you're a betting man I'd put money down that they are not implementing this in C, are not using custom cuda kernels, TensorRT, nor are they quantizing their model. They might not even be using fp16.
I don't know why everyone expects everything to be TRL 8+. What's wrong with low TRL research? Especially from academia. This is definitely somewhere in TRL 3-5. You wanna see cool things? Take cool research and turn them into products. It's a whole other challenge. But we've even seen LLMs and diffusion models get a lot faster in such a short time. Why not this too?
- prox 3y agoWhat if you mix splatting and diffusion models, so you get point cloud like type semi-3d image (re)construction?
- godelski 3y agoSure, why not? Or why not splatting + GAN since the GAN will be much more computationally light and you get high diversity from the splatting already? All worth a shot, right? Lots of things to explore.
- rbanffy 3y agoBesides, while transistors aren’t shrinking that much, Moore’s Law is still a thing. I’d expect 50fps at 2K in less than 2 years.