3 ms·
The architecture is the same but it has an additional model trained on the sampling trajectory computation, speeding up the process. So instead of 20 iterations
by orbital-decay 3y ago
The architecture is the same but it has an additional model trained on the sampling trajectory computation, speeding up the process. So instead of 20 iterations with the original sampler that solves a SDE, you could use 8, 4, or sometimes even 2 iterations with this new ML-based sampler. That's not a new idea, but the author of this adapter figured out how to train it efficiently using very few GPU-hours.
- jiggawatts 3y agoI wonder if they could do a real-time preview using 2-4 iterations, and then 'refine' the quality with a higher number of iterations over time, much like how 3D editors preview ray-tracing.
- orbital-decay 3y agoI've lost the link, but IIRC someone on twitter demoed exactly that in Blender. "Neural rendering" in a 3D editor has some nice properties - the scene can be rendered into separate layers with separate depth maps and openpose skeletons generated from the rigs to serve as input for controlnets, objects can be tagged and tags composed into the prompt for the particular tile in case of tiled rendering, etc.
- dvrp 3y agoprobably not but are you talking about the guy who used control net and animatediff to make a video that looked relatively consistent?
- orbital-decay 3y agoNo, it was running in the Blender viewport using the original LCM Dreamshaper model.
- dvrp 3y agowe’re thinking of doing that as well! but first we want to make sure we can get this in the hands of a tight cohort of creatives and see how they use it.