3 ms·
Looks more like 4 s/iter which is actually quite good for CPU inference in Automatic111. It is an apples-to-oranges comparison to the LCM models used by fastsdc
by samsartor 3y ago
Looks more like 4 s/iter which is actually quite good for CPU inference in Automatic111. It is an apples-to-oranges comparison to the LCM models used by fastsdcpu (since those can get away with fewer sampling steps) or to the quantized models favored by apple (since they use totally different underlying compute capabilities). Currently automatic111 and other popular GUIs are pretty focused on users with beefy GPUs that can sample images in seconds, so they haven't yet implemented some of these really significant CPU-friendly speedups. But we'll get there!
- mosselman 3y agoAre those cpu speedups out there? Where can I find them?
- brucethemoose2 3y agoNot for A1111's SAI backend, as far as I know. VoltaML has some CPU optimized options for the torch.compile optimization.