3 ms·
Yes, you can run any inference with any model on CPU, but here some examples: To create a single frame with Stable Diffusion 4-5B parameters, 512x512, 20 itera
by novaRom 4y ago
Yes, you can run any inference with any model on CPU, but here some examples:
To create a single frame with Stable Diffusion 4-5B parameters, 512x512, 20 iterations takes 5-30 minutes depending on your CPU. On any modern GPU it's only 0.1-20 seconds!
Similarly with LLMs, to produce one token with a transformer of 30-50 layers 7-12B parameters you will wait several CPU minutes while it takes few seconds on a Pascal-generation GPU and tiny fraction of second on Ampere.
- probablynish 4y agoThe *.cpp adaptations of popular models appear to have optimized them for the CPU, for example llama.cpp and alpaca.cpp let me generate several tokens in a matter of seconds.
- gpderetta 4y ago> To create a single frame with Stable Diffusion 4-5B parameters, 512x512, 20 iterations takes 5-30 minutes depending on your CPU Depends a lot on the cpu. Are you specifically talking about Text2Video or SD in general? IIRC, last time I tried SD on my CPU (10 core 10850k, not exactly cutting edge) it did take less than one minute for more than 20 iters. This was about 4-5 months ago, things might have gotten better. The GPU (even a vintage 1070) was faster still of course.
- tjoff 4y agoThat is much better than expected, depending on how easy it is to setup it is very much worthwhile if you don't have access to a decent GPU.