3 ms·
Sufficient for LLMs and image/video gen.
by treprinum 2y ago
Sufficient for LLMs and image/video gen.
- m00x 2y agoFLUX.1 D generation is about a minute at 20 steps on a 4080, but takes 35 minutes on the CPU.
- treprinum 2y ago4080 won't do video due to low RAM. The GPU doesn't have to be as fast there, it can be 5x slower which is still way faster than a CPU. And Intel can iterate from there.
- m00x 2y agoIt won't be 5x slower, it would be 20-50x slower if you would implement it as you said. You can't just "add more ram" to GPUs and have them work the same way. Memory access is completely different than on CPUs.
- vunderba 2y agoYep. Any large GenAI image model (beyond SD 1.5) is hideously slow on Mac's irrespective of how much RAM you cram in - whereas I can spit out a 1024x1024 image from Flux.1 Dev model in ~15 seconds on a RTX 4090.
- Der_Einzige 2y agoNot even close. Llama.cpp isn't even close to a production ready LLM inference engine, and it runs overwhelmingly faster when using CUDA
- pjmlp 2y agoA fraction of what a GPU is used for.