3 ms·
LLM? Probably ok. Image diffusion? Not so hot, but better than CPU.
by weebull 3y ago
LLM? Probably ok.
Image diffusion? Not so hot, but better than CPU.
- stevenhuang 3y agoYou got that switched around. Text generation is harder than image generation.
- lostmsu 3y agoIt is also memory bandwidth bound for non-batched cases.