4 ms·
Ugh. No, we can now run a decent model through CPU. Not an expensive video card. Just try out the standard deepseek-r1 or even the deepseek-r1:1.5B through oll
by NicoJuicy 2y ago
Ugh. No, we can now run a decent model through CPU. Not an expensive video card.
Just try out the standard deepseek-r1 or even the deepseek-r1:1.5B through ollama.
No need for expensive hardware anymore locally. My PC ( without Nvidia card/expensive hardware) runs the a deepseek 1.5 b query fast enough - 2 - 9 seconds until it's finished.
- aurareturn 2y agoThe world isn't satisfied with a "decent model". The world is trying to reach AGI. Further more, reasoning models require more tokens. The faster the GPU, the more thinking it can do in a set amount of time. This means the faster the hardware, the smarter the model output. Again, that reinforces the need for faster hardware. More thinking = smarter models Faster hardware = more thinking Therefore, faster hardware = smarter models
- Jlagreen 2y agoI would shoot myself if I had to wait 9 seconds for a query. I sometimes even would like to kill by browser taking seconds to open a page...
- NicoJuicy 2y ago2-4 seconds without streaming on a non GPU machine