3 ms·
Is four years really 'long tail' these days? Our VM host box is from 2010 (and I had to rebuild llama.cpp locally without AVX to get it working :P )
by taneq 3y ago
Is four years really 'long tail' these days? Our VM host box is from 2010 (and I had to rebuild llama.cpp locally without AVX to get it working :P )
- yjftsjthsd-h 3y agoFor cutting-edge LLM work, probably? I mean, I run mine on older hardware than that, but I'm a total hobbyist...
- d416 3y agoIt should be noted that while the HP Prodesk was released in 2020, the CPU’s Skylake architecture was designed in 2014. Architecture is a significant factor in this style of engineering gymnastics to squeeze the most out of silicon.
- refulgentis 3y agoFor LLMs...yeah. I imagine you're measuring in tokens/minute with that setup. So its possible, but...do you use it much? :)