I'm shocked - before I seen it working myself, I was not believe, so large (smart) LLM could run on desktop cpu. - core i7, 4 cores, 3.3GHz, 64G ddr4-2400, 70b

I'm shocked - before I seen it working myself, I was not believe, so large (smart) LLM could run on desktop cpu. - core i7, 4 cores, 3.3GHz, 64G ddr4-2400, 70b model, ~0.5 tokens/s; 30b model ~1 tokens/s.