One can run a llama 13B finetune on a 2020 16GB laptop, at (for me) ~230 ms/token, and hook it up to duckdb with off the shelf open source UIs. But it is kind

One can run a llama 13B finetune on a 2020 16GB laptop, at (for me) ~230 ms/token, and hook it up to duckdb with off the shelf open source UIs. But it is kind of "stupid" compared to gpt3.5 unless you make the jump to Falcon 40B or LLaMA 30B/65B. And (right now) you need a pretty beefy PC for those models.