For the interested, I've been having fun with llama.cpp. I've got it running well, if a bit slowly, on my 2019 intel MacBook Pro. There's configs you can tweak

For the interested, I've been having fun with llama.cpp. I've got it running well, if a bit slowly, on my 2019 intel MacBook Pro. There's configs you can tweak that get it to run faster, mainly dealing with memory mapping. I've been working on documenting my findings but haven't published yet. I say all that to encourage people to try it out with different models. It's not production quality but totally usable for testing things out even on older machines.