When llamaCPP came out, I was running 13B at 100ms/token on a base model MacBook Pro 14". Edit: apparently llama.cpp supports running on GPU, so I imagine it's

When llamaCPP came out, I was running 13B at 100ms/token on a base model MacBook Pro 14". Edit: apparently llama.cpp supports running on GPU, so I imagine it's gonna be a bit faster. Maybe a fun evening project for me to get going.