> Do you know if that hardware is capable of running LLM or is that optimized for something else? I do! Checkout llama.cpp [0] and related projects. The game c

> Do you know if that hardware is capable of running LLM or is that optimized for something else? I do! Checkout llama.cpp [0] and related projects. The game changer with this was Gerganov's ggml format that makes inference on CPUs realistic. I'm currently running alpaca.cpp and llama.cpp, using 7B models 4-bit quantized (so they're the lowest powered, lightest weight models) on an M1 Pro with 16gig RAM. You could run same on an M1 Macbook Air with 8gig RAM. Gerganov shows the Whisper model running on an Android phone. [0]https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp