The large Alpaca models can run on a M1 Pro MacBook or similar performance Linux PC with llama.cpp. You will need 32-64G of RAM and fast SSD for the large model

The large Alpaca models can run on a M1 Pro MacBook or similar performance Linux PC with llama.cpp. You will need 32-64G of RAM and fast SSD for the large models. They show performance that reminds me of GPT-3 (not quite 3.5). There are some newer models out there I have not tried yet like the open llama, GPT4all, etc. so I’m not sure how good they are. I get the sense they are still GPT-3 level but are achieving that with less RAM. There’s a race on both for raw capability and optimization via pruning and quantization. The latter is important to make these things runnable locally without gigantic hardware. Lots of people have stuff with GPUs, fast CPUs, and 32-64G RAM. Few have huge workstations with hundreds of gigs of RAM. Unless progress stagnates I can see something approaching GPT-4 that can run on under $5k worth of hardware in a year or so. Open model progress seems to be lagging only 1-2 years behind big cloud hosted models.