Some have had fun running some of the larger Llama, Vicuna & Guanaco models on their MacBooks. Mostly to experiment with local inference. Although quantisation

Some have had fun running some of the larger Llama, Vicuna & Guanaco models on their MacBooks. Mostly to experiment with local inference. Although quantisation has now lowered the required memory, I wouldn’t be surprised if it comes in handy again in the near future. But today that’s quite a niche use case.