4 ms·
It is also the performance of 4 Mac Pros. GP was talking about tasks that dedicated GPUs supposedly could not handle. Who is is running models needing 60+ GB of
by incrudible 3y ago
It is also the performance of 4 Mac Pros. GP was talking about tasks that dedicated GPUs supposedly could not handle. Who is is running models needing 60+ GB of VRAM on their laptops?
- ElFitz 3y agoSome have had fun running some of the larger Llama, Vicuna & Guanaco models on their MacBooks. Mostly to experiment with local inference. Although quantisation has now lowered the required memory, I wouldn’t be surprised if it comes in handy again in the near future. But today that’s quite a niche use case.