4 ms·
Sure, but then that’s the price of multiple entire laptops. Or 2 Mac Pros. Not sure what point you’re trying to make here.
by ElFitz 3y ago
Sure, but then that’s the price of multiple entire laptops. Or 2 Mac Pros. Not sure what point you’re trying to make here.
- incrudible 3y agoIt is also the performance of 4 Mac Pros. GP was talking about tasks that dedicated GPUs supposedly could not handle. Who is is running models needing 60+ GB of VRAM on their laptops?
- ElFitz 3y agoSome have had fun running some of the larger Llama, Vicuna & Guanaco models on their MacBooks. Mostly to experiment with local inference. Although quantisation has now lowered the required memory, I wouldn’t be surprised if it comes in handy again in the near future. But today that’s quite a niche use case.