3 ms·
The 65b model now runs on the 14" MBP with 64GB of RAM for $3,500. The base config is $2500 to begin with. So not thousands plural, it is one grand more than t
by recuter 4y ago
The 65b model now runs on the 14" MBP with 64GB of RAM for $3,500.
The base config is $2500 to begin with. So not thousands plural, it is one grand more than the base config and a lot of devs/pros go for the 32GB anyhow.
Apple laptops also have atypically high resale value, I usually sell mine after a couple of years for 50%+ of what I paid for it. By the by, Apple have a two week return policy. ;)
I think only your first catch applies. But if you can come up with the right mvp even that might fall away.
- Taek 4y agoDon't need a macbook anymore, any AVX2 CPU can do more than one token per second. You could spend $600 and build your own PC that runs LLaMA 65b.
- recuter 4y agoIt is still much faster/cost-effective thanks to the unified memory. Would be interested in building out a PC as soon as something comparable becomes available.
- Taek 4y agoIf you are going to spend $3500 on macbook because its more cost effective, might as well go all the way to buying an a6000 for $4095 which is iirc 5-10x faster than macbook
- recuter 4y agoI am agnostic about Mac/PC but people sure like to chime in with these sort of comments. First of all it is $4,700 on Newegg and secondly it is out of stock. This also leaves me with the rest of the computer to put together which is not $0. And finally it would absolutely not be 10x faster for this use case than a 64GB MBP. The advantage of going with Apple is that I can pick one up immediately, use it as a laptop (soon with Asahi linux when it matriculates fully) and if need be easily sell it. You might not see the value in that, that's alright, people have different use cases and needs.
- dougmwne 4y agoHow are you going to do training on that machine in a reasonable amount of time? Most of the utility of these models are unlocked after fine tuning.
- recuter 4y agoThrough sheer tyranny of will!