4 ms·

I understand the preference to run models locally rather than in a rented space, but given the speed of development and the amount of money involved, you should

I understand the preference to run models locally rather than in a rented space, but given the speed of development and the amount of money involved, you should have a clear reason why you need to run these models locally. As far as I know, the most RAM you can get in a MacBook Pro (which is what you said you're shopping for) is 48G.* The base price for new one with that much unified RAM is around $4000. The Mac Pro towers (not MacBook) have up to 192G unified RAM. The base price for that configuration is around $8600. The smaller LLMs are getting quite good. A lightly quantized Llama-8B should comfortably run on a MacBook Pro with with 16G of RAM which you can get for around $2000. The money you save on a cheaper machine will go a very long way renting compute from a datacenter. If you need to run locally, then high end Macs are excellent machines. Though at those prices you might get better value buying a second hand crypto-mining rig with multiple Nvidia 4090's. EDIT: I was wrong about the MBP unified RAM. You can get an M3 Max with 128GB for around $4700.

Terretta 2y ago
> As far as I know, the most RAM you can get in a MacBook Pro (which is what you said you're shopping for) is 48G. My Macbook Pro has 2.5x that (128GB), and I run models that use 2x that RAM (96GB) with no impact to my IDE, browser, or other apps running at the time, they act like they're on a 32GB machine. LM Studio makes it easy for newcomers to on-device LLMs to dip your toe into this, both for turning on Metal and helping suggest which models will fit entirely in RAM.
- loudmax 2y ago
  You're right, you can get an M3 Max MBP with 128GB, starting at $4700. My main point is that if your objective is dipping your toe into this, you can do it with smaller models for far less. That is a really sweet machine, but for the amount of money involved you should be clear about what your needs are.
coder543 2y ago
> As far as I know, the most RAM you can get in a MacBook Pro (which is what you said you're shopping for) is 48G Huh? They have options up to 128GB… https://www.apple.com/shop/buy-mac/macbook-pro/14-inch-space-black-apple-m3-max-with-14-core-cpu-and-30-core-gpu-36gb-memory-1tb https://www.apple.com/shop/buy-mac/macbook-pro/14-inch-space...

Terretta 2y ago

> As far as I know, the most RAM you can get in a MacBook Pro (which is what you said you're shopping for) is 48G. My Macbook Pro has 2.5x that (128GB), and I run models that use 2x that RAM (96GB) with no impact to my IDE, browser, or other apps running at the time, they act like they're on a 32GB machine. LM Studio makes it easy for newcomers to on-device LLMs to dip your toe into this, both for turning on Metal and helping suggest which models will fit entirely in RAM.

loudmax 2y ago
You're right, you can get an M3 Max MBP with 128GB, starting at $4700. My main point is that if your objective is dipping your toe into this, you can do it with smaller models for far less. That is a really sweet machine, but for the amount of money involved you should be clear about what your needs are.

coder543 2y ago

> As far as I know, the most RAM you can get in a MacBook Pro (which is what you said you're shopping for) is 48G Huh? They have options up to 128GB… https://www.apple.com/shop/buy-mac/macbook-pro/14-inch-space-black-apple-m3-max-with-14-core-cpu-and-30-core-gpu-36gb-memory-1tb https://www.apple.com/shop/buy-mac/macbook-pro/14-inch-space...