3 ms·

Framework offers the AI Ryzen Max with ̶1̶9̶6̶G̶B̶ 128GB of unified RAM for 2,699$ That's a pretty good deal I would think https://frame.work/de/de/products/

Framework offers the AI Ryzen Max with ̶1̶9̶6̶G̶B̶ 128GB of unified RAM for 2,699$ That's a pretty good deal I would think https://frame.work/de/de/products/desktop-diy-amd-aimax300/configuration/new https://frame.work/de/de/products/desktop-diy-amd-aimax300/c...

rl3 6mo ago
128GB*
- hermanzegerman 6mo ago
  Thanks for spotting the mistake. No Idea how I got to 192
  - rl3 6mo ago
    For what it's worth, I really wish that was the actual number.
eigenspace 6mo ago
The framework desktop is quite cool, but those Ryzen Max CPUs are still a pretty poor competitor to Apple's chips if what you care about it running an LLM. Ryzen Max tops out at 256 GB/s of memory bandwidth, whereas an M4 Max can hit 560 GB/s of bandwidth. So even if the model fits in the memory buffer on the Ryzen Max, you're still going to hit something like half the tokens/second just because the GPU will be sitting around waiting for data. Personally, I'd rather have the Framework machine, but if running local LLMs is your main goal, the offerings from Apple are very compelling, even when you adjust for the higher price on the Apple machine.
- rl3 6mo ago
  There's also the DGX Spark. Granted, its price has been going up recently alongside everything else that has memory in it.
  - eigenspace 6mo ago
    I haven't heard a single good think about DGX Spark from anyone using it, so I'd be pretty wary about that.
  - xienze 6mo ago
    That also has pretty poor memory bandwidth. 283GB/s I think.
    - rl3 6mo ago
      Yeah. The main selling point I'd say is the onboard ConnectX-7 hardware.
freeqaz 6mo ago
128gb is the max RAM that the current Strix Halo supports with ~250GB/s of bandwidth. The Mac Studio is 256GB max and ~900GB/s of memory bandwidth. They are in different categories of performance, even price-per-dollar is worse. (~$2700 for Framework Desktop vs $7500 for Mac Studio M3 Ultra)

rl3 6mo ago

128GB*

hermanzegerman 6mo ago
Thanks for spotting the mistake. No Idea how I got to 192
- rl3 6mo ago
  For what it's worth, I really wish that was the actual number.

eigenspace 6mo ago

The framework desktop is quite cool, but those Ryzen Max CPUs are still a pretty poor competitor to Apple's chips if what you care about it running an LLM. Ryzen Max tops out at 256 GB/s of memory bandwidth, whereas an M4 Max can hit 560 GB/s of bandwidth. So even if the model fits in the memory buffer on the Ryzen Max, you're still going to hit something like half the tokens/second just because the GPU will be sitting around waiting for data. Personally, I'd rather have the Framework machine, but if running local LLMs is your main goal, the offerings from Apple are very compelling, even when you adjust for the higher price on the Apple machine.

rl3 6mo ago
There's also the DGX Spark. Granted, its price has been going up recently alongside everything else that has memory in it.
- eigenspace 6mo ago
  I haven't heard a single good think about DGX Spark from anyone using it, so I'd be pretty wary about that.
- xienze 6mo ago
  That also has pretty poor memory bandwidth. 283GB/s I think.
  - rl3 6mo ago
    Yeah. The main selling point I'd say is the onboard ConnectX-7 hardware.

freeqaz 6mo ago

128gb is the max RAM that the current Strix Halo supports with ~250GB/s of bandwidth. The Mac Studio is 256GB max and ~900GB/s of memory bandwidth. They are in different categories of performance, even price-per-dollar is worse. (~$2700 for Framework Desktop vs $7500 for Mac Studio M3 Ultra)