4 ms·
There is no socketable Unified Memory platform. The entire point of this SoC is that it has Unified Memory. The Framework Desktop has a Mini-ITX board, but yo
by tristor 3mo ago
There is no socketable Unified Memory platform. The entire point of this SoC is that it has Unified Memory. The Framework Desktop has a Mini-ITX board, but you MUST use a solderable SoC with HBM stacked memory to get a Unified Memory outcome. For local AI workloads, that's the only part that matters, which is the entire point of wanting 192GB of RAM available on an APU.
I like the repairability and modularity of my Framework 13 laptop, and I still bought a maxed out MBP M5 Max because for local LLM, unified memory is all that matters.
- artisinal 3mo agoIf this CPU is not socketable by design, why not pick a CPU that is? There are plenty of options. Repairability is more important than raw speed. That is why people pick Framework. It’s not the best machine at everything but it’s easy to repair.
- tristor 3mo agoYou fundamentally cannot do things on the system you are envisioning that you can do on a Framework Desktop, or DGX Spark, or MBP M5 Max. Simple as that. Repairable modular laptops were a fairly rare thing when Framework came out. A repairable modular desktop is just a normal desktop computer. A desktop computer in a compact form factor that can run AI workloads though is a much rarer thing.
- artisinal 3mo agoMy computer with socketed CPU had no issues being a computer. It’s so strange for Framework to make a machine you can’t repair. It’s the Framework Desktop. Why not create the best possible repairable Desktop instead of using some niche CPU soldered to the board? Not everyone has AI workloads. Some people just browse the web a little.
- tristor 3mo agoCan it run a local LLM with 35B+ parameters using GPU acceleration for reasonable token performance? If not, then you're being obtuse.
- artisinal 3mo agoHow many people use a desktop for a local LLM with 35B+ parameters? Maybe 0.0001% of computer users?
- tristor 3mo agoThe people who buy a Framework Desktop with 192GB of RAM (or in my case an M5 Max MBP with 128GB of RAM). I don't know what argument you're trying to make here? You are saying this product is pointless, the point of this product is clearly stated and obvious, and now you're acting like its not relevant. If you don't want to engage in a reasonable discussion, you can get bent.
- LargoLasskhyfv 3mo agoAny SoC currently available does not use HBM for its unified memory. That's just stacked DRAM, wired somewhat more optimally because of shorter distances, making it able to go a little bit faster, and maybe wider. But not as massively wide as HBM is, with its load of many lanes. This may shift in the near future, in favor of more traditional stacking, cost/production-wise, vs. real HBM with massive speed improvements, at much higher costs. That aside, ever heard of https://en.wikipedia.org/wiki/CAMM_(memory_module) https://en.wikipedia.org/wiki/CAMM_(memory_module) ?