5 ms·
109B at Q6 is also nice for Framework Desktop 128GB.
by tuukkah 2y ago
109B at Q6 is also nice for Framework Desktop 128GB.
- echelon 2y agoI don't understand Framework's desktop offerings. For laptops their open approach makes sense, but desktops are already about as hackable and DIY as they come.
- nrp 2y agoWe took the Ryzen AI Max, which is nominally a high-end laptop processor, and built it into a standard PC form factor (Mini-ITX). It’s a more open/extensible mini PC using mobile technology.
- mdp2021 2y agoAnd given that some people are afraid of malicious software in some brands of mini-PCs on the market, to have some more trusted product around will also be an asset.
- kybernetikos 2y agoI love the look of it and if I were in the market right now it would be high on the list, but I do understand the confusion here - is it just a cool product you wanted to make or does it somehow link to what I assumed your mission was - to reduce e-waste?
- nrp 2y agoA big part of our mission is accessibility and consumer empowerment. We were able to build a smaller/simpler PC for gamers new to it that still leverages PC standards, and the processor we used also makes local interference of large models more accessible to people who want to tinker with them.
- bavell 2y agoConsidering the framework desktop or something like it for a combo homelab / home assistant / HTPC. The new gen of AMD APUs looks to be the sweet spot for a lot of really interesting products. Love what you guys are doing!!
- elorant 2y agoIt’s an x86 PC with unified RAM based on AMD’s new AI cpus. Pretty unique offering. Similar to Mac studio but you can run Linux or Windows on it, and it’s cheaper too.
- aurareturn 2y agoIt's a lot slower than a Mac Studio. Significantly slower CPU, GPU, memory bandwidth.
- tw1984 2y agointeresting to know, thanks. any link to some concrete benchmarks to share?
- aurareturn 2y agoYes. Geekbench 6 for CPU. Notebookcheck for GPU. Youtube/X for LLM inference.
- nrp 2y agoYes, this announcement was a nice surprise for us. We’re going to test out exactly that setup.
- rubymamis 2y agoAwesome, where can we find out the results?
- nrp 2y agoWe’ll likely post on our social accounts to start with, but eventually we plan to write more blog posts about using Framework Desktop for inference.
- rcarmo 2y agoThat would be great. I’ve been hacking at ROCm and using Ryzen iGPUs for industrial scenarios, and the HX chipsets look like a massive improvement over what you’d get from folk like AsRock Industrial.
- rcarmo 2y agoCan’t wait.
- theptip 2y agoIs the AMD GPU stack reliable for running models like llama these days?
- rubatuga 2y agoRunning yes, training is questionable