3 ms·
hoping nvidia gets some competition, especially for vram. i’d be willing to get an m3 ultra with 256gb ram just to avoid nvidia. their monopoly has allowed them
by moneycantbuy 3y ago
hoping nvidia gets some competition, especially for vram. i’d be willing to get an m3 ultra with 256gb ram just to avoid nvidia. their monopoly has allowed them to get away with being greedy jerks to the very customers who made them a trillion dollar company.
- whywhywhywhy 3y ago> i’d be willing to get an m3 ultra with 256gb ram just to avoid nvidia The performance just isn't there unfortunately if you're comparing the Nvidia's high end where you'd be considering this. 3090 leaves the entire M range in the dust and a lot of things the XX90 series doesn't really break a sweat for like SDXL you can feel the M series choking under the weight of it and significantly longer processing even if it does have more ram. I'd say the difference is closer with the llama.cpp stuff but image gen it's night and day.
- behnamoh 3y agoto run local LLMs, Mx Ultra is fine, esp. with parallel decoding. It can run 70B models easily. What's the point of faster speeds if the GPU can't even load the model?
- smoldesu 3y agoWhat's the point of running unquantized models? If you want to run 70b knock yourself out, but you're wasting electricity and life-hours by using 16-bit float at full precision. Even if you could accelerate such a model well, it's not going to perform any better than a decently trained finetune. The line seems pretty clearly drawn, to me. Datacenter customers flat-out get better treatment, and the gaming segment is going to have to cope with that. Mellanox networking is not coming to gaming hardware, and Grace CPUs are probably not going to get crammed on an ATX motherboard. They aren't the same market, full stop. Maybe Nvidia will have something for you when their desktop chips launch.