3 ms·
Can I ask what did you buy? I don't have that money now but I wouldn't knpw what to buy anyway.
by amunozo 3mo ago
Can I ask what did you buy? I don't have that money now but I wouldn't knpw what to buy anyway.
- jliptzin 3mo agoBlackwell 96gb + 128gb system ram. Not powerful enough for GLM 5.2 but I may upgrade.
- amunozo 3mo agoThanks! What a beast.
- aetherspawn 3mo agoI can run most models 100B and under on my MacBook Pro with Ultra 3 and 128GB of RAM at 25-70tok/sec-ish, and it was around $5k. I don’t think I can run GLM 5.2 since it requires around 256GB of memory and the inference is probably too slow, but the future (planned) Apple M7 may be able to. The leaks say it will support up to ~700GB of RAM. The models under 100B are kind of dumb as a brick and aren’t that useful unless you’re really bad at coding imo. They can’t really be trusted not to hallucinate so they’re not even good for data processing.
- amunozo 3mo agoI'll probably buy something like this in the future, but given the rapid advances and that I bought a laptop recently, I don't think is urgent.