4 ms·
$10k to run a 4 bit quantized model. Ouch.
by nightski 2y ago
$10k to run a 4 bit quantized model. Ouch.
- vaxman 2y ago[flagged]
- vaxman 2y ago[flagged]
- 1R053 2y agoare you comparing the same models? How did you calculate the TOPS for M3 Ultra?
- vaxman 2y agoAn M3 Ultra is two M3 Max chips connected via fabric, so physics. Did not mean to shit on anyone's parade, but it's a trap for novices, with the caveat that you reportedly can't buy a GB10 until "May 2025" and the expectation that it will be severely supply constrained. For some (overfunded startups running on AI monkey code? Youtube Influencers?), that timing is an unacceptable risk, so I do expect these things to fly off the shelves and then hit eBay this Summer.
- fredoliveira 2y agowhat?
- vaxman 2y agoSorry, an apostrophe got lost in "PO's"
- titanomachy 2y agoI'm downvoting you because your use of language is so annoying, not because I work for Apple.
- vaxman 2y agoSo, Microsoft?
- water9 2y agoThe M4 MacBook Pro 128GB can run a 32B perimeter model with an 8 bit quantized model just fine
- OriginalMrPink 2y agoThat's today. What about tomorrow?