4 ms·
There is no "future proof" for >1T param models, there is no present or future where you can run a model that size on consumer hardware.
by root_axis 1mo ago
There is no "future proof" for >1T param models, there is no present or future where you can run a model that size on consumer hardware.
- bewareofscams 1mo ago6 Mac Studios for 100k USD. Consider it a rule of thumb now - 100k to run 1T params, scales linearly.
- MPSimmons 1mo agoOut of curiosity, what do you do to distribute the weights and inference across the multiple machines?
- fzzzy 1mo agoThunderbolt 5, and software
- NamlchakKhandro 1mo agoat a 100k$ you're not buying apple products to be limited by the 120gb/s thunderbolt port.
- root_axis 1mo agoIf you have 100k you can just buy a few b200s.
- blints 1mo agoOne does not simply "buy a few b200". They come in 8 packs minimally, at >5x that budget.
- blints 1mo agoI don't know that "consumer hardware" is a useful distinction anymore, it's just "what's your budget and what's your speed requirement".
- kridsdale1 1mo agoConsumer hardware means: - 120v input plug - not rack-mounted - has a video out port
- iAMkenough 1mo agoAh, a MacBook Professional is Consumer /s
- kridsdale1 1mo agoYes?
- iAMkenough 1mo agoPro?
- oblio 1mo agoWant to bet that consumers with enough money buy it?
- deleted 1mo ago[deleted]
- kllrnohj 1mo agoApple's use of "Professional" is just a fancy alias for high end or premium, it in no way indicates anything about being "for professionals" (most extreme example: "professional" iPhone models) Hence why they had to make up the "Studio" brand for the workstation market, because they'd already fully removed any meaning from "Professional"
- 1mo ago