5 ms·
A single 3090 will deliver more tflops than the m2 ultra.
by sva_ 2y ago
A single 3090 will deliver more tflops than the m2 ultra.
- angoragoats 2y agoYep. The 8x3090 setup should be at least 10 times faster than the Mac.
- wkat4242 2y agoYes though for inference memory bandwidth is more important than tflops. Not sure how Apple compares in that regard. The OP does train models too though which is more compute heavy.
- angoragoats 2y agoEach of the 3090s has something like 15% higher memory bandwidth than the whole M2 Ultra system does.
- evilduck 2y agoThe M2 Ultra doesn't require doing electrical work on your house like this 8x 3090 setup did though.
- ein0p 2y agoYes, for 20x the FLOPS at about twice the price you might need to do some work.
- bongodongobob 2y agoYeah cause it's doing like 50x less work.
- NavinF 2y agoYou could power limit the 3090s to fit a standard 120V*20A = 2400W outlet if you really want to. The default power limit is 350W each so you'll only lose a little perf. Also most rooms have multiple circuits. Just connect half the GPUs to each outlet. I already do this with my desktop PC because it has 2 PSUs. Also most homes in the US have 30A*240V = 7200W dryer/stove outlets in the kitchen, laundry room, garage, etc. That's where you want to keep loud computers.
- jrflowers 2y agoBrb replacing my stove with a bunch of GPUs
- lvncelot 2y agoInference Oven
- philsnow 2y agoEasy-bake cryptominer
- qingcharles 2y agoYou can cook on the GPUs from now on.
- Dylan16807 2y agoA standard outlet is 15A, and you're only allowed to use 80% continuously. So now you're dropping to 150W each. Using two circuits is viable.
- NavinF 2y agoIn the bay area every home I've lived in had 20A circuits for everything. The NEC 80% rule is only relevant in data centers where someone enforces it. The time-current curve for any 20A breaker shows that it will take infinite time to trip when you draw 20A. This is correct behavior because 12awg wire can handle 20A indefinitely
- Dylan16807 2y agoDo they actually install the 20A outlets with the sideways part? I don't know if I've ever seen one in person. I guess the outlet doesn't matter a whole lot, especially if you take multiple cables to the wall. But I still don't want to be doing unenforced electrical violations.
- 2y ago
- santoshalper 2y agoSure, neither does just using ChatGPT, but we don't do things because they are easy, we do them because they are difficult.
- angoragoats 2y agoYou can run 8 or even more 3090s without doing electrical work on a standard US residential power feed, you just need outlets connected to multiple circuits. This is not always convenient, which is why some may choose to do electrical work, but it’s absolutely not required.
- korhojoa 2y agoLocation specific issue. Higher voltage resolves this and a regular household socket will power this just fine. Then again, if you're spending that much money to get the job done, new wiring is probably one of the cheaper parts of the build.
- fallingsquirrel 2y ago1x 3090 would be faster than the mac, cheaper than the mac, and run on a normal residential circuit. How many breakers would you need for a cluster of 20 macs running at full throttle to match the perf of OP's build?
- paulmd 2y agoA single 3090 won’t even fit the model. OP is talking about running a 405B model, it needs close to 200gb of memory just to open it which is why we’re talking about mac studio and it’s 192gb unified memory. This is exactly like when the AMD fanboys got a burr up their ass about the “$50k Mac Pro” with 2tb of memory… when you could the same thing with a threadripper with 256gb of memory for $5k, and it’s just as fast in Cinebench! https://old.reddit.com/r/Amd/comments/f1a0qp/15000_mac_pro_vs_5000_threadripper_sorry_apple/ https://old.reddit.com/r/Amd/comments/f1a0qp/15000_mac_pro_v... your gaming scores on your 3090 with 24gb are just as irrelevant to this 200gb workload as the threadripper is to those Mac Pro workloads lol
- mysteria 2y agoThe 3090s also give you access to all those CUDA inference and training frameworks used in industry. With Apple you're pretty much stuck with either MLX or llama.cpp. Honestly though I'd be curious to see a cost analysis of Apple vs. Nvidia for commercial batched inference. The Nvidia system can obviously spit out more tokens/s but for the same price you could have multiple Mac Studios running the same model (and users would be dispatched to one of them).