Obviously gb200 and m2 studios are very different price points and power envelopes, per other comments. I agree that Apple does not have a competitive server o

Obviously gb200 and m2 studios are very different price points and power envelopes, per other comments. I agree that Apple does not have a competitive server offering right now. But I think I'd still be spending toward that goal if I were reserving all the 2nm chips in the world next year. Your claim that NVIDIA has better performance per watt really surprised me, so I did some digging. My prior is that opencl is medium support at NVIDIA and very low support at Apple, so an OpenCL benchmark might not be fair. Reporting tokens/s, using llamafile's llama.cpp benchmarks for Mistral 7b f16 from Justine tunney's website (so this is non-batched): 192GB Mac Studio gives 79 prompt tok/sec and 15 eval tok/sec. https://www.baseten.co/blog/benchmarking-fast-mistral-7b-inference/ https://www.baseten.co/blog/benchmarking-fast-mistral-7b-inf... benchmarks an H100 on Mistral 7B, and claims different providers range from 38-170 TPS depending on their proprietary configurations. A used H100 server is roughly $80k according to google. H100 power draw alone is 700W, and with server is likely on the order of 1,000W. A new Mac Studio with 192GB of RAM is <$7k. The Studio power draw is max 295W. So, Apple in Cost / 150 tokens/s: $70,000. Power draw: 3kw. Nvidia in cost / 150 tokens/s using highly tuned proprietary software: $80k. Power draw: 1kw. Nvidia in cost / 150 tokens/s using Mistrals own cloud performance: $150k+, Power draw: 2kw-3kw. I don't think you're correct that Apples M2 chips get smoked by Nvidia's chips for these workloads.