3 ms·
Apple really should license the M chip IP to someone to make a server chip out of it, or do it themselves. It's money on the table for them and would not cannib
by api 3y ago
Apple really should license the M chip IP to someone to make a server chip out of it, or do it themselves. It's money on the table for them and would not cannibalize their Mac business at all. It's a very nice core.
- mschuster91 3y agoFully agree. It's time for some serious competition not just against Intel/x86 but also in the ARM space.
- dzhiurgis 3y agoIDK about ML part, but equivalent performing Ryzen mini pc cost me 3x less over m1 macbook (yes im aware you get more with macbook)
- solardev 3y agoIt's easy to outperform Apple Silicon on pure power, but what about efficiency & heat? (like FLOPS/watt or whatever). Does anything else come close yet?
- blacksmith_tb 3y agoThat matters a lot in a laptop, but not so much in a 1U rack? Not that datacenters love heat, but the competition isn't extra hot, it's just hotter than Apple Silicon?
- solardev 3y agoI guess it depends on what you're doing with them? If you're running them 24/7 to train or model something, the energy costs might add up. Even if you're not, having more efficient chips might mean more data centers don't need as complex cooling equipment.
- ska 3y agoA 1U rack is a small machine, but a few hundred of them+ in a constrained space is a different story. At larger scales, moving electricity in and heat out are usually the defining factors.
- dzhiurgis 3y agoYou're quite right - 6800H TDP is 45W, M1 is 30W
- GeekyBear 3y ago> IDK about ML part, but equivalent performing Ryzen mini pc cost me 3x less over m1 macbook When running an ML workload, the Nvidea A100 has massive GPU compute resources, and a large amount of GPU local high bandwidth memory, so it's ideal, but is nowhere near low cost. A consumer Ryzen chip is inexpensive, but lacks in both memory bandwidth and GPU resources. The M2 Ultra has access to way more RAM than consumer GPUs, many times the memory bandwidth of a Ryzen (800GB/s vs Ryzen 7 1800X at 40 GB/s) with a large amount of local GPU resources. Even stepping up to a Threadripper Pro would only get you a quarter of the memory bandwidth, and those aren't exactly cheap either.
- sitkack 3y agoThere are multiple riscv based solutions that will be out in 12-18 months. But for now, getting Apple hardware is the best solution.
- starcraft2wol 3y agoyou can buy rack mount Mac pro
- Aurornis 3y agoApple silicon is great for low power desktops and laptops, but they don't actually have groundbreaking performance relative to what we've got in the server space. If you dropped the M2 Ultra from the $4000 into a server, it would perform about the same as a $1500 AMD 7950X3D based server (this is a common budget server setup with ECC) in CPU tasks. Stick a common GPU in there and you're running circles around the M2 Silicon in GPU tasks. The Apple Silicon is great at really low power work, but if you dial desktop or server GPU power limits down they also become quite efficient. The marginal cost of electricity is cheaper than buying more hardware, so nVidia and others run their parts deep into the diminishing returns part of the curve to maximize performance at the expense of power efficiency.
- cyber_kinetist 3y agoWhat Apple Silicon brings to the table is not simply just performance, but a large amount of unified memory that can be used by the GPU (which are needed for inference of large deep neural networks like LLMs). A top-of-the-line Mac Studio will give you 192 GB of unified RAM in less than $7000. Meanwhile a H100 from NVIDIA with 80 GB of VRAM will cost you like $30000...
- api 3y agoIf the software were a little better Apple Silicon would be by far the most cost effective rig today for DIY LLM research or training. 192GiB RAM is enough to train or inference Falcon 180B in RAM at 8-bit resolution.