4 ms·
> gone are the days of PCIe. My GPU, NVMe drives and motherboard might disagree.
by magic_hamster 7mo ago
> gone are the days of PCIe.
My GPU, NVMe drives and motherboard might disagree.
- aprilnya 7mo ago- GPU is integrated into the SoC - Surprisingly, it is possible to plug a drive into a TB/USB port …so what do you actually need PCIe for?
- gizajob 7mo agoThings that aren’t graphics cards, such very high bandwidth video capture cards and any other equipment that needs a lot of lanes of PCI data at low latency.
- HeWhoLurksLate 7mo agobut what about second GPU?
- wtallis 7mo agoMultiple GPUs was tried, by the whole industry including Apple (most notably with the trash can Mac Pro). Despite significant investment, it was ultimately a failure for consumer workloads like gaming, and was relegated to the datacenter and some very high-end workstations depending on the workload. Multi-GPU has recently experienced a resurgence due to the discovery of new workloads with broader appeal (LLMs), but that's too new to have significantly influenced hardware architectures, and LLM inference isn't the most natural thing to scale across many GPUs. Everybody's still competing with more or less the architectures they had on hand when LLMs arrived, with new low-precision matrix math units squeezed in wherever room can be made. It's not at all clear yet what the long-term outcome will be in terms of the balance between local vs cloud compute for inference, whether there will be any local training/fine-tuning at all, and which use cases are ultimately profitable in the long run. All of that influences whether it would be worthwhile for Apple to abandon their current client-first architecture that standardizes on a single integrated GPU and omits/rejects the complexity of multi-GPU setups.
- jltsiren 7mo agoHigh-end Macs have moved to PCIe 5.0 speeds in their internal drives. Thunderbolt 5 is not fast enough to get the same performance from external ones. Thunderbolt is also too slow for higher-end networks. A single port is already insufficient for 100-gigabit speeds.
- vlovich123 7mo agoWhen people talk about 100gigabit networks for Macs, im really curious what kind of network you run at home and how much money you spent on it. Even at work I’m generally seeing 10gigabit network ports with 100gigabit+ only in data centers where macs don’t have a presence
- Forgeties79 7mo agoI work in media production and I have the same thought constantly. Hell I curse in church as far as my industry is concerned because I find 2.5 to be fine for most of us. 10 absolutely.
- nine_k 7mo agoI suppose the throughput is not the key, latency is. When you split ann operation that normally ran within one machine between two machines, anything that crosses the boundary becomes orders of magnitude slower. Even with careful structuring, there are limits of how little and how rarely you can send data between nodes. I suppose that splitting an LLM workload is pretty sensitive to that.
- AdamN 7mo ago100gbps is going to be for mesh networks supporting clusters (4 Mac Studios let's just say) - not for LAN type networks (unless it's in an actual datacenter).
- jltsiren 7mo agoLocal AI is probably the most common application these days. Apple recently added support for InfiniBand over Thunderbolt. And now almost all decent Mac Studio configurations have sold out. Those two may be connected.
- deleted 7mo ago[deleted]
- wpm 7mo agoVideo capture I/O expansion Networking
- pjmlp 7mo agoTo have lots of them plugged together, high end audio cards, electronics integrations, disks with having cables all over the place.
- rayiner 7mo agoThe top Mac Studio has six thunderbolt 5 ports, each of which is a PCIe 4.0 x4 link. Each is a 8GB/sec link in each direction, which is a lot. Going from x16 down to x4 has less than a 10% hit on games: https://www.reddit.com/r/buildapc/comments/sbegpb/gpu_in_pcie_40_x4_slot/ https://www.reddit.com/r/buildapc/comments/sbegpb/gpu_in_pci...
- washadjeffmad 7mo agoGaming isn't what people are using Mac Studios for. Thunderbolt also isn't a substitute for OCuLink.
- rayiner 7mo agoSure, but it’s probably reflective of the fact that GPUs generally aren’t PCIe-bandwidth bound. Also, TB5 and Oculink2 both use PCI 4.0 x4 links.
- angoragoats 7mo agoOculink is generally faster than TB5 despite them both using PCIe 4.0, because Oculink provides direct PCIe access whereas Thunderbolt has to route all PCIe traffic through its controller. The benchmarks show that the overhead introduced by the TB5 controller slows down GPU performance.
- rayiner 7mo agoApple Silicon has an integrated thunderbolt controller so that should have less latency than PCs that use a discrete thunderbolt controller.
- adrian_b 7mo agoMany recent laptop CPUs from Intel and AMD have integrated Thunderbolt controllers (i.e. USB 4), so that has not been a difference for a long time.