4 ms·

Apple Silicon Mac Pro uses PCIe switches to support all slots

59 points by zdw 3y ago

teruakohatu 3y ago
What is the consequence of this? Does it limit the theoretical ability to add GPUs?
- rerx 3y ago
  If you add multiple GPUs to the same PCIe switch, their bandwidth to main memory of the SOC will be shared. For some use cases that bandwidth was already the bottle neck.
  - detourdog 3y ago
    Using the older PCI bus every three or 4 slots represented a dedicated controller. We would install the high bandwidth cards on seperate controllers. PCI-E exists in a faster timing bus which may make a difference. The visionOS demo apparently shows sub 20 nanosecond processing and graphics blitting to appear imperceptibly. Finally Woz’s design for the Apple II color graphics was based on unified memory. Dramatic shifts in technology sometimes obviates all our past learning and experience.
    - c00lio 3y ago
      The defining characteristic of PCIe isn't faster bus timings. It is that PCIe isn't even a "bus" in the traditional sense. With a bus, you have a number of shared physical wires for addresses and data, and an arbitration system that assigns those shared wires to one device at a time. The available bandwidth is just the bandwidth of the bus, divided into the time you can reserve it for yourself. PCIe is a tree-shaped network (root is the root hub in the CPU, inner nodes are switches, leafs are devices) with point-to-point links (called lanes) that transmit packets. Bandwidth is determined by the number of lanes that a device's path up to the root hub has at the narrowest node.
      - detourdog 3y ago
        That sounds like a good description but do you think the term serial should be worked in there?
        c00lio 3y ago
        On some level yes, on some level no. The physical interface for each point-to-point link is serial, but one device can have several of those links and use them (send/receive packets on them) in parallel.
        detourdog 3y ago
        Does the lane count correspond to the quantity of parallel links? would the switches be single transistors?
        c00lio 3y ago
        Yes, lane count is the number of parallel links. Switches are far more complex. The protocol specifies addressing, bandwidth reservations, flow control, priorities and message ordering requirements, which the switches have to handle. Not a switch thing per se, but also relevant for "single transistors won't do": On the physical layer, each port/link is also relatively complicated to initialize and operate. The transmission encoding, due to the relatively high speeds involved, requires a learning phase where both ends measure the link characteristics and adjust their (relatively complex) signal processing accordingly. The physical layer also handles different link speeds for down-/upward compatibility.
        detourdog 3y ago
        This sounds like an ideal config for unified memory. I suppose that one can count all the lanes to figure out the actually total throughput. Does every vendor come up with their own implementation of a "busmaster"? This may explains how they achieve the graphics blitting in the Vision Pro.
- goosedragons 3y ago
  You already can't add GPUs but since there's only 16 real lanes and a GPU can use 16 lanes it could cause issues. I would think this would lead to bottlenecks for instance if you were trying to write to two different 16x PCIe SSD RAID cards simultaneously.
  - Zetobal 3y ago
    No, GPU is even close to saturate 16 lanes.
    - ohgodplsno 3y ago
      Saturating 16 lanes is very doable, although for a very short amount of time. You'll never see it last more than half a second at absolute worst on normal usage, maybe more if you're trying to upload on a 192GB GPU or something. But since most uses evict data out of GPU RAM before writing instead of blindly overwriting, it's not a problem you could ever reasonably have.
- c00lio 3y ago
  Actually, no. It only limits the available I/O bandwidth to those GPU(s), that is, data transfers from and to the CPU will be far slower. You'll notice in games when the "loading next level" screen takes longer, that's usually a large geometry and texture copy to the GPU. As an extreme example, when and where it was/is still viable, cryptocurrency mining operations use external PCIe cases with lots of slots, attached to only a few or even just one lane per case. For their purpose this was still viable, since mining uses very little I/O from/to the GPU, essentially only a small block of setup data into the GPU ("try hashing this block, vary those bytes within that range, report when result matches this pattern", a few kB) and a few bytes of results out ("found a match, varied bytes were as follows").
  - phire 3y ago
    In the case of multi-gpu gaming, loading might not be an issue. You are sending the exact same data to both GPUs. If the the PCIe switch and GPU drivers support multicast, then you would actually get the full 16x bandwidth to both. Which is actually a huge advantage over the most common multi-gpu setup, which is forced to statically split the 16 PCIe lanes into two independent 8x slots. Even if you were playing a game that only used one GPU, simply having the second gpu plugged in meant your total bandwidth to each gpu was limited to 8x speed. I really wish you could find x86 motherboards with PCIe switch chips... but switch chips are really expensive so nobody makes them.
- nottorp 3y ago
  Well mac os doesnt support non Apple gpus any more. And if you want to run Linux with a bunch of compute GPUs Im sure there are more appropriate solutions than Apple hardware. Not sure how this affects the use case of multiple video capture cards writing to fast internal storage that I see mentioned here and there. Anyone close to that industry around to comment? Edit: i think I’ve seen mentioned that the PSU in this thing is “only” 1400W. That only covers one modern nvidia gpu right? :)
  - detourdog 3y ago
    If one looks at all the encode/decode streams that Apple claims happen on processor I wonder how significant this is. The graphics and processing speed of the Vision Pro makes me wonder if the GPUs are truly needed. Most of the cards for the M2 Mac Pro are used to access the analog world. It seems that there is power inside the die and if you need an analog signal use a PCI-e port.
    - nottorp 3y ago
      It's probably good or even powerful for professional video editing. For GPU compute, I'm sure it can't beat 15 kW of nvidia dedicated cards. But then you don't want 15 kW of cooling on your desk.
      - detourdog 3y ago
        Where I get confused is what is the extra kW doing? It has been at least a decade since I did any intense projects but doesn't 15kW represent the power of an average American household circuit. The entire Mac Pro only supports 3 8k displays with built-in hardware. One would need to add SDI cards to add extra displays but the computer only draws 300 Watts. I could get 50 Mac Pros and give up which would give 150 8k displays without expansion. Maybe one gives up the convenience of a single card solution but there maybe other advantages. I'm mostly curious as to what this work load consists of? but when I read the number 8k encode and decode streams 15 kW represents How much of the power is just getting the data in and out of the card to the processor. Apple Silicon doing the same work on die is going to use less power. I don't doubt that there are faster processing units.
c00lio 3y ago
Apple Silicon is low-power low-end laptop hardware. Of course their I/O capabilities are limited, because their users do not need lots of I/O. If you are looking for stuff with lots of I/O capabilities you have to buy proper server hardware.
- wkat4242 3y ago
  Of course Apple positions this as a proper workstation.
  - varispeed 3y ago
    I remember being downvoted for saying this is essentially a phone CPU and motherboard, just without consideration for constraints imposed by the phone's footprint. It means it can be designed with higher TDP in mind etc. and yet, Apple charges for it as if it was a real workstation equipment rather than a phone in a massive enclosure.
    - shoonmcgregor 3y ago
      [dead]
    - martini333 3y ago
      Do you think there is only phone CPU's and real workstation CPU's? Nothing in-between? If so, I get why you got downvoted.
    - daneel_w 3y ago
      Why is it "essentially" a phone CPU? Please don't say "because of the ISA". It has 24 CPU cores, 60+ GPU cores, an insanely fast memory bus and a purported 60 watt power envelope. There's nothing "phone" about it.
      - microtonal 3y ago
        And Thunderbolt. Show me all those phones with Thunderbolt :).
Tsiklon 3y ago
The last Intel Mac Pro also has a sizeable number of PCIe switches did it not? At least I recall seeing them in the hardware tear down
sprash 3y ago
If you have switched PCIe anyways I would have loved a Mac Pro with a "passive" switched PCIe backplane where you can hotplug PCIe devices in including M2 CPU PCIe cards. Also polish the OS accordingly that you can upgrade/switch your CPU in live operations without shutting down the OS or may be even run Apple silicon CPUs and x86 CPUs alongside.
- dijit 3y ago
  I think we collectively decided that we didn't want hot-swap CPUs and instead wanted inexpensive compute (so we could have a lot more of it) when we let SPARC die.
  - hosteur 3y ago
    Why must hot swap cpu inherently be expensive?
    - dijit 3y ago
      Because engineering effort is not free.
      - hosteur 3y ago
        > Because engineering effort is not free. Obviously, but that does not explain why CPU hot swap is inherently expensive. Engineering effort is required to make any CPU. Why must the engineering effort specifically relating to CPU hot swap inherently cause architectures that supports CPU hot swap significantly expensive compared to architectures that does not? As a lay man without any experience with actual semi conductors outside of university, I would have thought that this would mostly be something that needs to be designed and specified once and then it could be mass produced. Clearly it will take some effort and you would have to spend some gates on it in the chip design. Considering all the other stuff that are in modern CPUs and chipsets, I would have thought it would be possible to do rather cheaply if mass produced.
  - wongarsu 3y ago
walrus01 3y ago
For the kind of money that a Mac Pro costs, one would expect comparable full bandwidth, non-oversubscribed PCI-E lane capacity to a current gen Threadripper.
bradfa 3y ago
A nice advantage of PCIe switches is that device-to-device transfers are required to be supported. Many/most PCIe root ports don't support device-to-device transfers through the root port. So if your use-case requires PCIe devices to talk directly to each other without involving the root port, having switches is super beneficial.
sys_64738 3y ago
Switching is a terrible hardware restriction leading to shared bandwidth.
- juancn 3y ago
  Unless you want to do device to device transfers, in which case you can move it inside the switch. You could for example transfer texture data from an SSD straight to GPU memory, a la [NVidia's GPUDirect](https://developer.nvidia.com/blog/gpudirect-storage/ https://developer.nvidia.com/blog/gpudirect-storage/)
juancn 3y ago
This could be bad or great depending on the use case. - Bad: bottlenecked access to the CPU and its memory - Good: Bypass the CPU on device to device transfer, in the style of NVidia's GPUDirect (https://developer.nvidia.com/blog/gpudirect-storage/ https://developer.nvidia.com/blog/gpudirect-storage/)

59 points by zdw 3y ago

teruakohatu 3y ago

What is the consequence of this? Does it limit the theoretical ability to add GPUs?

rerx 3y ago
If you add multiple GPUs to the same PCIe switch, their bandwidth to main memory of the SOC will be shared. For some use cases that bandwidth was already the bottle neck.
- detourdog 3y ago
  Using the older PCI bus every three or 4 slots represented a dedicated controller. We would install the high bandwidth cards on seperate controllers. PCI-E exists in a faster timing bus which may make a difference. The visionOS demo apparently shows sub 20 nanosecond processing and graphics blitting to appear imperceptibly. Finally Woz’s design for the Apple II color graphics was based on unified memory. Dramatic shifts in technology sometimes obviates all our past learning and experience.
  - c00lio 3y ago
    The defining characteristic of PCIe isn't faster bus timings. It is that PCIe isn't even a "bus" in the traditional sense. With a bus, you have a number of shared physical wires for addresses and data, and an arbitration system that assigns those shared wires to one device at a time. The available bandwidth is just the bandwidth of the bus, divided into the time you can reserve it for yourself. PCIe is a tree-shaped network (root is the root hub in the CPU, inner nodes are switches, leafs are devices) with point-to-point links (called lanes) that transmit packets. Bandwidth is determined by the number of lanes that a device's path up to the root hub has at the narrowest node.
    - detourdog 3y ago
      That sounds like a good description but do you think the term serial should be worked in there?
      - c00lio 3y ago
        On some level yes, on some level no. The physical interface for each point-to-point link is serial, but one device can have several of those links and use them (send/receive packets on them) in parallel.
        detourdog 3y ago
        Does the lane count correspond to the quantity of parallel links? would the switches be single transistors?
        c00lio 3y ago
        Yes, lane count is the number of parallel links. Switches are far more complex. The protocol specifies addressing, bandwidth reservations, flow control, priorities and message ordering requirements, which the switches have to handle. Not a switch thing per se, but also relevant for "single transistors won't do": On the physical layer, each port/link is also relatively complicated to initialize and operate. The transmission encoding, due to the relatively high speeds involved, requires a learning phase where both ends measure the link characteristics and adjust their (relatively complex) signal processing accordingly. The physical layer also handles different link speeds for down-/upward compatibility.
        detourdog 3y ago
        This sounds like an ideal config for unified memory. I suppose that one can count all the lanes to figure out the actually total throughput. Does every vendor come up with their own implementation of a "busmaster"? This may explains how they achieve the graphics blitting in the Vision Pro.
goosedragons 3y ago
You already can't add GPUs but since there's only 16 real lanes and a GPU can use 16 lanes it could cause issues. I would think this would lead to bottlenecks for instance if you were trying to write to two different 16x PCIe SSD RAID cards simultaneously.
- Zetobal 3y ago
  No, GPU is even close to saturate 16 lanes.
  - ohgodplsno 3y ago
    Saturating 16 lanes is very doable, although for a very short amount of time. You'll never see it last more than half a second at absolute worst on normal usage, maybe more if you're trying to upload on a 192GB GPU or something. But since most uses evict data out of GPU RAM before writing instead of blindly overwriting, it's not a problem you could ever reasonably have.
c00lio 3y ago
Actually, no. It only limits the available I/O bandwidth to those GPU(s), that is, data transfers from and to the CPU will be far slower. You'll notice in games when the "loading next level" screen takes longer, that's usually a large geometry and texture copy to the GPU. As an extreme example, when and where it was/is still viable, cryptocurrency mining operations use external PCIe cases with lots of slots, attached to only a few or even just one lane per case. For their purpose this was still viable, since mining uses very little I/O from/to the GPU, essentially only a small block of setup data into the GPU ("try hashing this block, vary those bytes within that range, report when result matches this pattern", a few kB) and a few bytes of results out ("found a match, varied bytes were as follows").
- phire 3y ago
  In the case of multi-gpu gaming, loading might not be an issue. You are sending the exact same data to both GPUs. If the the PCIe switch and GPU drivers support multicast, then you would actually get the full 16x bandwidth to both. Which is actually a huge advantage over the most common multi-gpu setup, which is forced to statically split the 16 PCIe lanes into two independent 8x slots. Even if you were playing a game that only used one GPU, simply having the second gpu plugged in meant your total bandwidth to each gpu was limited to 8x speed. I really wish you could find x86 motherboards with PCIe switch chips... but switch chips are really expensive so nobody makes them.
nottorp 3y ago
Well mac os doesnt support non Apple gpus any more. And if you want to run Linux with a bunch of compute GPUs Im sure there are more appropriate solutions than Apple hardware. Not sure how this affects the use case of multiple video capture cards writing to fast internal storage that I see mentioned here and there. Anyone close to that industry around to comment? Edit: i think I’ve seen mentioned that the PSU in this thing is “only” 1400W. That only covers one modern nvidia gpu right? :)
- detourdog 3y ago
  If one looks at all the encode/decode streams that Apple claims happen on processor I wonder how significant this is. The graphics and processing speed of the Vision Pro makes me wonder if the GPUs are truly needed. Most of the cards for the M2 Mac Pro are used to access the analog world. It seems that there is power inside the die and if you need an analog signal use a PCI-e port.
  - nottorp 3y ago
    It's probably good or even powerful for professional video editing. For GPU compute, I'm sure it can't beat 15 kW of nvidia dedicated cards. But then you don't want 15 kW of cooling on your desk.
    - detourdog 3y ago
      Where I get confused is what is the extra kW doing? It has been at least a decade since I did any intense projects but doesn't 15kW represent the power of an average American household circuit. The entire Mac Pro only supports 3 8k displays with built-in hardware. One would need to add SDI cards to add extra displays but the computer only draws 300 Watts. I could get 50 Mac Pros and give up which would give 150 8k displays without expansion. Maybe one gives up the convenience of a single card solution but there maybe other advantages. I'm mostly curious as to what this work load consists of? but when I read the number 8k encode and decode streams 15 kW represents How much of the power is just getting the data in and out of the card to the processor. Apple Silicon doing the same work on die is going to use less power. I don't doubt that there are faster processing units.

c00lio 3y ago

Apple Silicon is low-power low-end laptop hardware. Of course their I/O capabilities are limited, because their users do not need lots of I/O. If you are looking for stuff with lots of I/O capabilities you have to buy proper server hardware.

wkat4242 3y ago
Of course Apple positions this as a proper workstation.
- varispeed 3y ago
  I remember being downvoted for saying this is essentially a phone CPU and motherboard, just without consideration for constraints imposed by the phone's footprint. It means it can be designed with higher TDP in mind etc. and yet, Apple charges for it as if it was a real workstation equipment rather than a phone in a massive enclosure.
  - shoonmcgregor 3y ago
    [dead]
  - martini333 3y ago
    Do you think there is only phone CPU's and real workstation CPU's? Nothing in-between? If so, I get why you got downvoted.
  - daneel_w 3y ago
    Why is it "essentially" a phone CPU? Please don't say "because of the ISA". It has 24 CPU cores, 60+ GPU cores, an insanely fast memory bus and a purported 60 watt power envelope. There's nothing "phone" about it.
    - microtonal 3y ago
      And Thunderbolt. Show me all those phones with Thunderbolt :).

Tsiklon 3y ago

The last Intel Mac Pro also has a sizeable number of PCIe switches did it not? At least I recall seeing them in the hardware tear down

sprash 3y ago

If you have switched PCIe anyways I would have loved a Mac Pro with a "passive" switched PCIe backplane where you can hotplug PCIe devices in including M2 CPU PCIe cards. Also polish the OS accordingly that you can upgrade/switch your CPU in live operations without shutting down the OS or may be even run Apple silicon CPUs and x86 CPUs alongside.

dijit 3y ago
I think we collectively decided that we didn't want hot-swap CPUs and instead wanted inexpensive compute (so we could have a lot more of it) when we let SPARC die.
- hosteur 3y ago
  Why must hot swap cpu inherently be expensive?
  - dijit 3y ago
    Because engineering effort is not free.
    - hosteur 3y ago
      > Because engineering effort is not free. Obviously, but that does not explain why CPU hot swap is inherently expensive. Engineering effort is required to make any CPU. Why must the engineering effort specifically relating to CPU hot swap inherently cause architectures that supports CPU hot swap significantly expensive compared to architectures that does not? As a lay man without any experience with actual semi conductors outside of university, I would have thought that this would mostly be something that needs to be designed and specified once and then it could be mass produced. Clearly it will take some effort and you would have to spend some gates on it in the chip design. Considering all the other stuff that are in modern CPUs and chipsets, I would have thought it would be possible to do rather cheaply if mass produced.
- wongarsu 3y ago

walrus01 3y ago

For the kind of money that a Mac Pro costs, one would expect comparable full bandwidth, non-oversubscribed PCI-E lane capacity to a current gen Threadripper.

bradfa 3y ago

A nice advantage of PCIe switches is that device-to-device transfers are required to be supported. Many/most PCIe root ports don't support device-to-device transfers through the root port. So if your use-case requires PCIe devices to talk directly to each other without involving the root port, having switches is super beneficial.

sys_64738 3y ago

Switching is a terrible hardware restriction leading to shared bandwidth.

juancn 3y ago
Unless you want to do device to device transfers, in which case you can move it inside the switch. You could for example transfer texture data from an SSD straight to GPU memory, a la [NVidia's GPUDirect](https://developer.nvidia.com/blog/gpudirect-storage/ https://developer.nvidia.com/blog/gpudirect-storage/)

juancn 3y ago

This could be bad or great depending on the use case. - Bad: bottlenecked access to the CPU and its memory - Good: Bypass the CPU on device to device transfer, in the style of NVidia's GPUDirect (https://developer.nvidia.com/blog/gpudirect-storage/ https://developer.nvidia.com/blog/gpudirect-storage/)