5 ms·

What is the consequence of this? Does it limit the theoretical ability to add GPUs?

rerx 3y ago
If you add multiple GPUs to the same PCIe switch, their bandwidth to main memory of the SOC will be shared. For some use cases that bandwidth was already the bottle neck.
- detourdog 3y ago
  Using the older PCI bus every three or 4 slots represented a dedicated controller. We would install the high bandwidth cards on seperate controllers. PCI-E exists in a faster timing bus which may make a difference. The visionOS demo apparently shows sub 20 nanosecond processing and graphics blitting to appear imperceptibly. Finally Woz’s design for the Apple II color graphics was based on unified memory. Dramatic shifts in technology sometimes obviates all our past learning and experience.
  - c00lio 3y ago
    The defining characteristic of PCIe isn't faster bus timings. It is that PCIe isn't even a "bus" in the traditional sense. With a bus, you have a number of shared physical wires for addresses and data, and an arbitration system that assigns those shared wires to one device at a time. The available bandwidth is just the bandwidth of the bus, divided into the time you can reserve it for yourself. PCIe is a tree-shaped network (root is the root hub in the CPU, inner nodes are switches, leafs are devices) with point-to-point links (called lanes) that transmit packets. Bandwidth is determined by the number of lanes that a device's path up to the root hub has at the narrowest node.
    - detourdog 3y ago
      That sounds like a good description but do you think the term serial should be worked in there?
      - c00lio 3y ago
        On some level yes, on some level no. The physical interface for each point-to-point link is serial, but one device can have several of those links and use them (send/receive packets on them) in parallel.
        detourdog 3y ago
        Does the lane count correspond to the quantity of parallel links? would the switches be single transistors?
        c00lio 3y ago
        Yes, lane count is the number of parallel links. Switches are far more complex. The protocol specifies addressing, bandwidth reservations, flow control, priorities and message ordering requirements, which the switches have to handle. Not a switch thing per se, but also relevant for "single transistors won't do": On the physical layer, each port/link is also relatively complicated to initialize and operate. The transmission encoding, due to the relatively high speeds involved, requires a learning phase where both ends measure the link characteristics and adjust their (relatively complex) signal processing accordingly. The physical layer also handles different link speeds for down-/upward compatibility.
        detourdog 3y ago
        This sounds like an ideal config for unified memory. I suppose that one can count all the lanes to figure out the actually total throughput. Does every vendor come up with their own implementation of a "busmaster"? This may explains how they achieve the graphics blitting in the Vision Pro.
goosedragons 3y ago
You already can't add GPUs but since there's only 16 real lanes and a GPU can use 16 lanes it could cause issues. I would think this would lead to bottlenecks for instance if you were trying to write to two different 16x PCIe SSD RAID cards simultaneously.
- Zetobal 3y ago
  No, GPU is even close to saturate 16 lanes.
  - ohgodplsno 3y ago
    Saturating 16 lanes is very doable, although for a very short amount of time. You'll never see it last more than half a second at absolute worst on normal usage, maybe more if you're trying to upload on a 192GB GPU or something. But since most uses evict data out of GPU RAM before writing instead of blindly overwriting, it's not a problem you could ever reasonably have.
c00lio 3y ago
Actually, no. It only limits the available I/O bandwidth to those GPU(s), that is, data transfers from and to the CPU will be far slower. You'll notice in games when the "loading next level" screen takes longer, that's usually a large geometry and texture copy to the GPU. As an extreme example, when and where it was/is still viable, cryptocurrency mining operations use external PCIe cases with lots of slots, attached to only a few or even just one lane per case. For their purpose this was still viable, since mining uses very little I/O from/to the GPU, essentially only a small block of setup data into the GPU ("try hashing this block, vary those bytes within that range, report when result matches this pattern", a few kB) and a few bytes of results out ("found a match, varied bytes were as follows").
- phire 3y ago
  In the case of multi-gpu gaming, loading might not be an issue. You are sending the exact same data to both GPUs. If the the PCIe switch and GPU drivers support multicast, then you would actually get the full 16x bandwidth to both. Which is actually a huge advantage over the most common multi-gpu setup, which is forced to statically split the 16 PCIe lanes into two independent 8x slots. Even if you were playing a game that only used one GPU, simply having the second gpu plugged in meant your total bandwidth to each gpu was limited to 8x speed. I really wish you could find x86 motherboards with PCIe switch chips... but switch chips are really expensive so nobody makes them.
nottorp 3y ago
Well mac os doesnt support non Apple gpus any more. And if you want to run Linux with a bunch of compute GPUs Im sure there are more appropriate solutions than Apple hardware. Not sure how this affects the use case of multiple video capture cards writing to fast internal storage that I see mentioned here and there. Anyone close to that industry around to comment? Edit: i think I’ve seen mentioned that the PSU in this thing is “only” 1400W. That only covers one modern nvidia gpu right? :)
- detourdog 3y ago
  If one looks at all the encode/decode streams that Apple claims happen on processor I wonder how significant this is. The graphics and processing speed of the Vision Pro makes me wonder if the GPUs are truly needed. Most of the cards for the M2 Mac Pro are used to access the analog world. It seems that there is power inside the die and if you need an analog signal use a PCI-e port.
  - nottorp 3y ago
    It's probably good or even powerful for professional video editing. For GPU compute, I'm sure it can't beat 15 kW of nvidia dedicated cards. But then you don't want 15 kW of cooling on your desk.
    - detourdog 3y ago
      Where I get confused is what is the extra kW doing? It has been at least a decade since I did any intense projects but doesn't 15kW represent the power of an average American household circuit. The entire Mac Pro only supports 3 8k displays with built-in hardware. One would need to add SDI cards to add extra displays but the computer only draws 300 Watts. I could get 50 Mac Pros and give up which would give 150 8k displays without expansion. Maybe one gives up the convenience of a single card solution but there maybe other advantages. I'm mostly curious as to what this work load consists of? but when I read the number 8k encode and decode streams 15 kW represents How much of the power is just getting the data in and out of the card to the processor. Apple Silicon doing the same work on die is going to use less power. I don't doubt that there are faster processing units.

rerx 3y ago

If you add multiple GPUs to the same PCIe switch, their bandwidth to main memory of the SOC will be shared. For some use cases that bandwidth was already the bottle neck.

detourdog 3y ago
Using the older PCI bus every three or 4 slots represented a dedicated controller. We would install the high bandwidth cards on seperate controllers. PCI-E exists in a faster timing bus which may make a difference. The visionOS demo apparently shows sub 20 nanosecond processing and graphics blitting to appear imperceptibly. Finally Woz’s design for the Apple II color graphics was based on unified memory. Dramatic shifts in technology sometimes obviates all our past learning and experience.
- c00lio 3y ago
  The defining characteristic of PCIe isn't faster bus timings. It is that PCIe isn't even a "bus" in the traditional sense. With a bus, you have a number of shared physical wires for addresses and data, and an arbitration system that assigns those shared wires to one device at a time. The available bandwidth is just the bandwidth of the bus, divided into the time you can reserve it for yourself. PCIe is a tree-shaped network (root is the root hub in the CPU, inner nodes are switches, leafs are devices) with point-to-point links (called lanes) that transmit packets. Bandwidth is determined by the number of lanes that a device's path up to the root hub has at the narrowest node.
  - detourdog 3y ago
    That sounds like a good description but do you think the term serial should be worked in there?
    - c00lio 3y ago
      On some level yes, on some level no. The physical interface for each point-to-point link is serial, but one device can have several of those links and use them (send/receive packets on them) in parallel.
      - detourdog 3y ago
        Does the lane count correspond to the quantity of parallel links? would the switches be single transistors?
        c00lio 3y ago
        Yes, lane count is the number of parallel links. Switches are far more complex. The protocol specifies addressing, bandwidth reservations, flow control, priorities and message ordering requirements, which the switches have to handle. Not a switch thing per se, but also relevant for "single transistors won't do": On the physical layer, each port/link is also relatively complicated to initialize and operate. The transmission encoding, due to the relatively high speeds involved, requires a learning phase where both ends measure the link characteristics and adjust their (relatively complex) signal processing accordingly. The physical layer also handles different link speeds for down-/upward compatibility.
        detourdog 3y ago
        This sounds like an ideal config for unified memory. I suppose that one can count all the lanes to figure out the actually total throughput. Does every vendor come up with their own implementation of a "busmaster"? This may explains how they achieve the graphics blitting in the Vision Pro.

goosedragons 3y ago

You already can't add GPUs but since there's only 16 real lanes and a GPU can use 16 lanes it could cause issues. I would think this would lead to bottlenecks for instance if you were trying to write to two different 16x PCIe SSD RAID cards simultaneously.

Zetobal 3y ago
No, GPU is even close to saturate 16 lanes.
- ohgodplsno 3y ago
  Saturating 16 lanes is very doable, although for a very short amount of time. You'll never see it last more than half a second at absolute worst on normal usage, maybe more if you're trying to upload on a 192GB GPU or something. But since most uses evict data out of GPU RAM before writing instead of blindly overwriting, it's not a problem you could ever reasonably have.

c00lio 3y ago

Actually, no. It only limits the available I/O bandwidth to those GPU(s), that is, data transfers from and to the CPU will be far slower. You'll notice in games when the "loading next level" screen takes longer, that's usually a large geometry and texture copy to the GPU. As an extreme example, when and where it was/is still viable, cryptocurrency mining operations use external PCIe cases with lots of slots, attached to only a few or even just one lane per case. For their purpose this was still viable, since mining uses very little I/O from/to the GPU, essentially only a small block of setup data into the GPU ("try hashing this block, vary those bytes within that range, report when result matches this pattern", a few kB) and a few bytes of results out ("found a match, varied bytes were as follows").

phire 3y ago
In the case of multi-gpu gaming, loading might not be an issue. You are sending the exact same data to both GPUs. If the the PCIe switch and GPU drivers support multicast, then you would actually get the full 16x bandwidth to both. Which is actually a huge advantage over the most common multi-gpu setup, which is forced to statically split the 16 PCIe lanes into two independent 8x slots. Even if you were playing a game that only used one GPU, simply having the second gpu plugged in meant your total bandwidth to each gpu was limited to 8x speed. I really wish you could find x86 motherboards with PCIe switch chips... but switch chips are really expensive so nobody makes them.

nottorp 3y ago

Well mac os doesnt support non Apple gpus any more. And if you want to run Linux with a bunch of compute GPUs Im sure there are more appropriate solutions than Apple hardware. Not sure how this affects the use case of multiple video capture cards writing to fast internal storage that I see mentioned here and there. Anyone close to that industry around to comment? Edit: i think I’ve seen mentioned that the PSU in this thing is “only” 1400W. That only covers one modern nvidia gpu right? :)

detourdog 3y ago
If one looks at all the encode/decode streams that Apple claims happen on processor I wonder how significant this is. The graphics and processing speed of the Vision Pro makes me wonder if the GPUs are truly needed. Most of the cards for the M2 Mac Pro are used to access the analog world. It seems that there is power inside the die and if you need an analog signal use a PCI-e port.
- nottorp 3y ago
  It's probably good or even powerful for professional video editing. For GPU compute, I'm sure it can't beat 15 kW of nvidia dedicated cards. But then you don't want 15 kW of cooling on your desk.
  - detourdog 3y ago
    Where I get confused is what is the extra kW doing? It has been at least a decade since I did any intense projects but doesn't 15kW represent the power of an average American household circuit. The entire Mac Pro only supports 3 8k displays with built-in hardware. One would need to add SDI cards to add extra displays but the computer only draws 300 Watts. I could get 50 Mac Pros and give up which would give 150 8k displays without expansion. Maybe one gives up the convenience of a single card solution but there maybe other advantages. I'm mostly curious as to what this work load consists of? but when I read the number 8k encode and decode streams 15 kW represents How much of the power is just getting the data in and out of the card to the processor. Apple Silicon doing the same work on die is going to use less power. I don't doubt that there are faster processing units.