4 ms·

I have the same question! They double the performance over the same physical interface generation after generation. Why haven't we seen the need for a PCI-X or

I have the same question! They double the performance over the same physical interface generation after generation. Why haven't we seen the need for a PCI-X or VLB-style PCIe interface expansion?

Night_Thastus 3y ago
They explain on the website: >To achieve its impressive data transfer rates, PCIe 7.0 doubles the bus frequency at the physical layer compared to PCIe 5.0 and 6.0. Otherwise, the standard retains pulse amplitude modulation with four level signaling (PAM4), 1b/1b FLIT mode encoding, and the forward error correction (FEC) technologies that are already used for PCIe 6.0. Otherwise, PCI-SIG says that the PCIe 7.0 speicification also focuses on enhanced channel parameters and reach as well as improved power efficiency. So it sounds like they doubled the frequency and kept the encoding the same. PCIe 6 can get up to 256 GB/s, and 2x 256 = 512. In any case, it'll be a long time before the standard is finished, and far longer before any real hardware is around that actually uses PCIe 7.
- ksec 3y ago
  >In any case, it'll be a long time before the standard is finished, and far longer before any real hardware is around that actually uses PCIe 7. The standard is expected to finish in 2025, and hardware / IP for PCIe 7 are already in the work. Since there is so much resemblance of it and PCIe 6. In terms of schedule it perhaps may be the least lead time of recent PCIe from Standard 1.0 to IP available. The HPC industry is really pushing for this ASAP.
- Retr0id 3y ago
  This answers the question in the literal sense, but I'm nonetheless surprised (as a relative outsider to the world of high-speed signalling). "just double the frequency" isn't something we're used to seeing elsewhere these days (e.g. CPU clock speeds for the last couple of decades). What are the fundamental technological advances that allow them to do so? Or in other words, what stopped from from achieving this in the previous generation?
  - Night_Thastus 3y ago
    PCIe doesn't have to do anything as complex as a general-purpose CPU. Increasing the frequency is a lot easier when you don't need to worry about things like heat, pipelining, caching, branch prediction, multithreading, etc. It's just encoding data and sending it back and forth. We've gotten very, very good at that. It wasn't really like it was impossible before now - it's just more that it wasn't in demand. With the proliferation of SSDs transferring data over PCIe, it's become much more important - so the extra cost of better signaling hardware is worth it. Not to dismiss it completely, it's still a hard problem. But it's far easier than doubling the frequency of a CPU.
    - p1esk 3y ago
      Why would pipelining, caching, branch prediction make increasing the frequency difficult? Why would heat be less of a problem for a pcie controller than for a cpu?
      - kanetw 3y ago
        The short, stupid answer is: transistor count and size. Explaining it in detail requires more background in electronics, but that's ultimately what it boils down to. High-end analog front ends can reach the three-digit GHz (non-silicon processes admittedly, but still).
        p1esk 3y ago
        Explaining it in detail Please do
        0xcde4c3db 3y ago
        I see you don't have any responses, so I'll give it a shot. I don't remember the actual math from my coursework, and have no experience with real-world custom ASIC design (let alone anything at all in the "three-digit GHz" range), but the major variables look something like this (in no particular order): 1) transistor size (smaller = faster, but process is more expensive) 2) gate dielectric thickness (smaller = faster, but easier to damage the gate; there's a corollary of this with electric fields across various PCB layer counts, microstrips, etc. that I'm nowhere near prepared to articulate) 3) logic voltage swing (smaller = faster, but less noise immunity) 4) number of logic/buffer gates traversed on the critical path (fewer = faster, but requires wider internal buses or simpler logic) 5) output drive strength (higher = faster, but usually larger, less efficient, and more prone to generating EMI) 6) fanout (how many inputs must be driven by a single output) on the critical path (lower = faster) Most of these have significant ties to the manufacturing process and gate library, and so aren't necessarily different between a link like PCIe and a CPU or GPU. Some things can be tweaked for I/O pads, but broadly speaking a lot of these vary together across a chip. The biggest exceptions are points #4 and #6. Doing general-purpose math, having flexible program control flow constructs, and being able to optimize legacy instruction sequences on-the-fly (but I repeat myself) unavoidably requires somewhat complex logic with state that needs to be observed in multiple places. Modern processors mitigate this with pipelining, which splits the processing into smaller stages separated by registers that hold on to the intermediate state (pipeline registers). This increases the maximum frequency of the circuit at the cost of requiring multiple clock cycles for an operation to proceed from initiation to completion (but allowing multiple operations to be "in flight" at once). That being said, what's the simplest possible example of a pipeline stage? Conceptually, it's just a pair of 1-bit registers with no logic between the input of one register and the input of the next one. When the clock ticks, a bit is moved from one register to the next. Chain a bunch of these stages together, and you have something called a shift register. Add some extra wires to read or write the stages in parallel, and a shift register lets you convert between serial and parallel connections. The big advantage that PCIe (and SATA/SAS, HDMI, DisplayPort, etc.) has over CPUs is that the actual part hooked up to the pairs that needs to run at the link rate is "just" a pair of big shift registers and some analog voodoo to get synchronization between two ends that are probably not running from the same physical oscillator (aka a SERDES block). In some sense it's the absolute optimal case of the "make my CPU go faster" design strategy. Actually designing one of these to reliably run at multi-gigabit rates is a considerable task, but any given foundry will generally have a validated block that can be licensed and pasted into your chip for a given process node. Does that make sense?
Tuna-Fish 3y ago
> the same physical interface It's not really the same physical interface. The connector is the same, but quality requirements for the traces have gotten much more strict over time. > Why haven't we seen the need for a PCI-X or VLB-style PCIe interface expansion? x32 PCIe does exist, it's just rarely used.

I have the same question! They double the performance over the same physical interface generation after generation. Why haven't we seen the need for a PCI-X or VLB-style PCIe interface expansion?

Night_Thastus 3y ago

They explain on the website: >To achieve its impressive data transfer rates, PCIe 7.0 doubles the bus frequency at the physical layer compared to PCIe 5.0 and 6.0. Otherwise, the standard retains pulse amplitude modulation with four level signaling (PAM4), 1b/1b FLIT mode encoding, and the forward error correction (FEC) technologies that are already used for PCIe 6.0. Otherwise, PCI-SIG says that the PCIe 7.0 speicification also focuses on enhanced channel parameters and reach as well as improved power efficiency. So it sounds like they doubled the frequency and kept the encoding the same. PCIe 6 can get up to 256 GB/s, and 2x 256 = 512. In any case, it'll be a long time before the standard is finished, and far longer before any real hardware is around that actually uses PCIe 7.

ksec 3y ago
>In any case, it'll be a long time before the standard is finished, and far longer before any real hardware is around that actually uses PCIe 7. The standard is expected to finish in 2025, and hardware / IP for PCIe 7 are already in the work. Since there is so much resemblance of it and PCIe 6. In terms of schedule it perhaps may be the least lead time of recent PCIe from Standard 1.0 to IP available. The HPC industry is really pushing for this ASAP.
Retr0id 3y ago
This answers the question in the literal sense, but I'm nonetheless surprised (as a relative outsider to the world of high-speed signalling). "just double the frequency" isn't something we're used to seeing elsewhere these days (e.g. CPU clock speeds for the last couple of decades). What are the fundamental technological advances that allow them to do so? Or in other words, what stopped from from achieving this in the previous generation?
- Night_Thastus 3y ago
  PCIe doesn't have to do anything as complex as a general-purpose CPU. Increasing the frequency is a lot easier when you don't need to worry about things like heat, pipelining, caching, branch prediction, multithreading, etc. It's just encoding data and sending it back and forth. We've gotten very, very good at that. It wasn't really like it was impossible before now - it's just more that it wasn't in demand. With the proliferation of SSDs transferring data over PCIe, it's become much more important - so the extra cost of better signaling hardware is worth it. Not to dismiss it completely, it's still a hard problem. But it's far easier than doubling the frequency of a CPU.
  - p1esk 3y ago
    Why would pipelining, caching, branch prediction make increasing the frequency difficult? Why would heat be less of a problem for a pcie controller than for a cpu?
    - kanetw 3y ago
      The short, stupid answer is: transistor count and size. Explaining it in detail requires more background in electronics, but that's ultimately what it boils down to. High-end analog front ends can reach the three-digit GHz (non-silicon processes admittedly, but still).
      - p1esk 3y ago
        Explaining it in detail Please do
        0xcde4c3db 3y ago
        I see you don't have any responses, so I'll give it a shot. I don't remember the actual math from my coursework, and have no experience with real-world custom ASIC design (let alone anything at all in the "three-digit GHz" range), but the major variables look something like this (in no particular order): 1) transistor size (smaller = faster, but process is more expensive) 2) gate dielectric thickness (smaller = faster, but easier to damage the gate; there's a corollary of this with electric fields across various PCB layer counts, microstrips, etc. that I'm nowhere near prepared to articulate) 3) logic voltage swing (smaller = faster, but less noise immunity) 4) number of logic/buffer gates traversed on the critical path (fewer = faster, but requires wider internal buses or simpler logic) 5) output drive strength (higher = faster, but usually larger, less efficient, and more prone to generating EMI) 6) fanout (how many inputs must be driven by a single output) on the critical path (lower = faster) Most of these have significant ties to the manufacturing process and gate library, and so aren't necessarily different between a link like PCIe and a CPU or GPU. Some things can be tweaked for I/O pads, but broadly speaking a lot of these vary together across a chip. The biggest exceptions are points #4 and #6. Doing general-purpose math, having flexible program control flow constructs, and being able to optimize legacy instruction sequences on-the-fly (but I repeat myself) unavoidably requires somewhat complex logic with state that needs to be observed in multiple places. Modern processors mitigate this with pipelining, which splits the processing into smaller stages separated by registers that hold on to the intermediate state (pipeline registers). This increases the maximum frequency of the circuit at the cost of requiring multiple clock cycles for an operation to proceed from initiation to completion (but allowing multiple operations to be "in flight" at once). That being said, what's the simplest possible example of a pipeline stage? Conceptually, it's just a pair of 1-bit registers with no logic between the input of one register and the input of the next one. When the clock ticks, a bit is moved from one register to the next. Chain a bunch of these stages together, and you have something called a shift register. Add some extra wires to read or write the stages in parallel, and a shift register lets you convert between serial and parallel connections. The big advantage that PCIe (and SATA/SAS, HDMI, DisplayPort, etc.) has over CPUs is that the actual part hooked up to the pairs that needs to run at the link rate is "just" a pair of big shift registers and some analog voodoo to get synchronization between two ends that are probably not running from the same physical oscillator (aka a SERDES block). In some sense it's the absolute optimal case of the "make my CPU go faster" design strategy. Actually designing one of these to reliably run at multi-gigabit rates is a considerable task, but any given foundry will generally have a validated block that can be licensed and pasted into your chip for a given process node. Does that make sense?

Tuna-Fish 3y ago

> the same physical interface It's not really the same physical interface. The connector is the same, but quality requirements for the traces have gotten much more strict over time. > Why haven't we seen the need for a PCI-X or VLB-style PCIe interface expansion? x32 PCIe does exist, it's just rarely used.