3 ms·

GT/s is a measure of latency (not total system latency, but the bus itself is only adding 128 billionth of a second). In fact it does not say anything about ban

by ZiiS 2y ago

GT/s is a measure of latency (not total system latency, but the bus itself is only adding 128 billionth of a second). In fact it does not say anything about bandwidth if you don't know how many bits in a transfer.

out_of_protocol 2y ago
Throughput != latency, and often is tradeoff to latency (e.g. if you send stuff in big batches, database can process 100k tx/sec, but one by one it's 1k tx/sec at most)
eqvinox 2y ago
I'm sorry, but you're multiply wrong. First, a "transfer" is not a term in the PCIe spec; if anything, there's "transaction". But GT/s does not refer to transactions as you seem to be implying, and in fact "GT" does not have an assigned long form in the PCIe base specification. The term is introduced / defined like this: | The primary Link attributes for PCI Express Link are: | · The basic Link – PCI Express Link consists of dual unidirectional differential Links, implemented as a Transmit pair and a Receive pair. A data clock is embedded using an encoding scheme (see Chapter 4) to achieve very high data rates. | · Signaling rate – Once initialized, each Link must only operate at one of the supported signaling levels. For the first generation of PCI Express technology, there is only one signaling rate defined, which provides an effective 2.5 Gigabits/second/Lane/direction of raw bandwidth. The second generation provides an effective 5.0 Gigabits/second/Lane/direction of raw bandwidth. The third generation provides an effective 8.0 Gigabits/second/Lane/direction of raw bandwidth. The data rate is expected to increase with technology advances in the future. | · Lanes – A Link must support at least one Lane – each Lane represents a set of differential signal pairs (one pair for transmission, one pair for reception). To scale bandwidth, a Link may aggregate multiple Lanes denoted by xN where N may be any of the supported Link widths. A x8 Link operating at the 2.5 GT/s data rate represents an aggregate bandwidth of 20 Gigabits/second of raw bandwidth in each direction. This specification describes operations for x1, x2, x4, x8, x12, x16, and x32 Lane widths. (from PCIe 4.0 base specification) So, GT/s is used to be less ambiguous on multi-lane links. Next, > the bus itself is only adding 128 billionth of a second). no, the bus does actually add more latency since almost all receivers need to reassemble the whole transaction (generally tens to hundreds of bytes) to checksum validate and then dispatch further to continue. This latency can show up multiple times if you have PCIe switches, but (unlike endpoints) these are frequently cut-through. However, that latency is seriously negligible compared to anything else in your system. > In fact it does not say anything about bandwidth if you don't know how many bits in a transfer. How many bits are in a transaction does in fact influence that latency mentioned right above, but has no impact on bandwidth. What does have an impact on available end-user bandwidth is how small you chunk longer transactions since each of them has per-transaction overhead. And finally — > GT/s is a measure of latency — absolutely not. It is a measure of raw bandwidth. It indirectly influences minimum and maximum latency, but those are complicated relationships especially on multi-lane links, and especially maximum latency depends on a whole host of factors from hardware capabilities, to BIOS and OS settings in PCIe config, to driver behavior.
- vinay_ys 2y ago
  PCIe 1.0 to 5.0 used NRZ (non return to zero) electrical signaling for transmission. In NRZ signaling at high rate and over long distances, there are challenges w.r.t clock recovery, DC balance and error correction. To deal with this, encoding is used. Encoding is basically a block of data (a sequence of zeros and ones) is represented as a sequence of electrical voltage changes (a block of symbols). GT/s does stand for Giga Transfers per second. Here, the transfers are referring to number of symbols transferred per second, and not actual usable data bits per second. We say GT/s instead of Gbps, because the actual usable bits/sec is determined by the encoding scheme used. PCIe 1.0 and 2.0 encoded 8 data bits in 10 symbols (NRZ electrical signals). That's 20% overhead. PCIe 3.0 to 5.0 encoded 128 bits of data in 130 symbols. That's a much lower overhead of 1.54%. PCIe 6.0 (& yet to be standardized PCIe 7.0) use PAM4 for signaling and doesn't required any encoding on top –hence it is written as 1b/1b. (btw, in PAM4, each symbol is 2 bits). You can see similar NRZ signaling with encoding in SATA, Ethernet, Fiber Channel etc. Btw, PAM4 with NRZ is used as well! Coming to latency, latency is the time it takes for a single bit of usable data to transfer from A to B. Many factors affect this latency. Signaling medium's speed of transmission (a fraction of speed of light), signaling medium's length, signaling frequency (Mhz, Ghz etc of voltage switching), encoding scheme (encoding overhead, clock recovery or its failure and hence retransmissions, error detection/correction quality or its failure and hence retransmissions) - each of these things affect the latency of usable data. GT/s = Signal Frequency x Bits per cycle. Remember, PAM4 encoding in PCIe6.0 has 2 bits per cycle (2 bits per symbol).
  - Dylan16807 2y ago
    > PCIe 6.0 (& yet to be standardized PCIe 7.0) use PAM4 for signaling and doesn't required any encoding on top –hence it is written as 1b/1b. (btw, in PAM4, each symbol is 2 bits). Nothing on top if you exclude the error correction bits, which I don't think you should.
  - eqvinox 2y ago
    You're confusing GT and Gbaud. Gbaud/s = Symbol rate × bits per symbol. GT/s is in fact G"bit"/s, before line coding (I left that can of worms unopened because line coding wasn't relevant to the bandwidth vs. latency discussion.) PCIe 6.0 is "64GT/s", but only 32 Gbaud, since as you correctly point out it uses PAM-4. > GT/s does stand for Giga Transfers per second. If you have a citable source for this, that'd be nice — it's not in the PCIe spec, and AFAIK the term is not used elsewhere.

by ZiiS 2y ago

out_of_protocol 2y ago

Throughput != latency, and often is tradeoff to latency (e.g. if you send stuff in big batches, database can process 100k tx/sec, but one by one it's 1k tx/sec at most)

eqvinox 2y ago

I'm sorry, but you're multiply wrong. First, a "transfer" is not a term in the PCIe spec; if anything, there's "transaction". But GT/s does not refer to transactions as you seem to be implying, and in fact "GT" does not have an assigned long form in the PCIe base specification. The term is introduced / defined like this: | The primary Link attributes for PCI Express Link are: | · The basic Link – PCI Express Link consists of dual unidirectional differential Links, implemented as a Transmit pair and a Receive pair. A data clock is embedded using an encoding scheme (see Chapter 4) to achieve very high data rates. | · Signaling rate – Once initialized, each Link must only operate at one of the supported signaling levels. For the first generation of PCI Express technology, there is only one signaling rate defined, which provides an effective 2.5 Gigabits/second/Lane/direction of raw bandwidth. The second generation provides an effective 5.0 Gigabits/second/Lane/direction of raw bandwidth. The third generation provides an effective 8.0 Gigabits/second/Lane/direction of raw bandwidth. The data rate is expected to increase with technology advances in the future. | · Lanes – A Link must support at least one Lane – each Lane represents a set of differential signal pairs (one pair for transmission, one pair for reception). To scale bandwidth, a Link may aggregate multiple Lanes denoted by xN where N may be any of the supported Link widths. A x8 Link operating at the 2.5 GT/s data rate represents an aggregate bandwidth of 20 Gigabits/second of raw bandwidth in each direction. This specification describes operations for x1, x2, x4, x8, x12, x16, and x32 Lane widths. (from PCIe 4.0 base specification) So, GT/s is used to be less ambiguous on multi-lane links. Next, > the bus itself is only adding 128 billionth of a second). no, the bus does actually add more latency since almost all receivers need to reassemble the whole transaction (generally tens to hundreds of bytes) to checksum validate and then dispatch further to continue. This latency can show up multiple times if you have PCIe switches, but (unlike endpoints) these are frequently cut-through. However, that latency is seriously negligible compared to anything else in your system. > In fact it does not say anything about bandwidth if you don't know how many bits in a transfer. How many bits are in a transaction does in fact influence that latency mentioned right above, but has no impact on bandwidth. What does have an impact on available end-user bandwidth is how small you chunk longer transactions since each of them has per-transaction overhead. And finally — > GT/s is a measure of latency — absolutely not. It is a measure of raw bandwidth. It indirectly influences minimum and maximum latency, but those are complicated relationships especially on multi-lane links, and especially maximum latency depends on a whole host of factors from hardware capabilities, to BIOS and OS settings in PCIe config, to driver behavior.

vinay_ys 2y ago
PCIe 1.0 to 5.0 used NRZ (non return to zero) electrical signaling for transmission. In NRZ signaling at high rate and over long distances, there are challenges w.r.t clock recovery, DC balance and error correction. To deal with this, encoding is used. Encoding is basically a block of data (a sequence of zeros and ones) is represented as a sequence of electrical voltage changes (a block of symbols). GT/s does stand for Giga Transfers per second. Here, the transfers are referring to number of symbols transferred per second, and not actual usable data bits per second. We say GT/s instead of Gbps, because the actual usable bits/sec is determined by the encoding scheme used. PCIe 1.0 and 2.0 encoded 8 data bits in 10 symbols (NRZ electrical signals). That's 20% overhead. PCIe 3.0 to 5.0 encoded 128 bits of data in 130 symbols. That's a much lower overhead of 1.54%. PCIe 6.0 (& yet to be standardized PCIe 7.0) use PAM4 for signaling and doesn't required any encoding on top –hence it is written as 1b/1b. (btw, in PAM4, each symbol is 2 bits). You can see similar NRZ signaling with encoding in SATA, Ethernet, Fiber Channel etc. Btw, PAM4 with NRZ is used as well! Coming to latency, latency is the time it takes for a single bit of usable data to transfer from A to B. Many factors affect this latency. Signaling medium's speed of transmission (a fraction of speed of light), signaling medium's length, signaling frequency (Mhz, Ghz etc of voltage switching), encoding scheme (encoding overhead, clock recovery or its failure and hence retransmissions, error detection/correction quality or its failure and hence retransmissions) - each of these things affect the latency of usable data. GT/s = Signal Frequency x Bits per cycle. Remember, PAM4 encoding in PCIe6.0 has 2 bits per cycle (2 bits per symbol).
- Dylan16807 2y ago
  > PCIe 6.0 (& yet to be standardized PCIe 7.0) use PAM4 for signaling and doesn't required any encoding on top –hence it is written as 1b/1b. (btw, in PAM4, each symbol is 2 bits). Nothing on top if you exclude the error correction bits, which I don't think you should.
- eqvinox 2y ago
  You're confusing GT and Gbaud. Gbaud/s = Symbol rate × bits per symbol. GT/s is in fact G"bit"/s, before line coding (I left that can of worms unopened because line coding wasn't relevant to the bandwidth vs. latency discussion.) PCIe 6.0 is "64GT/s", but only 32 Gbaud, since as you correctly point out it uses PAM-4. > GT/s does stand for Giga Transfers per second. If you have a citable source for this, that'd be nice — it's not in the PCIe spec, and AFAIK the term is not used elsewhere.