10 ms·

PCIe 7.0 Draft 0.5 Spec: 512 GB/s over PCIe x16 On Track For 2025

141 points by ksec 3y ago

ksec 3y ago
A PCIe 7.0 x4 NVMe Drive would give ~64GB/s. I am not sure if we could unlock any new paradigm with it. But in terms of Gaming even with hardware compression the PS5 with 8GB/s already spends 95%+ of its time in CPU. Or are we going to do heavy Paging and stick with 8GB of RAM. My Mac currently does about 200-500GB write per day simply due to Kernel_Task Paging from Safari. Both the SSD Controller and NAND will continue to get cheaper and faster. While Production Cost of GB on DRAM hasn't moved much.
- sph 3y ago
  The thing that strikes me the most any time there's such a technologically jump is that, judging by the past 30 years, it is very likely that the most common apps, such as Chrome or Microsoft Office, will require these 64 GB/s NVMe drives to work correctly in a decade or so. We will look back at our meagre 4 GB/s NVMe of today as old junk. I truly wonder what abomination will we create to require that much disk bandwidth in 10 years to run a web browser.
  - internet101010 3y ago
    A new React component
    - TheTxT 3y ago
      Maybe we could have react components containing other frameworks. So you could use all the JavaScript frameworks for the most simple landing page.
    - sph 3y ago
      Chrome pushing for their future WebAI standard, which will allow people to write extremely advanced AGI systems... in pure Javascript. People are concerned about Bitcoin's energy usage, and do not stop to think how many cycles and joules are wasted running the world, very slowly and inefficiently, on Javascript.
  - pdimitar 3y ago
    > We will look back at our meagre 4 GB/s NVMe of today as old junk. Nah, I don't think so. There is a natural resistance against the newer more expensive machines. The non-tech people largely started looking on the area as always chasing the new shinies, and they are absolutely correct. I as a programmer, had to replace my gaming PC last year because the old one (which was 11 year old) had its motherboard give up. And I settled for 5600x with 1080 Ti and DDR4 RAM and a 4-7 GB/s NVMe SSD. Some of the more demanding games struggle because the GPU is old but outside of that that machine is impressively and hugely powerful. Even in those games it still keeps 45-60 FPS just fine. I would want to get a better GPU but honestly, the price jump from a 1080 Ti all the way to the 3070 and beyond is just insane and I can't justify it. I am gaming something like 4h a week, 12h in rare cases, usually about 6h. I really don't see the draw of all these innovations. Servers, surely, okay, but even for workstations I'd think we are much more in a need of machines with more memory channels -- that the Threadripper Pro stations offer -- and not RAM and SSDs that go into the higher double-digits of GB bandwidth. It's very obvious that most programmer tooling is not making full use of hardware resources. You have to go as low as a Celeron J CPU to actually see programmer tooling monopolize your machine. Even i3 CPUs and old Ryzens are flying. Unless you are crunching data all day every day then a normal NVMe SSD that tops at 7 GB/s during sequential bursts but is otherwise working stably at 2 GB/s, is good enough and then some more.
- lmeyerov 3y ago
  With GDS, we get to skip the CPU and go straight to parallel arrays of network/ssd cards, where 100GB/s is very doable We don't have a lot of our customers doing that yet... but they're definitely some of the more fun ones!
jl6 3y ago
I would love to get more information on the cost implications of this generational doubling. Are we going to get this “for free”, or does it come with, say, twice the power usage, or a requirement for tighter tolerances on physical components that are thus going to be more expensive to manufacture? We should pay more attention to the distinction between technologies which are a fundamental advancement, and technologies which are just a way to spend more resources. Intel have offered us many great examples of the latter: CPUs which are faster, but at great power and cooling cost. Or, to pick another industry, supercars with incredible acceleration and top speed, but with billionaire-only price tags. I don’t think those are really advanced cars. The true advancements lie in figuring out how to do more with less (like we saw with Apple Silicon), not more with more. I hope this is all irrelevant to PCIe 7.0 and it does in fact raise the baseline for everyone, but we can no longer take this for granted in this industry.
teaearlgraycold 3y ago
How do they keep pulling this off?
- nullindividual 3y ago
  I have the same question! They double the performance over the same physical interface generation after generation. Why haven't we seen the need for a PCI-X or VLB-style PCIe interface expansion?
  - Night_Thastus 3y ago
    They explain on the website: >To achieve its impressive data transfer rates, PCIe 7.0 doubles the bus frequency at the physical layer compared to PCIe 5.0 and 6.0. Otherwise, the standard retains pulse amplitude modulation with four level signaling (PAM4), 1b/1b FLIT mode encoding, and the forward error correction (FEC) technologies that are already used for PCIe 6.0. Otherwise, PCI-SIG says that the PCIe 7.0 speicification also focuses on enhanced channel parameters and reach as well as improved power efficiency. So it sounds like they doubled the frequency and kept the encoding the same. PCIe 6 can get up to 256 GB/s, and 2x 256 = 512. In any case, it'll be a long time before the standard is finished, and far longer before any real hardware is around that actually uses PCIe 7.
    - ksec 3y ago
      >In any case, it'll be a long time before the standard is finished, and far longer before any real hardware is around that actually uses PCIe 7. The standard is expected to finish in 2025, and hardware / IP for PCIe 7 are already in the work. Since there is so much resemblance of it and PCIe 6. In terms of schedule it perhaps may be the least lead time of recent PCIe from Standard 1.0 to IP available. The HPC industry is really pushing for this ASAP.
    - Retr0id 3y ago
      This answers the question in the literal sense, but I'm nonetheless surprised (as a relative outsider to the world of high-speed signalling). "just double the frequency" isn't something we're used to seeing elsewhere these days (e.g. CPU clock speeds for the last couple of decades). What are the fundamental technological advances that allow them to do so? Or in other words, what stopped from from achieving this in the previous generation?
vardump 3y ago
Good. IMHO, PCIe bandwidth is problematic on consumer devices. Can’t have multiple high bandwidth devices.
- Night_Thastus 3y ago
  Where is it problematic? On PCIe 4, you can run a high end graphics card and multiple SSDs over PCIe no problem. Bandwidth is not an issue there. If you're doing some server stuff, then maybe. But for normal consumers it's not an issue.
  - michaelt 3y ago
    Modern consumer motherboards are in a bit of a weird place at the moment. Even if you buy the fanciest consumer motherboard out there, you'll find it only comes with one PCIe slot that actually runs at 16x speed, one that's maybe 4x speed (but 16x size), and that's about it. You want a motherboard with two 16x slots, to run a dual-GPU setup at full speed? Buy a threadripper. From a market segmentation perspective this makes sense from the processor manufacturers' point of view - get the people who want lots of IO onto your high-margin server/workstation product range. Back in the 1990s you needed to plug in a graphics card, a sound card, an ethernet card, maybe a separate wifi card, and so on, each of which would use exactly one slot - but these days other than the GPU, it's all integrated on the motherboard already, and the GPU might block 2 or 3 slots anyway. So modern consumer motherboards still sell OK despite having fewer slots than ever before!
    - Alupis 3y ago
      > Even if you buy the fanciest consumer motherboard out there, you'll find it only comes with one PCIe slot that actually runs at 16x speed, one that's maybe 4x speed (but 16x size), and that's about it. Maybe in the Intel World, this is true. Intel has always played games at consumer's expense... Here's the cheapest AMD AM5 socket ATX mobo sold by NewEgg, complete with two (2) 16x PCIe gen 4 slots[1]. Here's an AM5 mATX board with two (2) 16x PCIe gen 4 slots[2]. Come join team Red - the water is warm. [1] https://www.newegg.com/msi-pro-b650-s-wifi/p/N82E16813144642 https://www.newegg.com/msi-pro-b650-s-wifi/p/N82E16813144642 [2] https://www.newegg.com/msi-pro-b650m-a-wifi/p/13-144-559 https://www.newegg.com/msi-pro-b650m-a-wifi/p/13-144-559
anonymousDan 3y ago
So will it close the gap with NVLink?
- p1esk 3y ago
  What makes you think nvlink will stay the same?
  - anonymousDan 3y ago
    I don't know either way, hence my question! Is there some fundamental design issue that means NVLink (or ultra Ethernet in the future) are likely to maintain their bandwidth advantage? Or is PCIe likely to close the gap?
    - transpute 3y ago
      Recent thread: https://news.ycombinator.com/item?id=39729509 https://news.ycombinator.com/item?id=39729509
- touisteur 3y ago
  Recent announcement during GTC was Blackwell would do x2 on nvlink, 900GB/s => 1800GB/s. And iirc nvlink can do multi-node (gpus in different servers) too now... so... no?
  - z4y5f3 3y ago
    NVLink advertises combined bandwidth in both direction, so the 1800 GBps NVLink on Blackwell is actually 900 GBps for everyone else. PCIe can also do multi-node direct transfer via PCIe switches and has been already widely adopted. NVLink still has the power and chip arena advantage even if the bandwidth is similar.
    - touisteur 3y ago
      I was just saying they've announced x2 already, around the same pace as PCIe. And it's G-Byte-ps on nvlink, while G-bit-ps on PCIe, right? I'm probably missing something... Anyway, good read here https://community.fs.com/article/an-overview-of-nvidia-nvlink.html https://community.fs.com/article/an-overview-of-nvidia-nvlin...
Veserv 3y ago
That is significantly more bandwidth than a DDR5 memory bus. Is there something about PCIe that makes it infeasible to use as a memory bus? Otherwise that just seems like a ton of free memory bandwidth just being left on the table.
- O5vYtytb 3y ago
  Isn't that what CXL is?
  - crest 3y ago
    CXL supports cache-coherent memories and even the numbers claimed in marketing materials are ~5x when it comes than DDR5 access times measured on desktop CPUs in microbenchmarks. It's still a lot better than having software handle the coherency protocol and could be really useful to allow either very large ccNUMA systems or memory capacity and bandwidth hungry accelerators to access the host DRAM as long they can tolerate the latency e.g. something along the lines of a several SX-Aurora cards hooked into a big host with a few TiB of DRAM spread over ≥16 DDR5 or later channels.
- redleader55 3y ago
  PCIe is a pretty complicated protocol based on messages. Using this for RAM would mean the CPU and everything will be required to speak this protocol. Using PCIe for RAM will also have impact on latency, which is undesired. Speed is not always the only thing.
  - Quarrel 3y ago
    > Speed is not always the only thing. or perhaps more accurately, bandwidth is not the only goal.
    - alexgartrell 3y ago
      Or perhaps more accurately, throughput is not the only goal.
loudmax 3y ago
I wonder how this might affect GPU access to system RAM. Right now, people wanting to large language models on GPUs are constrained by the amount of VRAM the cards have. GPU access to system RAM is enough of a bottleneck that you might as well run your application on the regular CPU. But GPU access to system RAM is fast enough, then this opens more possibilities for GPU acceleration for large models.
- appplication 3y ago
  I know Apple doesn’t really do GPUs the same way other companies do, but isn’t this generally the idea behind their integrated memory SOCs?
- alexgartrell 3y ago
  Latency and cache coherency are the other things that make this hard. Cache coherency can theoretically be resolved by CXL, so maybe we’ll get there that way.
  - Tuna-Fish 3y ago
    AI models do not need coherent memory, the access pattern is regular enough that you can make do with explicit barriers. The bigger problem is that by the time PCIe 7.0 will be actually available, 242GB/s per direction will probably not be sufficient for anything interesting.
    - paulmd 3y ago
      > AI models do not need coherent memory, the access pattern is regular enough that you can make do with explicit barriers. response to both this and the sibling: even for training, I remember some speculation that explicit synchronization might not even be needed, especially in the middle stages of training (past the early part, before fine tuning). it's not "correct" but the gradient descent will eventually fix it anyway, as long as the error signal doesn't exceed the rate of gradient descent. And "error signal" here isn't just the noise itself but the error in gradient descent caused by incorrect sync - if the average delta in the descent is very slow, so is the amount of error it introduces, right? actually iirc there was some thought that the noise might help bounce the model out of local optimums. little bit of a Simulated Annealing idea there. > The bigger problem is that by the time PCIe 7.0 will be actually available, 242GB/s per direction will probably not be sufficient for anything interesting. yea it's this - SemiAnalysis/Dylan Patel actually has been doing some great pieces on this. background: networks really don't scale past about 8-16 nodes. 8 is a hypercube, that's easy. You can do 16 with ringbus or xy-grid arrangements (although I don't think xy-grid has proven satisfactory for anything except systolic arrays imo). But as you increase the number of nodes past 8, link count blows out tremendously, bisection bandwidth stagnates, worst-case distance blows out tremendously, etc. So you want tiers, and you want them to be composed of nodes that are as large as you can make them, because you can't scale the node count infinitely. https://www.cs.cmu.edu/afs/cs/academic/class/15418-s12/www/lectures/18_interconnects.pdf https://www.cs.cmu.edu/afs/cs/academic/class/15418-s12/www/l... Past about 16 nodes you just go to a switched-fabric thing that's a crossbar or a butterfly tree or whatever - and it's still a big fat chip itself, the 2nd gen nvswitch (pascal era) was about as many transistors as a Xeon 1650v1. They are on gen 3 or 4 now, and they have three of these in a giant mixing network (butterfly or fat tree or something) for just titanic amounts of interconnect between 2 racks. I don't even want to know what the switching itself pulls, it's not 300kW but it's definitely not insignificant either. any HPC interconnect really needs to be a meaningful fraction of memory bandwidth speed if you want to treat it like a "single system". Doesn't have to be 100%, but like, needs to be 1/3 or 1/4 of normal memory BW at least. One of the theories around AMD's MCM patents was that the cache port and the interconnect port should be more or less the same thing - because you need to talk to interconnect at pretty much the same rate you talk to cache. So a cache chiplet and an interconnect could be pretty much the same thing in silicon (I guess today we'd say they're both Infinity Link - which is not the same as Infinity Fabric BTW, again the key difference being coherency). But that kinda frames the discussion in terms of the requirements here. https://hexus.net/tech/news/graphics/147643-amd-patent-outlines-gpu-active-bridge-chiplet-design/ https://hexus.net/tech/news/graphics/147643-amd-patent-outli... https://community.amd.com/t5/general-discussions/amd-files-patent-for-its-own-gpu-chiplet-implementation/m-p/436624 https://community.amd.com/t5/general-discussions/amd-files-p... anyway, to your point: pcie/cxl will never be as fast as ethernet because the signal integrity requirements are orders of magnitude tighter, pcie is a very short-range link etc and requires comparatively larger PHY to drive it than ethernet does for the same bandwidth. ethernet serdes apparently have 3x the bandwidth as PCIe (and CXL) serdes per mm of beachfront, and GPU networking highly favors bandwidth above almost any other concern (and they utterly don't care about latency). The denser you make the bandwidth, the more links (or fatter links) you can fit on a given chip. And more links basically translates to larger networks, meaning more total capacity, better TCO, etc. Sort of a gustafson's law thing. (and it goes without saying that regardless, this all burns a tremendous amount of power. data movement is expensive.) https://www.semianalysis.com/p/cxl-is-dead-in-the-ai-era https://www.semianalysis.com/p/cxl-is-dead-in-the-ai-era the shape this is taking is basically computronium. giant chips, massive interconnects between them. It's not that chiplet is a bad idea, but what's better than lots of little chiplets fused into a single processor? lots of big chiplets fused into a single processor. And in fact that pattern gets repeated fractally. MI300X and B200 both take two big dies and fuse them together into what feels like a single GPU. Then you take a bunch of those GPUs and fuse those together into a local node via NVSwitch. https://www.semianalysis.com/p/nvidias-optical-boogeyman-nvl72-infiniband https://www.semianalysis.com/p/nvidias-optical-boogeyman-nvl... Standard HPC stuff... other than the density. They are thinking it might actually scale to at least 300 kW per rack... and efficiency actually improves when you do this (just like packaging!) because data movement is hideously expensive. You absolutely want to keep everything "local" (at every level) and talk over the interconnects as little as possible. https://www.fabricatedknowledge.com/p/the-data-center-is-the-new-compute https://www.fabricatedknowledge.com/p/the-data-center-is-the... MLID interviewed an NVIDIA engineer after RDNA3 came out, iirc they more or less said "they looked at chiplet, they didn't think it was worth it yet, so they didn't do it. and gonna do chiplets in their own way, and aren't going to be constrained or limited to chasing the approaches AMD uses". And my interpretation of that quote has always been - they see what they are doing as building a giant GPU out of tons of "chiplets", where each chiplet is a H100 board or whatever. NVLink is their Infinity Link, Ethernet is their Infinity Fabric/IFOP. The idea of a processor as a bunch of disaggregated tiny chiplets is great for yields, it's terrible for performance and efficiency. "tile" in the sense of having 64 tiles on your server processor is dead dead dead, tiles need to be decent-sized chunks of silicon on their own, because that reduces data movement a ton (and network node count etc). And while of course packaging lets you stack a couple dies... it also blows up power consumption in other areas, because if each chiplet is slower then you are moving more data around. The chip might be more efficient, but the system isn't as efficient as just building computronium out of big chunks. it's been an obvious lesson from the start even with ryzen, and RDNA3 should have really driven it home: data movement (cross-CCX/cross-CCD) is both performance-bottlenecking and power-intensive, so making the chiplets too small is a mistake. Navi 32 is barely a viable product etc, and that's without even hitting the prices that most people want to see from it. Driving 7700XT down to $349 or $329 or whatever is really really tough (it'll get there during clearance but they can't do it during the prime of its life), and idle power/low-load power sucks. You want the chunks to be at least medium sized - and really as big as you can afford to make them. Yields get lower the bigger you get, of course, but does anybody care about yielded price right now? Frankly I am surprised NVIDIA isn't pursing Cerebras-style wafer-scale right now tbh. again, lots of words to say: you want the nodes to be as fat as possible, because you probably only get 8 or 16 nodes per tier anyway. So the smaller you make the nodes, the less performance available at each tier. And that means slower systems with more energy spent moving data. The analyst (not NVIDIA)'s claim is that he thinks water-cooled 300kw would be more efficient than current systems. it's CRAY time! (no nvidia, no!!!) https://en.wikipedia.org/wiki/Cray-2#/media/File:Cray2.jpg https://en.wikipedia.org/wiki/Cray-2#/media/File:Cray2.jpg https://en.wikipedia.org/wiki/File:EPFL_CRAY-II_2.jpg https://en.wikipedia.org/wiki/File:EPFL_CRAY-II_2.jpg https://en.wikipedia.org/wiki/File:Cray-2_module_side_view.jpg https://en.wikipedia.org/wiki/File:Cray-2_module_side_view.j... (e: power consumption was 150-200 kW for Cray-2 so NVIDIA's got a ways to go (currently 100 kW, rumored 200 kW) to even to reach the peak of historical "make it bent so there's less data movement" style hyperdense designs. Tbh that makes me suspect that analyst is probably right, it's both possible and might well improve efficiency, but due to the data movement factors this time, rather than latency. Ironic.)
westurner 3y ago
PCI Express > PCIe 7.0: https://en.wikipedia.org/wiki/PCI_Express#PCI_Express_7.0 https://en.wikipedia.org/wiki/PCI_Express#PCI_Express_7.0
- SpecialistK 3y ago
  From your link: > PCI Express (Peripheral Component Interconnect Express), officially abbreviated as PCIe or PCI-e So I'm not sure what you're trying to say?
  - westurner 3y ago
    I linked the section of the PCIe Wikipedia article that could be updated with this latest 512 GB/s spec ann, which is newer than the 242 GB/s max spec listed in the infobox. No idea what the problem with that was.
    - pezezin 3y ago
      The 242 GB/s in the Wikipedia article is the per-direction bandwidth available after removing the encoding overhead. The 512 GB/s in the headline is the raw bi-directional bandwidth. Both are actually the same number.
      - westurner 3y ago
        Good thing I linked to Wikipedia on this for later reference instead of trying to update the article.
ShakataGaNai 3y ago
This one of those things were the default reaction is to say "Cool! This will be super handy" but then realize your existing computer is still running PCIE4 and and PCIE5 devices are still rare. I didn't realize that PCIE6 was already done and "out".
- Tuna-Fish 3y ago
  The standard needs to be developed well in advance of the devices that support it. PCIe 7.0 is not coming particularly quickly when compared to past standards. PCIe 5.0 first draft was out in 2017, final spec released in 2019, platform support first available in 2021, devices that actually supported it on the market in late 2022. PCIe 6.0 first draft out in 2020, final spec released in 2022, first devices (and hopefully platform support?) out this year. PCIe 7.0 first draft out in 2024, spec scheduled to be released 2025, devices and platform support out probably a few years after that. It's not that the standard is moving super quickly, but that because everyone needs to collaborate for the standard to actually be interoperable, you get to see how the sausage is made and so the news about it seems to be out very early.
  - rkagerer 3y ago
    Any reason a few companies can't leapfrog to the fastest one?
    - Tuna-Fish 3y ago
      It takes time to design and manufacture chips.
- zamadatix 3y ago
  Consumer devices don't really need the bandwidth doubling, outside power users treating it as a way to have less lane contention instead of increased bandwidth per device, yet. Most of the attraction is for large systems hoping to remotely access/share memory in a cluster. On the consumer side the only thing really close to mattering is "higher speed sequential SSD reads/writes on an x4 m.2 NVMe slot" and even then the speeds are already a bit silly for any practical single drive purpose.
_zoltan_ 3y ago
is the expectation that Zen 5 will be PCIe 6?
- trynumber9 3y ago
  Not for Granite Ridge, rumor is it reuses the same IO die as Raphael.
deleted 3y ago
[deleted]
m3kw9 3y ago
Can still play games maxed out on a 13 year old pc running i5. You just stick with 1080p, you are really not missing out much by chasing 4K
- imtringued 3y ago
  You clearly wouldn't be able to play Minecraft due to a lack of RAM. I had to upgrade in 2019, because of RAM alone.
  - m3kw9 3y ago
    Yes I did upgrade ssd, ram and gpu, not cpu/mb. I guess what I was really referring to is how the PCI express didn’t matter.
- korhojoa 3y ago
  This really depends on what you're playing. I used to use a pc built in 2013 with the then latest-and-greatest, but it did really stop being viable in my games ~early 2022 when they had some updates and four haswell cores just weren't enough anymore. Upgraded to 12th gen intel at that time and quadrupled my fps. (Notably cpu-bound games) Games that are less demanding can be fine, but if you're playing something where frames matter might be harder to play.
m3kw9 3y ago
This stuff tells me it could be only useful for AI things if they get the cost and power down so you don’t need to run the AC in the winter with it on.

141 points by ksec 3y ago

ksec 3y ago

A PCIe 7.0 x4 NVMe Drive would give ~64GB/s. I am not sure if we could unlock any new paradigm with it. But in terms of Gaming even with hardware compression the PS5 with 8GB/s already spends 95%+ of its time in CPU. Or are we going to do heavy Paging and stick with 8GB of RAM. My Mac currently does about 200-500GB write per day simply due to Kernel_Task Paging from Safari. Both the SSD Controller and NAND will continue to get cheaper and faster. While Production Cost of GB on DRAM hasn't moved much.

sph 3y ago
The thing that strikes me the most any time there's such a technologically jump is that, judging by the past 30 years, it is very likely that the most common apps, such as Chrome or Microsoft Office, will require these 64 GB/s NVMe drives to work correctly in a decade or so. We will look back at our meagre 4 GB/s NVMe of today as old junk. I truly wonder what abomination will we create to require that much disk bandwidth in 10 years to run a web browser.
- internet101010 3y ago
  A new React component
  - TheTxT 3y ago
    Maybe we could have react components containing other frameworks. So you could use all the JavaScript frameworks for the most simple landing page.
  - sph 3y ago
    Chrome pushing for their future WebAI standard, which will allow people to write extremely advanced AGI systems... in pure Javascript. People are concerned about Bitcoin's energy usage, and do not stop to think how many cycles and joules are wasted running the world, very slowly and inefficiently, on Javascript.
- pdimitar 3y ago
  > We will look back at our meagre 4 GB/s NVMe of today as old junk. Nah, I don't think so. There is a natural resistance against the newer more expensive machines. The non-tech people largely started looking on the area as always chasing the new shinies, and they are absolutely correct. I as a programmer, had to replace my gaming PC last year because the old one (which was 11 year old) had its motherboard give up. And I settled for 5600x with 1080 Ti and DDR4 RAM and a 4-7 GB/s NVMe SSD. Some of the more demanding games struggle because the GPU is old but outside of that that machine is impressively and hugely powerful. Even in those games it still keeps 45-60 FPS just fine. I would want to get a better GPU but honestly, the price jump from a 1080 Ti all the way to the 3070 and beyond is just insane and I can't justify it. I am gaming something like 4h a week, 12h in rare cases, usually about 6h. I really don't see the draw of all these innovations. Servers, surely, okay, but even for workstations I'd think we are much more in a need of machines with more memory channels -- that the Threadripper Pro stations offer -- and not RAM and SSDs that go into the higher double-digits of GB bandwidth. It's very obvious that most programmer tooling is not making full use of hardware resources. You have to go as low as a Celeron J CPU to actually see programmer tooling monopolize your machine. Even i3 CPUs and old Ryzens are flying. Unless you are crunching data all day every day then a normal NVMe SSD that tops at 7 GB/s during sequential bursts but is otherwise working stably at 2 GB/s, is good enough and then some more.
lmeyerov 3y ago
With GDS, we get to skip the CPU and go straight to parallel arrays of network/ssd cards, where 100GB/s is very doable We don't have a lot of our customers doing that yet... but they're definitely some of the more fun ones!

jl6 3y ago

I would love to get more information on the cost implications of this generational doubling. Are we going to get this “for free”, or does it come with, say, twice the power usage, or a requirement for tighter tolerances on physical components that are thus going to be more expensive to manufacture? We should pay more attention to the distinction between technologies which are a fundamental advancement, and technologies which are just a way to spend more resources. Intel have offered us many great examples of the latter: CPUs which are faster, but at great power and cooling cost. Or, to pick another industry, supercars with incredible acceleration and top speed, but with billionaire-only price tags. I don’t think those are really advanced cars. The true advancements lie in figuring out how to do more with less (like we saw with Apple Silicon), not more with more. I hope this is all irrelevant to PCIe 7.0 and it does in fact raise the baseline for everyone, but we can no longer take this for granted in this industry.

teaearlgraycold 3y ago

How do they keep pulling this off?

nullindividual 3y ago
I have the same question! They double the performance over the same physical interface generation after generation. Why haven't we seen the need for a PCI-X or VLB-style PCIe interface expansion?
- Night_Thastus 3y ago
  They explain on the website: >To achieve its impressive data transfer rates, PCIe 7.0 doubles the bus frequency at the physical layer compared to PCIe 5.0 and 6.0. Otherwise, the standard retains pulse amplitude modulation with four level signaling (PAM4), 1b/1b FLIT mode encoding, and the forward error correction (FEC) technologies that are already used for PCIe 6.0. Otherwise, PCI-SIG says that the PCIe 7.0 speicification also focuses on enhanced channel parameters and reach as well as improved power efficiency. So it sounds like they doubled the frequency and kept the encoding the same. PCIe 6 can get up to 256 GB/s, and 2x 256 = 512. In any case, it'll be a long time before the standard is finished, and far longer before any real hardware is around that actually uses PCIe 7.
  - ksec 3y ago
    >In any case, it'll be a long time before the standard is finished, and far longer before any real hardware is around that actually uses PCIe 7. The standard is expected to finish in 2025, and hardware / IP for PCIe 7 are already in the work. Since there is so much resemblance of it and PCIe 6. In terms of schedule it perhaps may be the least lead time of recent PCIe from Standard 1.0 to IP available. The HPC industry is really pushing for this ASAP.
  - Retr0id 3y ago
    This answers the question in the literal sense, but I'm nonetheless surprised (as a relative outsider to the world of high-speed signalling). "just double the frequency" isn't something we're used to seeing elsewhere these days (e.g. CPU clock speeds for the last couple of decades). What are the fundamental technological advances that allow them to do so? Or in other words, what stopped from from achieving this in the previous generation?

vardump 3y ago

Good. IMHO, PCIe bandwidth is problematic on consumer devices. Can’t have multiple high bandwidth devices.

Night_Thastus 3y ago
Where is it problematic? On PCIe 4, you can run a high end graphics card and multiple SSDs over PCIe no problem. Bandwidth is not an issue there. If you're doing some server stuff, then maybe. But for normal consumers it's not an issue.
- michaelt 3y ago
  Modern consumer motherboards are in a bit of a weird place at the moment. Even if you buy the fanciest consumer motherboard out there, you'll find it only comes with one PCIe slot that actually runs at 16x speed, one that's maybe 4x speed (but 16x size), and that's about it. You want a motherboard with two 16x slots, to run a dual-GPU setup at full speed? Buy a threadripper. From a market segmentation perspective this makes sense from the processor manufacturers' point of view - get the people who want lots of IO onto your high-margin server/workstation product range. Back in the 1990s you needed to plug in a graphics card, a sound card, an ethernet card, maybe a separate wifi card, and so on, each of which would use exactly one slot - but these days other than the GPU, it's all integrated on the motherboard already, and the GPU might block 2 or 3 slots anyway. So modern consumer motherboards still sell OK despite having fewer slots than ever before!
  - Alupis 3y ago
    > Even if you buy the fanciest consumer motherboard out there, you'll find it only comes with one PCIe slot that actually runs at 16x speed, one that's maybe 4x speed (but 16x size), and that's about it. Maybe in the Intel World, this is true. Intel has always played games at consumer's expense... Here's the cheapest AMD AM5 socket ATX mobo sold by NewEgg, complete with two (2) 16x PCIe gen 4 slots[1]. Here's an AM5 mATX board with two (2) 16x PCIe gen 4 slots[2]. Come join team Red - the water is warm. [1] https://www.newegg.com/msi-pro-b650-s-wifi/p/N82E16813144642 https://www.newegg.com/msi-pro-b650-s-wifi/p/N82E16813144642 [2] https://www.newegg.com/msi-pro-b650m-a-wifi/p/13-144-559 https://www.newegg.com/msi-pro-b650m-a-wifi/p/13-144-559

anonymousDan 3y ago

So will it close the gap with NVLink?

p1esk 3y ago
What makes you think nvlink will stay the same?
- anonymousDan 3y ago
  I don't know either way, hence my question! Is there some fundamental design issue that means NVLink (or ultra Ethernet in the future) are likely to maintain their bandwidth advantage? Or is PCIe likely to close the gap?
  - transpute 3y ago
    Recent thread: https://news.ycombinator.com/item?id=39729509 https://news.ycombinator.com/item?id=39729509
touisteur 3y ago
Recent announcement during GTC was Blackwell would do x2 on nvlink, 900GB/s => 1800GB/s. And iirc nvlink can do multi-node (gpus in different servers) too now... so... no?
- z4y5f3 3y ago
  NVLink advertises combined bandwidth in both direction, so the 1800 GBps NVLink on Blackwell is actually 900 GBps for everyone else. PCIe can also do multi-node direct transfer via PCIe switches and has been already widely adopted. NVLink still has the power and chip arena advantage even if the bandwidth is similar.
  - touisteur 3y ago
    I was just saying they've announced x2 already, around the same pace as PCIe. And it's G-Byte-ps on nvlink, while G-bit-ps on PCIe, right? I'm probably missing something... Anyway, good read here https://community.fs.com/article/an-overview-of-nvidia-nvlink.html https://community.fs.com/article/an-overview-of-nvidia-nvlin...

Veserv 3y ago

That is significantly more bandwidth than a DDR5 memory bus. Is there something about PCIe that makes it infeasible to use as a memory bus? Otherwise that just seems like a ton of free memory bandwidth just being left on the table.

O5vYtytb 3y ago
Isn't that what CXL is?
- crest 3y ago
  CXL supports cache-coherent memories and even the numbers claimed in marketing materials are ~5x when it comes than DDR5 access times measured on desktop CPUs in microbenchmarks. It's still a lot better than having software handle the coherency protocol and could be really useful to allow either very large ccNUMA systems or memory capacity and bandwidth hungry accelerators to access the host DRAM as long they can tolerate the latency e.g. something along the lines of a several SX-Aurora cards hooked into a big host with a few TiB of DRAM spread over ≥16 DDR5 or later channels.
redleader55 3y ago
PCIe is a pretty complicated protocol based on messages. Using this for RAM would mean the CPU and everything will be required to speak this protocol. Using PCIe for RAM will also have impact on latency, which is undesired. Speed is not always the only thing.
- Quarrel 3y ago
  > Speed is not always the only thing. or perhaps more accurately, bandwidth is not the only goal.
  - alexgartrell 3y ago
    Or perhaps more accurately, throughput is not the only goal.

loudmax 3y ago

I wonder how this might affect GPU access to system RAM. Right now, people wanting to large language models on GPUs are constrained by the amount of VRAM the cards have. GPU access to system RAM is enough of a bottleneck that you might as well run your application on the regular CPU. But GPU access to system RAM is fast enough, then this opens more possibilities for GPU acceleration for large models.

appplication 3y ago
I know Apple doesn’t really do GPUs the same way other companies do, but isn’t this generally the idea behind their integrated memory SOCs?
alexgartrell 3y ago
Latency and cache coherency are the other things that make this hard. Cache coherency can theoretically be resolved by CXL, so maybe we’ll get there that way.
- Tuna-Fish 3y ago
  AI models do not need coherent memory, the access pattern is regular enough that you can make do with explicit barriers. The bigger problem is that by the time PCIe 7.0 will be actually available, 242GB/s per direction will probably not be sufficient for anything interesting.
  - paulmd 3y ago
    > AI models do not need coherent memory, the access pattern is regular enough that you can make do with explicit barriers. response to both this and the sibling: even for training, I remember some speculation that explicit synchronization might not even be needed, especially in the middle stages of training (past the early part, before fine tuning). it's not "correct" but the gradient descent will eventually fix it anyway, as long as the error signal doesn't exceed the rate of gradient descent. And "error signal" here isn't just the noise itself but the error in gradient descent caused by incorrect sync - if the average delta in the descent is very slow, so is the amount of error it introduces, right? actually iirc there was some thought that the noise might help bounce the model out of local optimums. little bit of a Simulated Annealing idea there. > The bigger problem is that by the time PCIe 7.0 will be actually available, 242GB/s per direction will probably not be sufficient for anything interesting. yea it's this - SemiAnalysis/Dylan Patel actually has been doing some great pieces on this. background: networks really don't scale past about 8-16 nodes. 8 is a hypercube, that's easy. You can do 16 with ringbus or xy-grid arrangements (although I don't think xy-grid has proven satisfactory for anything except systolic arrays imo). But as you increase the number of nodes past 8, link count blows out tremendously, bisection bandwidth stagnates, worst-case distance blows out tremendously, etc. So you want tiers, and you want them to be composed of nodes that are as large as you can make them, because you can't scale the node count infinitely. https://www.cs.cmu.edu/afs/cs/academic/class/15418-s12/www/lectures/18_interconnects.pdf https://www.cs.cmu.edu/afs/cs/academic/class/15418-s12/www/l... Past about 16 nodes you just go to a switched-fabric thing that's a crossbar or a butterfly tree or whatever - and it's still a big fat chip itself, the 2nd gen nvswitch (pascal era) was about as many transistors as a Xeon 1650v1. They are on gen 3 or 4 now, and they have three of these in a giant mixing network (butterfly or fat tree or something) for just titanic amounts of interconnect between 2 racks. I don't even want to know what the switching itself pulls, it's not 300kW but it's definitely not insignificant either. any HPC interconnect really needs to be a meaningful fraction of memory bandwidth speed if you want to treat it like a "single system". Doesn't have to be 100%, but like, needs to be 1/3 or 1/4 of normal memory BW at least. One of the theories around AMD's MCM patents was that the cache port and the interconnect port should be more or less the same thing - because you need to talk to interconnect at pretty much the same rate you talk to cache. So a cache chiplet and an interconnect could be pretty much the same thing in silicon (I guess today we'd say they're both Infinity Link - which is not the same as Infinity Fabric BTW, again the key difference being coherency). But that kinda frames the discussion in terms of the requirements here. https://hexus.net/tech/news/graphics/147643-amd-patent-outlines-gpu-active-bridge-chiplet-design/ https://hexus.net/tech/news/graphics/147643-amd-patent-outli... https://community.amd.com/t5/general-discussions/amd-files-patent-for-its-own-gpu-chiplet-implementation/m-p/436624 https://community.amd.com/t5/general-discussions/amd-files-p... anyway, to your point: pcie/cxl will never be as fast as ethernet because the signal integrity requirements are orders of magnitude tighter, pcie is a very short-range link etc and requires comparatively larger PHY to drive it than ethernet does for the same bandwidth. ethernet serdes apparently have 3x the bandwidth as PCIe (and CXL) serdes per mm of beachfront, and GPU networking highly favors bandwidth above almost any other concern (and they utterly don't care about latency). The denser you make the bandwidth, the more links (or fatter links) you can fit on a given chip. And more links basically translates to larger networks, meaning more total capacity, better TCO, etc. Sort of a gustafson's law thing. (and it goes without saying that regardless, this all burns a tremendous amount of power. data movement is expensive.) https://www.semianalysis.com/p/cxl-is-dead-in-the-ai-era https://www.semianalysis.com/p/cxl-is-dead-in-the-ai-era the shape this is taking is basically computronium. giant chips, massive interconnects between them. It's not that chiplet is a bad idea, but what's better than lots of little chiplets fused into a single processor? lots of big chiplets fused into a single processor. And in fact that pattern gets repeated fractally. MI300X and B200 both take two big dies and fuse them together into what feels like a single GPU. Then you take a bunch of those GPUs and fuse those together into a local node via NVSwitch. https://www.semianalysis.com/p/nvidias-optical-boogeyman-nvl72-infiniband https://www.semianalysis.com/p/nvidias-optical-boogeyman-nvl... Standard HPC stuff... other than the density. They are thinking it might actually scale to at least 300 kW per rack... and efficiency actually improves when you do this (just like packaging!) because data movement is hideously expensive. You absolutely want to keep everything "local" (at every level) and talk over the interconnects as little as possible. https://www.fabricatedknowledge.com/p/the-data-center-is-the-new-compute https://www.fabricatedknowledge.com/p/the-data-center-is-the... MLID interviewed an NVIDIA engineer after RDNA3 came out, iirc they more or less said "they looked at chiplet, they didn't think it was worth it yet, so they didn't do it. and gonna do chiplets in their own way, and aren't going to be constrained or limited to chasing the approaches AMD uses". And my interpretation of that quote has always been - they see what they are doing as building a giant GPU out of tons of "chiplets", where each chiplet is a H100 board or whatever. NVLink is their Infinity Link, Ethernet is their Infinity Fabric/IFOP. The idea of a processor as a bunch of disaggregated tiny chiplets is great for yields, it's terrible for performance and efficiency. "tile" in the sense of having 64 tiles on your server processor is dead dead dead, tiles need to be decent-sized chunks of silicon on their own, because that reduces data movement a ton (and network node count etc). And while of course packaging lets you stack a couple dies... it also blows up power consumption in other areas, because if each chiplet is slower then you are moving more data around. The chip might be more efficient, but the system isn't as efficient as just building computronium out of big chunks. it's been an obvious lesson from the start even with ryzen, and RDNA3 should have really driven it home: data movement (cross-CCX/cross-CCD) is both performance-bottlenecking and power-intensive, so making the chiplets too small is a mistake. Navi 32 is barely a viable product etc, and that's without even hitting the prices that most people want to see from it. Driving 7700XT down to $349 or $329 or whatever is really really tough (it'll get there during clearance but they can't do it during the prime of its life), and idle power/low-load power sucks. You want the chunks to be at least medium sized - and really as big as you can afford to make them. Yields get lower the bigger you get, of course, but does anybody care about yielded price right now? Frankly I am surprised NVIDIA isn't pursing Cerebras-style wafer-scale right now tbh. again, lots of words to say: you want the nodes to be as fat as possible, because you probably only get 8 or 16 nodes per tier anyway. So the smaller you make the nodes, the less performance available at each tier. And that means slower systems with more energy spent moving data. The analyst (not NVIDIA)'s claim is that he thinks water-cooled 300kw would be more efficient than current systems. it's CRAY time! (no nvidia, no!!!) https://en.wikipedia.org/wiki/Cray-2#/media/File:Cray2.jpg https://en.wikipedia.org/wiki/Cray-2#/media/File:Cray2.jpg https://en.wikipedia.org/wiki/File:EPFL_CRAY-II_2.jpg https://en.wikipedia.org/wiki/File:EPFL_CRAY-II_2.jpg https://en.wikipedia.org/wiki/File:Cray-2_module_side_view.jpg https://en.wikipedia.org/wiki/File:Cray-2_module_side_view.j... (e: power consumption was 150-200 kW for Cray-2 so NVIDIA's got a ways to go (currently 100 kW, rumored 200 kW) to even to reach the peak of historical "make it bent so there's less data movement" style hyperdense designs. Tbh that makes me suspect that analyst is probably right, it's both possible and might well improve efficiency, but due to the data movement factors this time, rather than latency. Ironic.)

westurner 3y ago

PCI Express > PCIe 7.0: https://en.wikipedia.org/wiki/PCI_Express#PCI_Express_7.0 https://en.wikipedia.org/wiki/PCI_Express#PCI_Express_7.0

SpecialistK 3y ago
From your link: > PCI Express (Peripheral Component Interconnect Express), officially abbreviated as PCIe or PCI-e So I'm not sure what you're trying to say?
- westurner 3y ago
  I linked the section of the PCIe Wikipedia article that could be updated with this latest 512 GB/s spec ann, which is newer than the 242 GB/s max spec listed in the infobox. No idea what the problem with that was.
  - pezezin 3y ago
    The 242 GB/s in the Wikipedia article is the per-direction bandwidth available after removing the encoding overhead. The 512 GB/s in the headline is the raw bi-directional bandwidth. Both are actually the same number.
    - westurner 3y ago
      Good thing I linked to Wikipedia on this for later reference instead of trying to update the article.

ShakataGaNai 3y ago

This one of those things were the default reaction is to say "Cool! This will be super handy" but then realize your existing computer is still running PCIE4 and and PCIE5 devices are still rare. I didn't realize that PCIE6 was already done and "out".

Tuna-Fish 3y ago
The standard needs to be developed well in advance of the devices that support it. PCIe 7.0 is not coming particularly quickly when compared to past standards. PCIe 5.0 first draft was out in 2017, final spec released in 2019, platform support first available in 2021, devices that actually supported it on the market in late 2022. PCIe 6.0 first draft out in 2020, final spec released in 2022, first devices (and hopefully platform support?) out this year. PCIe 7.0 first draft out in 2024, spec scheduled to be released 2025, devices and platform support out probably a few years after that. It's not that the standard is moving super quickly, but that because everyone needs to collaborate for the standard to actually be interoperable, you get to see how the sausage is made and so the news about it seems to be out very early.
- rkagerer 3y ago
  Any reason a few companies can't leapfrog to the fastest one?
  - Tuna-Fish 3y ago
    It takes time to design and manufacture chips.
zamadatix 3y ago
Consumer devices don't really need the bandwidth doubling, outside power users treating it as a way to have less lane contention instead of increased bandwidth per device, yet. Most of the attraction is for large systems hoping to remotely access/share memory in a cluster. On the consumer side the only thing really close to mattering is "higher speed sequential SSD reads/writes on an x4 m.2 NVMe slot" and even then the speeds are already a bit silly for any practical single drive purpose.

_zoltan_ 3y ago

is the expectation that Zen 5 will be PCIe 6?

trynumber9 3y ago
Not for Granite Ridge, rumor is it reuses the same IO die as Raphael.

deleted 3y ago

[deleted]

m3kw9 3y ago

Can still play games maxed out on a 13 year old pc running i5. You just stick with 1080p, you are really not missing out much by chasing 4K

imtringued 3y ago
You clearly wouldn't be able to play Minecraft due to a lack of RAM. I had to upgrade in 2019, because of RAM alone.
- m3kw9 3y ago
  Yes I did upgrade ssd, ram and gpu, not cpu/mb. I guess what I was really referring to is how the PCI express didn’t matter.
korhojoa 3y ago
This really depends on what you're playing. I used to use a pc built in 2013 with the then latest-and-greatest, but it did really stop being viable in my games ~early 2022 when they had some updates and four haswell cores just weren't enough anymore. Upgraded to 12th gen intel at that time and quadrupled my fps. (Notably cpu-bound games) Games that are less demanding can be fine, but if you're playing something where frames matter might be harder to play.

m3kw9 3y ago

This stuff tells me it could be only useful for AI things if they get the cost and power down so you don’t need to run the AC in the winter with it on.