17 ms·
AMD Demonstrates Stacked 3D V-Cache Technology: 192 MB at 2 TB/SEC
- yarg 5y agoTheir computex presentation: https://www.youtube.com/watch?v=gqAYMx34euU&t=1995s https://www.youtube.com/watch?v=gqAYMx34euU&t=1995s
- quelsolaar 5y agoI absolutely love that AMD is making technology open and cross platform on the GPU end. They don't get enough credit for doing that.
- DaiPlusPlus 5y agoI'm sure Intel and NVIDIA would also like to make more of their platform and driver-stack open, but they are bound by confidentiality clauses in third-party technology that they've licensed. That's part of why huge chunks of Windows aren't open-source: because Microsoft doesn't "own" all of it, there's a lot of third-party licensed code compiled from source in Windows.
- snicker7 5y agoNot fully owning your own drivers/OS is something only a galaxy-brained MBA could think of.
- rawoke083600 5y agoUnderrated comment :)
- znpy 5y agoNot necessarily. Bryan Cantrill talked extensively about how long it took for Sun Microsystems to open source Solaris, and a lot came from code that was often outsources because it was "boring" and basically non-core tech (the example was i18n/l10n)
- deleted 5y ago[deleted]
- ChuckMcM 5y agoPresumably Intel or nVidia could buy out any confidentiality, after all it really is just about dollars at that level.
- yaomtc 5y agoIntel works on developing Mesa, together with AMD and VMWare. Intel graphics work quite well on the open source graphics stack.
- IntelMiner 5y agoIntel aggressively pushes their chips into Mesa, likely since they use Linux as the testbed for unreleased chips in development AMD is something like 10% of the total Linux kernel size now which is ridiculous. But their commitment in recent years is admirable as hell NVIDIA need to pull their fucking head out and just start working on Nouveau
- atq2119 5y agoMost of it is header files full of redundant register names. It's pretty ridiculous. The driver code itself also has lots of code duplication. It's just strange.
- vbezhenar 5y agoWhat's strange is why Linus allows that kind of code in the kernel.
- ClumsyPilot 5y agoBecause done is better than perfect. I've had enough issues with ideologues precenting any meaning full progress
- snuxoll 5y agoIt’s also not the worst way of making sure new chips get supported relatively quickly. Those “giant” header files are mechanically generated from the RTL source for each GPU, it’s not like it’s some horrendously inefficient bloat form an outsource developer that doesn’t care about writing clean code or something.
- 5y ago
- hobofan 5y agoEven with a proprietary driver they didn't have to kill OpenCL once they got the slightest edge on it with CUDA.
- 01100011 5y agoWho killed OpenCL? OpenCL 3.0 is supported on Nvidia's latest GPUs.
- hobofan 5y agoYes, now, after letting the whole ecosystem die by not supporting OpenCL 2.x for ~10 years, which in turn caused the whole ML field to turn into a Nvidia/CUDA monoculture.
- my123 5y agoOpenCL 2.x wasn’t well supported by others either, AMD’s implementation was pretty unusable and only had printf as a debug mechanism. There was no hardware at all on which you could rely on it, on top of it being much worse than CUDA.
- 0-_-0 5y agoI believe OpenCL 2.0 had a mandatory feature that nvidia couldn't support (maybe something about sharing pointers between GPU and CPU?). OpenCL 3.0 solved this by making many features optional.
- hobofan 5y agoI tried to create a ML framework[0] that would work on both CUDA and OpenCL (and natively on the CPU) around 2015/2016, which included creating FFI wrappers for both CUDA and OpenCL. This is where my experience on the subject (and my contempt for NVIDIA) comes from. Me memory isn't perfect, but IIRC the situation was roughly the following: We were quite short on resources (both devtime and money), which meant that we had to choose our scope wisely. Optimally we would have implemented both CUDA and OpenCL 2.0, but we had to settle for OpenCL 1.2 (which offered reduced performance, but was "good enough" for inference). IIRC OpenCL 2.0 was very very similar in what capabilities it assumed and offered to the CUDA version at the time, and cards like the GTX Titan X had "compute capabilities" that supported features like shared virtual memory between CPU and GPU in CUDA at the time. In fact the advances around memory management (and async copying) that were present in CUDA and not in OpenCL 1.x were the main source for the performance differences between the two. From everything that I can tell at that point in time, if NVIDIA would have wanted to support OpenCL 2.0 they could have done so based on technical requirements. What the reason for not doing so is, is just pure speculation (lack of internal resources due to focusing on devtools?), but to me it always looked like they were using the edge they got via their proprietary libraries like cuDNN to get a foot into the field of ML and then purposefully neglected OpenCL to prevent any competitors from catching up. Classic Embrace, Extend, Extinguish. [0]: https://github.com/autumnai/leaf https://github.com/autumnai/leaf
- schmorptron 5y agoNah, nVidia massively profits from the marketability of stuff like HairWorks / GameWorks a couple of years back being proprietary, and now it's DLSS. It's super cool tech, but kinda sad that [Speculation] it'll be replaced with a slightly worse version that works on all cards down the line just because nVidia wants to market it's exclusivity while it lasts.
- Hikikomori 5y agoNvidia likes to use Gameworks etc as a weapon against AMD. Like when they turned up tessellation to ridiculous levels because AMDs cards didn't have the hardware for it at the time.
- beebeepka 5y agoCrysis 2 simulating an entire ocean to render a puddle comes to mind. Also tessalating perfectly leveled floors IIRC.
- andromeduck 5y agoThat has been throughly debunked a million times over. The ocean only renders in wireframe mode because occlusion culling doesn't happen in wire frame mode.
- bazooka_penguin 5y agoNvidia never did that though.
- zucker42 5y agoNVIDIA doesn't give a crap about it's driver stack being open source. In fact, they actively want it to be not open. They refuse to even release signed firmware so the community can build an independent driver. Intel's stack is already pretty open on Linux.
- moogly 5y agoI'm not sure they have much choice, since they're not the ones driving the innovation.
- keanebean86 5y agoSo the next logical step would be to remove L3 cache from the compute chiplet all together? That would let AMD either save money since the chiplet is smaller or add more logic for the same die space. This could also mean a GPU chiplet on package with compute. Each chiplet gets at least one cache layer. The next few years could be pretty crazy.
- Scene_Cast2 5y agoLatency is still unclear, and probably worse than L3.
- coreblocks 5y agoGoing over TSV will definitely incur at least one clock cycle.
- Dylan16807 5y agoIf you're on a high end chip then you already have to talk to the other chiplet for half your L3. A stacked chip should be a lot faster than that.
- ece 5y agoThe additional L3 is per chiplet, so you're still going to have to talk to other chiplets and their L3. Latency numbers would be good to see. This is definitely better than eDRAM L4 on the I/O die, and that's still something they could do, so props for that. The power cost will really determine if we see this in Laptops or not though.
- Dylan16807 5y ago> The additional L3 is per chiplet, so you're still going to have to talk to other chiplets and their L3. You will sometimes, but traffic that still overflows to the other chiplet is probably almost all traffic that would have gone to RAM before. And as a reference point, cross-chiplet L3 is about 50 nanoseconds slower than local L3. That will dwarf most things.
- 8note 5y agoHow do you get more bits per second than is there? Can you do many writes to the same byte?
- Gh0stRAT 5y agoIt's a rate, so the same way you can drive 60 miles per hour between two points that are less than 60 miles apart.
- baybal2 5y agoHere comes another knockdown punch for Intel. Now they got a solution to add hundreds of megs of cache for cheap. More importantly SRAM can be binned/tested/KGDed separately! And it can be fabbed on its own customised process! And also, they can throw in an eDRAM there instead on a moment notice. And as they already have silicon interposer here, adding HBM2/3 will be also a triffle.
- fomine3 5y agoSRAM / for cheap?
- baybal2 5y agoYes, it would certainly be slower than an on-die cache, but very cheap. They can seriously reduce the L3 cache area, which is like 50% of the die currently, or get rid of it alltogether. Imagine, 2 times more dies per wafer at near no cost except for extra packaging.
- fomine3 5y agoSeparating SRAM cache would improve chip yield so possibly decrease some cost, but I don't believe it make SRAM "very cheap" or "at near no cost".
- baybal2 5y agoSee how the defect rate work: If a particle kills a repairable part of the die (which can be downgraded, or cut-off) you still get from a quarter, to a half of die area dead, and a second grade chip. But if you can make 2x dies from wafer even with the same defect rate (it will usually be less.) Even if the die will completely die from a local defect, you still get much more perfect dies in total, which is more important for your bottom line. Standalone SRAM can be very repairable, and high yield with a custom process. Adding few spare columns, or SRAM banks should be covering for much more defects than if binning was done per entire CPU die.
- inasio 5y agoSmart matter! So far we've only really been able to make smart paper (all chips are basically 2D). I remember reading in Stephenson's Diamond Age about a brick of compute, when I understood what it was it blew my mind...
- z3t4 5y agoMany chips today are multilayer (many sheets of paper)
- acidbaseextract 5y agoMultilayer interconnect or multiple layers of transistors?
- formerly_proven 5y agoMany chips as in flash (and recently also DRAM), but stacking logic is fairly new.
- inasio 5y agoExactly! Stacked logic is the key to the fabled smart matter. Heat dissipation is going to be an issue, although I think IBM had some ideas on microfluidics. Also, there's been a ton of work on dilution fridges for quantum chips, and some startups are doing cryo logic, these two could complement
- IshKebab 5y agoMultiple layers of transistors - at least for RAM (HBM). Multiple logic layers are coming really soon.
- fulafel 5y agoThis is packaging, the chips are still square (though they do have several layers). A solid brick of compute would have to run pretty slow, square chips have the surface area for heat removal.
- z3t4 5y agoMaybe the os core/kernel can live on the cpu and be able to make live optimizations and have its own assembly code...
- pdimitar 5y agoNot sure why you are downvoted, IMO this can really help with security in server environments -- or help some companies close down their kernel even more (like Google's Android closed-source extensions).
- formerly_proven 5y agoHow do the thermals work out? The compute dies are already one heck of a hotplate, putting an entire layer of insulating silicon on top isn't going to improve things?
- deleted 5y ago[deleted]
- ido 5y agoI recall years ago IBM had an experimental solution to that that involved liquid in between the layers to transport the heat (vague memory, I think they used an analogy to blood). A quick googling brings up this article, which I think may be the same thing: https://www.bbc.com/news/science-environment-24571219 https://www.bbc.com/news/science-environment-24571219
- baybal2 5y agoThere are copper to copper connections in between dies.
- bullen 5y agoI'm pretty sure the longevity of chips manufactured now is going to be alot shorter than for older chips if you can run them at the same temperature! Both litography and the desperate search for infinite growth is going to hurt in the long run! I'm stacking up on 14nm Atom that I can passively cool at 42 degrees celcius full blast on 8 cores and 50nm (100.000 writes per bit) X25-E SSDs!
- nicolapcweek94 5y agoGot any specs on those Atoms? A passively cooled homelab build sounds pretty interesting...
- timeinput 5y agoI have [1] which has a similar chip to what the parent described. Its a fun little computer. [1] https://www.newegg.com/supermicro-mbd-a1sri-2758f-o-intel-atom-c2758-2-4ghz-cpu-tdp-20w/p/N82E16813182855 https://www.newegg.com/supermicro-mbd-a1sri-2758f-o-intel-at...
- runeks 5y agoImagine 32GB of this V-Cache instead of RAM. Does each additional layer add latency? If so, I wonder how many layers you could have until it would reach RAM-equivalent latencies. Also, it would require a compiler change if you had 500 sections of RAM (at 64MB per section) with latency increases for each section.
- Cthulhu_ 5y agoI'm confident we'll see memory-on-chip within the next few years. Actually isn't that already the case for mobile CPU's?
- baybal2 5y agoMain memory, if you mean DRAM, no. DRAM is incompatible with the logic family used in CPU making, though attempts were numerous at trying to work around that. Every few years, there are somebody coming with claims of passable CMOS+DRAM tech, but none got adoption so far.
- my123 5y agoIt is not actually incompatible within the same die, IBM z15 uses eDRAM for the L1 caches even.
- baybal2 5y agoNot L1, but L2 and on. A very curious case, and a very clever hackery with a custom SOI process. It's still very, very far from coming to mainstream chips.
- jahnu 5y agoIt says L1 here "Both it and all levels of cache in the main processor from level 1 use eDRAM, instead of the traditionally used SRAM." https://en.m.wikipedia.org/wiki/IBM_z15_(microprocessor) https://en.m.wikipedia.org/wiki/IBM_z15_(microprocessor)
- torginus 5y agoThis is just an aside but almost every post about writing high-performance code stresses the importance of data locality, that related data must be packed together in memory, and a cache miss that goes to main memory costs thousands of instructions worth of executed code. This implies that languages with tons of pointer indirection like Java will never be as fast as well-written C++ code. I wonder how the continuous growth of caches in the recent past, as well as this approach will affect this performance requirement.
- bsenftner 5y agoTalk to game developers - game code lives or dies with it's ability to persist on cache.
- gameswithgo 5y agoHave they grown? I don’t recall that l1 has grown much, and the bigger ones likely have more latency. AMD has some big l3 lately but their l3 has a bit more latency for it these big caches will help indirect code be a few % less bad but it will still be better by a lot to keep things in l1. now maybe one day with some unforeseen developments, linked lists will be cool again.
- torginus 5y agoL1, not so much, but comparing my old Haswell and my new Zen 2 proc, the L2 size has doubled, and the L3 has has increased from 6MB to a whopping 32MB
- gameswithgo 5y agoright but I bet the ratios of cpu latency to l2 and l3 latency have both gotten worse with that growth.
- torginus 5y agoI checked, and it's about the same according to these: https://www.7-cpu.com/cpu/Haswell.html https://www.7-cpu.com/cpu/Haswell.html https://www.7-cpu.com/cpu/Zen.html https://www.7-cpu.com/cpu/Zen.html I think latency has a lot to do with the speed of light - since both of these CPUs run at similar frequencies, you can cover more SRAM cells worth of distance in the one with a denser process geometry, hence you can have access to a larger cache with the same latency.
- xvilka 5y agoStill not enough for Electron.
- LeanderK 5y agoI don't understand. Is it just cache put above the compute-chip (so "layered" and the connections are still just 2D) or is it really 3D-interconnected?
- flumpcakes 5y ago3D interconnected. Only 2 layers so perhaps more "2.5D" but they are different chips (potentially) from different wafers or processes.
- rajmann 5y agoAnyone here has experience with Xeon Phi (Knights landing) processor which had 16GB L4 cache. Was it useful. Too bad that Intel botched up their process tech, and this line of processors had to be stopped.
- dragontamer 5y ago> processor which had 16GB L4 cache That was HMC (a competitor to HBM), a DRAM technology that IIRC was slightly higher latency than the other DRAM on their board. So it was kind of hard to use. HMC had more bandwidth but worse latency, so it was only sometimes faster. This V-Cache is SRAM, and therefore likely to be lower-latency than any DRAM technology, and therefore actually useful as a cache. I don't have experience with Xeon Phi. I just remember people talking about it back then.
- deepsun 5y agoSide-question: is there a way to write code to be sure my code and data is stored in L3 cache? I understand it's done automatically from RAM, but what if I want to hack it out so that it's guaranteed to be in cache? What about L2 cache?
- deleted 5y ago[deleted]
- deleted 5y ago[deleted]
- aidenn0 5y agoIntel had extensions to let you encourage that to happen CAT and CDP. However they require OS support. It's a common feature on embedded processors, where portability is less of a concern.
- ksec 5y agoProbably need an VM professional to answer this. AFAIK Most of the Dynamic Lang VM optimisation basically comes down to prediction and cache hit. Would a 512MB of Cache, or say a 4GB Cache dramatically change the performance of these Dynamic Languages? It still wouldn't be C like performance, but instead of the current 10-20x for Python or Ruby, it would be closer to 3-5x? Or would it not be much difference at all? I could imagine in the future there will be a System Level Cache built on top the IOD on EPYC where it has 4-8GB capacity.
- AtlasBarfed 5y agoI think the most interesting thing about this is combining a 7nm process for the vast L3 cache expansion with presumably 3-5nm processes. Cheap silicon enhancing expensive silicon with high-bandwidth interconnects.