3 ms·

If you need a ton of memory bandwidth and have the money then you build your product around HBM2 or similar. There are routers, switches, IPS, etc built like th

• comprambler 4y ago
  Indeed, any spec of HBM is not going to be comparable to an ondie memory controller having to reach out to individual DIMMS on a traditional motherboard. I appreciate the work this person is putting in to Asahi linux, but on the whole this is an unfair comparison.
• rektide 4y ago
  > If you need a ton of memory bandwidth and have the money then you build your product around HBM2 or similar What does similar mean? I'm not sure I agree. The M1 is basically a classic big-iron architecture, miniaturized with chiplets. The top M1 Max config is effectively 8x channel/512-bit-wide, just like a current epyc. It's faster though because it's 5.2GT/s LPDDR5 memory (400GBps) vs 3.2GT/s DDR5, and it's much smaller because it appears as 4x stacks on package. (Next gen Genoa is up to 12x channels of 5.2GT/s DDR5 fwiw.) But it's still a conventional play, basically, with new packaging. > Intel is married to the commodity server market and their laptop/mobile products have always been trickle-down, because they are a giant megacorp whose roadmaps are dictated by their biggest profit centers. Generally, yes! Notably sometimes innovation is first delivered in mobile and "trickles up," but basically it's because mobile is a safer easier space to innovate/explore/learn in (as it's much less investment cost to make the much smaller/lighter chips there). Generally you're right about what leads the business. > Even if they designed this kind of system, they wouldn't be able to capture most of the profit from it like Apple can, since they don't sell memory or systems, they sell CPUs. It's definitely a little intimidating figuring out how to sell bundled on ram. The technical benefits are hard to ignore though. Total cost of ownership should improve for customers notably, via less intense/simpler motherboards, via low power consumption stemming from much more direct channels & stronger signal integrity to main memory. But you're right, I think there's a lot of unappealing & unwanted aspects to becoming a company that ships ram. It'll be hard to resist/ignore for long though. AMD hasn't shown DRAM related interest yet, but their 5800X3D has 96MB stacked L3 v-cache on a separate chiplet. Their upcoming Genoa cpus will have ~1GB stacked L3 V-cache, again at absurd terabits-per-second speed, also via chiplets. AMD is pushing the memory heirarchy wider, is doing something like this M1-ening, but at a at higher-throughput lower-latency but smaller-capacity points, while leaving the conventional RAM channels also available. But they could, pretty easily I'd wager, snap back & deliver integrated DDR if they wanted, saw the need. Now that there are groups like Universal Chiplet Interconnect Express, this tech is kind of semi-commodified, available for purchase, to a degree. Yes, the mainstream market may lag. But with the capabilities arriving, it's hard to imagine this tech staying out of the data-centers for too long. Once we see some hyperscalers or upstarts delivering chips with integrated DDR, I expect the appeal will be irresistible/unsuppressable. It'll become obvious/known that this is good tech that is table stakes for selling chips, like it or not.
  • verall 4y ago
    > What does similar mean? I was thinking that about HMC and thinking that HBM was still Samsung / SK's thing. I see now everyone is making HBM. You'd build your system around HBM. > Notably sometimes innovation is first delivered in mobile and "trickles up," Not at Intel :P but I agree in general. Apple would never use HBM because consumer margins and they have been building SoC based on LPDDR for years. Hyperscalers already do their compute on "bundled ram" systems when they buy NV Tesla or AMD ROCm cards, both based on HBM. AMD is targeting the slow&wide "million VMs" market, and DDR is doing great in their niche. I do agree we will start seeing weirder server boards sold with soldered DDR for cost and power reasons, probably by Intel or AMD. I think it will be an even more specific niche, sitting between off the shelf commodity servers and expensive appliances like NV DGX. Again Intel/AMD are in a bad spot to capture the fat margin here. UCIE is classic giving away old stuff to extinguish competition. It's good for bespoke builders but meant to ward off actual competition from selling something similar and then eventually becoming competitive with IF or NVLink etc. Note Apple made their own fancy interconnect thing too for M1.
    • rektide 4y ago
      I've seen HBM as growingly niche & less interesting, but upcoming HBM3 seems to be quite a turn towards a more DDR like behavior: 5.2+ GT/s from 2.0GT/s, a huge jump, and channel width dropping from 8x gargantuan 128-bit channels to 16x reasonable-size-ish 64-bit channels. I'm not sure what if anything else would be required to escape HBM2's reputation for being high latency, but hopefully the situation is just better. AMD hasn't really sold HBM as far as I've seen since ~2018. Counter-point, upcoming Intel data-center GPUs seem loaded with HBM. I tend to believe this use case will be more permissive of the the higher latency, wider width HBM, that deep number crunching is less affected. Nvidia has HBM on their contemporary Axx Tesla accelerators- again, a case where aggregate number crunching with some internal latency is ok, unlike general GPU and CPU work.