4 ms·

> What does similar mean? I was thinking that about HMC and thinking that HBM was still Samsung / SK's thing. I see now everyone is making HBM. You'd build you

by verall 4y ago

> What does similar mean? I was thinking that about HMC and thinking that HBM was still Samsung / SK's thing. I see now everyone is making HBM. You'd build your system around HBM. > Notably sometimes innovation is first delivered in mobile and "trickles up," Not at Intel :P but I agree in general. Apple would never use HBM because consumer margins and they have been building SoC based on LPDDR for years. Hyperscalers already do their compute on "bundled ram" systems when they buy NV Tesla or AMD ROCm cards, both based on HBM. AMD is targeting the slow&wide "million VMs" market, and DDR is doing great in their niche. I do agree we will start seeing weirder server boards sold with soldered DDR for cost and power reasons, probably by Intel or AMD. I think it will be an even more specific niche, sitting between off the shelf commodity servers and expensive appliances like NV DGX. Again Intel/AMD are in a bad spot to capture the fat margin here. UCIE is classic giving away old stuff to extinguish competition. It's good for bespoke builders but meant to ward off actual competition from selling something similar and then eventually becoming competitive with IF or NVLink etc. Note Apple made their own fancy interconnect thing too for M1.

rektide 4y ago
I've seen HBM as growingly niche & less interesting, but upcoming HBM3 seems to be quite a turn towards a more DDR like behavior: 5.2+ GT/s from 2.0GT/s, a huge jump, and channel width dropping from 8x gargantuan 128-bit channels to 16x reasonable-size-ish 64-bit channels. I'm not sure what if anything else would be required to escape HBM2's reputation for being high latency, but hopefully the situation is just better. AMD hasn't really sold HBM as far as I've seen since ~2018. Counter-point, upcoming Intel data-center GPUs seem loaded with HBM. I tend to believe this use case will be more permissive of the the higher latency, wider width HBM, that deep number crunching is less affected. Nvidia has HBM on their contemporary Axx Tesla accelerators- again, a case where aggregate number crunching with some internal latency is ok, unlike general GPU and CPU work.

by verall 4y ago

rektide 4y ago

I've seen HBM as growingly niche & less interesting, but upcoming HBM3 seems to be quite a turn towards a more DDR like behavior: 5.2+ GT/s from 2.0GT/s, a huge jump, and channel width dropping from 8x gargantuan 128-bit channels to 16x reasonable-size-ish 64-bit channels. I'm not sure what if anything else would be required to escape HBM2's reputation for being high latency, but hopefully the situation is just better. AMD hasn't really sold HBM as far as I've seen since ~2018. Counter-point, upcoming Intel data-center GPUs seem loaded with HBM. I tend to believe this use case will be more permissive of the the higher latency, wider width HBM, that deep number crunching is less affected. Nvidia has HBM on their contemporary Axx Tesla accelerators- again, a case where aggregate number crunching with some internal latency is ok, unlike general GPU and CPU work.