6 ms·
Microsoft is first to get HBM-juiced AMD CPUs
- alias_neo 2y agoAnother article source that uses a headline initialisation, "HBM"[0] in this case, and almost 30 times at that, and yet doesn't spell out what it stands for even once. I will point this out every time I see it, and continue to refuse to read from places that don't follow this simple etiquette. Be better. [0] High Bandwidth Memory
- sroussey 2y agoThey don't define HPC either, but I think the audience of this site knows these acronyms.
- alias_neo 2y agoThere were two reasons it was drilled into me in engineering-school; it provides context and avoidance of doubt about the topic, particularly when there are so many overlapping initialisms these days, often in the same space. The second is that you should never make assumptions about the audience of your writing, and their understanding of the topic; provide any and all information that might be pertinent for a non-subject-matter-specialist to understand, or at least find the information they need to understand.
- switchbak 2y ago> Be better They're almost certainly not on this forum, and they're not reading your post. So who is that quip directed at?
- rcthompson 2y agoPresumably it's directed at anyone writing an article for public consumption.
- alias_neo 2y ago> They're almost certainly not on this forum, and they're not reading your post I don't know much about the site in the OP, but I work on the assumption that almost anyone could be reading comments on links to their site on this forum. It's directed at them, you and even myself.
- switchbak 2y agoWhy don't you email them directly then? You're just posturing here. And it's directed at me? You don't know the first thing about me. PFO.
- dang 2y ago"Please don't pick the most provocative thing in an article or post to complain about in the thread. Find something interesting to respond to instead." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- Dylan16807 2y agoI don't feel like that rule works here? If you cut out part of the second sentence to get "Find something interesting to respond to", that's a good point, but the full context is "instead [of the most provocative thing in the article]" and that doesn't fit a complaint about acronyms.
- dang 2y agoTo paraphrase McLuhan, you don't like that guideline? We got others: "Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting." The point, in any case, is to avoid off-topic indignation about tangential things, even annoying ones.
- Dylan16807 2y agoYeah that one works.
- alias_neo 2y agoWhile I understand, I can't find something interesting to respond to in this case because I refused to read it. I'd argue that I also provided value by solving the complaint I made by spelling out what it stood for, for those who might not know.
- dang 2y agoI hear you and agree there's benefit in that; it's just that the cost (what it does to the thread) is larger than the benefit.
- sroussey 2y agoThis is very unlikely, but it would be interesting if Apple included HBM memory interfaces in the Max series of Apple Silicon, to be used in MacPro (and maybe the studio, but the Pro needs some more differentiation like HBM or a NUMA layout).
- throwaway48476 2y agoThey'd have to redesign the on die memory controller and tape out a new die all of which is expensive. Apple is a consumer technology company not a cutting tech tech company making high cost products for niche markets. There's just no way to make HBM work in the consumer space at the current price.
- sroussey 2y agoWell, they could put in a memory controller for both DDR5 and HBM on the die, so they would only have one die to tape out. The Max variant is something they are using in their own datacenters. It would be possible that they would use an HBM solely for themselves, but it would be cheaper overall if they did the same thing for workstations.
- nsteel 2y agoHBM has a very wide, relatively slow interface. A HBM phy is physically large and takes up a lot of beachfront, a massive waste of area (money) if you're not going to use it. It also (currently) requires you to use a silicon interposer, another huge extra expense in your design.
- whatever1 2y agoSo currently our consumer grade CPUs with DDR5 are limited to less than 100GB/s. Meanwhile Apple is shipping computers with multiples of that.
- oDot 2y agoStrix Halo is rumored to be about twice as fast but unfortunately not near Apple's speed.
- mananaysiempre 2y agoOn the other hand, I bought an 8C/16T Zen 4 laptop with 64GB RAM and an 4TB SSD for less than $2000 total including tax. I’ll take that trade.
- sroussey 2y agoHow are 70b LLMs running on that?
- jocaal 2y agoThe free version of chatgpt is better than your 70b LLM, whats the point?
- cma 2y agoQwen coder 32b instruct is the state of the art for local LLM coding and will run with a smallish context with that on a 64GB laptop with partial GPU offload. Probably around .8 tok/sec. With a quantization of it you can run larger contexts and go a bit faster. 1.4 tok/sec at 8b quant with offload to a 6GB laptop GPU. Speculative decoding has been being added to lots of the runtimes recently and can give a 20-30% boost with a 1 billion weight model running the speculative token stream.
- YetAnotherNick 2y agoWhy do you need 64GB RAM?
- 2y ago
- _bare_metal 2y agoHBM or not, those latest server chips are crazy fast and efficient. You can probably condense 8 servers from just a few years ago into one latest-gen Epyc. I run BareMetalSavings.com[0], a toy for ballpark-estimating bare-metal/cloud savings, and the things you can do with just a few servers today are pretty crazy. [0]: https://www.BareMetalSavings.com https://www.BareMetalSavings.com
- 1oooqooq 2y agoa graph showing this against cloud instance costs and aws profits would be funny.
- tame3902 2y agoCore counts have increased dramatically. The latest AMD server CPUs have up to 192 cores. The Zen1 top model had only 32 cores and that was already a lot compared to Intel. However, the power consumption has also increased: the current top model has a TDP of 500W.
- Guzba 2y agoDoes absolute power consumption matter or would it not be better to focus on per-core power consumption? Eg running 6 32-core CPUs seems unlikely to be better than 1 192-core.
- tame3902 2y agoYes, per core power consumption or better performance per Watt is usually more relevant than the total power consumption. And 1 high-core CPU is usually better than the same number of cores on multiple CPUs. (That is unless you are trying to maximize memory bandwidth per Watt.) What I wanted to get at is that the pure core count can be misleading if you care about power consumption. If you don't and just look at performance, the current CPU generations are monsters. But if you care about performance/Watt, the improvement isn't that large. The Zen1 CPU I was talking about had a TDP of 180 W. So you get 6x as many cores, but the power consumption increases by 2.7x.
- AnotherGoodName 2y agoI’m having trouble parsing the article even though I know fully what the mi300 is and what hbm memory is. I’m not alone right? This article seems to be complete ai nonsense at various points confusing the gpu and cpu portions of the product and not at all giving clarity on which parts of the product have hbm memory.
- kristianp 2y agoI agree, the article lacks clarity, jumping between 3 different AMD models and an Intel one. I'd suggest it's flaws hint at a human writer more than an AI.
- Tepix 2y agoPeople are buying dual Epyc Zen5 systems to get 24 DDR5-6000 memory channel bandwidth for inferemcing large LLMs on CPU. Clearly there is a demand for very fast memory.
- bobim 2y agoSure, implicit finite elements analysis scales up to two cores per DDR4 channel. Core density just grew up faster than bandwidth and it makes all those high cores cpus a waste for this kinds of workloads.
- pixelpoet 2y agoThis website and its spend-128-hours-disabling-1024-separate-cookies-and-vendors is pure cancer, I wish HN would just ban all these disgusting data hoovering leeches. By now I get that no one else cares and I should just stop coming here.
- codr7 2y agoAgreed, I wouldn't mind banning paywall crap while we're at it.
- zelphirkalt 2y agoNope, you are not alone. Without uBlock Origin I wouldn't go much anywhere these days.
- scouw 2y agoIt's far from an ideal solution but I've started to just have JS disabled by default in uBlock Origin and then enabling it manually on a per-site basis. Bit of a hassle but many sites, including this one, render just fine witbout JS and are arguably better without it.
- AnotherGoodName 2y agoThat entire site is 100% ai generated click farming. The fact that the top comments here not even talking about the content of the article but instead more general ‘hbm is great’ worries me. For anyone that read the article which product has hbm attached? The cpu or gpu? What is the name of this product? There’s literally nothing specific in here and the article is rambling ai nonsense. The whole site is a machine gun of such articles.
- pixelpoet 2y ago1000% agreed, especially the silent carrying on with the topic without acknowledging the cancer actual link. I know e.g. Deng is a real person who works hard to make HN decent, but wtf is this, are we just going to accept Daily Mail links too? I weep for the internet we had as children.
- 2y ago
- gessha 2y agoDoes it make sense to put HBM memory on mobile computing like laptops and smartphones?
- nsteel 2y agoThey'd be very expensive. Is there really a consumer market for large amounts (tens of GBs) of RAM with super high (800+GB/s) bandwidth? I guess you'll say AI applications but doing that amount of work on a mobile seems mad.
- gessha 2y agoYeah, I feel similarly about the development of NPUs. I guess it might be useful if we find more maybe non-AI uses for high-bandwidth memory that’s needed on the edge and not in centralized servers.