4 ms·
New Huawei 96GB GPU
- commandersaki 1y agoI'm not really familiar with GPUs that are not Intel, AMD, and Nvidia, but would these work fine for gaming (realising it is not the primary purpose).
- ZiiS 1y agoThey would need very complex drivers writing.
- abracadaniel 1y agoAt this point it seems like most of what goes into the driver must be per-game tweaks to fix mistakes or optimize unoptimized code.
- chickenzzzzu 1y agoThats from the opengl days, that has mostly been shifted to vulkan and dx12 now. still, you end up having the hw vendor employees do the work in the game company's codebase
- smallmancontrov 1y agoGaming has an enormous API surface compared to AI, so almost certainly not.
- deleted 1y ago[deleted]
- jsheard 1y agoGaming also wants a bunch of fixed-function graphics units that a dedicated accelerator like this has no reason to include. I guess you could try to make a GPU with just programmable compute but Intel attempted that and it didn't go very well.
- OsrsNeedsf2P 1y agoCertainly big for China, but if it can't run CUDA then it's not going to help them catch up in the AI race
- aswanson 1y agoMaybe they'll make a better abstraction layer than cuda.
- chickenzzzzu 1y agoVulkan already achieves 95% of cuda, with the remaining 5% being scheduling.
- LeoPanthera 1y agoMacs can't run CUDA but everyone is buying those for AI anyway.
- jszymborski 1y agoGP was likely referring to training, not inference.
- wolfgangK 1y agoOnly those who don't care/know about prompt processing speed are buying Macs for LLM inference.
- esseph 1y agoDon't know and don't care are definitely things that I could be, but it also makes sense if they want to keep lookups private.
- com2kid 1y agoEven 40 tokens per second is plenty enough for real time usage. The average person reads at ~4 words per second, 40 tokens per second is going to be 15-20 words per second. Even useful models like gemma3 27b are hitting 22 t/s on 4bit quants. You aren't going to be reformatting gigabytes of PDFs or anything, but for a lot of common use cases, those speeds are fine.
- GodelNumbering 1y agoLPDDR4X 96GB, total bandwidth 408GB/s
- sciencesama 1y agoBut the price !
- nine_k 1y agoI wonder if Huawei are making any money off this, or are they trying to get a foot in the door first.
- moralestapia 1y agoWhich is?
- deleted 1y ago[deleted]
- spectre3d 1y agoI took a gander: “Available for around $1,250 to $1,500 before shipping and potential import fees … if your primary bottleneck is VRAM for loading massive models … it is not an RTX 4090 replacement for high-speed inference on smaller models, but it is a VRAM barrier-breaker for running the giants of the open-source world.” https://www.hardware-corner.net/huawei-atlas-300i-duo-96gb-llm-20250830/ https://www.hardware-corner.net/huawei-atlas-300i-duo-96gb-l...
- jsheard 1y agoFor comparison, Nvidia's 96GB card uses GDDR7 for 1.8TB/sec bandwidth. This Huawei card is more in the league of an M4 Max bandwidth-wise.
- spectre3d 1y agoYes, similar to the slower M4 Max option. The Mac Studio with either M4 Max or M3 Ultra uses 8533 MT/s LPDDR5X. Memory bandwidth for the M4 Max is 410 GB/s with the 32-core GPU or 546 GB/s with the 40-core GPU. It’s 819 GB/s in the M3 Ultra config.
- hereme888 1y agoLet's see how it performs in the real world. The website makes some outlandish claims, like 1.86 TOPS/W leads the industry... NVIDIA's comparable RTX 6000 Pro gives out 6.67 TOPS/W. Qualcomm's Cloud AI 100 Ultra has like 100 TOPS/W.
- koolala 1y agoWhy is 1.86 TOPS/W outlandish? Why does 1.86 TOPS/W lead the industry?
- Insanity 1y agoI guess the outlandish claim is that it’s industry leading while it’s not, as other GPUs outperform it.
- sho_hn 1y agoI think they are saying the website claims that 1.86 TOPS/W is industry-leading while it is not.
- hereme888 1y agoBecause, as I showed, 1.86 is nowhere near industry-leading.
- rvz 1y agoNo need to panic, given that it is not compatible and cannot run CUDA....yet. But don't ignore this at all. This industry can change in 1 or 2 years very quickly.
- rowanG077 1y agoWhy panic? It's great that finally there is AI hardware coming out of china. This can only be good for everyone.
- beeflet 1y agoexcept for humans
- artninja1988 1y agoWhy? I'm a human and I like competitive markets and hate monopolies
- jiggawatts 1y agoSo… five to ten times worse TOPS/W compared to NVIDIA, if I’m reading this correctly. It wouldn’t have a market if it wasn’t for the tensions between the USA and China, unless it’s super cheap.
- enlyth 1y agoIt has shitty LPDDR4X memory
- LargoLasskhyfv 1y agoLPDDR4X 96GB or 48GB, total bandwidth 408GB/s Support for ECC @150Watts
- diggan 1y ago> It wouldn’t have a market if it wasn’t for the tensions between the USA and China, unless it’s super cheap. It is super cheap, compared to what's available if you want 96GB for ML inference in a single card (ignoring the other aspects one might care about). I'm seeing it on Alibaba for 1200-1500 EUR which is like 7-8 times cheaper than I can buy a RTX Pro 6000 for locally.
- kittikitti 1y agoThis is really great. Things like CUDA support are not a requirement for my purposes. The DDR4 speeds seem like a bottleneck but the VRAM in GPU's are most often also a bottleneck. I look forward to more technical reports on their AI accelerator.
- jjcm 1y agoThis feels comparable to the intel's battlemage offering of GPUs. Competitive when it comes to price, but simply not at the level of nvidia's offerings nor are they usable for AI. I highly doubt these will overtake the modded 48gb 4090 usage in China, but still it's a clear indicator that the chip embargo has lit a fire under Chinese industry to accelerate their own offerings in the semiconductor sector. It'll be interesting to see how these evolve in the coming months/years, as I suspect their iteration cycles will be shorter than US counterparts.
- sho_hn 1y agoThis is a first-gen effort from Huawei, no? Plenty of resources and time to iterate.
- wakawaka28 1y agoIsn't it also an early effort for Intel? They are not known for their GPUs. Also, I'm sure Huawei has some chip tech, and GPUs are mostly just a ton of little processors arranged in a grid.
- com2kid 1y agoYou can do a lot with a lot of VRAM and a reasonable amount of TOPS. A 24GB affordable GPU can easily power an entire house worth of AI work, from real time voice chat, image generation, simple tool calls and task running, reminders, alerts, smart home integrations, etc. IMHO a large set of potential use cases is being held back by Nvidia's high prices.
- jauntywundrkind 1y agoHuawei doing what the West won't, smashing the product segmentation niches. 96GB is what counts here. Being able to fit a good model is more important than speed, for many folks. Also worth trying to better compare on efficiency. I don't know how this shakes up, but TOps/W is another figure of merit that also matters a lot, maybe more than absolute TOps count. I don't know how good 1.86TOps/W measures up here, but knowing that this is 150W is quite time where-as a 4090 or whatever is way way more reasonable than how most cards get built.
- trasirinc 1y ago[flagged]
- echelon 1y agoMaybe the Chinese can break the CUDA monopoly? They're the ones writing most of the open source AI code.
- xadhominemx 1y agoExtremely large and fragmented community of AI researchers and developers in China. CUDA competitors in China face the same hurdles as those in the West.
- deleted 1y ago[deleted]
- trasirinc 1y agoBecause it's not just "CUDA" by itself, it's the whole ecosystem. Most AI developers don't actually use CUDA directly, they use programming libraries like PyTorch that use CUDA under the hood to communicate with the GPU to run tasks in parallel. CUDA is pretty much the standard and is supported anywhere it's relevant. Just creating an alternative is pretty meaningless if it isn't actually supported anywhere. Adding support isn't easy, and there's also stability issues, bugs, etc. People want something that works and is reliable (= CUDA, since it's battle-tested). That's the same flawed argument that people have used to expect Huawei to replace Android/EUV.
- diggan 1y agoInteresting approach of taking 2-3 comments from reddit and posting a comment joining them together here, with a brand new account no less. https://old.reddit.com/r/LocalLLaMA/comments/1n46ify/finally_china_entering_the_gpu_market_to_destroy/nbip09q/ https://old.reddit.com/r/LocalLLaMA/comments/1n46ify/finally...
- nsxwolf 1y agoThat is creepy and I do not like it.
- washadjeffmad 1y agoSeems like not a bad add-in card for OpenCV/2.
- cuuupid 1y agoI predict the tech reaction will be underwhelmed/disappointed and the wall street reaction will be panic selling NVDA
- nextworddev 1y agoJust a matter of time till they steal the recipe
- Escapado 1y agoNaive question: Are the current (from what I have heard not very effective) export restrictions of HPC GPUs to china truly productive in the long run if the goal is to retain an edge? As in, to me it seems that it just fuels an expansion of domestic capabilities and in the car and solar sector my impression is that china had already proven that it can absolutely perform on par or even better in many different metrics compared to western countries, given time and pressure. So while these chips are not on par with current or even last gen GPUs, I would not be surprised if china would catch up and even have a much higher incentive to do so, now that other countries try to control their access to key technologies. I am not saying whether retaining an edge is good or bad or that I have a different answer if one thought it was good. Just curious what you guys think.
- sho_hn 1y agoI would not be surprised if most of us are running Chinese silicon a decade or two from now, unless China invades Taiwan, and I also think recent events have certainly spurned CCP tech strategy and accelerated this timeline. There's a few hurdles for China to overcome first, most notably catching up on high-end manufacturing processes, but it's naive to assume that won't happen eventually. For consumer and prosumer gear that they can get it done is already obvious, cf. people generally having no problem with buying DJI, BambuLabs or Anker.
- b112 1y agoChina will 100% invade Taiwan. This is why both parties in the US are spending so much to get domestic chip production running. I would be astonished if the backroom deal wasn't "If you take Taiwan now, we'll have to stop you, if you wait until we're self sufficient, we won't interfere."
- neurostimulant 1y agoNo need to invade when all China have to do is to help the opposition party in Taiwan win: https://www.bbc.com/news/articles/cn8185e19l4o https://www.bbc.com/news/articles/cn8185e19l4o
- userbinator 1y agoIf they release the full hardware documentation they will already be ahead of the incumbent in openness, just like Intel and AMD did; and IMHO that will also help gain a following.
- wakawaka28 1y agoIf openness becomes the killer feature, their competitors will also release docs or whatever else.
- deleted 1y ago[deleted]
- atlgator 1y agoHow many do I need to run the full DeepSeek 671B?
- robotnikman 1y agoUsing a rough estimate where 1B parameters requires 1GB of VRAM, you would need at least 7
- sciencesama 1y agoThere was an nvidia hack that was related to china, they got all the technology they need now !
- bababalamba 1y agoMany are so focused on the narrative that the media runs with so that they seem all but blind to see the more logical macro-view of the situation which is just sitting there in plain sight.. 1. If you look at a world map you will quickly notice that the island of Taiwan is adjacent to the asian subcontinent and mainland China. If you had one of those physical globes with a world map on them, you would not even see the United states of America when looking at Taiwan. This fact alone should make you realize that the US has no business whatsoever interfering or warmongering about anything going on over there. 2. The reason for all this is to, by any means, try prevent any increase in "power" for other countries in an attempt to prevent their time in the sun on the global economic field from fading away. 3. The reason why this will fail is because the strategy of trying to force others to be your friends and to blackmail/strongarm them into submissive allies does not work in the long run. Anyone who has attended the social battleground of kindergarten should have seen this to be a human fact. 4. My tip to all world leaders is as follows. Try not act like hyperagressive mentally challenged warmongering pssies and everything will be fine. /Michael Ah. Sweden
- bababalamba 1y ago[dupe]
- doe88 1y agoNow US has just to copy & paste the reasons for export restrictions and implement brand new import restrictions, hence China will then just flood EU with their cheap GPUs.