12 ms·
Nvidia investigating reports of RTX 4090 power cables burning or melting
- rx_tx 4y agoFor context, the RTX4090's power budget is 450W, and if all 4 plugs are connected to the cable adapter, the system can allow up 600W (at 12V) to the GPU. That's 6A to 8A per pin (there's 6 pins for current, 6 for GND plus 4 signaling pins), so pretty sensitive to resistive heating in case of a bad contact.
- vbezhenar 4y agoSeems like GPUs need more voltage. Something like 120V would be more appropriate.
- ChuckNorris89 4y agoHouses should come with a separate "Gaming PC" circuit breaker and power plug just like your dryer or oven. /s
- 112233 4y agoHa ha only serious. There were cases on local news when homes burned down because PC caught fire. Not only circuit breaker — please give me something (IR sensor?) that will trip and cut power if anything in the room and in the PC enclosure exceeds, say, 120°C
- undefinedzero 4y agoOr a simpler setup would be a smoke detector that turns off a power strip when triggered. Kind of like some 3d printers have.
- Hamuko 4y agoNot sure how useful that would be considering the fire has probably already started.
- undefinedzero 4y agoThings generally smoke before they burn
- masklinn 4y agoThe problem is not external, 500 or 1000 W is not that much in the grand scheme of things (a euro plug allows up to 625W, unearthed). The issue is internal: because the top PSU rail is only 12V, high-power devices (mostly GPU) need to push very high amperage totals. And because PC internal layouts, they can’t do that through large wires with huge connectors. Also a legacy of cheap friction fit.
- Eisenstein 4y agoExactly. This is why high wattage USB-C cables rely on the power supply negotiating a higher voltage in order to work. Sending 100W over 5V would be ridiculous (20A). Instead the device tells the power supply to switch to 20V and then the cable only needs to be rated for 5A. Something like that for GPUs would make sense.
- TylerE 4y agoIt is KINDA external. 1000+w is starting to get into “trip the breaker” pretty easily if you have anything else on that circuit with any kind of transient load - like say a minifridge.
- lmm 4y agoI mean that's half the power of a decent kettle...
- masklinn 4y ago> 1000+w is starting to get into “trip the breaker” pretty easily I was wondering where you'd get that from, and then I remembered that you're probably american so your baseline is only 1800W, in which case... yeah that's true. If you're installing a really powerful PC in the US you probably want either a dedicated circuit or a 20A circuit (or both).
- kennend3 4y ago> I was wondering where you'd get that from, and then I remembered that you're probably american so your baseline is only 1800W, in which case.. In North America, you cant pull 1800W continuous either, it is limited to 1440 by regulation. "This regulation is called the maximum continuous load, which is 80% of the total wattage computed. " This is why space heaters and such are 1200 watt. So while 120v * 15A is 1,800W no electrician would wire a setup like this. if you need 1800W they would use probably use a 20A circuit and wiring rated for 20A. 80% load of this would be 1920W. NOTE: Not an electrician.. if i am mistaken please let me know.
- izacus 4y agoA gaming PC uses maybe 20-30% of power of an average kitchen appliance like a kettle or a blender. Those are plugged into standard wall sockets just fine.
- Hamuko 4y agoYeah, a kettle is gonna pull like 2200 W when it's switched on. I'll be impressed if a gaming PC can pull half of that during a gaming load.
- nigerianbrince 4y agoFfs don't give them ideas.
- gruez 4y ago>Yeah, a kettle is gonna pull like 2200 W when it's switched on ...in areas that use 220/240V. In north america appliances are limited to around 1500W, because the mains voltage is 120V.
- andylynch 4y agoMight depend on where you are and how seriously they take tea. Our very ordinary British kettle is 3kW.
- jcpham2 4y agoDepends on how many GPUs are in the PC. Circa 2011 I had quad-Radeon 6970 machines pulling ~1000w each from 1200 watt power supplies, measured with a kill-a-watt. Those machines ran 24/7 for at least 36 months before I had to start servicing stuff like GPU fans and heat sinks. This stuff can be designed NOT to melt the dang 6 pin 12v rail
- jcpham2 4y agoDepends on how many GPUs are in the PC. Circa 2011 I had quad-Radeon 6970 machines pulling ~1000w each from 1200 watt power supplies, measured with a kill-a-watt. Those machines ran 24/7 for at least 36 months before I had to start servicing stuff like GPU fans and heat sinks. This stuff can be designed NOT to melt the dang 12v rail
- de6u99er 4y agoMaybe it's own power supply? Although this would require electric decoupling from the PC because of switching power supplies. I see here an opportunity for Jensen and his marketing people to come up with a fancy name for it.
- onethought 4y agoCountries like Australia/China 240v is the standard. You need step-down transformers for 110v devices. China has both voltages/plugs running in its cities - not sure how they manage that.
- mkl 4y agoMost of the world is on 220-240V: https://theverbalized.com/why-do-we-have-two-standards-of-electricity-supply/ https://theverbalized.com/why-do-we-have-two-standards-of-el... Also the power supply to a GPU is DC anyway, so the wall voltage probably doesn't matter much.
- sgjohnson 4y agoThe United States is also a 240V country. It’s just split phase. But yes, mains at the wall would be 120V.
- Covzire 4y agoA common misconception about the US is that the US is only 110V. 220V service in homes in America is standard, it's just that standard power plugs around the house get 110V because of split phase breaker boxes. This lets 220V be available for large appliances like Washers, Dryers etc using the full phases, and 110-120V to the rest of the house using only half phase. In my next house, I think I'll request a 220V outlet in my kitchen with a UK style plug, so I can have my fast tea kettle.
- onethought 4y agoDoesn't seem like a misconception. It seems like y'all have 110v power outlets. Having 220-240v somewhere else is kind of moot... The power socket IS the interface, so if the interface is 110v, then the switcher box, the power line, the high voltage power line, the power plants voltage aren't really relevant.
- onethought 4y agoHow can that be true? I plug a kettle cable from my a/c socket to the GPU... is there a rectifier involved in a standard PC setup?
- namibj 4y agoYeah, sending 48V would be more appropriate, but unfortunately that's not really backwards-compatible (well, the way to do that is typically to allow the PSU to be farther from the use-site, like having a datacenter with tall adult high cabinets (racks; they have 4 posts for holding the equipment, which is standardized as occupying one-or-more slots of 1¾ inch height and free to occupy the entire length) and putting 4 AC/DC converter blocks into each such cabinet, to power a few layers of servers/equipment both above and below the PSU layer. They do this so that the PSU can be made up of multiple redundant modules, and to integrate some batteries that can cover a generator starting up. This way they don't need to pay for a dedicated UPS battery with chargers and inverters just to convert it back to lower-voltage DC near the server. They can make it so that for example 5-8 PSUs spread over different electrical circuits can share the load of the servers, even if 2-3 of them fail (so they, for example, only need 2 extra on top of the 5 that can power the servers and the 1 that recharges the battery (62.5% load if everything works and the servers are running full-power), or even relying on the servers occasionally running below full power to spread the full rated server power over 6 modules with 2 spares (75% load if everything works and the servers are running full-power)). A big issue in this situation here is though that Nvidia skimps heavily on board space for the 4090 Founders Edition cards, seemingly to use the back for heat sink fins/air-flow cross-section. It would take up some board area, but in theory, they could add a 48V power option that makes it all sleek and pretty, possibly with a 12V->48V adapter for the people who don't already have a 48V supply.
- kube-system 4y agoOften, anything above 48 or 50 volts is considered high voltage and in some places might be subject to regulation. Not to mention just being more dangerous to handle.
- de6u99er 4y agoYour comment reminds me of this meme: https://static01.nyt.com/images/2016/08/05/us/05onfire1_xp/05onfire1_xp-superJumbo-v2.jpg https://static01.nyt.com/images/2016/08/05/us/05onfire1_xp/0...
- MarkusWandel 4y agoMore interestingly, modern ASICs run their logic at something like 1V (I don't know the exact voltage of this particular one of course), so the current going into the actual chip (after voltage downconversion on the board) is hundreds of amperes.
- kevingadd 4y agoAFAIK the top-end for modern Ryzen chips is in the 1.5-1.6V range, usual operating levels are lower. My 3xxx Threadripper is at 1.3V right now. In comparison, my 3xxx-series RTX card is at 1.06V (but only operating at 40% of max power). The EVGA software only allows me to set a +100mV voltage offset, but the default voltage curve looks like it peaks at 1.100V for ~1950mhz.
- willis936 4y agoThere isn't an issue with resistive heating. The safety margin is down a few factors from the 8-pin PCIe adapter, but there is still a healthy safety factor. https://twitter.com/hms1193/status/1585257428291325958 https://twitter.com/hms1193/status/1585257428291325958 This isn't like running kA of current where the threat of thermal runaway is always present, even with active cooling. I worked on a stellarator that had a coil feed blow (years before I started). Pushing even 10% more current meant a complete overhaul of the feeds. That's what thin margins looks like.
- Retric 4y agoIt’s only a healthy safety factor if absolutely nothing goes wrong. Connectors are normally a weak point and internal PC components aren’t supposed to be plugged and unplugged regularly. Simply testing the multiple hardware configurations can quickly result in failures.
- Varloom 4y agoWho's idea to feed monstrous 450W GPU with power using this small choke point?
- metadat 4y agoAlso discussed 2 days ago: Users report Nvidia RTX 4090 GPUs with melted 16-pin power connectors https://news.ycombinator.com/item?id=33327515 https://news.ycombinator.com/item?id=33327515 (116 points, 98 comments)
- dang 4y agoThanks! Macroexpanded: Nvidia’s hot adapter for the GeForce RTX 4090 with a built-in breaking point - https://news.ycombinator.com/item?id=33373729 https://news.ycombinator.com/item?id=33373729 - Oct 2022 (274 comments) Users report Nvidia RTX 4090 GPUs with melted 16-pin power connectors - https://news.ycombinator.com/item?id=33327515 https://news.ycombinator.com/item?id=33327515 - Oct 2022 (101 comments)
- terom 4y agohttps://news.ycombinator.com/item?id=33373729 https://news.ycombinator.com/item?id=33373729 details about the problematic 12VHPWR adapter shipped by nVidia, with extremely questionable engineering (14AWG wires soldered to thin tabs), which should absolutely be recalled as a fire hazard. I believe it is entirely possible to implement a solid and safe 12VHPWR connector, and its reputation is being destroyed by a shoddy implementation... but that's very much nVidia shooting themselves in the foot.
- echelon 4y agoMaybe they shouldn't have pissed off EVGA? https://news.ycombinator.com/item?id=32870677 https://news.ycombinator.com/item?id=32870677
- Kognito 4y agoWhat does this have to do with EVGA?
- raisin_churn 4y agoMaybe nothing, maybe EVGA, with their reputation for quality and sound engineering, would have looked at the garbage adapter cable Nvidia shipped to board partners and said "Hell no, we're not shipping that to customers" and sounded the alarm on it before thousands of these got into people's homes and offices where they may yet kill someone. I don't personally like EVGA, had a bad experience trying (and failing) to get warranty replacement for a defective PSU, but I can certainly follow GP's thinking about how EVGA's involvement with this launch might have made a difference.
- echelon 4y agoThat was my thinking. I'm sure Nvidia will get it sorted in time, but they're trying to do a lot right now, and it's noticeably bumpy.
- incrudible 4y ago
- ilyt 4y agoI know it's not a "problem" but I wonder why they didn't just went with some actual high power connector instead of just adding pins over and over again. XT60 have been well tested in RC industry and it's 720W sustained and 3x that in peak. Just add sense connector and you're done
- Eisenstein 4y agoComputer power supplies have (usually) multiple rails, which are multiple voltage regulator circuits outputting the same voltage. You wouldn't be able to do that with two wires. Also, I think the connector needs to be larger to distribute load, and the ATX style pin connectors are already used and no one wants to add a completely new connector type on their assembly line.
- formerly_proven 4y agoModern ATX PSUs are all single-rail, including the highest power ones. E.g. Seasonic Prime 1600 W produces over 130 A on its singular 12V rail.
- Karliss 4y agoThe current connector 12VHPWR connector already joins all the wires together on GPU end of connector. So multiple PSU power rails isn't an argument why they couldn't have used two beefier wires and connector.
- namibj 4y agoWhy even sense connector? Like, what's that aspect needed for, here? I understand it for the lower-voltage rails, but the 12V GPU bulk power supply shouldn't need remote sensing, just mandate appropriate limits on voltage drop and rate cable resistance in a way that DIY builders can test their parts (not like we need high accuracy here, just need to 4-wire measure resistance to like +-10%). But to answer your question on "why adding pins": electrical regulations regarding fire risks eschew PSUs that don't trip over current protection when you hit like 20-ish A on a single output, due to the risk of starting a fire if you were to dump that much power into a small place and happen to have enough resistance in that "short" to not trip the over current protection. That's why GPUs feed multiple power rails from the PSU and control themselves to balance their draw across their input rails to not overload any single input. And IMO as long as we let people build computers in cases that won't protect the room/building from e.g. a broken capacitor on the GPU's 12V side drawing 600W until the GPU is burnt to a crisp and the case fan has blown the flames out the back against the e.g. wooden shelf on/near the desk, we maybe should keep up the regulations that such fire-non-containing cases can't use PSUs where individual power rails willingly deliver enough power for an electrical fire to escape the case.
- evanb 4y agoWhen the BlueGene/Q came online for testing and early science in Livermore, some of my colleagues burned out a number of power supplies. IBM wanted to make sure they weren't doing some "trivial" task; I guess while(true){x+=1} or something? No, they were really running lattice QCD calculations. Anyway they were not prepared for kernels in hand-optimized assembly, and IBM had to beef up the power supplies.
- formerly_proven 4y agoProblems like that seem to plague almost every new platform.
- smcl 4y agoI don't think the problem here is that users are running heavier calcs on their cards than nvidia intended - there's been a teardown showing that some of the plugs on those power cables have weak connections
- Manu40 4y agoMy landlord recently told me about his other tenants have 600$ power bills before swapping to a locked rate. I wonder if the lad with the epic gaming PC is running a 4090. Should I tell him this is a possibility if he is?
- deleted 4y ago[deleted]
- colechristensen 4y agoSeems very much like a situation where you should keep out of other people’s business.
- Manu40 4y agoRight. So when the house somehow burns down and it turns out the fella's super rig had one of the first 4090's; am I supposed to go "Oops. Sorry landlord. I knew this GPU had issues, and you asked me how much power these sorts of things use, but I decided to ignore the possible consequences; just to keep my nose out of others business?" How about the families of the 5 people in that house? What should they be told? "Sorry folks. I knew this GPU has issues and could possibly cause a fire, but people on the internet decided I was a bad person for doing the right thing, so I ignored the potential issue and didn't let the landlord know." I hope you have a great life.
- colechristensen 4y agoYeah, no. Every gamer with a nosy neighbor does not need to be “protected” by people like you telling the landlord they’re going to burn the house down. Mind your own business.
- Manu40 4y agoExcept it was made my business when the landlord asked my opinion on why their house was using so much power, and he told me they had a high end gaming rig, multiple fish tanks, and Air conditioners going.
- frou_dh 4y agoSeems like a bit of an insult to the colleagues that manage to successfully design a 50+ billion transistor GPU that their product gets messed up by something like this. Talk about the last mile.
- colechristensen 4y agoThis is what failure is like, there’s a lot of surface area and sometimes it’s things you never thought of worrying about. I would bet the cause coming down to minor manufacturing defects and testing that was “too good” at plugging things in not accounting for sloppy end user assembly. Combined with good old inadequate safety margins.
- adhesive_wombat 4y agoI agree with that, but, glibly, I'd argue with this: > things you never thought of worrying about. In my experience, it's nearly always the connectors that you suspect first.
- lazide 4y agoAlso, power supply design (including power cables) is a really foundational piece for any electronic device, and the one that typically causes complete failure/letting the blue smoke out if not done carefully. And of course, safety/fire issues if screwed up particularly badly. It is weird that in their push to all time high TDP they didn’t spend more time thinking about it. That said, cutting edge often means bleeding edge, something something.
- amelius 4y agoYou are right of course, but that 50+ billion transistor design is mostly copy-pasted from a small number of blocks. For example, the RTX 4090 has 16,384 cores, so you have to divide 50+ billion by that number. Then inside every core you have lots of similar repetition, etc. The biggest challenge is to create a die that size with little to no defects, which is of course the foundry's achievement.
- jiwidi 4y agohttps://www.youtube.com/watch?v=-NGUov5Zb_0 https://www.youtube.com/watch?v=-NGUov5Zb_0 Here there is a vid about what is the issue
- snvzz 4y ago
- dvngnt_ 4y agodoesn't matter what AMD is going to do if you value ray tracing, dlss, rtx voice, shadowplay, good drivers
- snvzz 4y ago>if you value ray tracing AMD has Ray Tracing, too. In RDNA3, it might even perform really well. Nov 3rd is close, and we'll know by then. >dlss Current FSR (open, works on AMD, NVIDIA and Intel) is considered equivalent by reviewers. Differences exist, but neither is better than the other. These differences are now up to individual taste. >rtx voice AMD Noise Suppression. >shadowplay AMD ReLive. But you should be using OBS: It's better than either, and open source. That's what most streamers use, pro and else. >good drivers AMD are better in my experience. YMMV.
- dvngnt_ 4y agoI've never seen a single reviewer say FSR is equal/better than DLSS especially now with 3.0. I would like to see a source for that. I do like that it's open. AMD's ray tracing isn't good yet and is years behind NVIDIA
- Aerroon 4y agoEven if you use OBS nvenc is way better than AMD's offering. You'd just use nvenc through OBS, unless you want to dedicate half of your CPU to recording. Even if you do that you'll still take a performance impact in anything else you're doing.
- snvzz 4y agoAMD's hardware encoder can of course be used with OBS, and it is actually good as of OBS 28.
- nigerianbrince 4y agoNow that high end cpus are pulling as much power as gpus, are they going to cause this issue as well?
- Varloom 4y agoCPUs are not fed 600W through a tiny overloaded connector.
- saati 4y agoWhich CPU would that be? An i9-13900K has a max power draw of 253W, a 3990X is 280W, these are both peak values, the 4090 peak is 600W.
- philliphaydon 4y agoIt draws more than 253 tho.
- nigerianbrince 4y ago350
- dontbenebby 4y agoIs it unusual to have that severe an issue with a power cable? I thought the whole point of going direct to the manufacturer or someplace less shady than Amazon like Monoprice. (I haven't ordered from them in a while, but while they don't ship as fast as Amazon I never had a failure, versus Amazon constantly selling me shit that breaks, then demanding I go print a shipping label and send them their less than twenty dollar hunk of whatever -- it felt like a very purposeful strategy to exhaust folks who were more than willing to do some research. Then again, I have an uneasy alliance with "the gamers" since I switched to being a film snob in 2009 -- that's why they took out all the payphones -- it's a long story, but apparently someone formally complained to the FBI that I said they should ring the Vatican with Italian tanks and start shelling if they don't join NATO and pay the victims, so they're putting off legalizing Jenkem and taxing it for revenue for another 69 or so years.)
- deng 4y agoGamers Nexus has released their video on the issue: https://www.youtube.com/watch?v=EIKjZ1djp8c https://www.youtube.com/watch?v=EIKjZ1djp8c In short: They couldn't replicate it, no conclusion yet, further testing needed. What's weird is that all their adapter cables are clearly different (better) than the one from "Igor's Lab".
- phreenet 4y agoFor anyone who doesn’t want to watch the video the key differences that made Gamers Nexus cables better than what Igor showed was GNs cables were 300V rated vs 150V and the solder joints where the cable meets the connector bus bar was higher quality.
- newsclues 4y agoDifferent suppliers? Igor is EU and Gamers Nexus is American so perhaps different adapters for different regions?
- Someone1234 4y agoSomeone in /r/nvidia in the US got one similar to the Ignor one, so it isn't clear if it will be regional or not.
- newsclues 4y agoIf so, that seems like perhaps it was an early revision that was corrected. Which is problematic because the situation was not fully handled.
- deleted 4y ago[deleted]
- bob1029 4y agoImagine trying to run a LAN party at a typical residence with a room full of modern, high-end gaming PCs... I thought we were saved on power consumption when we went from CRT to LCD, but nothing can compensate for the madness we are working with now. Back in the early 00's, I've had 20A breakers trip with ~8 high-end PCs on the same circuit. Today, I think we would struggle to handle half the number of PCs. My entire panel is only good for 150A of service, so I am wondering if I could even fill up a battlefield match before my main trips.
- Retric 4y agoThe high end is down from the peak. Quad SLI GTX 980 used to require a separate power supply just for the graphics cards.
- oblvious-earth 4y agoI think it depends how you count it. According to this setup their under load power consumption of the entire system was just shy of 700Ws https://pcper.com/2014/10/nvidia-gtx-980-3-way-and-4-way-sli-performance/10/ https://pcper.com/2014/10/nvidia-gtx-980-3-way-and-4-way-sli... Back then components didn't tend to be as power bursty so I think it matters whether you compare peak power consumption or average power consumption under "typical" load. A wall load from a gaming PC with a 4090 and 13900k is definitely going to burst passed 700Ws, but average while playing a game is probably lower than that.
- Retric 4y agoYea, Stock GTX 980 wasn’t that bad however the GTX 980 Ti was significantly more power hungry. That said, that review mentioned several very power hungry SLI setups.
- jhot 4y agoThe high end these days is insane. Huge diminishing returns in performance/watt and performance/cost. My current mid-range gaming PC (Ryzen 5600X, Radeon 6700XT) uses about the same amount of power as my 2008 mid-range PC (Core2 Q6600, GeForce 8800 GTS 512) while of course running laps around it in performance. Unfortunately this current generation across the board seems to be throwing GHz at the problem as if we didn't learn our lesson from Pentium 4.
- mansilladev 4y agoJust looking at the article photos of the burnt connectors had tricked my brain into smelling the melted/burning plastic. And if you ever worked on your own rig, it's the worst smell ever.
- js2 4y agoWhen I built my most recent NAS, I used a bunch of Molex to SATA power adapters. Then I came across posts about how they are liable to short circuit: https://www.crucial.com/support/articles-faq-ssd/dangerous-molex-to-sata-cables https://www.crucial.com/support/articles-faq-ssd/dangerous-m... https://old.reddit.com/r/buildapc/comments/9mq6e4/molex_to_sata_is_it_safe/ https://old.reddit.com/r/buildapc/comments/9mq6e4/molex_to_s... That was cause for concern so I removed them all and used splitters instead so I didn't need any of those adapters.
- mistrial9 4y agothe only fire I have ever had in a local box was exactly this connector type - cheap, from China. I had no warning it was a hazard at the time, busy with thoughts of my (data science) project at hand. The event was of course at the wrong time, and could have been very, very serious. Please know your connectors !
- natch 4y agoFor a little more than the price of a 4090 you can get double the memory with two 3090s. I’m still struggling to understand which is the better deal but the lower power draw of 3090 is nice as well. And I mean just 3090, not 3090 Ti.
- dinvlad 4y agoCurious what RDNA 3 will have to offer, and how they would approach the same problem.
- noahtallen 4y agoAMD has said they won’t use the new power cable, and since the existing approach of using 2x or 3x 8pin works fine, I imagine they’ll continue doing that :) They are also rumored to be more power efficient which would help things