4 ms·
Dell exec reveals Nvidia has a 1,000-watt GPU in the works
- bayindirh 3y agoThe big question is "Will it blend (with the datacenter dynamics, power distribution and cooling)?". Considering a conservative 4GPU system adds a 1.2KW additional overhead, packing these together densely in racks is a huge ordeal.
- agilob 3y agoThe big questions is, my heat pump is 2.6 kW, can I remove the heat pump and run 2 of these GPUs instead?
- TeMPOraL 3y agoIf you find a way to extract compute from the work done by a compressor, sure. But then you'll be able to afford the heatpump and any number of GPUs, for the money you get from overturning laws of thermodynamics.
- agilob 3y agoI better start working on it then!
- c_o_n_v_e_x 3y agoConsidering that heat pumps have a coefficient of performance of greater than 1 (meaning they are moving more than 2.6 kw of heat), no. You'd probably need 3 to 5 GPUs to get comparable levels of warmth.
- michaelt 3y agoThese days you can already get water-cooled servers [1] and rack-mount water distribution manifolds [2] [1] https://www.supermicro.com/en/solutions/liquid-cooling https://www.supermicro.com/en/solutions/liquid-cooling [2] https://www.ibm.com/docs/en/power8?topic=POWER8/p8had/p8had_83x_watercool.html https://www.ibm.com/docs/en/power8?topic=POWER8/p8had/p8had_...
- bayindirh 3y agoI mean, we have these, but changing to a bigger CDU just because you decided to get a rack of GPU servers is not always fun/easy/doable. Even if you just upgrade them, then there's the primary loop's capacity.
- agilob 3y agoI can't find it anymore, but I remember reading a few years ago about a watercooled server farm that used an outdoor water fountain. It wasn't a single rack, but a few 100s of them
- Timber-6539 3y agoLooks like a solution in search of a problem.
- bayindirh 3y agoAI workloads are currently going through their "developers are expensive, hardware is cheap(ish)" era. Nobody is trying very hard to fit models to GPUs, and optimize utilization and whatnot. Instead they throw in more GPUs. So B100 will fit right into the bill.
- hardware2win 3y ago>"developers are expensive, hardware is cheap(ish)" Is it even true? H100 is worth 1 year of salary of mid engineer in eastern eu - 40k usd before tax
- bayindirh 3y agoGenerally it's thought in the hourly cost, plus how much productivity it adds to your workflow. IOW, you pay yearly price of an engineer, plus some hourly cost, but it generally adds more productivity rather than optimizing what you have at hand. > H100 is worth 1 year of salary of mid engineer in eastern eu If you can have them supplied to you, in the first place.
- dpacmittal 3y agoThere's a lot of work being put in to make LLMs run on consumer hardware, albeit it's being done by the open source community and not the companies themselves.
- yau8edq12i 3y agoNice, we'll be able to consume the Earth's resources even faster. Congrats everyone!
- eru 3y agoHuh? This is an AI accelerator card. So you should be looking at performance per Watt, instead of absolute numbers. The linked article doesn't mention anything about that.
- amelius 3y agoYeah so they should increase the efficiency instead of the power.
- epolanski 3y agoWell, there's no yeah, because there's no ratio... For all you could know energy went twofold and performance threefold..
- amelius 3y agoThey said nothing about performance, and if you think it increased without them saying anything about it, then you're being highly optimistic. Right now I can only assume they made it easier to burn more energy.
- hulitu 3y ago> They said nothing about performance 1kW says a lot about perfornance. You can heat a house with it. /s
- hulitu 3y agoYes, but this usually happens when they cannot increase the power anymore. (remember CPUs ?)
- hresvelgr 3y agoWith the recent breakthrough in ternary weight models it'll be interesting to see how needed this kind of bruteforce power will be.
- throwaway4good 3y agoYou would still use floating point for training afaik.
- Aardwolf 3y agoWhy not have both software and hardware improvements? This is like saying we shouldn't have any new CPUs after the the PDP-1 because quicksort got invented
- TeMPOraL 3y agoYeah, there's this weird notion that OSS work on running LLMs on consumer hardware, and researching more efficient architectures, is somehow in opposition or competition to the Evil AI Wielding Corporations. It's not. Like always, whatever tool an individual has, a corporation can afford to use more of it, and use it better.
- hresvelgr 3y ago> This is like saying we shouldn't have any new CPUs after the the PDP-1 because quicksort got invented I didn't assert anything here, I merely pondered. However, lets pretend that I did say that we shouldn't need heavy GPUs anymore. I don't think that's an entirely bad take because it would be fantastic if we could run larger models with more parameters on lower spec hardware. Especially phones that aren't flagship models.
- Aardwolf 3y agoSure, the analogy was a bit exaggerated perhaps :) I also of course didn't necessarily want to imply whether it's actually a good idea to have a 1000W hogging monster card, vs e.g. putting more memory on consumer cards
- bigpaw 3y agoThose dedicated gpus draw way too much power. Why isnt there more focus on low power cards? Same on cpus. Electricity prices are set to drastically increase in my area.
- deleted 3y ago[deleted]