5 ms·
Enterprises in for a shock when they realize power and cooling demands of AI
- grajaganDev 2y agoThere is not enough water or power to support the current AI stock prices.
- crimsoneer 2y agoIt's worth recognising that a year ago gpt-4 was the cutting edge, and today you can now run a gpt-4 class model on a macbook pro (with lots of ram, sure).
- EVa5I7bHFq9mnYK 2y agoA year ago gpt-4 was the cutting edge, the new cutting edge model takes 11.2 MWh of energy to answer a single question (enough to power a home for 4 years).
- PittleyDunkin 2y agoYea but gpt-4 is also only very slightly marginally useful in terms of productivity as it stands today. I highly doubt we're going to get returns on this without huge structural changes to how chatbots work.
- crimsoneer 2y agoSure, I think that's fair - the tech is only one part of the whole equation. Just pointing out that power demands are really quite a small (and actually quite surmountable) part of the problem.
- gopher_space 2y agoIt's interesting to look at how hardware requirements change alongside acceptable response time. Knowing when you need a reply seems like it might save a lot of money.
- agilob 2y agoThe executive order talks about building new geothermal power plants to support AI infra
- PittleyDunkin 2y agoHow about we cut the AI bit and just get a nice C02 decrease, i.e. the actual thing that will support future humans? THat's one more coal plant we can shut down.
- grajaganDev 2y agoWhat about the water?
- codingwagie 2y agoWhat is it with society producing content that predicts the downfall of some new trend
- AnimalMuppet 2y ago"If something cannot go on forever, it will stop." - Herb Stein Point is, we can look at the situation, and recognize "that can't go on forever". And then we can say so, sometimes in public places. And there's nothing wrong with that.
- mlepath 2y agoAll companies that are doing something with the current iteration of "AI" are underwater. Sam Altman says that the pro tier of ChatGPT is losing money, Adobe is losing tons on firefly, ... This is pretty typical for silicon valley though, we always burn investor money to corner the market and then tech usually catches up. Most enterprises don't need to be first adopters.
- __loam 2y ago"We don't make money off the $200/mo option" is embarrassing.
- ForHackernews 2y agoAh, the WeWork strategy...
- DeepYogurt 2y ago> This is pretty typical for silicon valley though, we always burn investor money to corner the market and then tech usually catches up. Yes though the cost breakdown has traditionally been large upfront development costs and low to moderate running costs. This time around the running costs are astronomical and Moore's law ain't what it used to be.
- enragedcacti 2y agoIt seems like a repeat of Uber's play but at 10x scale. Lose 10s of billions of dollars scaling a product that loses you money on every sale in the hopes that it will position you well for the massive disrupter of [self-driving | AGI]. Uber's play didn't shake out so now they are very very slowly digging out of a 30bn hole. I guess its just a question of how big the AI hole gets before they either make AGI or give up and start shoveling.
- marcosdumay 2y agoWhile there is no reasonable explanation for why Uber can't just turn a profit (and it looks like they have for the last few years), deep-learning models have very hard physical constraints on how much they cost.
- nbuujocjut 2y agoWill most organisations be building out their own AI Infrastructure? I would guess in most cases they would either use APIs to existing models (OpenAI etc) or use cloud providers where they need to run and train their own models. Is there a risk that cloud providers cannot meet the demand?
- pavel_lishin 2y agoCertain companies have very strict data handling requirements, and can't just send data off-site.
- cptskippy 2y agoYes, and those companies are a minority. I wouldn't be surprised if AI providers developed solutions for them. Almost immediately we saw tinybox emerge, old Tesla cards doubled and tripled in price on eBay, and Nvidia announced Digits so there's definitely interest in offline solutions.
- deleted 2y ago[deleted]
- PittleyDunkin 2y ago> Is there a risk that cloud providers cannot meet the demand? Waste? Cloud providers are super expensive and can't generally justify this expense with the benefits of outsourcing.
- ForHackernews 2y agoDon't worry, they can just convert all those Bitcoin mining racks to AI. Er, wait... what do you mean it only does SHA256?
- the_sleaze_ 2y agoBitcoin miners are purchased with full expectation of a life-span and an ROI within that, just like all data centers. Not sure why you're bringing this up anyways.
- add-sub-mul-div 2y agoSame reason we teach history.
- EVa5I7bHFq9mnYK 2y agoBitcoin miners provide a valuable commodity - secure and unrestricted storage and transfer of value. AI provides ugly cat pictures, bad code and a lot of spam.
- janderson215 2y agoYeah, AI has never produced anything of value and nobody in the crypto world has ever scammed anybody.
- EVa5I7bHFq9mnYK 2y agoYeah, careful around this internetz thing - someone scammed somebody over there, I've heard.
- mattlondon 2y agoWhich enterprises will seriously be rolling their own and not just using cloud? All the clouds offer sovereignty controls now IIRC. If your data is so sensitive that it can't even leave the building, then you're probably going to have a bad time using it with your own home-grown AI solutions due to hallucinations.
- crimsoneer 2y agoyeah, this whole article makes no sense. Nobody is going to be running their own datacentres, everyone is just going to be running it on their existing aws/gcp/azure.
- philipwhiuk 2y agoDepends how much AI you're doing. At some point cloud no longer makes economic sense. Additionally Finance needs to co-locate compute for latency reasons meaning there's a limit to what you can off-load to the cloud.
- add-sub-mul-div 2y agoAnd the respective cloud providers will absorb the increased cost of this power usage forever?
- deleted 2y ago[deleted]
- croes 2y agoThanks to the CloudAct there is no sovereignty. Why are homegrown hallucinations worse than cloud hallucinations?
- mattlondon 2y agoLess training than someone like Google that has ~everything and are pouring billions into research. I don't think anyone apart from a FAANG-like corp really stands a chance in keeping up with SOTA.
- sebazzz 2y agoIts pretty ironic though. Last few years were focused on reducing CO2 output by reducing power usage etc. Now with all the AI hype we suddenly don't hear so much about that.
- oytis 2y agoBut once we have AGI it will solve climate change forever. Probably.
- anticorporate 2y agoI hope this is /s
- ajmurmann 2y agoSalvatore Sanlfilippo, here between known as Antirez, wrotea book about humans attempting exactly this: https://www.goodreads.com/book/show/61657571 https://www.goodreads.com/book/show/61657571 Fun read!
- smallmancontrov 2y agoBy killing all the humans? Or by setting up nice enclaves for the owners and keeping the rest of in squalor with a swarm of murderbots?
- PittleyDunkin 2y ago...if AGI is even a meaningful concept. I'm deeply skeptical.
- exe34 2y agodo you not believe human level intelligence is possible?
- marcosdumay 2y agoHuman level intelligence is clearly not enough to solve global warming all by itself just by existing. What people mean when they say "AGI" is something different.
- Vox_Leone 2y agoDecentralizing computation [making it mostly on-premises again] could potentially mitigate some of the problems the article points out, in several ways: *Localizing Power Usage*: By moving some computational tasks to local systems (on-premises), you could reduce the reliance on large, centralized data centers, which are often constrained by power and cooling limitations. This would also reduce the need for extensive retrofitting of data centers to support higher power densities. *Efficiency Gains*: Decentralized computation, especially for non-essential services, could be more energy-efficient on a per-unit basis. Local infrastructure might allow for better optimization of energy use. Smaller, distributed systems could have specialized power and cooling needs that are more manageable and tailored to the task at hand. *Reduced Data Transmission and Latency*: With computing spread out across local or edge-based facilities, energy costs could be further reduced by minimizing the need for long-distance data transmission, which itself requires significant energy, especially with AI workloads.
- sobellian 2y agoI usually have the opposite intuition, that centralizing gives greater economies of scale. If a DC could be made more efficient by splitting it into N micro-DCs, then couldn't AWS just do that? That they do not seems telling. AFAICT centralization offers strictly more freedom in the solution space than if everyone rolls their own micro-DC.
- bokohut 2y agoCan we proactively speculate that as businesses port to use artificial intelligence at an ever increasing rate in time they will remove human roles in their short term efforts to decrease business costs? In reality however over the longer term they will just be locking themselves into a greater expense of energy, either indirect from cloud use or directly when buying from a centralized energy provider. With salaried employees a business can “excuse away" their raise or fire them for another person that will take equal pay however one must pay their cloud/energy bill to operate tomorrow. Are we as a species to be unintelligent and artificially believe that prices will not continue to rise, most certainly for energy? Interesting immediate times as the technology Kool-A.i.d propagates with little regard for the long term impacts.