4 ms·Energy efficient LLM inference (for now) is as realistic as existence of perpetual motion.by bayindirh 1y agoEnergy efficient LLM inference (for now) is as realistic as existence of perpetual motion.qwertox 1y agoI wouldn't mind if it burns 200 watts while it does the task, as long as it idles at below 30Wbayindirh 1y agoNVIDIA H200 idles at 75 watts. I'm not keeping my hopes high on that, either.janstice 1y agoTo be fair, if I paid $30k+ for an H200, I’d want it to be making money 24/7 rather than idling, so the idle power draw would be strictly theoretical.bayindirh 1y agoIt's not always about money.