3 ms·

Yeah, an incredibly dumb analysis. I did a back of the hand estimate based off of the following assumptions: gpt-4o: 10 H100s = ~7000W power requirement, 30 s

Yeah, an incredibly dumb analysis. I did a back of the hand estimate based off of the following assumptions: gpt-4o: 10 H100s = ~7000W power requirement, 30 seconds to generate a summary for an hour-long call human: ~10.7 kWh/year energy requirement (avg per American), 20 minutes to generate the same summary The numbers come out to 60 Wh for 4o and 400 Wh for a human

claytongulick 2y ago
Except that the human will be using that energy regardless, it's sunk cost. The big question is whether the additional 60Wh are worth it.
- chaos_emergent 2y ago
  I agree! I think it’s “worth it”. A parallel: it costs an OOM more to drive places than, say, bike places, but it allows us to do the things we did before an OOM faster and expands the types of things we can do. You can make the argument that it’s not “worth it” in terms of cost to our environment, but you have to acknowledge the upside of it simultaneously. And the dynamics are different in that we’ve reached an efficiency asymptote with cars and are still in the middle (or perhaps at the start) of the S curve with AI systems.
FireBeyond 2y ago
Speaking of suboptimal analyses. The human isn't a single core, 100% pegged on that task for those 20 minutes. Even actively working at something that energy consumption is likely to be less than 10% of that. There's a whole lot of energy being burned by the cardiovascular system, respiratory, digestive and renal, endocrine, muscular system (all that typing, moving and focusing of the eyes), and we haven't even got to the energy of the "OS" execution in the brain to control all that. I don't think your numbers are the insight you think they are, nor accurate - not even particularly close.
- chaos_emergent 2y ago
  We're neither disembodied brains nor bodies in isolation of basic needs like shelter or desires for energy-intensive conveniences, so I challenge that we should try to assess energy consumption in the way you're proposing. The outcome and the inputs required to get to it are the two relevant variables IMO. Being consistent, though, I should've included the amortized cost of training and the energy cost of the rest of the data center to the 4o analysis. I don't think that those costs would be more than a factor constant greater. When we can create disembodied brains that don't have needs for shelter and desires for energy-intensive conveniences, I buy that we can do 10% of the energy accounting.

claytongulick 2y ago

Except that the human will be using that energy regardless, it's sunk cost. The big question is whether the additional 60Wh are worth it.

chaos_emergent 2y ago
I agree! I think it’s “worth it”. A parallel: it costs an OOM more to drive places than, say, bike places, but it allows us to do the things we did before an OOM faster and expands the types of things we can do. You can make the argument that it’s not “worth it” in terms of cost to our environment, but you have to acknowledge the upside of it simultaneously. And the dynamics are different in that we’ve reached an efficiency asymptote with cars and are still in the middle (or perhaps at the start) of the S curve with AI systems.

FireBeyond 2y ago

Speaking of suboptimal analyses. The human isn't a single core, 100% pegged on that task for those 20 minutes. Even actively working at something that energy consumption is likely to be less than 10% of that. There's a whole lot of energy being burned by the cardiovascular system, respiratory, digestive and renal, endocrine, muscular system (all that typing, moving and focusing of the eyes), and we haven't even got to the energy of the "OS" execution in the brain to control all that. I don't think your numbers are the insight you think they are, nor accurate - not even particularly close.

chaos_emergent 2y ago
We're neither disembodied brains nor bodies in isolation of basic needs like shelter or desires for energy-intensive conveniences, so I challenge that we should try to assess energy consumption in the way you're proposing. The outcome and the inputs required to get to it are the two relevant variables IMO. Being consistent, though, I should've included the amortized cost of training and the energy cost of the rest of the data center to the 4o analysis. I don't think that those costs would be more than a factor constant greater. When we can create disembodied brains that don't have needs for shelter and desires for energy-intensive conveniences, I buy that we can do 10% of the energy accounting.