3 ms·

It's very doubtful that they'd have any kind of magical breakthrough that makes the model anything other than incrementally better right now.

sosodev 1y ago
How do you figure? They’ve hinted that the reasoning breakthrough used to achieve gold in the IMO will be here in GPT-5.
- andai 1y ago
  The gold which Google won too, right?
- gowld 1y ago
  What breakthrough? The self-awarded "gold" IMO result was achieved by running the model for over 1hr per question.
  - sosodev 1y ago
    That sounds like a breakthrough to me. I don’t think GPT-4 could accomplish the same thing given several hours to try.
  - vlovich123 1y ago
    Said another way, 30 min less than what humans get? It’s on average 90 min per question.
    - mtlmtlmtlmtl 1y ago
      And how much energy does a human being consume while spending 90 minutes on an IMO question?
      - jjmarr 1y ago
        Probably more. 200 kcal (a shrinkflated bag of chips) is about 232 watt hours. A typical 4o query is 0.3 to 3 watt hours. https://epoch.ai/gradient-updates/how-much-energy-does-chatgpt-use https://epoch.ai/gradient-updates/how-much-energy-does-chatg...
        SketchySeaBeast 1y ago
        But how much time does that 0.3 watt hour query take to run? They imply that an individual ChatGPT query takes 0.3-3 watt hours, but most queries come back in seconds, so we need to scale that over a whole hour of processing. Edit: Scrolling down: "one second of H100-time per query, 1500 watts per H100, and a 70% factor for power utilization gets us 1050 watt-seconds of energy", which is how they get down to 0.3 = 1050/60/60. OK, so if they run if for a full hour it's 1050*60*60 = 3.8 MW? That can't be right. Edit Edit: Wait, no, it's just 1050 Watt Hours, right (though let's be honest, the 70% power utilization is a bit goofy - the power is still used)? So it's 3x the power to solve the same question?
- famouswaffles 1y ago
  No Sam explicitly said that breakthrough wouldn't be in GPT-5

sosodev 1y ago

How do you figure? They’ve hinted that the reasoning breakthrough used to achieve gold in the IMO will be here in GPT-5.

andai 1y ago
The gold which Google won too, right?
gowld 1y ago
What breakthrough? The self-awarded "gold" IMO result was achieved by running the model for over 1hr per question.
- sosodev 1y ago
  That sounds like a breakthrough to me. I don’t think GPT-4 could accomplish the same thing given several hours to try.
- vlovich123 1y ago
  Said another way, 30 min less than what humans get? It’s on average 90 min per question.
  - mtlmtlmtlmtl 1y ago
    And how much energy does a human being consume while spending 90 minutes on an IMO question?
    - jjmarr 1y ago
      Probably more. 200 kcal (a shrinkflated bag of chips) is about 232 watt hours. A typical 4o query is 0.3 to 3 watt hours. https://epoch.ai/gradient-updates/how-much-energy-does-chatgpt-use https://epoch.ai/gradient-updates/how-much-energy-does-chatg...
      - SketchySeaBeast 1y ago
        But how much time does that 0.3 watt hour query take to run? They imply that an individual ChatGPT query takes 0.3-3 watt hours, but most queries come back in seconds, so we need to scale that over a whole hour of processing. Edit: Scrolling down: "one second of H100-time per query, 1500 watts per H100, and a 70% factor for power utilization gets us 1050 watt-seconds of energy", which is how they get down to 0.3 = 1050/60/60. OK, so if they run if for a full hour it's 1050*60*60 = 3.8 MW? That can't be right. Edit Edit: Wait, no, it's just 1050 Watt Hours, right (though let's be honest, the 70% power utilization is a bit goofy - the power is still used)? So it's 3x the power to solve the same question?
famouswaffles 1y ago
No Sam explicitly said that breakthrough wouldn't be in GPT-5