4 ms·

From the nvidia page, > 80 billion transistors > Hopper H100 .. generational leap > 9x at-scale training performance over A100 > 30x LLM inference throughpu

From the nvidia page, > 80 billion transistors > Hopper H100 .. generational leap > 9x at-scale training performance over A100 > 30x LLM inference throughput > Transformer Engine .. speed .. 6x without losing accuracy So another monster chip - same size of the Apple M1-max thingy .. I guess it comes down to pricing. The A100 is already ridiculously expensive at $10K. They can this one at $50K and it would sell out?

p1esk 5y ago
You can buy A100s in a server today, a number of integrators will happily sell it to you.
- chockchocschoir 5y ago
  As someone who've tried for some weeks, it really seems like it's out-of-stock literally everywhere. The demand seems to be a lot higher than the supply at the moment, so much that I'm considering buying one myself instead of renting servers with it.
  - p1esk 5y ago
    Did you check Lambda or Exxact?
    - chockchocschoir 5y ago
      Yes, nor Lambda Labs or Exxact Corporation have them available last time I checked (last week). Both citing high demand as the reason for it being unavailable.
      - sabalaba 5y ago
        We (Lambda) have all of the different NVIDIA GPUs in stock ---- can you send a message to sales@lambdalabs.com and check in again with your requirements? We're seeing a lot more stock these days as the supply chain crisis of 2021 comes to an end.
        chockchocschoir 5y ago
        I talked with you (Lambda Labs) just a week ago about the A100 specifically and you said that the demand was higher than the supply, and that people should check once a day or something like that to see if it's available in your dashboard. If you clearly have it available now, please say so outright instead of trying to push some other offer on me in emails :)
      - asciimike 5y ago
        Howdy, I run [Crusoe Cloud](https://crusoecloud.com/ https://crusoecloud.com/) and we just launched an alpha of an A100 and A40 Cloud offering--we've got capacity at a reasonable price! If you're interested in giving us a shot, feel free to shoot me an email at mike at crusoecloud dot com.
  - nqnielsen 5y ago
    And even if vendors does say they have it, or can get it, it ended up taking us 4-6 months before systems were online.
  - kovek 5y ago
    Does it make sense that all the GPUs are bought out? They each provide a return for mining in the short-term. In the long term, they can be used to run A(G)I models, which will be very very useful
    - fennecfoxen 5y ago
      This is the GPU the parent is talking about https://www.nvidia.com/en-us/data-center/a100/ https://www.nvidia.com/en-us/data-center/a100/
      - kovek 5y ago
        This still makes sense! TPUs are useful for AI, which itself will be very very useful. It’s almost like it’s the best investment. That’s why smart players buy them all. Maybe I’m going out-of-topic.
Uehreka 5y ago
Most people who use one of these will be doing so through an EC2 VM (or equivalent). Given that cloud platforms can spread load, keep these GPUs churning close to 24/7 and more easily predict/amortize costs, they’ll probably buy the amount that they know they need, and Nvidia probably has some approximately correct idea of what that number is.

p1esk 5y ago

You can buy A100s in a server today, a number of integrators will happily sell it to you.

chockchocschoir 5y ago
As someone who've tried for some weeks, it really seems like it's out-of-stock literally everywhere. The demand seems to be a lot higher than the supply at the moment, so much that I'm considering buying one myself instead of renting servers with it.
- p1esk 5y ago
  Did you check Lambda or Exxact?
  - chockchocschoir 5y ago
    Yes, nor Lambda Labs or Exxact Corporation have them available last time I checked (last week). Both citing high demand as the reason for it being unavailable.
    - sabalaba 5y ago
      We (Lambda) have all of the different NVIDIA GPUs in stock ---- can you send a message to sales@lambdalabs.com and check in again with your requirements? We're seeing a lot more stock these days as the supply chain crisis of 2021 comes to an end.
      - chockchocschoir 5y ago
        I talked with you (Lambda Labs) just a week ago about the A100 specifically and you said that the demand was higher than the supply, and that people should check once a day or something like that to see if it's available in your dashboard. If you clearly have it available now, please say so outright instead of trying to push some other offer on me in emails :)
    - asciimike 5y ago
      Howdy, I run [Crusoe Cloud](https://crusoecloud.com/ https://crusoecloud.com/) and we just launched an alpha of an A100 and A40 Cloud offering--we've got capacity at a reasonable price! If you're interested in giving us a shot, feel free to shoot me an email at mike at crusoecloud dot com.
- nqnielsen 5y ago
  And even if vendors does say they have it, or can get it, it ended up taking us 4-6 months before systems were online.
- kovek 5y ago
  Does it make sense that all the GPUs are bought out? They each provide a return for mining in the short-term. In the long term, they can be used to run A(G)I models, which will be very very useful
  - fennecfoxen 5y ago
    This is the GPU the parent is talking about https://www.nvidia.com/en-us/data-center/a100/ https://www.nvidia.com/en-us/data-center/a100/
    - kovek 5y ago
      This still makes sense! TPUs are useful for AI, which itself will be very very useful. It’s almost like it’s the best investment. That’s why smart players buy them all. Maybe I’m going out-of-topic.

Uehreka 5y ago

Most people who use one of these will be doing so through an EC2 VM (or equivalent). Given that cloud platforms can spread load, keep these GPUs churning close to 24/7 and more easily predict/amortize costs, they’ll probably buy the amount that they know they need, and Nvidia probably has some approximately correct idea of what that number is.