6 ms·

So the outage was caused by overuse?

by ape4 3y ago

weikju 3y ago
Creating demand/hype
- unsupp0rted 3y ago
  They haven’t needed to create demand thus far
  - weikju 3y ago
    Depends what agenda the news articles want to push, I guess (ie all the "chatGPT is losing users!" a while back). I have no idea one way or another.
nicce 3y ago
Maybe, but at the same time there were major changes for the platform, so who knows.
rvz 3y ago
Unable to handle Google scale usage, even with a login wall. Imagine if ChatGPT had no login wall just like Google doesn't require a login to use, then the whole site would be thrown straight into the ground.
- huytersd 3y ago
  I would think Google would require significantly less compute per query though.
- siva7 3y ago
  They certainly handle google scale compute. It's a scale even google couldn't handle at the moment uncontrolled. Hardware for this art of compute is very limited and not remotely comparable to google search
  - rvz 3y ago
    It is indeed comparable given that people here are talking about replacing Google with ChatGPT as a so-called 'search engine'. Even in that case, ChatGPT search queries are far more expensive than a Google search and requires more hardware to scale regardless. Google's scale of usage is 8.5 billion searches per day. For OpenAI, the costs for that scale will just skyrocket and OpenAI cannot handle that which explains why they are already pausing new sign ups even when they have a login wall. So it would be even worse for OpenAI had they opened ChatGPT up without a login just like Google doesn't require a login just like most search engines.
    - siva7 3y ago
      Let's not act like Google Bard isn't behind curtains. They face the same scaling issues like Openai/Microsoft. People are using search in a very different way than a LLM. Openai isn't simply replacing a search catalog to websites, it's replacing the internet as we know it so i don't see how this can be compared. Most people on earth won't have nearly unlimited access to this technology at the moment because of hardware limitations that all providers in this game face but those who have are at an advantage.
rich_sasha 3y ago
Possibly ridiculous question, but does GPU lifespan decrease with more intense use? I could imagine, if so, that they bought some hardware, which needs replacing sooner than they thought, and the running costs are catching up with them. Or not. Nigh impossible to speculate!
- singularity2001 3y ago
  Given the energy and heat involved the answer is most certainly yes
- nisegami 3y ago
  Haven't they left the hardware stuff to Azure/Microsoft?
  - rich_sasha 3y ago
    I would imagine at such scale this abstraction starts to fail. Eg say they wrecked Microsoft's GPUs, MS probs can't just provision more somewhere else at a blink of an eye.
  - ben_w 3y ago
    There's only so much you can leave to Azure. I have no idea how many tokens/second OpenAI gets[0], but 100 M users, each generating an average of (to pick an arbitrary number for a Fermi estimate) 1000 tokens per day, is 1e11 tokens/day or ~1,157,407/s, which could be anywhere from 36k severs to just 830 depending on which value[0] I use as the anchor to guess how many tokens/s their specific servers can generate with their specific models. At the upper end, they may have bought about 8% of all H100s currently in existence. The Microsoft investment (1e10 USD) correlates better with the higher range than the lower range of my estimates for how many machines are in their cluster ((36k * H100) * $30k/H100 ~= 1e9 USD). [0] a quick google said 30/s in the automated answers section (which is citing a May post on Reddit), while one of the links says ~300/s on an H100, and the Llama.cpp github page says ~1400/s on an M2 Ultra (but for a relatively small model).
- Yizahi 3y ago
  Mosfets have lifetime depending on the temperature, so yes. But on the other hand vendor can install components which won't fail for some long time even under max load. If I remember correctly this was the reason for the "New World destroys GPUs", because some vendors didn't read the Nvidia spec carefully and installed less robust mosfets, so the max load (which happened to be New World game) pushed them over the expected lifetime fast. But with proper design this seems not to be a case.

So the outage was caused by overuse?

by ape4 3y ago

So the outage was caused by overuse?

weikju 3y ago

Creating demand/hype

unsupp0rted 3y ago
They haven’t needed to create demand thus far
- weikju 3y ago
  Depends what agenda the news articles want to push, I guess (ie all the "chatGPT is losing users!" a while back). I have no idea one way or another.

nicce 3y ago

Maybe, but at the same time there were major changes for the platform, so who knows.

rvz 3y ago

Unable to handle Google scale usage, even with a login wall. Imagine if ChatGPT had no login wall just like Google doesn't require a login to use, then the whole site would be thrown straight into the ground.

huytersd 3y ago
I would think Google would require significantly less compute per query though.
siva7 3y ago
They certainly handle google scale compute. It's a scale even google couldn't handle at the moment uncontrolled. Hardware for this art of compute is very limited and not remotely comparable to google search
- rvz 3y ago
  It is indeed comparable given that people here are talking about replacing Google with ChatGPT as a so-called 'search engine'. Even in that case, ChatGPT search queries are far more expensive than a Google search and requires more hardware to scale regardless. Google's scale of usage is 8.5 billion searches per day. For OpenAI, the costs for that scale will just skyrocket and OpenAI cannot handle that which explains why they are already pausing new sign ups even when they have a login wall. So it would be even worse for OpenAI had they opened ChatGPT up without a login just like Google doesn't require a login just like most search engines.
  - siva7 3y ago
    Let's not act like Google Bard isn't behind curtains. They face the same scaling issues like Openai/Microsoft. People are using search in a very different way than a LLM. Openai isn't simply replacing a search catalog to websites, it's replacing the internet as we know it so i don't see how this can be compared. Most people on earth won't have nearly unlimited access to this technology at the moment because of hardware limitations that all providers in this game face but those who have are at an advantage.

rich_sasha 3y ago

Possibly ridiculous question, but does GPU lifespan decrease with more intense use? I could imagine, if so, that they bought some hardware, which needs replacing sooner than they thought, and the running costs are catching up with them. Or not. Nigh impossible to speculate!

singularity2001 3y ago
Given the energy and heat involved the answer is most certainly yes
nisegami 3y ago
Haven't they left the hardware stuff to Azure/Microsoft?
- rich_sasha 3y ago
  I would imagine at such scale this abstraction starts to fail. Eg say they wrecked Microsoft's GPUs, MS probs can't just provision more somewhere else at a blink of an eye.
- ben_w 3y ago
  There's only so much you can leave to Azure. I have no idea how many tokens/second OpenAI gets[0], but 100 M users, each generating an average of (to pick an arbitrary number for a Fermi estimate) 1000 tokens per day, is 1e11 tokens/day or ~1,157,407/s, which could be anywhere from 36k severs to just 830 depending on which value[0] I use as the anchor to guess how many tokens/s their specific servers can generate with their specific models. At the upper end, they may have bought about 8% of all H100s currently in existence. The Microsoft investment (1e10 USD) correlates better with the higher range than the lower range of my estimates for how many machines are in their cluster ((36k * H100) * $30k/H100 ~= 1e9 USD). [0] a quick google said 30/s in the automated answers section (which is citing a May post on Reddit), while one of the links says ~300/s on an H100, and the Llama.cpp github page says ~1400/s on an M2 Ultra (but for a relatively small model).
Yizahi 3y ago
Mosfets have lifetime depending on the temperature, so yes. But on the other hand vendor can install components which won't fail for some long time even under max load. If I remember correctly this was the reason for the "New World destroys GPUs", because some vendors didn't read the Nvidia spec carefully and installed less robust mosfets, so the max load (which happened to be New World game) pushed them over the expected lifetime fast. But with proper design this seems not to be a case.