5 ms·

xAI's Colossus: Most Powerful AI Cluster Online in 122 Days

27 points by gfortaine 2y ago

deleted 2y ago
[deleted]
somethoughts 2y ago
Is this notable due to the lack of HN comments?
blackeyeblitzar 2y ago
Is this really the most powerful “AI cluster” or is this an exaggeration? I would expect that hyperscalers like Google or Amazon would actually have the largest amount of compute power to throw at AI. Or is that no longer true? EDIT: I see a comment that has a graph of AI compute across different companies. I am not sure if this is trustworthy however, as the graph is itself just generated by an LLM (Claude): https://x.com/AnthonyEveryWhr/status/1830680977103794177 https://x.com/AnthonyEveryWhr/status/1830680977103794177
- danpalmer 2y ago
  I'd say this graph is absolute rubbish. The fact that the numbers are round numbers, decreasing in even steps, with no units, no comparison, incorrect scaling, and incorrect layout for a graph, suggests that this is little more than fanfiction. The truth is that the hyperscalers are very closely guarding information around things like this. Who has the biggest "cluster" is impossible to answer. Also most hyperscalers will have many clusters, and what's more useful, the SUM or MAX of those clusters? At a small scale probably the MAX, but at a large scale the SUM because you can distribute across clusters without as much loss in overheads and because you have a range of jobs to run.
- Havoc 2y ago
  No the chart is wrong.
- blitzar 2y ago
  For these numbers to be correct Nvidia would have to be committing securities fraud on a scale never seen before.
- MertsA 2y ago
  The data in that graph is just complete misinformation. https://engineering.fb.com/2024/03/12/data-center-engineering/building-metas-genai-infrastructure/ https://engineering.fb.com/2024/03/12/data-center-engineerin...
TkTech 2y ago
Is there a reputable source for these claims? Given source has been proven to lie about scale, delivery, and functionality, especially for topics that might boost share prices. https://elonmusk.today/ https://elonmusk.today/
- JumpCrisscross 2y ago
  “The utility company said that by August, xAI would have access to around 50 megawatts of power, or only enough for around 50,000 chips, and that an upcoming electric substation on the site would give him another 150 megawatts—enough to power 100,000 chips or more. But that wouldn’t happen until 2025, the utility company said” [1]. [1] https://www.theinformation.com/articles/why-musks-ai-rivals-are-alarmed-by-his-new-gpu-cluster?rc=gtdonx https://www.theinformation.com/articles/why-musks-ai-rivals-...
summerlight 2y ago
This looks like quite a futile PR to downplay other players? MS already had more than 100k H100 before the beginning of this year and the same thing for Meta. Google has its own TPU cluster so it's not even possible to do apple-to-apple comparison.
- Havoc 2y ago
  Yeah the charts posted in comments on x didn’t look right to me either
mellosouls 2y ago
This sounds like a notable achievement but they really ought to have chosen a different name. https://en.wikipedia.org/wiki/Colossus_computer https://en.wikipedia.org/wiki/Colossus_computer
XorNot 2y ago
Doesn't this literally not matter? We know that something Llama is being trained in about 3 months on Meta's current resources from their current datasets. If they had less GPU power available, it would take longer but not dramatically so. Building a "more powerful AI cluster" doesn't give you a more powerful AI currently. In fact none of the Transformer architecture's seem to, and their instantiations seem to all be targeted towards "single H100 executions" for inference. So this really does seem like nothing but hype: buying a lot of commercially available GPUs isn't accomplishing anything.
- latentsea 2y ago
  Right, but that means Meta can train 4 models a year. So, 4 opportunities for feedback. They however, need to release those to the public in order to get feedback on the limitations of the model, so that might limit it to training and releasing 2 models a year. If you had more GPUs you could iterate faster...
torlok 2y ago
Does xAI do anything interesting or are they just trying to catch up?
- Havoc 2y ago
  To be fair catching up is quite a feat in this game
  - torlok 2y ago
    That's undeniable, but it would be interesting to see something beyond "you can generate a picture" or "you can ask our model questions, though it may make things up".
    - nickthegreek 2y ago
      Generating pictures isn’t even xAI’s model. It uses flux.
- Vuizur 2y ago
  Grok-2 is rank 2 on LLM arena, it's basically as good as the best Gemini model. They already caught up. Only the latest ChatGPT model is a tiny bit better.
ado__dev 2y ago
(x) Doubt
jacknews 2y ago
Colossus ... didn't I watch a movie about this?
disposition2 2y ago
Apparently running on many gas turbines (enough to power 50,000 homes), without any permits for all that pollution [1]. Also, most likely, hoovering up water from the often low Mississippi River Just your modern day robber baron... 1. https://www.reuters.com/business/environment/musks-xai-operating-gas-turbines-without-permits-data-center-environmental-group-2024-08-28/ https://www.reuters.com/business/environment/musks-xai-opera...

27 points by gfortaine 2y ago

deleted 2y ago

[deleted]

somethoughts 2y ago

Is this notable due to the lack of HN comments?

blackeyeblitzar 2y ago

Is this really the most powerful “AI cluster” or is this an exaggeration? I would expect that hyperscalers like Google or Amazon would actually have the largest amount of compute power to throw at AI. Or is that no longer true? EDIT: I see a comment that has a graph of AI compute across different companies. I am not sure if this is trustworthy however, as the graph is itself just generated by an LLM (Claude): https://x.com/AnthonyEveryWhr/status/1830680977103794177 https://x.com/AnthonyEveryWhr/status/1830680977103794177

danpalmer 2y ago
I'd say this graph is absolute rubbish. The fact that the numbers are round numbers, decreasing in even steps, with no units, no comparison, incorrect scaling, and incorrect layout for a graph, suggests that this is little more than fanfiction. The truth is that the hyperscalers are very closely guarding information around things like this. Who has the biggest "cluster" is impossible to answer. Also most hyperscalers will have many clusters, and what's more useful, the SUM or MAX of those clusters? At a small scale probably the MAX, but at a large scale the SUM because you can distribute across clusters without as much loss in overheads and because you have a range of jobs to run.
Havoc 2y ago
No the chart is wrong.
blitzar 2y ago
For these numbers to be correct Nvidia would have to be committing securities fraud on a scale never seen before.
MertsA 2y ago
The data in that graph is just complete misinformation. https://engineering.fb.com/2024/03/12/data-center-engineering/building-metas-genai-infrastructure/ https://engineering.fb.com/2024/03/12/data-center-engineerin...

TkTech 2y ago

Is there a reputable source for these claims? Given source has been proven to lie about scale, delivery, and functionality, especially for topics that might boost share prices. https://elonmusk.today/ https://elonmusk.today/

JumpCrisscross 2y ago
“The utility company said that by August, xAI would have access to around 50 megawatts of power, or only enough for around 50,000 chips, and that an upcoming electric substation on the site would give him another 150 megawatts—enough to power 100,000 chips or more. But that wouldn’t happen until 2025, the utility company said” [1]. [1] https://www.theinformation.com/articles/why-musks-ai-rivals-are-alarmed-by-his-new-gpu-cluster?rc=gtdonx https://www.theinformation.com/articles/why-musks-ai-rivals-...

summerlight 2y ago

This looks like quite a futile PR to downplay other players? MS already had more than 100k H100 before the beginning of this year and the same thing for Meta. Google has its own TPU cluster so it's not even possible to do apple-to-apple comparison.

Havoc 2y ago
Yeah the charts posted in comments on x didn’t look right to me either

mellosouls 2y ago

This sounds like a notable achievement but they really ought to have chosen a different name. https://en.wikipedia.org/wiki/Colossus_computer https://en.wikipedia.org/wiki/Colossus_computer

XorNot 2y ago

Doesn't this literally not matter? We know that something Llama is being trained in about 3 months on Meta's current resources from their current datasets. If they had less GPU power available, it would take longer but not dramatically so. Building a "more powerful AI cluster" doesn't give you a more powerful AI currently. In fact none of the Transformer architecture's seem to, and their instantiations seem to all be targeted towards "single H100 executions" for inference. So this really does seem like nothing but hype: buying a lot of commercially available GPUs isn't accomplishing anything.

latentsea 2y ago
Right, but that means Meta can train 4 models a year. So, 4 opportunities for feedback. They however, need to release those to the public in order to get feedback on the limitations of the model, so that might limit it to training and releasing 2 models a year. If you had more GPUs you could iterate faster...

torlok 2y ago

Does xAI do anything interesting or are they just trying to catch up?

Havoc 2y ago
To be fair catching up is quite a feat in this game
- torlok 2y ago
  That's undeniable, but it would be interesting to see something beyond "you can generate a picture" or "you can ask our model questions, though it may make things up".
  - nickthegreek 2y ago
    Generating pictures isn’t even xAI’s model. It uses flux.
Vuizur 2y ago
Grok-2 is rank 2 on LLM arena, it's basically as good as the best Gemini model. They already caught up. Only the latest ChatGPT model is a tiny bit better.

ado__dev 2y ago

(x) Doubt

jacknews 2y ago

Colossus ... didn't I watch a movie about this?

disposition2 2y ago

Apparently running on many gas turbines (enough to power 50,000 homes), without any permits for all that pollution [1]. Also, most likely, hoovering up water from the often low Mississippi River Just your modern day robber baron... 1. https://www.reuters.com/business/environment/musks-xai-operating-gas-turbines-without-permits-data-center-environmental-group-2024-08-28/ https://www.reuters.com/business/environment/musks-xai-opera...