7 ms·

[flagged]

by gmerc 2y ago

[flagged]

tinyhouse 2y ago
DeepSeek wouldn't be able to train R1 without their ~600B parameters base model, so you should consider the cost of that model when you compare with Grok. In any case, Elon won't win this race cause the best talent will not work for him. He used to have good reputation and a lot of money, which is a deadly combination. Now he only has the latter -- not enough when leading AI people can make 7 figures in other companies. To be clear 1: I'm not saying that people who currently work on Grok are not great. It's not about hiring some great people. It's about competing in the long run - people with other options (e.g. offers from leading AI labs) are more likely to accept those offers than joining his research lab. To be clear 2: I'm not talking about Elon's reputation due to his politics. I'm only talking about his reputation as an employer. He has the vision and marketing skills but it's not going to be enough for leading the AI race.
- gmerc 2y ago
  It’s not like Grok3 didn’t have precious work to build on either, but point taken. I think the situations are a bit comparable given timelines however. A perfect analogy for AI … your ability to replace talent with money. And if you don’t have the talent, it’s gonna cost you 100x more.
  - tastyfreeze 2y ago
    > your ability to replace talent with money That sure seems to be the message given in Apple AI commercials. From those commercials the tag line for AI should be "enabling idiots everywhere".
- andy12_ 2y ago
  Actually, the 5 million figure is for the compute cost for the base 600B parameter model. Training R1 was just 8000 steps of reinforcement learning, so I expect that the vast, vast majority of the training cost is already included in the pretraining stage.
- deleted 2y ago
  [deleted]
screye 2y ago
It's a race for AGI, a VC's wet dream. The beauty of a failed investment is that it never goes below zero. So upside is the only thing they care about. Why invest in a near-zero chance for a random SAAS to take off, when you can invest in a near-zero chance of creating superhuman artificial life?
- mempko 2y ago
  A failed investment never goes below zero for the investor. For everyone else on the other hand...
- gmerc 2y ago
  Works well when you see the company stuffing dollar bills into their sports car to race on 1.25% fuel efficiency against a chinese family sedan with a hand tuned ICE.
- rtsil 2y ago
  > It's a race for AGI, a VC's wet dream. Yes but why? This is what I really don't understand. Say AGI is achieved within a reasonable timeframe. Odds are that no single company will achieve that, there will be no monopoly. If that's the case, where is the trillion dollars value for investors? From every claim we hear about it, AGI will lead to hundreds of millions of jobs disappearing (all white-collar jobs), and tens of millions of companies disappearing (all the companies that provide human-produced services). Who is going to buy your AGI-made products or services when nobody is paid anymore, when other companies, big and small, has ceased to exist? Sure, you can make extraordinary accomplishments and advance humanity far, far ahead, but who is going to pay for that? Even states won't be able to pay if their taxable population (individuals and corporations) disappear. So where will the money come from? How does it work?
  - bparsons 2y ago
    Also, profitability won't materialize in an environment with so many competitors offering comparable products. Perfect competition destroys profit. The good becomes a commodity, and the price people will pay simply becomes the marginal cost of production (or in this case, less, while the dumb money is still chasing the hype).
InkCanon 2y ago
I used to think the same way wrt Nvidia stock when it tanked - compute is clearly diminishing returns. Tech companies subsequently announced capex equal to or greater than expected in compute. I smacked myself on the forehead when I realized - I'd been think too much like an engineer. Tech CEOs badly want to believe they have an edge over every upstart from San Francisco to Shanghai. Unlimited spending on compute gives them that reassurance. In fact, the more threatened they feel, the more they spend to cling onto it. Kids have security blankets. Tech CEOs have security compute clusters.
- mrcwinn 2y ago
  This is the danger of being informed only by sensational headlines. Nvidia's stock has fully recovered and is again near an all-time high. You seem to be generalizing about "Tech CEOs" — but in this case, GPUs are the advantage. They are necessary to achieve the outcome, and yet they are severely supply constrained. It's smart to overpay now. Apple did something similar with NAND storage for the iPad mini. They took a bet that could have been wrong. It was not wrong. Competitors had a hard time because of it.
  - gmerc 2y ago
    Overpaying for using them is not smart. They depreciate fast under heavy load.
    - knowitnone 2y ago
      but nobody needs to know what load they were used so it's "barely used" on the listings
    - toolz 2y ago
      load isn't what causes degradation, it's heat and as someone who has mined crypto for years I'm aware that there are a lot of things that can be done to run hardware quite hard and keep thermals low. Whether or not that is what is being done, I have no idea. A GPU mining crypto for 5 years kept below 65C (rather easily done) is going to have far more life left than a GPU in some kids gaming PC that spikes frequently to 85C for even a year.
BluSyn 2y ago
You seem to be assuming that the full cost of the cluster is recouped by Grok 3. The real value will be in grok 5, 6, etc… xAI also announced a few days ago they are starting an internal video game studio. How long before AI companies take over Hollywood and Disney? The value available to be captured is massive. The cluster they’ve built is impressive compared to the competition, and grok 3 barely scratches what it’s capable of.
- doctorpangloss 2y ago
  > xAI also announced a few days ago they are starting an internal video game studio. Ha ha. I'm sure their play to claim airdrop idle game will be groundbreaking.
- Tycho 2y ago
  Yes. Why do get these replies on HN that seem to only consider the most shallow, surface details? It could well be that xAI wins the AI race by betting on hardware first and foremost - new ideas are quickly copied by everyone, but a compute edge is hard to match.
  - gmerc 2y ago
    DeepSeek just showed the compute edge is not that hard to match. They could have chosen to keep the gains proprietary but probably made good money playing the market instead, quants as they are. https://centreforaileadership.org/resources/deepseeks_narrative_attack/ https://centreforaileadership.org/resources/deepseeks_narrat... If you’re using your compute capacity at 1.25% efficiency, you are not going to win because your iteration time is just going to be too long to stay competitive.
    - Tycho 2y ago
      Sorry, you missed the point - DeepSeek tried some new software ideas, they did not manage to secure the same computation capacity.
      - 2y ago
belter 2y ago
> There is no monetisation potential. DOGE uses only X links, and I am sure Grok will be the next gov contract. After all he has all the data on everybody down to your IRS tax returns.
- nicce 2y ago
  How this is even legal. Don’t they have any sort of competitive tendering?
  - Larrikin 2y ago
    Why do you think legality matters?
    - nicce 2y ago
      I have still some hope. Everyone how is capable of, should challenge these in court.
      - CamperBob2 2y ago
        And then what? "They've made their decision, let's see if they can enforce it." Capturing the executive turns out to be the winning move. Maybe it's what Gödel saw coming (https://en.wikipedia.org/wiki/G%C3%B6del%27s_Loophole https://en.wikipedia.org/wiki/G%C3%B6del%27s_Loophole).
        matwood 2y ago
        Yep, hence 'constitutional crisis'.
        Hasu 2y ago
        If you think an executive that ignores court orders is going to survive for a long time in America, I am willing to bet any amount of dollars against you at any odds. It's a good bet for me, because if I lose, dollars won't be worth anything anyway.
gordonhart 2y ago
As a consumer, I'm just happy that base models are improving again after a ~quarter or more of relative stagnation (last big base model drop was Sonnet v2 in October). Many use cases can't use o1, r1, or o3[-mini] due to the additional reasoning latency.
- FergusArgyll 2y ago
  Yes, and the scaling laws survive! so, hopefully, more on the way... > due to the additional reasoning latency. They're also less creative for non-STEM topics
soheil 2y ago
[flagged]
- gmerc 2y ago
  [flagged]
nilkn 2y ago
xAI is not trying to make an immediate profit -- ironically, just like DeepSeek. They will undoubtedly use more efficient training processes in future runs and they will scale that across their massive GPU cluster. Just because they didn't cancel the training of Grok 3 and start over absolutely does not mean they will not incorporate all the work from R1 and more in the next run. What you're seeing right now is pure flex and a signal for the future and competition. A much maligned AI team that hasn't even been around for very long at all just matched or topped the competition without making use of the latest training techniques yet. The message this is intended to send is that xAI is a serious player in the space.
consumer451 2y ago
> For what? There is no ROI at that price point. There is no monetization potential. It appears that LLM chat interfaces will replace Google SERPs as the arbiters of truth. Getting people to use your LLM allows you to push your world view. Pushing his "unique" world view appears to be the most important thing to modern Musk. In that light, paying 40B for Twitter, and billions for Grok training makes perfect sense.
zhobbs 2y ago
>For what? There is no ROI at that price point. There is no monetisation potential. I think your whole argument is based on this being true, but you didn't give much argument about why there is no ROI. 400M USD isn't hard to generate...even a moderate ad engagement lift on X would generate ROI and that's just 1 customer. Imagine going back in time and showing every VC how great the search business will be in 20-30 years. The only rational response would be to make giant bets on 20 different Googles...and I think that's what's happening. These all seem like rational investments to me.
- gmerc 2y ago
  That's a red herring because it ignores the part where they could have done the same things spending a tiny fraction of the money.
  - gordonhart 2y ago
    _Could_ they have done the same thing with a tiny fraction of the money? Grok 3 benchmarks are SOTA for both base model and reasoning. By definition, nobody has been able to do the same thing with any amount of money (discounting o3 which has been teased but is unreleased). That may change in the future! But as of now this is the case.
    - gmerc 2y ago
      So apart from the part where SOTA doesn't mean anything in the real world (there is no monetisation, there's no moat), please, it's benchmarks, we all know how you beat those since 2023. Time to review https://arxiv.org/abs/2309.08632 https://arxiv.org/abs/2309.08632 AI-CEO.org's best friend (and actually o3-mini-high beat them in a bunch of benchmarks so they removed it from those charts in the livestream)
  - loandbehold 2y ago
    People like Demis Hasabis and Derio Amodei say that R1 efficiency gains are exaggerated. $5M training cost seems to be fake as sources suggest they own more GPUs.
submeta 2y ago
> until Claude4 snuffs it out later this month Any source? I’m a heavy user of Claude and pay for the Teams plan just for myself so I won’t get throttled. Love it. But I’ve been impressed with O1 Pro lately. That said, I don’t like paying both €166 for Claude Teams and €238 for OpenAI Pro. :)
deleted 2y ago
[deleted]
dragonwriter 2y ago
> This all by the man in charge of “government spending efficiency”. Per court filings by the administration, Musk is not in charge of DOGE, nor does he have any role in DOGE, nor any decision-making function in government at all, he is a White House advisor unconnected to DOGE.
shytey 2y ago
What is hilarious is your disdain for their achievements which occurred in less than two years. This is just the beginning.
jejeyyy77 2y ago
what makes you think there won't be an ROI?
- gmerc 2y ago
  I think at this point you're going to have to answer "what makes you think there will be any"
ctoth 2y ago
> DeeoSeek trained r1 for 1.25% (5M) of that money (using the same spot price) on 2048 crippled export H800s and is maybe a month behind. This is a great example of how a misleading narrative can take hold and dominate discussion even when it's fundamentally incorrect. SemiAnalysis documents that DeepSeek has spent well over $500M on GPUs alone, with total infrastructure costs around $2.5B when including operating costs[0]. The more-interesting question is probably why do people keep repeating this? Why do they want it to be true so badly? [0]: https://semianalysis.com/2025/01/31/deepseek-debates/#:~:text=Our%20analysis%20shows%20that%20the,associated%20with%20operating%20such%20clusters. https://semianalysis.com/2025/01/31/deepseek-debates/#:~:tex...
- tempusalaria 2y ago
  SemiAnalysis is wrong. They just made their numbers up (among many other things they have invented - they are not to be trusted). I have observed many errors of understanding, analysis and calculation in their writing. Deep Seek R1 is literally an open weight model. It has <40bln active parameters. We know that for a fact. That size of model is definitely roughly optimally trained over the time period and server times claimed. In fact, the 70bln parameter Llama 3 model used almost exactly the same compute as the DeepSeek V3/R1 claims (which makes sense, as you would expect a bit less efficiency for the H800 and for the complex DeepSeek MoE architecture).
  - FergusArgyll 2y ago
    Active parameters is definitely the wrong metric to use for evaluating the cost to train a model

[flagged]

by gmerc 2y ago

[flagged]

DeepSeek wouldn't be able to train R1 without their ~600B parameters base model, so you should consider the cost of that model when you compare with Grok. In any case, Elon won't win this race cause the best talent will not work for him. He used to have good reputation and a lot of money, which is a deadly combination. Now he only has the latter -- not enough when leading AI people can make 7 figures in other companies. To be clear 1: I'm not saying that people who currently work on Grok are not great. It's not about hiring some great people. It's about competing in the long run - people with other options (e.g. offers from leading AI labs) are more likely to accept those offers than joining his research lab. To be clear 2: I'm not talking about Elon's reputation due to his politics. I'm only talking about his reputation as an employer. He has the vision and marketing skills but it's not going to be enough for leading the AI race.

gmerc 2y ago
It’s not like Grok3 didn’t have precious work to build on either, but point taken. I think the situations are a bit comparable given timelines however. A perfect analogy for AI … your ability to replace talent with money. And if you don’t have the talent, it’s gonna cost you 100x more.
- tastyfreeze 2y ago
  > your ability to replace talent with money That sure seems to be the message given in Apple AI commercials. From those commercials the tag line for AI should be "enabling idiots everywhere".
andy12_ 2y ago
Actually, the 5 million figure is for the compute cost for the base 600B parameter model. Training R1 was just 8000 steps of reinforcement learning, so I expect that the vast, vast majority of the training cost is already included in the pretraining stage.
deleted 2y ago
[deleted]

screye 2y ago

It's a race for AGI, a VC's wet dream. The beauty of a failed investment is that it never goes below zero. So upside is the only thing they care about. Why invest in a near-zero chance for a random SAAS to take off, when you can invest in a near-zero chance of creating superhuman artificial life?

mempko 2y ago
A failed investment never goes below zero for the investor. For everyone else on the other hand...
gmerc 2y ago
Works well when you see the company stuffing dollar bills into their sports car to race on 1.25% fuel efficiency against a chinese family sedan with a hand tuned ICE.
rtsil 2y ago
> It's a race for AGI, a VC's wet dream. Yes but why? This is what I really don't understand. Say AGI is achieved within a reasonable timeframe. Odds are that no single company will achieve that, there will be no monopoly. If that's the case, where is the trillion dollars value for investors? From every claim we hear about it, AGI will lead to hundreds of millions of jobs disappearing (all white-collar jobs), and tens of millions of companies disappearing (all the companies that provide human-produced services). Who is going to buy your AGI-made products or services when nobody is paid anymore, when other companies, big and small, has ceased to exist? Sure, you can make extraordinary accomplishments and advance humanity far, far ahead, but who is going to pay for that? Even states won't be able to pay if their taxable population (individuals and corporations) disappear. So where will the money come from? How does it work?
- bparsons 2y ago
  Also, profitability won't materialize in an environment with so many competitors offering comparable products. Perfect competition destroys profit. The good becomes a commodity, and the price people will pay simply becomes the marginal cost of production (or in this case, less, while the dumb money is still chasing the hype).

InkCanon 2y ago

I used to think the same way wrt Nvidia stock when it tanked - compute is clearly diminishing returns. Tech companies subsequently announced capex equal to or greater than expected in compute. I smacked myself on the forehead when I realized - I'd been think too much like an engineer. Tech CEOs badly want to believe they have an edge over every upstart from San Francisco to Shanghai. Unlimited spending on compute gives them that reassurance. In fact, the more threatened they feel, the more they spend to cling onto it. Kids have security blankets. Tech CEOs have security compute clusters.

mrcwinn 2y ago
This is the danger of being informed only by sensational headlines. Nvidia's stock has fully recovered and is again near an all-time high. You seem to be generalizing about "Tech CEOs" — but in this case, GPUs are the advantage. They are necessary to achieve the outcome, and yet they are severely supply constrained. It's smart to overpay now. Apple did something similar with NAND storage for the iPad mini. They took a bet that could have been wrong. It was not wrong. Competitors had a hard time because of it.
- gmerc 2y ago
  Overpaying for using them is not smart. They depreciate fast under heavy load.
  - knowitnone 2y ago
    but nobody needs to know what load they were used so it's "barely used" on the listings
  - toolz 2y ago
    load isn't what causes degradation, it's heat and as someone who has mined crypto for years I'm aware that there are a lot of things that can be done to run hardware quite hard and keep thermals low. Whether or not that is what is being done, I have no idea. A GPU mining crypto for 5 years kept below 65C (rather easily done) is going to have far more life left than a GPU in some kids gaming PC that spikes frequently to 85C for even a year.

BluSyn 2y ago

You seem to be assuming that the full cost of the cluster is recouped by Grok 3. The real value will be in grok 5, 6, etc… xAI also announced a few days ago they are starting an internal video game studio. How long before AI companies take over Hollywood and Disney? The value available to be captured is massive. The cluster they’ve built is impressive compared to the competition, and grok 3 barely scratches what it’s capable of.

doctorpangloss 2y ago
> xAI also announced a few days ago they are starting an internal video game studio. Ha ha. I'm sure their play to claim airdrop idle game will be groundbreaking.
Tycho 2y ago
Yes. Why do get these replies on HN that seem to only consider the most shallow, surface details? It could well be that xAI wins the AI race by betting on hardware first and foremost - new ideas are quickly copied by everyone, but a compute edge is hard to match.
- gmerc 2y ago
  DeepSeek just showed the compute edge is not that hard to match. They could have chosen to keep the gains proprietary but probably made good money playing the market instead, quants as they are. https://centreforaileadership.org/resources/deepseeks_narrative_attack/ https://centreforaileadership.org/resources/deepseeks_narrat... If you’re using your compute capacity at 1.25% efficiency, you are not going to win because your iteration time is just going to be too long to stay competitive.
  - Tycho 2y ago
    Sorry, you missed the point - DeepSeek tried some new software ideas, they did not manage to secure the same computation capacity.
    - 2y ago

belter 2y ago

> There is no monetisation potential. DOGE uses only X links, and I am sure Grok will be the next gov contract. After all he has all the data on everybody down to your IRS tax returns.

nicce 2y ago
How this is even legal. Don’t they have any sort of competitive tendering?
- Larrikin 2y ago
  Why do you think legality matters?
  - nicce 2y ago
    I have still some hope. Everyone how is capable of, should challenge these in court.
    - CamperBob2 2y ago
      And then what? "They've made their decision, let's see if they can enforce it." Capturing the executive turns out to be the winning move. Maybe it's what Gödel saw coming (https://en.wikipedia.org/wiki/G%C3%B6del%27s_Loophole https://en.wikipedia.org/wiki/G%C3%B6del%27s_Loophole).
      - matwood 2y ago
        Yep, hence 'constitutional crisis'.
      - Hasu 2y ago
        If you think an executive that ignores court orders is going to survive for a long time in America, I am willing to bet any amount of dollars against you at any odds. It's a good bet for me, because if I lose, dollars won't be worth anything anyway.

gordonhart 2y ago

As a consumer, I'm just happy that base models are improving again after a ~quarter or more of relative stagnation (last big base model drop was Sonnet v2 in October). Many use cases can't use o1, r1, or o3[-mini] due to the additional reasoning latency.

FergusArgyll 2y ago
Yes, and the scaling laws survive! so, hopefully, more on the way... > due to the additional reasoning latency. They're also less creative for non-STEM topics

soheil 2y ago

[flagged]

gmerc 2y ago
[flagged]

nilkn 2y ago

xAI is not trying to make an immediate profit -- ironically, just like DeepSeek. They will undoubtedly use more efficient training processes in future runs and they will scale that across their massive GPU cluster. Just because they didn't cancel the training of Grok 3 and start over absolutely does not mean they will not incorporate all the work from R1 and more in the next run. What you're seeing right now is pure flex and a signal for the future and competition. A much maligned AI team that hasn't even been around for very long at all just matched or topped the competition without making use of the latest training techniques yet. The message this is intended to send is that xAI is a serious player in the space.

consumer451 2y ago

> For what? There is no ROI at that price point. There is no monetization potential. It appears that LLM chat interfaces will replace Google SERPs as the arbiters of truth. Getting people to use your LLM allows you to push your world view. Pushing his "unique" world view appears to be the most important thing to modern Musk. In that light, paying 40B for Twitter, and billions for Grok training makes perfect sense.

zhobbs 2y ago

>For what? There is no ROI at that price point. There is no monetisation potential. I think your whole argument is based on this being true, but you didn't give much argument about why there is no ROI. 400M USD isn't hard to generate...even a moderate ad engagement lift on X would generate ROI and that's just 1 customer. Imagine going back in time and showing every VC how great the search business will be in 20-30 years. The only rational response would be to make giant bets on 20 different Googles...and I think that's what's happening. These all seem like rational investments to me.

gmerc 2y ago
That's a red herring because it ignores the part where they could have done the same things spending a tiny fraction of the money.
- gordonhart 2y ago
  _Could_ they have done the same thing with a tiny fraction of the money? Grok 3 benchmarks are SOTA for both base model and reasoning. By definition, nobody has been able to do the same thing with any amount of money (discounting o3 which has been teased but is unreleased). That may change in the future! But as of now this is the case.
  - gmerc 2y ago
    So apart from the part where SOTA doesn't mean anything in the real world (there is no monetisation, there's no moat), please, it's benchmarks, we all know how you beat those since 2023. Time to review https://arxiv.org/abs/2309.08632 https://arxiv.org/abs/2309.08632 AI-CEO.org's best friend (and actually o3-mini-high beat them in a bunch of benchmarks so they removed it from those charts in the livestream)
- loandbehold 2y ago
  People like Demis Hasabis and Derio Amodei say that R1 efficiency gains are exaggerated. $5M training cost seems to be fake as sources suggest they own more GPUs.

submeta 2y ago

> until Claude4 snuffs it out later this month Any source? I’m a heavy user of Claude and pay for the Teams plan just for myself so I won’t get throttled. Love it. But I’ve been impressed with O1 Pro lately. That said, I don’t like paying both €166 for Claude Teams and €238 for OpenAI Pro. :)

deleted 2y ago

[deleted]

dragonwriter 2y ago

> This all by the man in charge of “government spending efficiency”. Per court filings by the administration, Musk is not in charge of DOGE, nor does he have any role in DOGE, nor any decision-making function in government at all, he is a White House advisor unconnected to DOGE.

shytey 2y ago

What is hilarious is your disdain for their achievements which occurred in less than two years. This is just the beginning.

jejeyyy77 2y ago

what makes you think there won't be an ROI?

gmerc 2y ago
I think at this point you're going to have to answer "what makes you think there will be any"

ctoth 2y ago

> DeeoSeek trained r1 for 1.25% (5M) of that money (using the same spot price) on 2048 crippled export H800s and is maybe a month behind. This is a great example of how a misleading narrative can take hold and dominate discussion even when it's fundamentally incorrect. SemiAnalysis documents that DeepSeek has spent well over $500M on GPUs alone, with total infrastructure costs around $2.5B when including operating costs[0]. The more-interesting question is probably why do people keep repeating this? Why do they want it to be true so badly? [0]: https://semianalysis.com/2025/01/31/deepseek-debates/#:~:text=Our%20analysis%20shows%20that%20the,associated%20with%20operating%20such%20clusters. https://semianalysis.com/2025/01/31/deepseek-debates/#:~:tex...

tempusalaria 2y ago
SemiAnalysis is wrong. They just made their numbers up (among many other things they have invented - they are not to be trusted). I have observed many errors of understanding, analysis and calculation in their writing. Deep Seek R1 is literally an open weight model. It has <40bln active parameters. We know that for a fact. That size of model is definitely roughly optimally trained over the time period and server times claimed. In fact, the 70bln parameter Llama 3 model used almost exactly the same compute as the DeepSeek V3/R1 claims (which makes sense, as you would expect a bit less efficiency for the H800 and for the complex DeepSeek MoE architecture).
- FergusArgyll 2y ago
  Active parameters is definitely the wrong metric to use for evaluating the cost to train a model