5 ms·

The wildest part is that the frontier models have a lifespan of 6 months or so. I don't see how it's sustainable to keep throwing this kind of money at trainin

The wildest part is that the frontier models have a lifespan of 6 months or so. I don't see how it's sustainable to keep throwing this kind of money at training new models that will be obsolete in the blink of an eye. Unless you believe that AGI is truly just a few model generations away and once achieved it's game over for everyone but the winner. I don't.

jononor 1y ago
It is being played like a winner-takes-it-all right now (it may or may not be such a market). So it is a game of being the one that is left standing, once the others fall off. In this kind of game, speeding more is done as a strategy to increase the chances of other competitors running out of cash or otherwise hitting a wall. Sustainability is the opposite of the goal being pursued... Whether one reaches "AGI" is not considered important either, as long as one can starve out most competitors. And for the newcomers, the scale needs to be bigger than what the incumbents (Google and Microsoft) have as discretionary spending - which is at least a few billion per year. Because at that rate, those companies can sustain it forever and would be default winners. So I think yearly expenditure is going to be 20B year++
- leptons 1y ago
  It's the Uber business plan - losing money until the competition loses more and goes out of business. So far Lyft seems to be doing okay, which proves the business plan doesn't really work.
  - Workaccount2 1y ago
    There are endless examples of that business model working...
    - oblio 1y ago
      Are there? Which ones? I'm especially interested in companies that weren't built to be sold.
  - jononor 1y ago
    Uber market cap makes places it in the top100 in the world, whereas Lyft is around 1/25th of Uber in market cap, and not even top1000. I would consider that a success... That is basically as much winner-takes-it-all one can realistically get in a global market. Cases where the top is just 5x the runner up would still be very winner oriented.
    - ViewTrick1002 1y ago
      And in Europe Bolt is winning in many markets. Taxi apps are a commodity today.
  - simianwords 1y ago
    Uber is profitable so why do you think it doesn't work?
    - oblio 1y ago
      Because the competition hasn't gone out of business (at least outside the US where tons of other ride hailing apps are available in most major locales) and because 16 (SIXTEEN!!!) years after founding Uber is still net profit negative: over its lifetime it has lost more money than it made. The only people that really benefited from Uber are: - Uber executives - early investors that saw the share price go up - early customers that got VC subsidized rides
      - simianwords 1y ago
        Are you predicting that they can't be net profitable?
        oblio 1y ago
        No, I'm predicting that: 1. opportunity costs are a thing. 2. if you add Uber's financial numbers since creation, the crazy amount of VC that was invested Uber would have provided better returns by investing it in the S&P 500. 3. Uber will settle in as a boring, profitable company that's going to be a side note in both the history of tech and also of transportation and will primarily be remembered for eroding worker rights.
        simianwords 1y ago
        I don't get your point. You would have still made more money investing in Uber than in S&P.
        oblio 1y ago
        No, you wouldn't have, unless you were one of handful VCs or Uber execs (ok, and a bunch of pre-IPO Uber employees). Uber IPO May 2019: market cap $82bn. Uber now: $193bn. 2.35x multiplier. S&P 500 May 2019: $2750. S&P 500 now: $6460. 2.35x multiplier. So the much, much riskier Uber investment has barely matched a passive S&P 500 investment over the same time frame. And the business itself has lost money, more money was put into it than has been gotten back so far. I'm not even sure why I'm in this conversation as it seems ideological. I bring up facts and you bring up... vibes?
- sdesol 1y ago
  > So it is a game of being the one that is left standing Or the last investor. When this type of money is raised, you can be sure the earlier investors are looking for ways to have a soft landing.
  - tim333 1y ago
    I'm not sure many investors are investing their own money. They are investing other people's money, maybe owned by shareholders of large companies in turn owned by our pension funds.
    - sdesol 1y ago
      It might not be their money, but they are paid a management fee and if they cannot provide some return, people will stop using them.
      - tim333 1y ago
        The kind of thing that happens is Joe Bloggs runs the Fidelity Hot Tech fund, up 50% over the last three years. Then when it crashes that's closed and Joe is switched to the Fidelity Safe Income fund with no down years for the last five years.
solomonb 1y ago
They are only getting deprecated this fast because the cost of training is in some sense sustainable. Once it is not, then they will no longer be deprecated so fast.
- utyop22 1y ago
  Is it though? Its only sustainable to the extent that there is easy access to funding on favourable terms...

jononor 1y ago

It is being played like a winner-takes-it-all right now (it may or may not be such a market). So it is a game of being the one that is left standing, once the others fall off. In this kind of game, speeding more is done as a strategy to increase the chances of other competitors running out of cash or otherwise hitting a wall. Sustainability is the opposite of the goal being pursued... Whether one reaches "AGI" is not considered important either, as long as one can starve out most competitors. And for the newcomers, the scale needs to be bigger than what the incumbents (Google and Microsoft) have as discretionary spending - which is at least a few billion per year. Because at that rate, those companies can sustain it forever and would be default winners. So I think yearly expenditure is going to be 20B year++

leptons 1y ago
It's the Uber business plan - losing money until the competition loses more and goes out of business. So far Lyft seems to be doing okay, which proves the business plan doesn't really work.
- Workaccount2 1y ago
  There are endless examples of that business model working...
  - oblio 1y ago
    Are there? Which ones? I'm especially interested in companies that weren't built to be sold.
- jononor 1y ago
  Uber market cap makes places it in the top100 in the world, whereas Lyft is around 1/25th of Uber in market cap, and not even top1000. I would consider that a success... That is basically as much winner-takes-it-all one can realistically get in a global market. Cases where the top is just 5x the runner up would still be very winner oriented.
  - ViewTrick1002 1y ago
    And in Europe Bolt is winning in many markets. Taxi apps are a commodity today.
- simianwords 1y ago
  Uber is profitable so why do you think it doesn't work?
  - oblio 1y ago
    Because the competition hasn't gone out of business (at least outside the US where tons of other ride hailing apps are available in most major locales) and because 16 (SIXTEEN!!!) years after founding Uber is still net profit negative: over its lifetime it has lost more money than it made. The only people that really benefited from Uber are: - Uber executives - early investors that saw the share price go up - early customers that got VC subsidized rides
    - simianwords 1y ago
      Are you predicting that they can't be net profitable?
      - oblio 1y ago
        No, I'm predicting that: 1. opportunity costs are a thing. 2. if you add Uber's financial numbers since creation, the crazy amount of VC that was invested Uber would have provided better returns by investing it in the S&P 500. 3. Uber will settle in as a boring, profitable company that's going to be a side note in both the history of tech and also of transportation and will primarily be remembered for eroding worker rights.
        simianwords 1y ago
        I don't get your point. You would have still made more money investing in Uber than in S&P.
        oblio 1y ago
        No, you wouldn't have, unless you were one of handful VCs or Uber execs (ok, and a bunch of pre-IPO Uber employees). Uber IPO May 2019: market cap $82bn. Uber now: $193bn. 2.35x multiplier. S&P 500 May 2019: $2750. S&P 500 now: $6460. 2.35x multiplier. So the much, much riskier Uber investment has barely matched a passive S&P 500 investment over the same time frame. And the business itself has lost money, more money was put into it than has been gotten back so far. I'm not even sure why I'm in this conversation as it seems ideological. I bring up facts and you bring up... vibes?
sdesol 1y ago
> So it is a game of being the one that is left standing Or the last investor. When this type of money is raised, you can be sure the earlier investors are looking for ways to have a soft landing.
- tim333 1y ago
  I'm not sure many investors are investing their own money. They are investing other people's money, maybe owned by shareholders of large companies in turn owned by our pension funds.
  - sdesol 1y ago
    It might not be their money, but they are paid a management fee and if they cannot provide some return, people will stop using them.
    - tim333 1y ago
      The kind of thing that happens is Joe Bloggs runs the Fidelity Hot Tech fund, up 50% over the last three years. Then when it crashes that's closed and Joe is switched to the Fidelity Safe Income fund with no down years for the last five years.

solomonb 1y ago

They are only getting deprecated this fast because the cost of training is in some sense sustainable. Once it is not, then they will no longer be deprecated so fast.

utyop22 1y ago
Is it though? Its only sustainable to the extent that there is easy access to funding on favourable terms...