8 ms·

AI models are in a race to the bottom and everybody inside Anthropic knows it. Besides OpenAI, with billions to spend plus a partnership with MS, there's also G

• Tepix 4y ago
  It's a bit early to tell, isn't it? If we get more unexpected emergent abilities by scaling the model further, things could get very interesting indeed.
  • bradleyjg 4y ago
    There’s needs to be some difficult barriers to entry beyond having the money to spend on training FLOPS in order for a startup to compete. I have no idea if there are or there aren’t, but that’s the big question.
    • dr_dshiv 4y ago
      I mean, this is just the beginning. Just wait till we get actual scifi robots in the next year or so. FWIW, I do find that Claude (Anthropic's GPT) is often better than GPT4 -- and very fast. Entrants can compete on price, safety, quality, etc.
      • ethbr0 4y ago
        And a big moat is going to be safety... and specifically configuration of safety. Wouldn't be surprised at all if the major API-based vendors start leaning in on making their safety config proprietary. If a business has already sunk XXXX hours into ensuring a model meets their safety criteria for public-facing use, they'd rather upgrade to a newer model from the same vendor that guarantees portability of that, versus having to reinvest and recertify. Ergo, the AI PaaS that dominate at the beginning will likely continue to dominate.
        • dr_dshiv 4y ago
          Excellent point. Fine tuning is at a low point now, but i expect this to create a moat for the same reasons.
      • comboy 4y ago
        Is it possible to test it somewhere?
        • dTal 4y ago
          https://poe.com/Claude-instant https://poe.com/Claude-instant It also provides a ChatGPT interface, and a number of other models.
      • dTal 4y ago
        I find that Claude is more conversational (better fine tuning), but not as smart as even ChatGPT. Prompt: The original titles and release years in the Harry Potter series are: Philosopher's Stone (1997) Chamber of Secrets (1998) Prisoner of Azkaban (1999) Goblet of Fire (2000) Order of the Phoenix (2003) Half-Blood Prince (2005) Deathly Hallows (2007) Given this, generate a new Harry Potter title, using only the words found in the existing titles. Avoid orderings in the original titles. You may add or remove plurals and possessives. Results: ChatGPT: Blood Chamber of the Phoenix's Prisoner Claude-instant: Chamber Prince Half-Blood Phoenix
        • dr_dshiv 4y ago
          The question for me is whether they understand complex concepts and can apply them in new areas. So when I’m doing quantum computing work, I go back and forth between Claude and GPT4 and both complement the other very well.
        • phillipcarter 4y ago
          I find the opposite, claude-instant seems to generally give me better results for my use case. FWIW gpt-3.5-turbo is good too, just not quite as good.
        • kalkin 4y ago
          ChatGPT is more comparable to what Quora/Poe calls Claude+ - slower/more expensive/smarter. Claude-instant is closer to GPT-turbo in that tradeoff space.
          • dTal 4y ago
            Both bots are free on poe.com, so one is not more expensive than the other.
  • sharemywin 4y ago
    Would you rather? invest in super intelligent AGI or NOT invest in super intelligent AGI. Especially if one of those emergent abilities is deciding your either with me or against me..lol
    • KineticLensman 4y ago
      That would be the AI version of Pascal's Wager [0] [0] https://en.wikipedia.org/wiki/Pascal%27s_wager https://en.wikipedia.org/wiki/Pascal%27s_wager
    • pharmakom 4y ago
      It still remains to be seen if LLMs will lead to AGI.
• rvz 4y ago
  Correct. Stability.ai (Stable Diffusion), Apple (Won't be surprising to see them announce on-device LLMs with Apple Silicon support), Meta (LLaMa), etc are already at the bottom and at the finish line with their AI models given for free. O̶p̶e̶n̶AI.com will eventually have to raise their prices which is bad news for businesses not making enough money and still are sitting on their APIs as O̶p̶e̶n̶AI.com themselves are running up huge costs for their AI models in the cloud for inferencing. Anthropic is just waiting to be acquired by big tech and the consolidation games will start again.
  • truetraveller 4y ago
    Aside: I love the O̶p̶e̶n̶AI.com thing. I got caught off guard at least twice!
    • iandanforth 4y ago
      Wow I didn't know ai.com redirects to chat.openai.com. How long has it been doing that?
      • dontwearitout 4y ago
        It's fairly recent, I think I saw an article here on their purchase of the domain for a few million
  • dmix 4y ago
    This sort of hand wavy generalizations about such a broad and ill defined market seems very naive/closed minded. If you're quabbling over how much OpenAI charges for an API today that barely just launched and from which we have barely scraped the surface for applications... I don't know that seems like a failure to think broadly and assumes the market today is what it will look like in 5yrs. There could be a ton of lucrative businesses which subsidize those operating costs. It doesn't have to be a mega-company like Google that floats it indefinitely off their ad business, or whatever other scheme. We have no idea what the value of those APIs are or if the API is the real business they (and others) are going to be relying on in the long term.
• naillo 4y ago
  The counter argument is that it's a growing market where any early entrants will be lifted with the tide and can probably yield enough profit from spillover hype for investors to make their investments back.
  • ninkendo 4y ago
    > can probably yield enough profit from spillover hype for investors to make their investments back. The correct term for this is “pyramid scheme”.
    • naillo 4y ago
      Nope
    • mc32 4y ago
      No, this is more "everyone is selling X, let's get in the business of X". On the other hand, yes, some will miss the boat and lose money.
      • ninkendo 4y ago
        I interpreted “spillover hype” as meaning “more investors coming in in future rounds” (ie pyramid scheme), but it’s possible that’s not what the commenter intended. But if early investors only profit due to late investors pouring money in, that’s by definition a pyramid scheme.
  • seydor 4y ago
    I can already run gpt4xalpaca on my PC, a model that is not-bad-at-all and is completely uncensored (i.e. does things that chatGPT can't do). I think it's true that LLMs are racing to the bottom and will be even more once they can fit as a peripheral to every computer. whoever is investing in this to monopolize has not thought it through
    • ethbr0 4y ago
      This is a repeat of the early GPU era. It's not the software or hardware that will "win" the race, it's who delivers the packaged end user capability (or centralizes and grabs most of the value along the chain). And end user capability is comprised of hardware + software + connectivity + standardized APIs for building software on top + integration into existing systems. If I were Nvidia, I'd be smiling. They've been here before.
• lightbendover 4y ago
  Truth is, and this applies to all companies regardless of size, is that you don't have to be first, best, biggest, fastest, or most well-known in order to win market share that out-paces your investment. The AI pie is going to be very, very big. To estimate this size, let's take McKinseys rough estimates of job displacement (~30% of ~60% of jobs, ~20% of work) and use that to estimate the actualized [US, apologies] GDP that can at some point be attributed to AI: it is in the 4-5 trillion range using today's figures. To say a market that large will be owned by only 4-5 companies doesn't make sense. Let's take the PC market for example: there are roughly 6 companies that make up ~80% of the market, sure. However, let's look at a tiny participant compared to the total market (~65B): iBuyPower at rank #77 had sales of 40MM or 0.06% (small, expected) of the market with a much smaller capital investment. If look at this percent compared to 5T, we would be at 3B. While the 5B investment stated in the headline could result in a lower ranking and smaller share, the point stands that there is still a lot of money to be made on the long tail. Even if Anthropic fails, there will be other companies with similar infusions that succeed.
  • RandomLensman 4y ago
    It is not clear if (i) a lot of the surplus will be captured by the AI providers and (ii) that the impact will be anywhere as big as people now guess/want it to be. Making a bet on the future is fine, of course.
    • carlmr 4y ago
      My question would also be what kind of insight McKinsey can provide here. What, if anything, do they know about AI that we don't know?
      • yunwal 4y ago
        If anything McKinsey has a lot to gain from exaggerating the numbers so more companies come to them for AI solutions or whatever their next consulting product is.
      • lightbendover 4y ago
• mxkopy 4y ago
  It all depends on if they're in the business of producing the whitepapers that drive ML advancements in the first place. AI is far from a solved problem and whoever gets to it first wins. We have GPT because of a billion dollars worth of data, not algorithms.
• blazespin 4y ago
  The fact is nobody can risk not owning a piece of the foundational models. There is waaaay too much upside risk that they will tots dominate the market. I mean, maybe they won't like you say, but what if they do? Then you're probably screwed. Better to gamble a few billion, imho.
• pera 4y ago
  > convince gullible investors that there's money to be made with foundational models. There's not. This is a ridiculously myopic statement. Foundation models are an extremely powerful technological advancement and they will shake the global economy as very few things did in human history. It's hard to imagine how this is not obvious to everyone right now, specially here in this forum.
  • dmix 4y ago
    Plus you're also investing in getting the talent together in the same building. Even if the foundational models aren't the money maker there's still a ton of opportunity having the best experts at building those models working together and figuring out which branches that LLMs spawns can turn into real markets. It's a high risk investment at this stage but the money is being thrown at the people as much as the current business plan.
  • Chabsff 4y ago
    No one here is disputing that. The question is whether whomever builds them can make a profit doing so, or will they just end up being the suckers that everyone who actually makes money piggybacks off. It's really not clear at the moment.
  • ben_w 4y ago
    None of which means the money will go to the people making the models. The game theory logic doesn't care about the labels "OpenAI" or "Anthropic" or any of the others, it's the same if you switch it around arbitrarily, but this is easier to write about if I focus on one of them: At some point, someone will reproduce GPT-3.5 and ChatGPT, given how much is known about them. When that happens, OpenAI can't make any significant profit from it. GPT-4 might remain sufficiently secret to avoid that, but the history of tech leaks and hacks suggests it too will become public, but even if it does itself remain behind closed doors, there is a further example in that DALL•E 2 is now the boring 3rd horse in the race between Stable Diffusion and Midjourney, and the same may happen with the GPT-series of LLMs. The models leaking or being superseded by others implies profit going to increased productivity in the general economy without investors getting a share.
• m3kw9 4y ago
  The best models will always be closely guarded and have the best outputs, it’s the watered down models that are fighting for scraps.
• HlessClaudesman 4y ago
  Ahh the old Lyft / Avis strat.
• dauertewigkeit 4y ago
  Who says that the AI model is the business?
• moneywoes 4y ago
  Will fine tuned models be lucrative then?
• 01100011 4y ago
  This sounds like saying "internet search engines are a race to the bottom" 20 years ago without realizing that someone may end up as Google and obtain market dominance for a decade or so. It also sounds like you believe you have defined the bounds for what AI will be, and figure we'll just iterate on that until it's a commodity. I don't think AI will be that static. We're all focused on stable diffusion and LLMs right now but the next thing will be something else, and something else after that. As each new technique comes out(assuming they are all published), we'll see quick progress to incorporate the new ideas into various implementations, but then we'll hit another wall, and suddenly big budgets and research teams may matter again. tldr is that it is way too early to make the cynical claim you are making.
• nr2x 4y ago
  The smart move could be an open-core approach. Release the models, but have the best engineering stack to run the APIs as a service.
  • pharmakom 4y ago
    But the models are the expensive part to train. Running the models is relatively easy.