9 ms·

The evidence shows that there is no methodological moat for LLMS. The moat of the frontier folks is just compute. xAI went in months from nothing to competing w

The evidence shows that there is no methodological moat for LLMS. The moat of the frontier folks is just compute. xAI went in months from nothing to competing with the top dogs. DeepSeek too. So why bother with splurging billions in talent when you can buy GPUs and energy instead and serve the compute needs of everyone? Also Amazon is in another capital intensive business. Retail. Spending billions on dubious AWS moonshots vs just buying more widgets and placing them across the houses of US customers for even faster deliveries does not make sense.

karterk 1y ago
> The moat of the frontier folks is just compute. This is not really true. Google has all the compute but in many dimensions they lag behind GPT-5 class (catching up, but it has not been a given). Amazon itself did try to train a model (so did Meta) and had limited success.
- jorisboris 1y ago
  Yes, or Apple who with all the talent don’t manage to pull off anything useful in AI xAI seems to be the exception, not the rule
  - rusk 1y ago
    Given Apple’s moat is their devices, their particular spin on AI is very much edge focussed, which isn’t as spectacular as the current wave of cloud based LLM. Apple’s cloud stuff is laughably poor.
- matusp 1y ago
  I switched to Gemini with my new phone and I literally couldn't tell a difference. It is actually crazy how small the cost of switching is for LLMs. It feels like AI is more like a commodity than a service.
  - lelanthran 1y ago
    > I switched to Gemini with my new phone and I literally couldn't tell a difference. It is actually crazy how small the cost of switching is for LLMs. It feels like AI is more like a commodity than a service. It is. It's wild to me that all these VCs pouring money into AI companies don't know what a value-chain is. Tokens are the bottom of the value-chain; it's where the lowest margins exist because the product at that level is a widely available commodity. I wrote about this already (shameless plug: https://www.rundata.co.za/blog/index.html?the-ai-value-chain https://www.rundata.co.za/blog/index.html?the-ai-value-chain )
  - physicsguy 1y ago
    On top of that, the on-device models have got stronger and stronger as the base models + RL has got better. You can do on your laptop now what 2 years ago was state of the art.
- Keyframe 1y ago
  " in many dimensions they lag behind GPT-5 class " - such as? Outside of computer, "the moat" is also data to train on. That's an even wider moat. Now, google has all the data. Data no one else has or ever will have. If anything, I'd expect them to outclass everyone by a fat margin. I think we're seeing that on video however.
  - willvarfar 1y ago
    not according to google: “We have no moat, and neither does OpenAI”: the big memo and the big HN thread on same https://news.ycombinator.com/item?id=35813322 https://news.ycombinator.com/item?id=35813322
    - Keyframe 1y ago
      a bit weird to think about it since google has literally internet.zip in multiple versions over the years, all of email, all of usenet, all of the videos, all of the music, all of the user's search interest, ads, everything..
      - lelanthran 1y ago
        > a bit weird to think about it since google has literally internet.zip in multiple versions over the years, all of email, all of usenet, all of the videos, all of the music, all of the user's search interest, ads, everything.. Yeah, Google totally has a moat. Them saying that they have no moat doesn't magically make that moat go away. They also own the entire vertical which none of the competitors do - all their competitors have to buy compute from someone who makes a profit just on compute (Nvidia, for example). Google owns the entire vertical, from silicon to end-user. It would be crazy if they can't make this work.
      - rvba 1y ago
        That's why robots make so much traffic now. Those other companies are trying to get data. Google theoretically has reddit access. I wonder if they have sort of an internet archive - data unpolutted by LLMs On a side note, funny how all the companies seem to train on book archivr which they just downloaded from the internet
- paulddraper 1y ago
  It doesn’t guarantee success, but the point stands about X and Deepseek
- gnfargbl 1y ago
  Which dimensions do you see Google lagging on? They seem broadly comparable on the usual leaderboard (https://lmarena.ai/leaderboard https://lmarena.ai/leaderboard) and anecdotally I can't tell the difference in quality. I tend personally to stick with ChatGPT most of the time, but only because I prefer the "tone" of the thing somehow. If you forced me to move to Gemini tomorrow I wouldn't be particularly upset.
  - motorest 1y ago
    > Which dimensions do you see Google lagging on? They seem broadly comparable on the usual leaderboard (https://lmarena.ai/leaderboard https://lmarena.ai/leaderboard) and anecdotally I can't tell the difference in quality. Gemini holds indeed the top spot, but I feel you framed your response quite well: they are all broadly comparable. The difference in the synthetic benchmark from the top spot and the 20th spot was something like 57 points on a scale of 0-1500
- jeanloolz 1y ago
  Depending on how you look at it I suppose but I believe Gemini surpasses OpenAI on many levels now. Better photo and video models. The leaderboard for text and embeddings are also putting Google on top of Openai.
- motorest 1y ago
  > This is not really true. Google has all the compute but in many dimensions they lag behind GPT-5 class (catching up, but it has not been a given). I don't know what you are talking about. I use Gemini on a daily basis and I honestly can't tell a difference. We are at a point where training corpus and hallucinations makes more of a difference than "model class".
- ebonnafoux 1y ago
  gemini-2.5-pro is ranked number 1 in llmarena (https://lmarena.ai/leaderboard https://lmarena.ai/leaderboard) before gpt-5-high. In the Text-to-Video and Image-to-video, google also have the highest places, OpenAI is nowhere.
  - IX-103 1y ago
    Yes, but they're also slower. As LLMs start to be used for more general purpose things, they are becoming a productivity bottle-neck. If I get a mostly right answer in a few seconds that's much better than a perfect answer in 5 minutes. Right now the delay for Google's AI coding assistant is high enough for humans to context switch and do something else while waiting. Particularly since one of the main features of AI code assistants is rapid iteration.
    - janalsncm 1y ago
      Anecdotally, Gemini pro is way faster than GPT 5 thinking. Flash is even faster. I have no numbers though.
jojobas 1y ago
Amazon retail runs on ridiculously low margins compared to AWS. Revenue-wise retail dwarfs AWS, profit-wise it's vice-versa.
VirusNewbie 1y ago
Are you arguing anthropic has more compute than Amazon? Are you saying the only reason Meta is behind everyone else is compute????
- motorest 1y ago
  > Are you arguing anthropic has more compute than Amazon? I wouldn't be surprised if the likes of Anthropic wasn't paying AWS for its compute. As the saying goes, the ones who got rich from the gold rush were the ones selling shovels.
  - ospray 1y ago
    I wouldn't be surprised if Amazon just buys Anthropic or another lab rather than competing for individuals.
- benterix 1y ago
  Think well: why should a platform provider get into a terribly expensive and unprofitable business when they can just provide hardware for those with money to spend? This was AWS strategy for years and it's been working well for them.
bhl 1y ago
The moat is people, data, and compute in that order. It’s not just compute. That has mostly plateaued. What matters now is quality of data and what type of experiments to run, which environments to build.
- sigmoid10 1y ago
  This "moat" is actually constantly shifting (which is why it isn't really a moat to begin with). Originally, it was all about quality data sources. But that saturated quite some time ago (at least for text). Before RLHF/RLAIF it was primarily a race who could throw more compute at a model and train longer on the same data. Then it was who could come up with the best RL approach. Now we're back to who can throw more compute at it since everyone is once again doing pretty much the same thing. With reasoning we now also opened a second avenue where it's all about who can throw more compute at it during runtime and not just while training. So in the end, it's mostly about compute. The last years have taught us that any significant algorithmic improvement will soon permeate across the entire field, no matter who originally invented it. So people are important for finding this stuff, but not for making the most of it. On top of that, I think we are very close to the point where LLMs can compete with humans on their own algorithmic development. Then it will be even more about who can spend more compute, because there will be tons of ideas to evaluate.
  - DrScientist 1y ago
    To put that into a scientific context - compute is capacity to do experiments and generate data ( about how best to build models ). However I do think you are missing an important aspect - and that's people who properly understand important solvable problems. ie I see quite a bit "we will solve this x, with AI' from startup's that don't fundamentally understand x.
    - sigmoid10 1y ago
      >we will solve this x, with AI You usually see this from startup techbro CEOs understand neither x nor AI. Those people are already replacable by AI today. The kind of people who think they can query ChatGPT once with "How to create a cutting edge model" and make millions. But when you go in on the deep end, there are very few people who still have enough tech knowledge to compete with your average modern LLM. And even the Math Olympiad gold medalists high-flyers at DeepSeek are about to have a run for their money with the next generation. Current AI engineers will shift more and more towards senior architecture and PM roles, because those will be the only ones that matter. But PM and architecture is already something that you could replace today.
Lyapunov_Lover 1y ago
> The evidence shows that there is no methodological moat for LLMS. Does it? Then how come Meta hasn't been able to release a SOTA model? It's not for a lack of trying. Or compute. And it's not like DeepSeek had access to vastly more compute than other Chinese AI companies. Alibaba and Baidu have been working on AI for a long time and have way more money and compute, but they haven't been able to do what DeepSeek did.
- postexitus 1y ago
  They may not have been leading (as in, releasing a SOTA model), but they definitely can match others - easily, as shown by llama 3/4, which proves the point - there is no moat. With enough money and resources, you can match others. Whether without SOTA models you can make a business out of it is a different question.
  - Lyapunov_Lover 1y ago
    Meta never matched the competition with their Llama models. They've never even come close. And Llama 4 was an actual disaster.
    - postexitus 1y ago
      I am not a daily user, so only rely on reviews and benchmarks - actual experience may be different.
      - YetAnotherNick 1y ago
        Even in reviews and benchmark, Llama wasn't close to frontier models. Also Llama 2/3 lead in open weight models wasn't more than few months.
  - ath3nd 1y ago
    > but they definitely can match others - easily, as shown by llama 3/4 Are we living in the same universe? LLAMA is universally recognized as one of the worst and least successful model releases. I am almost certain you haven't ever tried a LLAMA chat, because, by the beard of Thor, it's the worst experience anyone could ever had, with any LLM. LLAMA 4 (behemoth, whatever, whatever) is an absolute steaming pile of trash, not even close to ChatGPT 4o/4/5/, Gemini(any) and even not even close to cheaper ones like DeepSeek. And to think Meta pirated torrents to train it... What a bunch of criminal losers and what a bunch of waste of money, time and compute. Oh, at least the Metaverse is a success... https://www.pcgamer.com/gaming-industry/court-documents-show-not-only-did-meta-torrent-terabytes-of-pirated-books-to-train-ai-models-employees-wouldnt-stop-emailing-each-other-about-it-torrenting-from-a-corporate-laptop-doesnt-feel-right/ https://www.pcgamer.com/gaming-industry/court-documents-show... https://www.cnbc.com/2025/06/27/the-metaverse-as-we-knew-it-failed-but-its-resurrecting-in-new-ways.html https://www.cnbc.com/2025/06/27/the-metaverse-as-we-knew-it-...
cedws 1y ago
A lot of C-suite people seem to have an idea that if they just throw enough compute at LLMs that AGI will eventually emerge, even though it's pretty clear at this point that LLMs are never going to lead to general intelligence. In their view it makes sense to invest massive amounts of capital because it's like a lottery ticket to being the future AGI company that dominates the world. I recall Zuckerberg saying something about how there were early signs of AI "improving itself." I don't know what he was talking about but if he really believes that's true and that we're at the bottom of an exponential curve then Meta's rabid hiring and datacenter buildout makes sense.
- stuaxo 1y ago
  Its insane really, anyone who has worked with LLMs for a bit and has an idea of how they work shouldn't think its going to lead to "AGI". Hopefully some big players, like FB bankrupt themselves.
  - guardian5x 1y ago
    Just scaling them up might not leat to "AGI", but they can still lead to AGI as a bridge.
  - blackhaz 1y ago
    Is it widely accepted that LLMs won't lead to AGI? I've asked Gemini, so it came up with four primary arguments for this claim, commenting on them briefly: 1) LLMs as simple "next token predictors" so they just mimicry thinking: But can it be argued that current models operate on layers of multiple depth and are able to actually understand by building concepts and making connections on abstract levels? Also, don't we all mimicry? 2) Grounding problem: Yes, models build their world models on text data, but we have models operating on non-textual data already, so this appears to be a technical obstacle rather than fundamental. 3) Lack of World Model. But can anyone really claim they have a coherent model of reality? There are flat-earthers, yet I still wouldn't deny them having AGI. People hallucinate and make mistakes all the time. I'd argue hallucinations is in fact the sign of an emerging intelligence. 4) Fixed learning data sets. Looks like this is now being actively solved with self-improving models? I just couldn't find a strong argument supporting this claim. What am I missing?
abtinf 1y ago
The idea that models are copyrightable is also extremely dubious. So there probably isn’t even a legal moat.
ml-anon 1y ago
Lets not pretend this is strategy. Amazon has been trying and failing to hire top AI people. No-one in their right minds would join. Even Meta has to shell out 8-9 figures for top people, who with any modicum of talent or self respect would go to Amazon rather than Anthropic, OAI, GDM? They bought Adept, everyone left. AWS is also falling far behind Azure wrt serving AI needs at the frontier. GCP is also growing at a faster rate and has a way more promising future than AWS in this space.
- mikert89 1y ago
  AWS is very far behind, its already impacting the stock. Without a winning AI offering, all new cloud money is going to GCP and Azure. They have a huge problem
StopDisinfo910 1y ago
The barriers to entry for LLM are obvious: as you pointed, the field is extremely capital intensive. The only reason there are seemingly multiple players is because the amount of capital thrown at it at the moment is tremendous but that's unlikely to last forever. From my admittely poorly informed point of view, strategy-wise, it's hard to tell how wise it is investing in foundational work at the moment. As long as some players release competitive open weight models, the competitive advantage of being a leader in R&D will be limited. Amazon already has the compute power to place itself as a reseller without investing or having to share the revenue generated. Sure, they won't be at the forefront but they can still get their slice of the pie without exposing themselves too much to an eventual downturn.
DrScientist 1y ago
I think I'm right in saying that AWS, rather than deliveries, is by far the most profitable part of Amazon. Also a smart move is to be selling shovels in a gold rush - and that's exactly what Amazon is doing with AWS.
energy123 1y ago
There's not much of an architectural moat, but there is a methodological moat, such as with RL synthetic data.

karterk 1y ago

> The moat of the frontier folks is just compute. This is not really true. Google has all the compute but in many dimensions they lag behind GPT-5 class (catching up, but it has not been a given). Amazon itself did try to train a model (so did Meta) and had limited success.

jorisboris 1y ago
Yes, or Apple who with all the talent don’t manage to pull off anything useful in AI xAI seems to be the exception, not the rule
- rusk 1y ago
  Given Apple’s moat is their devices, their particular spin on AI is very much edge focussed, which isn’t as spectacular as the current wave of cloud based LLM. Apple’s cloud stuff is laughably poor.
matusp 1y ago
I switched to Gemini with my new phone and I literally couldn't tell a difference. It is actually crazy how small the cost of switching is for LLMs. It feels like AI is more like a commodity than a service.
- lelanthran 1y ago
  > I switched to Gemini with my new phone and I literally couldn't tell a difference. It is actually crazy how small the cost of switching is for LLMs. It feels like AI is more like a commodity than a service. It is. It's wild to me that all these VCs pouring money into AI companies don't know what a value-chain is. Tokens are the bottom of the value-chain; it's where the lowest margins exist because the product at that level is a widely available commodity. I wrote about this already (shameless plug: https://www.rundata.co.za/blog/index.html?the-ai-value-chain https://www.rundata.co.za/blog/index.html?the-ai-value-chain )
- physicsguy 1y ago
  On top of that, the on-device models have got stronger and stronger as the base models + RL has got better. You can do on your laptop now what 2 years ago was state of the art.
Keyframe 1y ago
" in many dimensions they lag behind GPT-5 class " - such as? Outside of computer, "the moat" is also data to train on. That's an even wider moat. Now, google has all the data. Data no one else has or ever will have. If anything, I'd expect them to outclass everyone by a fat margin. I think we're seeing that on video however.
- willvarfar 1y ago
  not according to google: “We have no moat, and neither does OpenAI”: the big memo and the big HN thread on same https://news.ycombinator.com/item?id=35813322 https://news.ycombinator.com/item?id=35813322
  - Keyframe 1y ago
    a bit weird to think about it since google has literally internet.zip in multiple versions over the years, all of email, all of usenet, all of the videos, all of the music, all of the user's search interest, ads, everything..
    - lelanthran 1y ago
      > a bit weird to think about it since google has literally internet.zip in multiple versions over the years, all of email, all of usenet, all of the videos, all of the music, all of the user's search interest, ads, everything.. Yeah, Google totally has a moat. Them saying that they have no moat doesn't magically make that moat go away. They also own the entire vertical which none of the competitors do - all their competitors have to buy compute from someone who makes a profit just on compute (Nvidia, for example). Google owns the entire vertical, from silicon to end-user. It would be crazy if they can't make this work.
    - rvba 1y ago
      That's why robots make so much traffic now. Those other companies are trying to get data. Google theoretically has reddit access. I wonder if they have sort of an internet archive - data unpolutted by LLMs On a side note, funny how all the companies seem to train on book archivr which they just downloaded from the internet
paulddraper 1y ago
It doesn’t guarantee success, but the point stands about X and Deepseek
gnfargbl 1y ago
Which dimensions do you see Google lagging on? They seem broadly comparable on the usual leaderboard (https://lmarena.ai/leaderboard https://lmarena.ai/leaderboard) and anecdotally I can't tell the difference in quality. I tend personally to stick with ChatGPT most of the time, but only because I prefer the "tone" of the thing somehow. If you forced me to move to Gemini tomorrow I wouldn't be particularly upset.
- motorest 1y ago
  > Which dimensions do you see Google lagging on? They seem broadly comparable on the usual leaderboard (https://lmarena.ai/leaderboard https://lmarena.ai/leaderboard) and anecdotally I can't tell the difference in quality. Gemini holds indeed the top spot, but I feel you framed your response quite well: they are all broadly comparable. The difference in the synthetic benchmark from the top spot and the 20th spot was something like 57 points on a scale of 0-1500
jeanloolz 1y ago
Depending on how you look at it I suppose but I believe Gemini surpasses OpenAI on many levels now. Better photo and video models. The leaderboard for text and embeddings are also putting Google on top of Openai.
motorest 1y ago
> This is not really true. Google has all the compute but in many dimensions they lag behind GPT-5 class (catching up, but it has not been a given). I don't know what you are talking about. I use Gemini on a daily basis and I honestly can't tell a difference. We are at a point where training corpus and hallucinations makes more of a difference than "model class".
ebonnafoux 1y ago
gemini-2.5-pro is ranked number 1 in llmarena (https://lmarena.ai/leaderboard https://lmarena.ai/leaderboard) before gpt-5-high. In the Text-to-Video and Image-to-video, google also have the highest places, OpenAI is nowhere.
- IX-103 1y ago
  Yes, but they're also slower. As LLMs start to be used for more general purpose things, they are becoming a productivity bottle-neck. If I get a mostly right answer in a few seconds that's much better than a perfect answer in 5 minutes. Right now the delay for Google's AI coding assistant is high enough for humans to context switch and do something else while waiting. Particularly since one of the main features of AI code assistants is rapid iteration.
  - janalsncm 1y ago
    Anecdotally, Gemini pro is way faster than GPT 5 thinking. Flash is even faster. I have no numbers though.

jojobas 1y ago

Amazon retail runs on ridiculously low margins compared to AWS. Revenue-wise retail dwarfs AWS, profit-wise it's vice-versa.

VirusNewbie 1y ago

Are you arguing anthropic has more compute than Amazon? Are you saying the only reason Meta is behind everyone else is compute????

motorest 1y ago
> Are you arguing anthropic has more compute than Amazon? I wouldn't be surprised if the likes of Anthropic wasn't paying AWS for its compute. As the saying goes, the ones who got rich from the gold rush were the ones selling shovels.
- ospray 1y ago
  I wouldn't be surprised if Amazon just buys Anthropic or another lab rather than competing for individuals.
benterix 1y ago
Think well: why should a platform provider get into a terribly expensive and unprofitable business when they can just provide hardware for those with money to spend? This was AWS strategy for years and it's been working well for them.

bhl 1y ago

The moat is people, data, and compute in that order. It’s not just compute. That has mostly plateaued. What matters now is quality of data and what type of experiments to run, which environments to build.

sigmoid10 1y ago
This "moat" is actually constantly shifting (which is why it isn't really a moat to begin with). Originally, it was all about quality data sources. But that saturated quite some time ago (at least for text). Before RLHF/RLAIF it was primarily a race who could throw more compute at a model and train longer on the same data. Then it was who could come up with the best RL approach. Now we're back to who can throw more compute at it since everyone is once again doing pretty much the same thing. With reasoning we now also opened a second avenue where it's all about who can throw more compute at it during runtime and not just while training. So in the end, it's mostly about compute. The last years have taught us that any significant algorithmic improvement will soon permeate across the entire field, no matter who originally invented it. So people are important for finding this stuff, but not for making the most of it. On top of that, I think we are very close to the point where LLMs can compete with humans on their own algorithmic development. Then it will be even more about who can spend more compute, because there will be tons of ideas to evaluate.
- DrScientist 1y ago
  To put that into a scientific context - compute is capacity to do experiments and generate data ( about how best to build models ). However I do think you are missing an important aspect - and that's people who properly understand important solvable problems. ie I see quite a bit "we will solve this x, with AI' from startup's that don't fundamentally understand x.
  - sigmoid10 1y ago
    >we will solve this x, with AI You usually see this from startup techbro CEOs understand neither x nor AI. Those people are already replacable by AI today. The kind of people who think they can query ChatGPT once with "How to create a cutting edge model" and make millions. But when you go in on the deep end, there are very few people who still have enough tech knowledge to compete with your average modern LLM. And even the Math Olympiad gold medalists high-flyers at DeepSeek are about to have a run for their money with the next generation. Current AI engineers will shift more and more towards senior architecture and PM roles, because those will be the only ones that matter. But PM and architecture is already something that you could replace today.

Lyapunov_Lover 1y ago

> The evidence shows that there is no methodological moat for LLMS. Does it? Then how come Meta hasn't been able to release a SOTA model? It's not for a lack of trying. Or compute. And it's not like DeepSeek had access to vastly more compute than other Chinese AI companies. Alibaba and Baidu have been working on AI for a long time and have way more money and compute, but they haven't been able to do what DeepSeek did.

postexitus 1y ago
They may not have been leading (as in, releasing a SOTA model), but they definitely can match others - easily, as shown by llama 3/4, which proves the point - there is no moat. With enough money and resources, you can match others. Whether without SOTA models you can make a business out of it is a different question.
- Lyapunov_Lover 1y ago
  Meta never matched the competition with their Llama models. They've never even come close. And Llama 4 was an actual disaster.
  - postexitus 1y ago
    I am not a daily user, so only rely on reviews and benchmarks - actual experience may be different.
    - YetAnotherNick 1y ago
      Even in reviews and benchmark, Llama wasn't close to frontier models. Also Llama 2/3 lead in open weight models wasn't more than few months.
- ath3nd 1y ago
  > but they definitely can match others - easily, as shown by llama 3/4 Are we living in the same universe? LLAMA is universally recognized as one of the worst and least successful model releases. I am almost certain you haven't ever tried a LLAMA chat, because, by the beard of Thor, it's the worst experience anyone could ever had, with any LLM. LLAMA 4 (behemoth, whatever, whatever) is an absolute steaming pile of trash, not even close to ChatGPT 4o/4/5/, Gemini(any) and even not even close to cheaper ones like DeepSeek. And to think Meta pirated torrents to train it... What a bunch of criminal losers and what a bunch of waste of money, time and compute. Oh, at least the Metaverse is a success... https://www.pcgamer.com/gaming-industry/court-documents-show-not-only-did-meta-torrent-terabytes-of-pirated-books-to-train-ai-models-employees-wouldnt-stop-emailing-each-other-about-it-torrenting-from-a-corporate-laptop-doesnt-feel-right/ https://www.pcgamer.com/gaming-industry/court-documents-show... https://www.cnbc.com/2025/06/27/the-metaverse-as-we-knew-it-failed-but-its-resurrecting-in-new-ways.html https://www.cnbc.com/2025/06/27/the-metaverse-as-we-knew-it-...

cedws 1y ago

A lot of C-suite people seem to have an idea that if they just throw enough compute at LLMs that AGI will eventually emerge, even though it's pretty clear at this point that LLMs are never going to lead to general intelligence. In their view it makes sense to invest massive amounts of capital because it's like a lottery ticket to being the future AGI company that dominates the world. I recall Zuckerberg saying something about how there were early signs of AI "improving itself." I don't know what he was talking about but if he really believes that's true and that we're at the bottom of an exponential curve then Meta's rabid hiring and datacenter buildout makes sense.

stuaxo 1y ago
Its insane really, anyone who has worked with LLMs for a bit and has an idea of how they work shouldn't think its going to lead to "AGI". Hopefully some big players, like FB bankrupt themselves.
- guardian5x 1y ago
  Just scaling them up might not leat to "AGI", but they can still lead to AGI as a bridge.
- blackhaz 1y ago
  Is it widely accepted that LLMs won't lead to AGI? I've asked Gemini, so it came up with four primary arguments for this claim, commenting on them briefly: 1) LLMs as simple "next token predictors" so they just mimicry thinking: But can it be argued that current models operate on layers of multiple depth and are able to actually understand by building concepts and making connections on abstract levels? Also, don't we all mimicry? 2) Grounding problem: Yes, models build their world models on text data, but we have models operating on non-textual data already, so this appears to be a technical obstacle rather than fundamental. 3) Lack of World Model. But can anyone really claim they have a coherent model of reality? There are flat-earthers, yet I still wouldn't deny them having AGI. People hallucinate and make mistakes all the time. I'd argue hallucinations is in fact the sign of an emerging intelligence. 4) Fixed learning data sets. Looks like this is now being actively solved with self-improving models? I just couldn't find a strong argument supporting this claim. What am I missing?

abtinf 1y ago

The idea that models are copyrightable is also extremely dubious. So there probably isn’t even a legal moat.

ml-anon 1y ago

Lets not pretend this is strategy. Amazon has been trying and failing to hire top AI people. No-one in their right minds would join. Even Meta has to shell out 8-9 figures for top people, who with any modicum of talent or self respect would go to Amazon rather than Anthropic, OAI, GDM? They bought Adept, everyone left. AWS is also falling far behind Azure wrt serving AI needs at the frontier. GCP is also growing at a faster rate and has a way more promising future than AWS in this space.

mikert89 1y ago
AWS is very far behind, its already impacting the stock. Without a winning AI offering, all new cloud money is going to GCP and Azure. They have a huge problem

StopDisinfo910 1y ago

The barriers to entry for LLM are obvious: as you pointed, the field is extremely capital intensive. The only reason there are seemingly multiple players is because the amount of capital thrown at it at the moment is tremendous but that's unlikely to last forever. From my admittely poorly informed point of view, strategy-wise, it's hard to tell how wise it is investing in foundational work at the moment. As long as some players release competitive open weight models, the competitive advantage of being a leader in R&D will be limited. Amazon already has the compute power to place itself as a reseller without investing or having to share the revenue generated. Sure, they won't be at the forefront but they can still get their slice of the pie without exposing themselves too much to an eventual downturn.

DrScientist 1y ago

I think I'm right in saying that AWS, rather than deliveries, is by far the most profitable part of Amazon. Also a smart move is to be selling shovels in a gold rush - and that's exactly what Amazon is doing with AWS.

energy123 1y ago

There's not much of an architectural moat, but there is a methodological moat, such as with RL synthetic data.