8 ms·

A lot of C-suite people seem to have an idea that if they just throw enough compute at LLMs that AGI will eventually emerge, even though it's pretty clear at th

• stuaxo 1y ago
  Its insane really, anyone who has worked with LLMs for a bit and has an idea of how they work shouldn't think its going to lead to "AGI". Hopefully some big players, like FB bankrupt themselves.
  • guardian5x 1y ago
    Just scaling them up might not leat to "AGI", but they can still lead to AGI as a bridge.
  • blackhaz 1y ago
    Is it widely accepted that LLMs won't lead to AGI? I've asked Gemini, so it came up with four primary arguments for this claim, commenting on them briefly: 1) LLMs as simple "next token predictors" so they just mimicry thinking: But can it be argued that current models operate on layers of multiple depth and are able to actually understand by building concepts and making connections on abstract levels? Also, don't we all mimicry? 2) Grounding problem: Yes, models build their world models on text data, but we have models operating on non-textual data already, so this appears to be a technical obstacle rather than fundamental. 3) Lack of World Model. But can anyone really claim they have a coherent model of reality? There are flat-earthers, yet I still wouldn't deny them having AGI. People hallucinate and make mistakes all the time. I'd argue hallucinations is in fact the sign of an emerging intelligence. 4) Fixed learning data sets. Looks like this is now being actively solved with self-improving models? I just couldn't find a strong argument supporting this claim. What am I missing?
    • globnomulous 1y ago
      Why on earth would you copy and paste an LLM's output into a comment? What does that accomplish or provide that just a simply stated argument doesn't accomplish more succinctly? If you don't know something, simply don't comment on it -- or ask a question.
      • blackhaz 1y ago
        None of the above is AI.
        • globnomulous 1y ago
          > I've asked Gemini, so it came up with four primary arguments for this claim, commenting on them briefly: This line means, and literally says, that everything that follows is a summary or direct quotation from an LLM's output. There's a more charitable but unintuitive interpretation, in which "commenting on them briefly" is intended to mean "I will comment on them briefly:". But this isn't a natural interpretation. It's one I could be expected to reach only after seeing your statement that 'none of the above is AI.' But even this more charitable interpretation actually contradicts your claim that it's not AI. So now I'm even less sure I know what you meant to communicate. Either I'm missing something really obvious or the writing doesn't communicate what you intended.
    • welferkj 1y ago
      Fur future reference, pasting llm slop feels exactly as patronizing as back when people pasted links to google searches in response to questions they considered beneath their dignity to answer. Except in this case, no-one asked to begin with.
  • amelius 1y ago
    The only way we'll have AGI is if people get dumber. Using modern tech like LLMs makes people dumber. Ergo, we might see AGI sooner than expected.
  • menaerus 1y ago
    > ... and has an idea of how they work shouldn't think its going to lead to "AGI" Not sure what level of understanding are you referring to but having learned and researched about the pretty much all LLM internals I think this has led me exactly to the opposite line of thinking. To me it's unbelievable what we have today.
  • IanCal 1y ago
    Tbh I find this view odd, and I wonder what people view as agi now. It used to be that we had extremely narrow pieces of AI and I remember being on a research project about architectures and just very basic “what’s going on?” was advanced. Understanding that someone asked a question, that would be solved by getting a book and being able to then go and navigate to the place the book was likely to be was fancy. Most systems could solve literally one type of problem. They weren’t just bad at other things they were fundamentally incapable of anything but an extremely narrow use case. I can throw wide ranging problems at things like gpt5 and get what seem like dramatically better answers than if I asked a random person. The amount of common sense is so far beyond what we had it’s hard to express. It used to be always pointed out that the things we had were below basic insect level. Now I have something that can research a charity, find grants and make coherent arguments for them, read matrix specs and debug error messages, and understand sarcasm. To me, it’s clear that agi is here. But then what I always pictured from it may be very different to you. What’s your image of it?
    • adwn 1y ago
      I think the discrepancy between different views on the matter mainly stems from the fact that state-of-the-art LLMs are better (sometimes extremely better) at some tasks, and worse (sometimes extremely worse) at other tasks, compared to average humans. For example, they're better at retrieving information from huge amounts of unstructured data. But they're also terrible at learning: any "experience" which falls out of the context window is lost forever, and the model can't learn from its mistakes. To actually make it learn something requires very many examples and a lot of compute, whereas a human can permanently learn from a single example.
      • andsoitis 1y ago
        > human can permanently learn from a single example This, to me at least, seems like an important ingredient to satisfying a practical definition / implementation of AGI. Another might be curiosity, and I think perhaps also agency.
    • AlienRobot 1y ago
      Nobody is saying that LLM's don't work like magic. I know how neural networks work and they still feel like voodoo to me. What we are saying is that LLM's can't become AGI. I don't know what AGI will look like, but it won't look like an LLM. There is a difference between being able to melt iron and being able to melt tungsten.
  • thaawyy33432434 1y ago
    Recently I realized that US are very close to a centrally planned economy. Meta wasted 50B on metaverse, which like how much Texas spends on healthcare. Now the "AI" investments seems dubious. You could fund 1000+ projects with this kinds of money. This is not an effective capital allocation.
  • meowface 1y ago
    This is not and has not been the consensus opinion. If you're not an AI researcher you shouldn't write as if you've set your confidence parameter to 0.95. Of course it might be the case, but it's not a thing that should be expressed with such confidence.
  • janalsncm 1y ago
    I think AI research is like anything else really. The smartest people are heads down working on their problems. The people going on podcasts are less connected to day to day work. It’s also pretty useless to talk about whether something is AGI without defining intelligence in the first place.
  • foobarian 1y ago
    I think something like we saw in the show "Devs" is much more likely, although what the developers did with it in the show was bonkers unrealistic. But some kind of big enough quantum device basically.
• CrossVR 1y ago
  I don't know if AGI will emerge from LLM, but I'm always reminded of the Chinese room thought experiment. With billions thrown at the idea it will certainly be the ultimate answer as to whether true understanding can emerge from a large enough dictionary.
  • torginus 1y ago
    Please stop refering to the Chinese Room - it's just magical/deist thinking in disguise. It postulates that humans have way of 'understanding' things that is impossible to replicate mechanically. The fact that philosophy hasn't recognized and rejected this argument based on this speaks volumes of the quality of arguments accepted there. (That doesn't mean LLMs are or will be AGI, its just this argument is tautological and meaningless)
    • simianparrot 1y ago
      It is still relevant because it hasn’t been disproven yet. So far all computer programs are Chinese Rooms, LLM’s included.
      • IanCal 1y ago
        If you’re talking about it being proven or disproven you’re misunderstanding the point of the thought experiment.
    • armada651 1y ago
      That some people use the Chinese Room to ascribe some magical properties to human consciousness says more about the person drawing that conclusion than the thought experiment itself. I think it's entirely valid to question whether a computer can form an understanding through deterministically processing instructions, whether that be through programming language or language training data. If the answer is no, that shouldn't lead to a deist conclusion. It can just as easily lead to the conclusion that a non-deterministic Turing machine is required.
      • 1y ago
• epolanski 1y ago
  I don't get it, I really don't. Even assuming a company gets to AGI first this doesn't mean another one will follow. Suppose that FooAI gets to it first: - competitors may get there too in a different or more efficient way - Some FooAI staff can leave and found their own company - Some FooAI staff can join a competitor - FooAI "secret sauce" can be figured out, or simply stolen, by a competitor At the end of the day, it really doesn't matter, the equation AI === commodity just does not change. There is no way to make money by going into this never ending frontier model war, price of training keeps getting higher and higher, but your competitors few months later can achieve your own results for a fraction of your $.
  • cedws 1y ago
    Some would say that the race to AGI is like the race to nuclear weapons and that the first to get there will hold all the cards (and be potentially able to stop others getting there.) It's a bit too sci-fi for me.
    • Yossarrian22 1y ago
      If AGI is reached it would be trivial for the competing superpowers to completely quarantine themselves network wise by cutting undersea cables long enough to develop competing AGI
• qcnguy 1y ago
  > I don't know what he was talking about There's a bunch of ways AI is improving itself, depending on how you want to interpret that. But it's been true since the start. 1. AI is used to train AI. RLHF uses this, curriculum learning is full of it, video model training pipelines are overflowing with it. AI gets used in pipelines to clean and upgrade training data a lot. 2. There are experimental AI agents that can patch their own code and explore a tree of possibilities to boost their own performance. However, at the moment they tap out after getting about as good as open source agents, but before they're as good as proprietary agents. There isn't exponential growth. There might be if you throw enough compute at it, but this tactic is very compute hungry. At current prices it's cheaper to pay an AI expert to implement your agent than use this.
  • Yoric 1y ago
    > There are experimental AI agents that can patch their own code and explore a tree of possibilities to boost their own performance. However, at the moment they tap out after getting about as good as open source agents, but before they're as good as proprietary agents. Interesting. Do you have links?
    • torbab 1y ago
      Not OP, but the Darwin Godel machine comes to mind: https://arxiv.org/abs/2505.22954 https://arxiv.org/abs/2505.22954
      • qcnguy 1y ago
        That's the one!
      • Yoric 1y ago
        Thanks!
  • franktankbank 1y ago
    I'm skeptical that RLHF really works. Doesn't it just patch the obvious holes so it looks better on paper? If it can't reason then it will continue to get 2nd and 3rd order difficulty problems wrong.
• hliyan 1y ago
  In early 2023, I remember someone breathlessly explaining that there are signs that LLMs that are seemingly good at chess/checkers moves may have a rudimentary model of the board within them, somehow magically encoded into the model weights through the training. I was stupid enough to briefly entertain the possibility until I actually bothered to develop a high level understanding of the transformer architecture. It's surprising how much mysticism this field seems to attract. Perhaps it being a non-deterministic, linguistically invoked black box, triggers the same internal impulses that draw some people to magic and spellcasting.
  • pegasus 1y ago
    Just because it's not that hard to reach a high-level understanding of the transformer pipeline doesn't mean we understand how these systems function, or that there can be no form of world model that they are developing. Recently there has been more evidence for that particular idea [1]. The feats of apparent intelligence LLMs sometimes display have taken even their creators by surprise. Sure, there's a lot of hype too, that's part and parcel of any new technology today, but we are far from understanding what makes them perform so well. In that sense, yeah you could say they are a bit "magical". [1] https://the-decoder.com/new-othello-experiment-supports-the-world-model-hypothesis-for-large-language-models/ https://the-decoder.com/new-othello-experiment-supports-the-...
    • ath3nd 1y ago
      > Just because it's not that hard to reach a high-level understanding of the transformer pipeline doesn't mean we understand how these systems function Mumbo jumbo magical thinking. They perform so well because they are trained on probabilistic token matching. Where they perform terribly, e.g math, reasoning, they are delegating to other approaches, and that's how you get the illusion that there is actually something there. But it's not. Faking intelligence is not intelligence. It's just text generation. > In that sense, yeah you could say they are a bit "magical" Nobody but the most unhinged hype pushers are calling it "magical". The LLM can never ever be AGI. Guessing the next word is not intelligence. > there can be no form of world model that they are developing Kind of impossible to form a world model if your foundation is probabilistic token guessing which is what LLMs are. LLMs are a dead end in achieving "intelligence", something novel as an approach needs to be discovered (or not) to go into the intelligence direction. But hey, at least we can generate text fast now!
• abraxas 1y ago
  > it's pretty clear at this point that LLMs are never going to lead to general intelligence. It is far from clear. There may well be emergent hierarchies of more abstract thought at much higher numbers of weights. We just don't know how a transformer will behave if one is built with 100T connections - something that would finally approach the connectome level of a human brain. Perhaps nothing interesting but we just do not know this and the current limitation in building such a beast is likely not software but hardware. At these scales the use of silicon transistors to approximate analog curve switching models just doesn't make sense. True neuromorphic chips may be needed to approach the numbers of weights necessary for general intelligence to emerge. I don't think there is anything in production at the moment that could rival the efficiency of biological neurons. Most likely we do not need that level of efficiency. But it's almost certain that stringing together a bunch of H100s isn't a path to the scale we should be aiming for.