4 ms·

> the singularity is happening [Citation needed] No LLM is yet being used effectively to improve LLM output in exponential ways. Personally, I'm skeptical tha

by danaris 9mo ago

> the singularity is happening [Citation needed] No LLM is yet being used effectively to improve LLM output in exponential ways. Personally, I'm skeptical that such a thing is possible. LLMs aren't AGI, and aren't a path to AGI. The Singularity is the Rapture for techbros.

Der_Einzige 9mo ago
If you look at the rapid acceleration of progress and conclude this way, well, de nile ain't just a river in egypt. Also yes LLMs are indeed AGI: https://www.noemamag.com/artificial-general-intelligence-is-already-here/ https://www.noemamag.com/artificial-general-intelligence-is-... This was Peter Norvig's take. AGI is a low bar because most humans are really stupid.
- danaris 9mo ago
  What rapid acceleration? I look at the trajectory of LLMs, and the shape I see is one of diminishing returns. The improvements in the first few generations came fast, and they were impressive. Then subsequent generations took longer, improved less over the previous generation, and required more and more (and more and more) resources to achieve. I'm not interested in one guy's take that LLMs are AGI, regardless of his computer science bonafides. I can look at what they do myself, and see that they aren't, by most very reasonable definitions of AGI. If you really believe that the singularity is happening now...well, then, shouldn't it take a very short time for the effects of that to be painfully obvious? Like, massive improvements in all kinds of technology coming in a matter of months? Come back in a few months and tell me what amazing new technologies this supposed AGI has created...or maybe the one in denial isn't me.
  - steveBK123 9mo ago
    > I look at the trajectory of LLMs, and the shape I see is one of diminishing returns It seems even more true if you look at OpenAI funding thru 2022 initial public release to how spending has exponentially increased to deliver improvements since. We’re now talking upwards of $600B/yr of spending on LLM based AI infrastructure across the industry in 2026.
  - thesz 9mo ago
    These diminishing returns are exponentially diminishing returns [1]. [1] https://arxiv.org/abs/2404.04125 https://arxiv.org/abs/2404.04125 " We consistently find that, far from exhibiting "zero-shot" generalization, multimodal models require exponentially more data to achieve linear improvements in downstream "zero-shot" performance, following a sample inefficient log-linear scaling trend. This trend persists even when controlling for sample-level similarity between pretraining and downstream datasets, and testing on purely synthetic data distributions. Furthermore, upon benchmarking models on long-tailed data sampled based on our analysis, we demonstrate that multimodal models across the board perform poorly."
- mattgreenrocks 9mo ago
  If you think AGI is at hand why are you trying to sway a bunch of internet randos who don’t get it? :) Use those god-like powers to make the life you want while it’s still under the radar.
  - Davidzheng 9mo ago
    how do you take over the world if you have access to 1000 normal people? if AGI is by the original definition (long forgotten by now) of surpassing MEDIAN human at almost all tasks. How the rebranding of ASI into AGI happened without anyone noticing is kind of insane
  - int_19h 9mo ago
    AGI is not a "godlike power" under this definition.
- sunrunner 9mo ago
  > rapid acceleration Who was it who stated that every exponential was just a sigmoid in disguise? > most humans are really stupid. Statistically, don't we all sort of fit somewhere along a bell curve?
  - pixl97 9mo ago
    The bell curve of IQ and being stupid probably don't have much to do with each other. Think of stupidity as the consequences of interacting with ones environment with negative outcomes. If you have a simple environment with few negative outcomes, then even someone with a 80 IQ may not be considered stupid. But if your environment rapidly grows more complex and the amount of thinking you have to do for positive outcomes increases then even someone with a 110 IQ may find themselves quickly in trouble.
- afavour 9mo ago
  > If you look at the rapid acceleration of progress I don’t understand this perspective. There are numerous examples of technical progress that then stalls out. Just look at batteries for example. Or ones where advancements are too expensive for widespread use (e.g. why no one flies Concorde any more) Why is previous progress a guaranteed indicator of future progress?
  - pixl97 9mo ago
    Just think of this as risk management. If AGI doesn't happen, then good. You get to keep working and playing and generally screwing off in the way that humans have for generations. On the other hand if AGI happens, especially any time soon, you are exceptionally fucked along with me. The world changes very rapidly and there is no getting off Mr Bones wild ride. >Why is previous progress a guaranteed indicator of future progress? In this case, because nature already did it. We're not just inventing and testing something whole cloth. And we know there are still massive efficiencies to be gained. For me the Concorde is an example of how people look at stuff incorrectly. In the past we had to send people places very quickly to do things. This was very expensive and inefficient. I don't need to get on a plane to have an effect just about anywhere else in the world now. The internet and digital mediums give me a presence at other locations that is very close to being there. We didn't need planes that fly at the speed of sound, we needed strings that communicate at the speed of light.
- coldtea 9mo ago
  >If you look at the rapid acceleration of progress and conclude this way There's no "rapid acceleration of progress". If anything there's a decline, and even an economic decline. Take away the financial bubbles based on deregulation and huge explosion of debt, and the last 40 years of "economic progress" are just a mirage filling a huge bubble with air in actual advancement terms - unlike the previous millenia.
  - azan_ 9mo ago
    That’s completely wrong. There was barely any progress in previous millennia. There was even economic Nobel prize for showing why!
    - coldtea 9mo ago
      The GDP per capita of the world has been slowly increasing for several millenia. Same for the advancements in core technology. The industrial revolution increased the pace, but it was already there, not flat or randomly flunctuating (think ancient hominids versus early agriculture vs bronge age, vs ancient Babylon and Assyrian empires, vs later Greece, and Persia, later Rome, later Renaissance and so on). Post 1970s most of the further increase has been based on mirages due to financialization, and doesn't reflect actual improvement.
      - azan_ 9mo ago
        > Post 1970s most of the further increase has been based on mirages due to financialization, and doesn't reflect actual improvement. Of course it does. It would be good if you would try to actually support such controversial claims with data.
      - inglor_cz 9mo ago
        If you got a random cancer in 1970, your chances of survival would be only somewhat better than during the Roman Empire, so slightly better than zero, but not very much so. Even Hodgkin's lymphoma, nowadays a very manageable disease, was reliably deadly. In 2026, unless you are really lucky your chances of being treated into remission are much better. That is a meaningful improvement. Medical science made a HUGE progress in the last 50 years alone. The oldest IVF baby is not even fifty yet.
- darkerside 9mo ago
  Yes, and that's why surpassing it doesn't lead to a singularity except over an infinite timeframe. This whole thing was stupid in the first place.
Recurecur 9mo ago
In my opinion, LLMs provide one piece of AGI. The only intelligence I’ve directly experienced is my own. I don’t consciously plan what I’m saying (or writing right now). Instead, a subconscious process assembles the words to support my stream of consciousness. I think that LLMs are very similar, if not identical. Stream of thought is accomplishing something superficially similar to consciousness, but without the ability to be innovative. At any rate, until there’s an artificial human level stream of consciousness in the mix for each AI, I doubt we’ll see a group of AIs collaborating to produce a significantly improved new generation of AI hardware and software minus human involvement. Once that does happen, the Singularity is at hand.
- Recurecur 9mo ago
  So, I was downvoted twice, yet neither of you doubtless brilliant individuals bothered to refute my points. I’ll emphasize my main point further, LLMs have no ability to innovate much beyond the vast data they’ve scraped. They’re almost entirely derivative. They are a definite improvement over traditional search engines though!
  - int_19h 9mo ago
    Your points seem to boil down to "it hasn't happened yet, so it will never happen". There really isn't much to engage with.
jodrellblank 9mo ago
LLMs aren't AGI and maybe aren't a path to AGI, but step back and look at the way the world is changing. Hard disks were invented by IBM in 1953 and now less than a hundred years later there's an estimated million terabytes a year of hard disks made and sold, and a total sold of Mega, Giga, Tera, Peta, Exa, Zetta ... 1.36 Zettabytes. In 2000, webcams were barely a thing, audio was often recorded to dictaphone tapes, and now you can find a recorded photo or video of roughly anyone and anything on Earth. Maybe a tenth of all humans, almost any place, animal, insect, or natural event, almost any machine, mechanism, invention, painting, and a large sampling of "indoors" both public and private, almost any festival or event or tradition, and a very large sampling of "people doing things" and people teaching things for all kinds of skills. And tons of measurements of locations, temperatures, movements, weather, experiment results, and so on. The ability of computers to process information jumped with punched card readers, with electronic computers in the 1950s, again with transistors in the 1970s, semiconductors in the 1980s, commodity computer clusters (Google) in the 1990s, maybe again with multi-core desktops for everyone in the 2000s, with general purpose GPUs in the 2010s, and with faster commodity networking from 10Mbit to 100Gbit and more, and with SATA, then SAS, then RAID, then SSDs. It's now completely normal to check Google Maps to look at road traffic and how busy stores are (picked up in near realtime from the movement of smartphones around the planet), to do face and object recognition and search in photos, to do realtime face editing/enhancement while recording on a smartphone, to track increasing amounts of exercise and health data from increasing numbers of people, to call and speak to people across the planet and have your voice transcribed automatically to text, to realtime face-swap or face-enhance on a mobile chip, to download gigabytes of compressed Wikipedia onto a laptop and play with it in a weekend in Python just for fun. "AI" stuff (LLMs, neural networks and other techniques, PyTorch, TensorFlow, cloud GPUs and TPUs), the increase in research money, in companies competing to hire the best researchers, the increase in tutorials and numbers of people around the world wanting to play with it and being able to do that ... do you predict that by 2030, 2035, 2040, 2045, 2050 ... 2100, we'll have manufactured more compute power and storage than has ever been made, several times over, and made it more and more accessible to more people, and nothing will change, nothing interesting or new will have been found deliberately or stumbled upon accidentally, nothing new will have been understood about human brains, biology, or cognition, no new insights or products or modelling or AI techniques developed or become normal, no once-in-a-lifetime geniuses having any flashes of insight?
thesz 9mo ago
You are right about "LLM improving themselves" is impossible. In fact, LLM make themselves worse, it is called knowledge collapse [1]. [1] https://arxiv.org/abs/2404.03502 https://arxiv.org/abs/2404.03502
- int_19h 9mo ago
  That paper again. LLMs have been trained on synthetic outputs for quite a while since then and they do get better. Turns out there's more to it than that.

by danaris 9mo ago

Der_Einzige 9mo ago

If you look at the rapid acceleration of progress and conclude this way, well, de nile ain't just a river in egypt. Also yes LLMs are indeed AGI: https://www.noemamag.com/artificial-general-intelligence-is-already-here/ https://www.noemamag.com/artificial-general-intelligence-is-... This was Peter Norvig's take. AGI is a low bar because most humans are really stupid.

danaris 9mo ago
What rapid acceleration? I look at the trajectory of LLMs, and the shape I see is one of diminishing returns. The improvements in the first few generations came fast, and they were impressive. Then subsequent generations took longer, improved less over the previous generation, and required more and more (and more and more) resources to achieve. I'm not interested in one guy's take that LLMs are AGI, regardless of his computer science bonafides. I can look at what they do myself, and see that they aren't, by most very reasonable definitions of AGI. If you really believe that the singularity is happening now...well, then, shouldn't it take a very short time for the effects of that to be painfully obvious? Like, massive improvements in all kinds of technology coming in a matter of months? Come back in a few months and tell me what amazing new technologies this supposed AGI has created...or maybe the one in denial isn't me.
- steveBK123 9mo ago
  > I look at the trajectory of LLMs, and the shape I see is one of diminishing returns It seems even more true if you look at OpenAI funding thru 2022 initial public release to how spending has exponentially increased to deliver improvements since. We’re now talking upwards of $600B/yr of spending on LLM based AI infrastructure across the industry in 2026.
- thesz 9mo ago
  These diminishing returns are exponentially diminishing returns [1]. [1] https://arxiv.org/abs/2404.04125 https://arxiv.org/abs/2404.04125 " We consistently find that, far from exhibiting "zero-shot" generalization, multimodal models require exponentially more data to achieve linear improvements in downstream "zero-shot" performance, following a sample inefficient log-linear scaling trend. This trend persists even when controlling for sample-level similarity between pretraining and downstream datasets, and testing on purely synthetic data distributions. Furthermore, upon benchmarking models on long-tailed data sampled based on our analysis, we demonstrate that multimodal models across the board perform poorly."
mattgreenrocks 9mo ago
If you think AGI is at hand why are you trying to sway a bunch of internet randos who don’t get it? :) Use those god-like powers to make the life you want while it’s still under the radar.
- Davidzheng 9mo ago
  how do you take over the world if you have access to 1000 normal people? if AGI is by the original definition (long forgotten by now) of surpassing MEDIAN human at almost all tasks. How the rebranding of ASI into AGI happened without anyone noticing is kind of insane
- int_19h 9mo ago
  AGI is not a "godlike power" under this definition.
sunrunner 9mo ago
> rapid acceleration Who was it who stated that every exponential was just a sigmoid in disguise? > most humans are really stupid. Statistically, don't we all sort of fit somewhere along a bell curve?
- pixl97 9mo ago
  The bell curve of IQ and being stupid probably don't have much to do with each other. Think of stupidity as the consequences of interacting with ones environment with negative outcomes. If you have a simple environment with few negative outcomes, then even someone with a 80 IQ may not be considered stupid. But if your environment rapidly grows more complex and the amount of thinking you have to do for positive outcomes increases then even someone with a 110 IQ may find themselves quickly in trouble.
afavour 9mo ago
> If you look at the rapid acceleration of progress I don’t understand this perspective. There are numerous examples of technical progress that then stalls out. Just look at batteries for example. Or ones where advancements are too expensive for widespread use (e.g. why no one flies Concorde any more) Why is previous progress a guaranteed indicator of future progress?
- pixl97 9mo ago
  Just think of this as risk management. If AGI doesn't happen, then good. You get to keep working and playing and generally screwing off in the way that humans have for generations. On the other hand if AGI happens, especially any time soon, you are exceptionally fucked along with me. The world changes very rapidly and there is no getting off Mr Bones wild ride. >Why is previous progress a guaranteed indicator of future progress? In this case, because nature already did it. We're not just inventing and testing something whole cloth. And we know there are still massive efficiencies to be gained. For me the Concorde is an example of how people look at stuff incorrectly. In the past we had to send people places very quickly to do things. This was very expensive and inefficient. I don't need to get on a plane to have an effect just about anywhere else in the world now. The internet and digital mediums give me a presence at other locations that is very close to being there. We didn't need planes that fly at the speed of sound, we needed strings that communicate at the speed of light.
coldtea 9mo ago
>If you look at the rapid acceleration of progress and conclude this way There's no "rapid acceleration of progress". If anything there's a decline, and even an economic decline. Take away the financial bubbles based on deregulation and huge explosion of debt, and the last 40 years of "economic progress" are just a mirage filling a huge bubble with air in actual advancement terms - unlike the previous millenia.
- azan_ 9mo ago
  That’s completely wrong. There was barely any progress in previous millennia. There was even economic Nobel prize for showing why!
  - coldtea 9mo ago
    The GDP per capita of the world has been slowly increasing for several millenia. Same for the advancements in core technology. The industrial revolution increased the pace, but it was already there, not flat or randomly flunctuating (think ancient hominids versus early agriculture vs bronge age, vs ancient Babylon and Assyrian empires, vs later Greece, and Persia, later Rome, later Renaissance and so on). Post 1970s most of the further increase has been based on mirages due to financialization, and doesn't reflect actual improvement.
    - azan_ 9mo ago
      > Post 1970s most of the further increase has been based on mirages due to financialization, and doesn't reflect actual improvement. Of course it does. It would be good if you would try to actually support such controversial claims with data.
    - inglor_cz 9mo ago
      If you got a random cancer in 1970, your chances of survival would be only somewhat better than during the Roman Empire, so slightly better than zero, but not very much so. Even Hodgkin's lymphoma, nowadays a very manageable disease, was reliably deadly. In 2026, unless you are really lucky your chances of being treated into remission are much better. That is a meaningful improvement. Medical science made a HUGE progress in the last 50 years alone. The oldest IVF baby is not even fifty yet.
darkerside 9mo ago
Yes, and that's why surpassing it doesn't lead to a singularity except over an infinite timeframe. This whole thing was stupid in the first place.

Recurecur 9mo ago

In my opinion, LLMs provide one piece of AGI. The only intelligence I’ve directly experienced is my own. I don’t consciously plan what I’m saying (or writing right now). Instead, a subconscious process assembles the words to support my stream of consciousness. I think that LLMs are very similar, if not identical. Stream of thought is accomplishing something superficially similar to consciousness, but without the ability to be innovative. At any rate, until there’s an artificial human level stream of consciousness in the mix for each AI, I doubt we’ll see a group of AIs collaborating to produce a significantly improved new generation of AI hardware and software minus human involvement. Once that does happen, the Singularity is at hand.

Recurecur 9mo ago
So, I was downvoted twice, yet neither of you doubtless brilliant individuals bothered to refute my points. I’ll emphasize my main point further, LLMs have no ability to innovate much beyond the vast data they’ve scraped. They’re almost entirely derivative. They are a definite improvement over traditional search engines though!
- int_19h 9mo ago
  Your points seem to boil down to "it hasn't happened yet, so it will never happen". There really isn't much to engage with.

jodrellblank 9mo ago

LLMs aren't AGI and maybe aren't a path to AGI, but step back and look at the way the world is changing. Hard disks were invented by IBM in 1953 and now less than a hundred years later there's an estimated million terabytes a year of hard disks made and sold, and a total sold of Mega, Giga, Tera, Peta, Exa, Zetta ... 1.36 Zettabytes. In 2000, webcams were barely a thing, audio was often recorded to dictaphone tapes, and now you can find a recorded photo or video of roughly anyone and anything on Earth. Maybe a tenth of all humans, almost any place, animal, insect, or natural event, almost any machine, mechanism, invention, painting, and a large sampling of "indoors" both public and private, almost any festival or event or tradition, and a very large sampling of "people doing things" and people teaching things for all kinds of skills. And tons of measurements of locations, temperatures, movements, weather, experiment results, and so on. The ability of computers to process information jumped with punched card readers, with electronic computers in the 1950s, again with transistors in the 1970s, semiconductors in the 1980s, commodity computer clusters (Google) in the 1990s, maybe again with multi-core desktops for everyone in the 2000s, with general purpose GPUs in the 2010s, and with faster commodity networking from 10Mbit to 100Gbit and more, and with SATA, then SAS, then RAID, then SSDs. It's now completely normal to check Google Maps to look at road traffic and how busy stores are (picked up in near realtime from the movement of smartphones around the planet), to do face and object recognition and search in photos, to do realtime face editing/enhancement while recording on a smartphone, to track increasing amounts of exercise and health data from increasing numbers of people, to call and speak to people across the planet and have your voice transcribed automatically to text, to realtime face-swap or face-enhance on a mobile chip, to download gigabytes of compressed Wikipedia onto a laptop and play with it in a weekend in Python just for fun. "AI" stuff (LLMs, neural networks and other techniques, PyTorch, TensorFlow, cloud GPUs and TPUs), the increase in research money, in companies competing to hire the best researchers, the increase in tutorials and numbers of people around the world wanting to play with it and being able to do that ... do you predict that by 2030, 2035, 2040, 2045, 2050 ... 2100, we'll have manufactured more compute power and storage than has ever been made, several times over, and made it more and more accessible to more people, and nothing will change, nothing interesting or new will have been found deliberately or stumbled upon accidentally, nothing new will have been understood about human brains, biology, or cognition, no new insights or products or modelling or AI techniques developed or become normal, no once-in-a-lifetime geniuses having any flashes of insight?

thesz 9mo ago

You are right about "LLM improving themselves" is impossible. In fact, LLM make themselves worse, it is called knowledge collapse [1]. [1] https://arxiv.org/abs/2404.03502 https://arxiv.org/abs/2404.03502

int_19h 9mo ago
That paper again. LLMs have been trained on synthetic outputs for quite a while since then and they do get better. Turns out there's more to it than that.