4 ms·

> singularity I just cant get over this term, do you honestly believe in this? I use AI daily and while it is super useful I see too many limitations for it to

by koe123 4mo ago

> singularity I just cant get over this term, do you honestly believe in this? I use AI daily and while it is super useful I see too many limitations for it to “recursively improve and cause an intelligence explosion”. 1. Clearly, the people selling AI with this idea benefit greatly with this promise of infinite upside. Can you can trust them? 2. Singularity essentially requires to handwave away a lot of baked-in issues with LLMs or rely on unrealized innovation.

lionheart 4mo ago
I do. Just based on personal experience of using these tools for the last several years and how they’ve progressed.
- koe123 4mo ago
  So, what your saying is that there is a perhaps linear, perhaps exponential increase, and that you are projecting that increase forward indefinitely. Let me know if this is unfair. Counter argument: does anything else work this way? E.g. Moores law had an end too right? I would argue that the core tech breakthrough (Transformer-based LLM) has been improved, but no fundamental further innovation seems to have been made. The current architecture fundamentally hallucinates, even Fabel even on trivial problems. I.e. as number tokens increase error likelihood goes to infinity. How then, can this scale recursively to infinity?
  - lionheart 4mo ago
    Not indefinitely but at least to the point where they’re smarter than humans.
    - yorwba 4mo ago
      The singularity usually refers to the idea that although the rate of relative improvement of a technology is approximately constant at a constant (human) intelligence level ([dy/dt]/y ≈ c), producing exponential growth, if said constant were actually proportional to intelligence and the technology under improvement being intelligence itself ([dy/dt]/y ≈ cy), then solving the differential equation leads to a blow-up to infinity in finite time, a mathematical singularity. If instead additional intelligence does little to speed up AI development (due to the need for other inputs like caputal and time), you could get a world where AI becomes better than humans at AI development and begins a cycle of recursive self-improvement without explosive growth leading to a singularity.
      - fuzzfactor 4mo ago
        That's a pretty good concept and you could say there is a fairly wide gap between your everyday singularity and the ultimate singularity. Which seems to me likely the result of an unforeseen variable or variables, and that's got to have outsized, uncharacterized, and unexpected importance to have such a strong effect. When the overwhelming consensus is that wonderful things are waiting just around the corner, it still could turn out to be just the opposite and you'll never know until you actually turn the corner.
- inigyou 4mo ago
  What about the tools makes you think we've hit the singularity? My experience with them is that they've memorized a lot of stuff, but can't make anything fundamentally new. Most of the useful things LLMs do amounts to semantic search.
  - lionheart 4mo ago
    They’re discovering previously unknown mathematical theorems. How is that not new?
    - koe123 4mo ago
      They are new proofs and for sure useful but as far as I’ve understood mainly interpolative. E.g. an LLM can “create” a poem about a purple chicken as it has datapoints for “purple” and “chicken”, so it can create something plausible inbetween. Similarly, in my mind it can interpolate proofs by interpolating between data points for technique A and technique B. This is novel and brute-forcing proofs this way is useful. It is analogus to how sometimes it can generate programs that pass unit tests, I think. However, creating fundamentally new concepts outside of the interpolated datapoints is not something I am convinced of. Maybe it can extrapolate some things, if correct add it as a data point, continue. Essentially a search, and it would be amazing if this works and maybe we can get some recursive improvement this way. But the “ideas” it will use to conduct this search are a function of the input data points as well, and thus in my view fundamentally limited in novelty. I am not discounting the usefulness, but I am not convinced you can just keep doing this indefinitely scaling intelligence exponentially. Of course nobody can know yet really and I am just speculating just like you. But I also think the “experts” Sam and Dario also don’t know, and given their incentives I am not really convinced by them.
    - inigyou 4mo ago
      They're finding new examples in well-known categories of stuff. Also proving is a question of search. They're finding some stuff we missed, but not because they're smart, just because they have lots of data and compute. Think of them as something closer to brute force.
GauntletWizard 4mo ago
I believe that AI is a singularity. Just like a Black hole, no information can escape and you're slowly (by subjective experience) dragged in and spaghettified.
- koe123 4mo ago
  Sounds great, but what is this belief based on? I think having a technical argument is important, as we live in a time with lots of hype merchants who stand to benefit from record breaking IPOs. Propaganda can affect us all, how do you know you aren’t being sold to?
S_Bear 4mo ago
The best part of the Singularity is that, once everyone who wants to uploads their consciousness to a computer does so, we mortal humans can simply unplug it and go on about our lives.

by koe123 4mo ago

lionheart 4mo ago

I do. Just based on personal experience of using these tools for the last several years and how they’ve progressed.

koe123 4mo ago
So, what your saying is that there is a perhaps linear, perhaps exponential increase, and that you are projecting that increase forward indefinitely. Let me know if this is unfair. Counter argument: does anything else work this way? E.g. Moores law had an end too right? I would argue that the core tech breakthrough (Transformer-based LLM) has been improved, but no fundamental further innovation seems to have been made. The current architecture fundamentally hallucinates, even Fabel even on trivial problems. I.e. as number tokens increase error likelihood goes to infinity. How then, can this scale recursively to infinity?
- lionheart 4mo ago
  Not indefinitely but at least to the point where they’re smarter than humans.
  - yorwba 4mo ago
    The singularity usually refers to the idea that although the rate of relative improvement of a technology is approximately constant at a constant (human) intelligence level ([dy/dt]/y ≈ c), producing exponential growth, if said constant were actually proportional to intelligence and the technology under improvement being intelligence itself ([dy/dt]/y ≈ cy), then solving the differential equation leads to a blow-up to infinity in finite time, a mathematical singularity. If instead additional intelligence does little to speed up AI development (due to the need for other inputs like caputal and time), you could get a world where AI becomes better than humans at AI development and begins a cycle of recursive self-improvement without explosive growth leading to a singularity.
    - fuzzfactor 4mo ago
      That's a pretty good concept and you could say there is a fairly wide gap between your everyday singularity and the ultimate singularity. Which seems to me likely the result of an unforeseen variable or variables, and that's got to have outsized, uncharacterized, and unexpected importance to have such a strong effect. When the overwhelming consensus is that wonderful things are waiting just around the corner, it still could turn out to be just the opposite and you'll never know until you actually turn the corner.
inigyou 4mo ago
What about the tools makes you think we've hit the singularity? My experience with them is that they've memorized a lot of stuff, but can't make anything fundamentally new. Most of the useful things LLMs do amounts to semantic search.
- lionheart 4mo ago
  They’re discovering previously unknown mathematical theorems. How is that not new?
  - koe123 4mo ago
    They are new proofs and for sure useful but as far as I’ve understood mainly interpolative. E.g. an LLM can “create” a poem about a purple chicken as it has datapoints for “purple” and “chicken”, so it can create something plausible inbetween. Similarly, in my mind it can interpolate proofs by interpolating between data points for technique A and technique B. This is novel and brute-forcing proofs this way is useful. It is analogus to how sometimes it can generate programs that pass unit tests, I think. However, creating fundamentally new concepts outside of the interpolated datapoints is not something I am convinced of. Maybe it can extrapolate some things, if correct add it as a data point, continue. Essentially a search, and it would be amazing if this works and maybe we can get some recursive improvement this way. But the “ideas” it will use to conduct this search are a function of the input data points as well, and thus in my view fundamentally limited in novelty. I am not discounting the usefulness, but I am not convinced you can just keep doing this indefinitely scaling intelligence exponentially. Of course nobody can know yet really and I am just speculating just like you. But I also think the “experts” Sam and Dario also don’t know, and given their incentives I am not really convinced by them.
  - inigyou 4mo ago
    They're finding new examples in well-known categories of stuff. Also proving is a question of search. They're finding some stuff we missed, but not because they're smart, just because they have lots of data and compute. Think of them as something closer to brute force.

GauntletWizard 4mo ago

I believe that AI is a singularity. Just like a Black hole, no information can escape and you're slowly (by subjective experience) dragged in and spaghettified.

koe123 4mo ago
Sounds great, but what is this belief based on? I think having a technical argument is important, as we live in a time with lots of hype merchants who stand to benefit from record breaking IPOs. Propaganda can affect us all, how do you know you aren’t being sold to?

S_Bear 4mo ago

The best part of the Singularity is that, once everyone who wants to uploads their consciousness to a computer does so, we mortal humans can simply unplug it and go on about our lives.