4 ms·

>And, for at least a while, it would also presumably be doing so at an exponentially increasing rate. Why would you presume this? I think part of a lot of peop

>And, for at least a while, it would also presumably be doing so at an exponentially increasing rate. Why would you presume this? I think part of a lot of people's AI skepticism is talk like this. You have no idea. Full stop. Why wouldn't progress be linear? As new breakthroughs come, newer ones will be harder to come by. Perhaps it's exponential. Perhaps it's linear. No one knows.

robwwilliams 1y ago
Or bottlenecked by data availability just like we humans are. Nothing will be exponential if a loop in the real world of science and engineering is involved.
- jatora 1y ago
  Coding and algorithmic advance does not require real world experimentation.
  - ceh123 1y ago
    Right but for self improving AI, training new models does have a real world bottleneck: energy and hardware. (Even if the data bottleneck is solved too)
- MarcelOlsz 1y ago
  Aren't we bottlenecked by not having any "prior art", as in not having reverse engineered any thinking machine like even a fly's brain? We can't even agree on a definition of consciousness and still don't understand the brain or how it works (to the extent that reverse engineering it can tell us something).
jatora 1y ago
It's a logical presumption. Researchers discover things. AGI is a researcher that can be scaled, research faster, and requires no downtime. Full stop if you dont find that obvious you should probably figure out where your bias is coming from. Coding and algorithmic advance does not require real world experimentation.
- throwaway48476 1y ago
  This is fundamentally flawed. There are upper bounds of efficiency that are laws of nature. To assume AI would be supernatural is magical thinking.
  - solumunus 1y ago
    And yet it seems to be the prevailing opinion even among very smart people. The “singularity” it’s just presumed. I’m highly skeptical to say the least. Look how much energy it’s taking to engineer these models which are still nowhere near AGI. When we get to AGI it won’t be immediately super intelligent and perhaps it never will be. Diminishing returns surely apply to anything that is energy based?
  - me-vs-cat 1y ago
    Natural intelligence appears supernatural from our current understanding, so it's not surprising that AGI also appears so.
    - goatlover 1y ago
      Neither appears supernatural from a scientific understanding.
- thinkingtoilet 1y ago
  Prove it.
- dkersten 1y ago
  If AGI has human capability, why would we think it could research any faster than a human? Sure, you can scale it, but if an LLM takes, say, $1 million a year to run an AGI instance, but it costs only $500k for one human researcher, then it still doesn’t get you anywhere faster than humans do. It might scale up, it might not, we don’t know. We won’t know until we reach it. We also don’t know if it scales linearly. Or if it’s learning capability and capacity will able to support exponential capability increase. Our current LLM’s don’t even have the capability of self improvement or learning even if they were capable: they can accumulate additional knowledge through the context window, but the models are static unless you fine tune or retrain them. What if our current models were ready for AGI but these limitations are stopping it? How would we ever know? Maybe it will be able to self improve but it will I’ll take exponentially larger amounts of training data. Or exponentially larger amounts of energy. Or maybe it can become “smarter” but at the cost of being larger to the point where the laws of physics mean it has to think slower, 2x the thinking but 2x the time, could happen! What if an AGI doesn’t want to improve? Far too many unknowns to say what will happen.
hennell 1y ago
No one knows, but it's a reasonable assumption surely. If you're theorising a AGI, that has recursive self improvement, exponential improvements seem almost unavoidable. AGI improves understanding of electronics, physics etc, that improves the AGI leading to new understandings and so on. Add in that new discoveries in one field might inspire the AGI/humans to find things in others and it seems hard to imagine a situation where theres not a lot of progress everywhere (at least theoretical progress, building new things might be slower / more costly then reasoning they would work.) Where I'm skeptical of AI would be in the idea an LLM can ever get to AGI level, if AGI is even really possible, and if the whole thing is actually viable. I'm also very skeptical that the discoveries of any AGI would be shared in ways that would allow exponential growth; licenses stopping using their AGI to make your own, copyright on the new laws of physics and royalties on any discovery you make from using those new laws etc.
- thinkingtoilet 1y ago
  >If you're theorising a AGI, that has recursive self improvement, exponential improvements seem almost unavoidable. Prove it. Also, AI will need resources. Hardware. Water. Electricity. Can those resources be supplied at an exponential rate? People need to calm down and stop stating things as truth when they literally have no idea.
  - johnisgood 1y ago
    Which is why I am an agnostic. :)
  - mangodrunk 1y ago
    Well said. It does seem that many who speculate on this are not taking into account limits that more/faster processing won’t actually help much. Say an algorithm is proven to be O(n!) for all cases, at a certain size of n, there’s not much that can be done if the algorithm is needed as is.
- deleted 1y ago
  [deleted]
frognumber 1y ago
I always consider different options when planning for the future, but I'll give the argument for exponential: Progress has been exponential in the generic. We made approximately the same progress in the past 100 years as the prior 1000 as the prior 30,000, as the prior million, and so on, all the way back to multicellular life evolving over 2 billion years or so. There's a question of the exponent, though. Living through that exponential growth circa 50AD felt at best linear, if not flat.
- FabHK 1y ago
  So you concede that there's nothing special about AI versus earlier innovations?
- ileonichwiesz 1y ago
  > We made approximately the same progress in the past 100 years as the prior 1000 as the prior 30,000 I hear this sort of argument all the time, but what is it even based on? There’s no clear definition of scientific and technological progress, much less something that’s measurable clearly enough to make claims like this. As I understand it, the idea is simply “Ooo, look, it took ten thousand years to go from fire to wheel, but only a couple hundred to go from printing press to airplane!!!”, and I guess that’s true (at least if you have a very juvenile, Sid Meier’s Civilization-like understanding of what history even is) but it’s also nonsense to try and extrapolate actual numbers from it.
  - thegrimmest 1y ago
    Plotting the highest observable assembly index over time will yield an exponential curve starting from the beginning of the universe. This is the closest I’m aware of to a mathematical model quantifying the distinct impression that local complexity has been increasing exponentially.
- qwertylicious 1y ago
  > Progress has been exponential in the generic. Has it? Really? Consider theoretical physics, which hasn't significantly advancement since the advent of general relativity and quantum theory. Or neurology, where we continue to have only the most basic understanding of how the human mind actually works (let alone the origin of consciousness). Heck, let's look at good ol' Moore's Law, which started off exponential but has slowed down dramatically. It's said that an S curve always starts out looking exponential, and I'd argue in all of those cases we're seeing exactly that. There's no reason to assume technological progress in general, whether via human or artificial intelligence, is necessarily any different.

robwwilliams 1y ago

Or bottlenecked by data availability just like we humans are. Nothing will be exponential if a loop in the real world of science and engineering is involved.

jatora 1y ago
Coding and algorithmic advance does not require real world experimentation.
- ceh123 1y ago
  Right but for self improving AI, training new models does have a real world bottleneck: energy and hardware. (Even if the data bottleneck is solved too)
MarcelOlsz 1y ago
Aren't we bottlenecked by not having any "prior art", as in not having reverse engineered any thinking machine like even a fly's brain? We can't even agree on a definition of consciousness and still don't understand the brain or how it works (to the extent that reverse engineering it can tell us something).

jatora 1y ago

It's a logical presumption. Researchers discover things. AGI is a researcher that can be scaled, research faster, and requires no downtime. Full stop if you dont find that obvious you should probably figure out where your bias is coming from. Coding and algorithmic advance does not require real world experimentation.

throwaway48476 1y ago
This is fundamentally flawed. There are upper bounds of efficiency that are laws of nature. To assume AI would be supernatural is magical thinking.
- solumunus 1y ago
  And yet it seems to be the prevailing opinion even among very smart people. The “singularity” it’s just presumed. I’m highly skeptical to say the least. Look how much energy it’s taking to engineer these models which are still nowhere near AGI. When we get to AGI it won’t be immediately super intelligent and perhaps it never will be. Diminishing returns surely apply to anything that is energy based?
- me-vs-cat 1y ago
  Natural intelligence appears supernatural from our current understanding, so it's not surprising that AGI also appears so.
  - goatlover 1y ago
    Neither appears supernatural from a scientific understanding.
thinkingtoilet 1y ago
Prove it.
dkersten 1y ago
If AGI has human capability, why would we think it could research any faster than a human? Sure, you can scale it, but if an LLM takes, say, $1 million a year to run an AGI instance, but it costs only $500k for one human researcher, then it still doesn’t get you anywhere faster than humans do. It might scale up, it might not, we don’t know. We won’t know until we reach it. We also don’t know if it scales linearly. Or if it’s learning capability and capacity will able to support exponential capability increase. Our current LLM’s don’t even have the capability of self improvement or learning even if they were capable: they can accumulate additional knowledge through the context window, but the models are static unless you fine tune or retrain them. What if our current models were ready for AGI but these limitations are stopping it? How would we ever know? Maybe it will be able to self improve but it will I’ll take exponentially larger amounts of training data. Or exponentially larger amounts of energy. Or maybe it can become “smarter” but at the cost of being larger to the point where the laws of physics mean it has to think slower, 2x the thinking but 2x the time, could happen! What if an AGI doesn’t want to improve? Far too many unknowns to say what will happen.

hennell 1y ago

No one knows, but it's a reasonable assumption surely. If you're theorising a AGI, that has recursive self improvement, exponential improvements seem almost unavoidable. AGI improves understanding of electronics, physics etc, that improves the AGI leading to new understandings and so on. Add in that new discoveries in one field might inspire the AGI/humans to find things in others and it seems hard to imagine a situation where theres not a lot of progress everywhere (at least theoretical progress, building new things might be slower / more costly then reasoning they would work.) Where I'm skeptical of AI would be in the idea an LLM can ever get to AGI level, if AGI is even really possible, and if the whole thing is actually viable. I'm also very skeptical that the discoveries of any AGI would be shared in ways that would allow exponential growth; licenses stopping using their AGI to make your own, copyright on the new laws of physics and royalties on any discovery you make from using those new laws etc.

thinkingtoilet 1y ago
>If you're theorising a AGI, that has recursive self improvement, exponential improvements seem almost unavoidable. Prove it. Also, AI will need resources. Hardware. Water. Electricity. Can those resources be supplied at an exponential rate? People need to calm down and stop stating things as truth when they literally have no idea.
- johnisgood 1y ago
  Which is why I am an agnostic. :)
- mangodrunk 1y ago
  Well said. It does seem that many who speculate on this are not taking into account limits that more/faster processing won’t actually help much. Say an algorithm is proven to be O(n!) for all cases, at a certain size of n, there’s not much that can be done if the algorithm is needed as is.
deleted 1y ago
[deleted]

frognumber 1y ago

I always consider different options when planning for the future, but I'll give the argument for exponential: Progress has been exponential in the generic. We made approximately the same progress in the past 100 years as the prior 1000 as the prior 30,000, as the prior million, and so on, all the way back to multicellular life evolving over 2 billion years or so. There's a question of the exponent, though. Living through that exponential growth circa 50AD felt at best linear, if not flat.

FabHK 1y ago
So you concede that there's nothing special about AI versus earlier innovations?
ileonichwiesz 1y ago
> We made approximately the same progress in the past 100 years as the prior 1000 as the prior 30,000 I hear this sort of argument all the time, but what is it even based on? There’s no clear definition of scientific and technological progress, much less something that’s measurable clearly enough to make claims like this. As I understand it, the idea is simply “Ooo, look, it took ten thousand years to go from fire to wheel, but only a couple hundred to go from printing press to airplane!!!”, and I guess that’s true (at least if you have a very juvenile, Sid Meier’s Civilization-like understanding of what history even is) but it’s also nonsense to try and extrapolate actual numbers from it.
- thegrimmest 1y ago
  Plotting the highest observable assembly index over time will yield an exponential curve starting from the beginning of the universe. This is the closest I’m aware of to a mathematical model quantifying the distinct impression that local complexity has been increasing exponentially.
qwertylicious 1y ago
> Progress has been exponential in the generic. Has it? Really? Consider theoretical physics, which hasn't significantly advancement since the advent of general relativity and quantum theory. Or neurology, where we continue to have only the most basic understanding of how the human mind actually works (let alone the origin of consciousness). Heck, let's look at good ol' Moore's Law, which started off exponential but has slowed down dramatically. It's said that an S curve always starts out looking exponential, and I'd argue in all of those cases we're seeing exactly that. There's no reason to assume technological progress in general, whether via human or artificial intelligence, is necessarily any different.