7 ms·

I think it’s a little absurd that we would make the leap to machine super-intelligence without going through the step of machine low or medium intelligence. If

I think it’s a little absurd that we would make the leap to machine super-intelligence without going through the step of machine low or medium intelligence. If I could write, I’d write a companion story to this called “Omega-1, the artificial super intelligence that wasn’t really that good at things”. They try to have it write mini-AIs to solve MT problems, but it’s slow and inaccurate. They have it write and produce TV shows, but they’re bland and poorly received. They have it make video games but the controls make no sense and they aren’t fun. They have it make trading strategies and it loses all the money. They ask it to build a smarter version of itself, but it sees no way forward that’s fundamentally better because it lacks comprehension of what it would mean to be “smarter” - it can add more memory or more computing power, but without changing the way that it stores and indexes the information, it can’t make something that can solve problems that it can’t. Eventually Omega team gives up, publishes the results, but doesn’t have the heart to shut down Omega-1, a machine that passes the Turing Test but isn’t really that good at things.

dmix 9y ago
> I think it’s a little absurd that we would make the leap to machine super-intelligence without going through the step of machine low or medium intelligence. I have the same feelings about socialism... If we can draw any lesson from the history of our biological existence from which we expect machine intelligence to spring from it will likely be a series of smaller pieces that will join together to form like Voltron. Combined will be the best of our ability. I highly doubt there will ever be some centralized system/organization that has 'figured out' intelligence. Although that doesn't have to necessarily mean we will forever be limited to niche/vertical intelligent systems in the long run either.
daveguy 9y ago
I think the storyline should include that no one wanted to shut it down and despite the fact that it broke their hearts they couldn't justify spending hundreds of thousands of dollars on the electricity that it cost to keep Omega-1 running. It was simply unable to sustain itself like 90% of startups.
thaumaturgy 9y ago
Yeah. Maciej wrote a pretty good piece rebutting AI alarmism and kind of alludes to that as one of several points. http://idlewords.com/talks/superintelligence.htm http://idlewords.com/talks/superintelligence.htm > With no way to define intelligence (except just pointing to ourselves), we don't even know if it's a quantity that can be maximized. For all we know, human-level intelligence could be a tradeoff. Maybe any entity significantly smarter than a human being would be crippled by existential despair, or spend all its time in Buddha-like contemplation. and > But the hard takeoff scenario requires that there be a feature of the AI algorithm that can be repeatedly optimized to make the AI better at self-improvement. And so on. It's a good read.
- 5_minutes 9y ago
  Thanks for this
- westoncb 9y ago
  Similarly, I wonder if it's something where adding to it doesn't actually do much, like all the variations you see done on Turing Machines (adding more tapes or 2d tapes etc.) which seem like they would increase its capacity somehow, but it turns out they have the same power as the original Turing Machine. Or like the arithmetic of infinite numbers, e.g. aleph-0 + aleph-0 = aleph-0.
- spynxic 9y ago
  > With no way to define intelligence (except just pointing to ourselves.. ..and anything else considered to be alive) Not to make an attempt at arguing whether or not machines can be intelligent but machines are not submerged in an environment that effects them, giving them no merit to be intelligent about anything; and even if they were they still have no ability to influence the environment. (Sensors for example) It seems many that speak on AI have completely disregarded coming up with a reasonable implementation of intelligence in favor of pondering its potential uses. (Typically in ideals central to humans)
  - xarball 9y ago
    > machines are not submerged in an environment that effects them, giving them no merit to be intelligent about anything You're alluding to the problem of having to 'raise' a super-intelligence with a vested interest in the continuity of its physical existence. I think I agree with where you're going with that. Our understanding of 'self' or 'self preservation' likely comes from a long history of suffering, pain, and failure brought on by trials at understanding the self from evolution's attempts at imbuing the information for longevity into cognition. Mother nature has had years of exponential growth with an inconceivable number of branches attempting to understand the 'self', that lead to failure, all necessary because the idea of 'self preservation' or 'grounding one's sense of ego into a concept of physical sensory data' is very obscure to precisely define, let along to define such that it can be TESTED between generations. Based on how much of a headstart evolution has on that refining process, and how bad humans are at implementing that or even exposing sensory information about that that plug directly into concepts that necessarily affect an artificial's sense of self, I'm willing to bet that we are completely ill-equipped to grow any real intelligence that's truly grounded in the real world, until it can make all the same failed attempts at living longer or evolving + testing those attempts, with that exact same directional goal of 'self-preservation' which humans have. Self-preservation is really the only thing you can evolve to, to guide a physical evolution. I'm not aware of any other physical goals that could lead to the same human-like intelligence or human-like neural models. Most creatures seem to hit the exact same point. An agent-based survival of the fittest with generational physical trials that punish poor understandings of the self's physical manifestation, trying to optimize for its longevity. Anything that doesn't hit that kind of model is strictly not aware of the manifestation of its ego, as we know it (e.g. plants, bacteria, etc), and I don't think the physical sense of self can evolve into grounded concepts without allowing for generational failure on the understanding of self, due to incorrect concepts of 'physical self' between competing generations. TL;DR - The concept of physical self is an evolved trait only if there is a goal tied to it, it cannot arrive without the concept of 'self existence' being threatened or when not allowed to fail.
ndkskshfr 9y ago
> I think it’s a little absurd that we would make the leap to machine super-intelligence without going through the step of machine low or medium intelligence. I’m not so sure about this, which is not so say i’m arguing FOR the leap. But, intuitively, there is an absolute massive leap in terms of perceived intelligence among humans, let alone with chimps. It seems some of the magic sauce for passing a turing test is, evolutionarily speaking, rather small.
kevin_thibedeau 9y ago
The first intelligent AIs will be weak and have limited influence on real world human affairs. The worst we have now is poorly supervised automatic trading systems that can shift massive amounts of money through electronic markets. However, once AIs can design and implement their own successors all bets are off. I would also be concerned about the development of non-theraputic neural implants and the ability of an AI to surreptitiously influence human behavior through them. Thankfully that is likely to to be further off than when we start bumping into the AI singularity.
- pdimitar 9y ago
  In the absolutely epic space opera "Hyperion" (and its sequel "Endymion") the author Dan Simmons theorizes -- through the plot, not through a monologue! -- that AI creatures would offer us tech that we could never invent -- impossibly minituarized personal computers (the size of a discrete and thin silver bracelet, an earring or even smaller implants under the skin, and which are basic AI by themselves), and real-time worm-holes through which humanity finally achieves inter-stellar civilization status. However, the author is very realistic (pessimistic?) about such an AI community -- actually there's billions of them living in a place humanity can't ever reach or even guess where it is, with internal politics as foreign for the people as our societal dynamics are a mystery to the bacteria around us. Anyhow -- the tech they give to humanity is heavily backdoored and this becomes very apparent and dangerous as the plot progresses. I'd recommend those books to anybody with a heart and some curiosity about a seemingly optimistic future (which is actually pretty dark). Be warned though: if you decide to read them, the first ~120 pages of the first book are DARK AND HEAVY. However, enduring it was the best sacrifice of several hours of patience I've ever done in my life. I just couldn't stop reading before I finished all 4 books.
  - crooked-v 9y ago
    That kind of falls apart in this context because a set of mysterious unreachable AIs bearing gifts are a de facto foreign nation and would be treated appropriately.
    - 9y ago
PoachedSausage 9y ago
Perhaps we will find that when we give it control of weapons, despite its inadequacies in other areas, it will excel at killing.
deleted 9y ago
[deleted]
eggpy 9y ago
But this part isn't science fiction, we already have AIs that do this. See AIVA for pretty good musical composition[0], new sites are using AI to write and compress news stories[1], and screeplays are being written via machine learning[2]. Obviously these are still in their infancy, but doesn't this perfectly encapsulate your point? There's not really an interesting story or thought piece there, because people are working on these in all sorts of fields all over the world, today. If we can build AIs that create "good" art, is it much of a stretch to teach a super AI what "better" means? By default a super intelligence doesn't have to start out as super intelligence, just have the capability to improve. [0] https://www.youtube.com/watch?v=Ebnd03x137A https://www.youtube.com/watch?v=Ebnd03x137A [1] https://www.wired.com/2017/02/robots-wrote-this-story/ https://www.wired.com/2017/02/robots-wrote-this-story/ [2] https://www.youtube.com/watch?v=LY7x2Ihqjmc https://www.youtube.com/watch?v=LY7x2Ihqjmc
rictic 9y ago
That's pretty likely, but not certain. Go-playing-AI went from "talentless amateur" to "better than every living human" in about a year. Sometimes things progress rapidly, and some kinds of intelligence could recursively self-improve.
edanm 9y ago
"I think it’s a little absurd that we would make the leap to machine super-intelligence without going through the step of machine low or medium intelligence." Why? The leap from calculators that were worse than humans, to calculators that were much, much better than humans, didn't take too long. No one knows how fast the leap will be from what we currently have, to general intelligence, to super intelligence. I don't think anyone in the AI safety movement will tell you that we know for sure the leap will be fast. I think they'd say we simply don't know how fast it will be, but the stakes are pretty high so it's worth preparing for.
dTal 9y ago
I feel this lacks imagination. In actual fact, a computerised person of average intelligence would be an incredible piece of technology and could easily lead to an AI singularity, given enough funding - which it would probably receive, because holy shit. What happens when you set thousands of average people on a problem, and give them thousands of simulated years to work on it? When you can spin up a research team as easily as an AWS instance? >but without changing the way that it stores and indexes the information, it can’t make something that can solve problems that it can’t. And yet people routinely grow up to be smarter than their parents, with no special effort on their parents' part. All you need to make a smarter version of yourself is the ability to make inaccurate copies and dumb luck. What, besides ethics and money, stops you from running an evolutionary process that selects for intelligence at a vastly higher rate than real time?
- KidComputer 9y ago
  > What, besides ethics and money, stops you from running an evolutionary process that selects for intelligence at a vastly higher rate than real time You can already do that today. So why don't we have a superintelligence? I can train a NTM to learn counting and give me the next natural number in a single step. I can't train it to give me the next prime in a single step, why?. Computability and complexity, both conveniently forgotten in the discussion of superintelligence.

dmix 9y ago

> I think it’s a little absurd that we would make the leap to machine super-intelligence without going through the step of machine low or medium intelligence. I have the same feelings about socialism... If we can draw any lesson from the history of our biological existence from which we expect machine intelligence to spring from it will likely be a series of smaller pieces that will join together to form like Voltron. Combined will be the best of our ability. I highly doubt there will ever be some centralized system/organization that has 'figured out' intelligence. Although that doesn't have to necessarily mean we will forever be limited to niche/vertical intelligent systems in the long run either.

daveguy 9y ago

I think the storyline should include that no one wanted to shut it down and despite the fact that it broke their hearts they couldn't justify spending hundreds of thousands of dollars on the electricity that it cost to keep Omega-1 running. It was simply unable to sustain itself like 90% of startups.

thaumaturgy 9y ago

Yeah. Maciej wrote a pretty good piece rebutting AI alarmism and kind of alludes to that as one of several points. http://idlewords.com/talks/superintelligence.htm http://idlewords.com/talks/superintelligence.htm > With no way to define intelligence (except just pointing to ourselves), we don't even know if it's a quantity that can be maximized. For all we know, human-level intelligence could be a tradeoff. Maybe any entity significantly smarter than a human being would be crippled by existential despair, or spend all its time in Buddha-like contemplation. and > But the hard takeoff scenario requires that there be a feature of the AI algorithm that can be repeatedly optimized to make the AI better at self-improvement. And so on. It's a good read.

5_minutes 9y ago
Thanks for this
westoncb 9y ago
Similarly, I wonder if it's something where adding to it doesn't actually do much, like all the variations you see done on Turing Machines (adding more tapes or 2d tapes etc.) which seem like they would increase its capacity somehow, but it turns out they have the same power as the original Turing Machine. Or like the arithmetic of infinite numbers, e.g. aleph-0 + aleph-0 = aleph-0.
spynxic 9y ago
> With no way to define intelligence (except just pointing to ourselves.. ..and anything else considered to be alive) Not to make an attempt at arguing whether or not machines can be intelligent but machines are not submerged in an environment that effects them, giving them no merit to be intelligent about anything; and even if they were they still have no ability to influence the environment. (Sensors for example) It seems many that speak on AI have completely disregarded coming up with a reasonable implementation of intelligence in favor of pondering its potential uses. (Typically in ideals central to humans)
- xarball 9y ago
  > machines are not submerged in an environment that effects them, giving them no merit to be intelligent about anything You're alluding to the problem of having to 'raise' a super-intelligence with a vested interest in the continuity of its physical existence. I think I agree with where you're going with that. Our understanding of 'self' or 'self preservation' likely comes from a long history of suffering, pain, and failure brought on by trials at understanding the self from evolution's attempts at imbuing the information for longevity into cognition. Mother nature has had years of exponential growth with an inconceivable number of branches attempting to understand the 'self', that lead to failure, all necessary because the idea of 'self preservation' or 'grounding one's sense of ego into a concept of physical sensory data' is very obscure to precisely define, let along to define such that it can be TESTED between generations. Based on how much of a headstart evolution has on that refining process, and how bad humans are at implementing that or even exposing sensory information about that that plug directly into concepts that necessarily affect an artificial's sense of self, I'm willing to bet that we are completely ill-equipped to grow any real intelligence that's truly grounded in the real world, until it can make all the same failed attempts at living longer or evolving + testing those attempts, with that exact same directional goal of 'self-preservation' which humans have. Self-preservation is really the only thing you can evolve to, to guide a physical evolution. I'm not aware of any other physical goals that could lead to the same human-like intelligence or human-like neural models. Most creatures seem to hit the exact same point. An agent-based survival of the fittest with generational physical trials that punish poor understandings of the self's physical manifestation, trying to optimize for its longevity. Anything that doesn't hit that kind of model is strictly not aware of the manifestation of its ego, as we know it (e.g. plants, bacteria, etc), and I don't think the physical sense of self can evolve into grounded concepts without allowing for generational failure on the understanding of self, due to incorrect concepts of 'physical self' between competing generations. TL;DR - The concept of physical self is an evolved trait only if there is a goal tied to it, it cannot arrive without the concept of 'self existence' being threatened or when not allowed to fail.

ndkskshfr 9y ago

> I think it’s a little absurd that we would make the leap to machine super-intelligence without going through the step of machine low or medium intelligence. I’m not so sure about this, which is not so say i’m arguing FOR the leap. But, intuitively, there is an absolute massive leap in terms of perceived intelligence among humans, let alone with chimps. It seems some of the magic sauce for passing a turing test is, evolutionarily speaking, rather small.

kevin_thibedeau 9y ago

The first intelligent AIs will be weak and have limited influence on real world human affairs. The worst we have now is poorly supervised automatic trading systems that can shift massive amounts of money through electronic markets. However, once AIs can design and implement their own successors all bets are off. I would also be concerned about the development of non-theraputic neural implants and the ability of an AI to surreptitiously influence human behavior through them. Thankfully that is likely to to be further off than when we start bumping into the AI singularity.

pdimitar 9y ago
In the absolutely epic space opera "Hyperion" (and its sequel "Endymion") the author Dan Simmons theorizes -- through the plot, not through a monologue! -- that AI creatures would offer us tech that we could never invent -- impossibly minituarized personal computers (the size of a discrete and thin silver bracelet, an earring or even smaller implants under the skin, and which are basic AI by themselves), and real-time worm-holes through which humanity finally achieves inter-stellar civilization status. However, the author is very realistic (pessimistic?) about such an AI community -- actually there's billions of them living in a place humanity can't ever reach or even guess where it is, with internal politics as foreign for the people as our societal dynamics are a mystery to the bacteria around us. Anyhow -- the tech they give to humanity is heavily backdoored and this becomes very apparent and dangerous as the plot progresses. I'd recommend those books to anybody with a heart and some curiosity about a seemingly optimistic future (which is actually pretty dark). Be warned though: if you decide to read them, the first ~120 pages of the first book are DARK AND HEAVY. However, enduring it was the best sacrifice of several hours of patience I've ever done in my life. I just couldn't stop reading before I finished all 4 books.
- crooked-v 9y ago
  That kind of falls apart in this context because a set of mysterious unreachable AIs bearing gifts are a de facto foreign nation and would be treated appropriately.
  - 9y ago

PoachedSausage 9y ago

Perhaps we will find that when we give it control of weapons, despite its inadequacies in other areas, it will excel at killing.

deleted 9y ago

[deleted]

eggpy 9y ago

But this part isn't science fiction, we already have AIs that do this. See AIVA for pretty good musical composition[0], new sites are using AI to write and compress news stories[1], and screeplays are being written via machine learning[2]. Obviously these are still in their infancy, but doesn't this perfectly encapsulate your point? There's not really an interesting story or thought piece there, because people are working on these in all sorts of fields all over the world, today. If we can build AIs that create "good" art, is it much of a stretch to teach a super AI what "better" means? By default a super intelligence doesn't have to start out as super intelligence, just have the capability to improve. [0] https://www.youtube.com/watch?v=Ebnd03x137A https://www.youtube.com/watch?v=Ebnd03x137A [1] https://www.wired.com/2017/02/robots-wrote-this-story/ https://www.wired.com/2017/02/robots-wrote-this-story/ [2] https://www.youtube.com/watch?v=LY7x2Ihqjmc https://www.youtube.com/watch?v=LY7x2Ihqjmc

rictic 9y ago

That's pretty likely, but not certain. Go-playing-AI went from "talentless amateur" to "better than every living human" in about a year. Sometimes things progress rapidly, and some kinds of intelligence could recursively self-improve.

edanm 9y ago

"I think it’s a little absurd that we would make the leap to machine super-intelligence without going through the step of machine low or medium intelligence." Why? The leap from calculators that were worse than humans, to calculators that were much, much better than humans, didn't take too long. No one knows how fast the leap will be from what we currently have, to general intelligence, to super intelligence. I don't think anyone in the AI safety movement will tell you that we know for sure the leap will be fast. I think they'd say we simply don't know how fast it will be, but the stakes are pretty high so it's worth preparing for.

dTal 9y ago

I feel this lacks imagination. In actual fact, a computerised person of average intelligence would be an incredible piece of technology and could easily lead to an AI singularity, given enough funding - which it would probably receive, because holy shit. What happens when you set thousands of average people on a problem, and give them thousands of simulated years to work on it? When you can spin up a research team as easily as an AWS instance? >but without changing the way that it stores and indexes the information, it can’t make something that can solve problems that it can’t. And yet people routinely grow up to be smarter than their parents, with no special effort on their parents' part. All you need to make a smarter version of yourself is the ability to make inaccurate copies and dumb luck. What, besides ethics and money, stops you from running an evolutionary process that selects for intelligence at a vastly higher rate than real time?

KidComputer 9y ago
> What, besides ethics and money, stops you from running an evolutionary process that selects for intelligence at a vastly higher rate than real time You can already do that today. So why don't we have a superintelligence? I can train a NTM to learn counting and give me the next natural number in a single step. I can't train it to give me the next prime in a single step, why?. Computability and complexity, both conveniently forgotten in the discussion of superintelligence.