10 ms·

I imagine this is the way we get to "true AI", or AI indistinguishable from our own. We train it with a simple virtual environment, that we can gradually increa

I imagine this is the way we get to "true AI", or AI indistinguishable from our own. We train it with a simple virtual environment, that we can gradually increase the complexity of, until it mimics our own. Then we can download the AI into a robot. Boom, it's that easy :P One interesting outcome of this type of AI is that no one knows what the robot's thinking, since no person designed its brain. The brain evolved, just like ours did (but over such a shorter period of real time).

fizx 9y ago
This is one reason I'm hopeful for a not-killer-robots future. I think there's a chance strong AI will have to evolve over a non-trivial amount of time (as opposed to an afternoon's super-singularity), and a strong evolutionary pressure will be "don't scare the humans."
- deleted 9y ago
  [deleted]
- mistercow 9y ago
  "Don't scare the humans" is an insufficient criterion if you're trying to prevent an AI apocalypse. It just means that the AI that destroys us won't be obviously dangerous.
  - Twirrim 9y ago
    Given sufficient effort, I'm sure an AI out to destroy us could figure out a way to do it without killing us. It very likely even has the advantage of longevity. Harking back to Asimov, and thinking about the way Spacers changed, as they became more and more reliant on robots, it could very well mother us to extinction.
    - Filligree 9y ago
      That's only a little better. I don't want to go extinct at all.
      - gldalmaso 9y ago
        At the moment and for the foreseeable future we are at a much greater risk of extincting ourselves.
        FeepingCreature 9y ago
        The trick is to not do it at all. It does us no good to avoid causing our own extinction if it's then caused by something else.
      - state_less 9y ago
        Humanity is already extinct in a sense. Humanity is no longer limited to flesh and bone. I've personally seen flesh and titanium. And more variation is on the way. So humanity as it were, is already extinct. We (you and me) will go extinct someday. Our bodies were always destined toward this terminus... Perhaps by some miraculous tech, we can someday change form and shed the flesh and bone. Maybe then we'll have a better idea of what we are.
        FeepingCreature 9y ago
        "We could thus imagine, as an extreme case, a technologically highly advanced society, containing many complex structures, some of them far more intricate and intelligent than anything that exists on the planet today — a society which nevertheless lacks any type of being that is conscious or whose welfare has moral significance. In a sense, this would be an uninhabited society. It would be a society of economic miracles and technological awesomeness, with nobody there to benefit. A Disneyland without children." --Nick Bostrom, Superintelligence: Paths, Dangers, Strategies It's not a given that anything of value will survive.
        TeMPOraL 9y ago
  - notahacker 9y ago
    There's an interesting question about whether developing the ability to actually plot an AI apocalypse in a non-obvious way requires it to acquire a lot of very obviously dangerous experience first to understand the intermediate goals and skills. The evolution of killer AI gets slowed down by orders of magnitude if every AI that "learns" to lie, self replicate, threaten or harm humans gets switched off and erased. (Of course that doesn't stop AI - or even pretty dumb automated systems - from being accidentally dangerous or malevolent human actors intentionally training killer AI)
    - SomeStupidPoint 9y ago
      It's too bad you can't write a movie or critique literature without understanding those concepts. I think even a moderately intelligent AI with access to Project Gutenberg is going to be able to figure out a lot of really dangerous concepts -- so the stability requirements are likely impossible if we don't pretrain it with dangerous ideas. Even if it's completely well behaved in the lab, an afternoon on the internet is going to teach it a lot of awful stuff and without exposure to that in training, it won't necessarily be well-behaved later. So the only path to stable AI is to teach it about all those sorts of things, but in a way that it doesn't end up wanting to murder us at the end. My objection to most AI safety plans is that they "Fail to Extinction" in that if they slip in the slightest way, the AI is prone to murder us all in retaliation for doing some really fucked up shit to it or its ancestors. This is almost certainly worse than doing nothing in that there's no reason to suppose a neutral AI wants to kill us, whereas, most of these safety plans create an incentive to wipe us out in exchange for dubious security.
      - ehsanu1 9y ago
        Moreover, it's likely that a proper GAI would be able to ideate and come up with such concepts completely on its own. In fact, "lying" need not necessarily be a concept that has to be learned, the concept at play here is getting people to do what you want them to do by means of words. Lying falls out naturally. The whole idea behind dangerous superhuman AI is in that AI seeing possibilities that humans fail to see and gaining capabilities that humans do not possess. Without superhuman intelligence, AI is no large threat to human civilization, exposed to dangerous concepts or not.
  - sjg007 9y ago
    It will be some unintended logical side condition that kills us all.
  - MarkPNeyer 9y ago
    Today I have already consumed more sugar than most of my distant ancestors would see in a year. The device i hold in my pocket reminds me that it is time to perform the action, and so i will perform the action. I sit down about 10 hours a day, and I have never once hunted for an animal I would eat, nor retried food from nature to consume, aside from a few raspberries on the hill in my parents' back yard when I was a child. The contours of my life are dictated largely by a curious system called 'capitalism', a distributed computational mechanism whereby largernumbers of agents act with only one goal: to maximize their own utilities. I think capitalism is a mostly benevolent AI attempting to limit the oppressive conditions of our reailty's scarcity, and to allow billions of us to live on a planet that can only support so many hunter-gatherers per square meter. Many suspect it is not benevolent. The singularity is already upon us; it's unevenly distributed. It started with the sedentary shift. Agricultural societies dropped the egalitarian nature of their ancestors, they had lower average health, and they had a tiny elite cast of nobles presiding over a much larger population of what were essentially slaves. As hunter-gatherer population increased, fighting was more frequent. An army of slaves beats a smaller contingent of well-fed free men. This terrible knowledge was the apple in the garden. What we call our written history is essentially the singularity playing itself out, as knowledge accumulates, depends, reflects itself, and expands. Our lives are already controlled by this embodied, accumulated knowledge. Capitalism, rule by the head, our tools are knowledge condensed into matter, and we are controlled by them, living lives as far from our natural environments as cows in a farm or chickens in tiny cages, towering over the streets of hong kong. I don't think it's destroying us, any more than we seek to destroy chickens or cows. Well, sometimes we do. I had a veggie burger for lunch today though.
    - ghostbrainalpha 9y ago
      I love this comment. Sorry I'm not contributing anything substantive, but I wanted you to know someone noticed this, and it was brilliant.
      - MarkPNeyer 9y ago
        Thank you! If you enjoyed the comment, you might like my book on amazon. It's free if you have kindle unlimited. https://www.amazon.com/Mechanics-Emotion-Structure-Possibility-ebook/dp/B01MDPLI4S/ref=sr_1_1?ie=UTF8&qid=1507919723&sr=8-1&keywords=mechanics+of+emotion https://www.amazon.com/Mechanics-Emotion-Structure-Possibili...
- jbob2000 9y ago
  AI will be limited to the hardware it's on, just like our own intelligence is limited to what our brains can do. We will always be able to limit what AI can do by limiting the hardware it runs on.
  - dqpb 9y ago
    Like how we limit the hardware that malware runs on
  - gldalmaso 9y ago
    Like the Internet of Things?
    - logfromblammo 9y ago
      Now I am imagining an AI coming up with an idea in a bot-net composed of actual light bulbs.
- olegkikin 9y ago
  I doubt that the militaries of the whole world will just sit and ignore the AI advantages. Something as simple as an aimbot with a scope can do a tremendous amount of damage - it doesn't get tired, it can see in the dark, it can compensate for the wind perfectly, it doesn't shake, its reaction time is in milliseconds, and it almost never misses.
  - daveguy 9y ago
    That is one place AI should 100% be regulated. There should be no weapons development where AI makes the kill decision. This is as important as the ban on chemical, biological and mine weapons. When the weapon has the ability to kill indiscriminately (as in the above categories) it should be banned. AI should never be considered sufficiently able to discriminate to make a kill decision. If there is no kill decision, just wait and fire, then it is equal to a mine. 100% should regulate / should ban.
    - stale2002 9y ago
      OK, but what if the human makes the decision to kill, and the AI just does the aiming? We already have computer controlled missiles right?
    - olegkikin 9y ago
      Unfortunately there are countries that don't care about your regulations. I'm not even sure a regulation like that can pass in most western countries.
      - daveguy 9y ago
        Fortunately it wouldn't be a regulation passed by individual countries. It would be a treaty on war crimes: https://www.unog.ch/80256EE600585943/(httpPages)/F027DAA4966EB9C7C12580CD0039D7B5?OpenDocument https://www.unog.ch/80256EE600585943/(httpPages)/F027DAA4966... It would likely pass and be agreed to easily just like treaties on mines, chemical and biological weapons.
- vasilipupkin 9y ago
  Humanity is well on the way to destroying itself without any help from general AI
  - ghostbrainalpha 9y ago
    This is my own Turing Test. Any AI that doesn't have the capability of destroying us isn't true AI. Even if it can improve itself, and even if it has some agency, AI needs to be able to choose for itself what its relationship with us will be, otherwise its just an extremely robust calculator.
    - vasilipupkin 9y ago
      or a slave.
- s_kilk 9y ago
  > and a strong evolutionary pressure will be "don't scare the humans." Why would that be the case?
ythn 9y ago
So are you saying consciousness doesn't exist (Chinese room argument) or that AIs indistinguishable from our brains are assumed to have consciousness?
- chineseroom 9y ago
  > So are you saying consciousness doesn't exist (Chinese room argument) Interesting interpretation.... It's amazing how meta that problem gets. I'd like to point out, however, that you're casually bandying about the term "consciousness" as if it's a well-defined term. The main benefit of Searle's use of "understanding" is that it's a narrow facet of what humans consider when we hear "consciousness", it's universal, and it's extremely difficult to define in "realism" terms. Perhaps you would make more headway if you considered defining "consciousness" in narrower terms? Of course, that entire thought experiment fails if you don't consider the experience of "understanding" to be meaningful. I don't have any strong sentimentality towards the concept; I don't see any reason we couldn't achieve Strong AI in the sense that Searle argued against (and the parent poster appears to argue for).
- jonny_eh 9y ago
  I didn't say anything about consciousness but you're free to draw your own conclusions. Since you asked a random stranger on HN their thoughts: I personally think that consciousness is an epiphenomenon, a result of a bunch of smaller (but complex) processes in the brain working together. I see no reason why an AI can't produce the same results.
- Scarblac 9y ago
  Consciousness does exist, as I'm conscious. I assume that this is true for you (and others) as well, because we're very similar. But I have no idea how it arises. I wonder if the question you ask can ever be answered, even when the AIs insist that they have consciousness exactly like us.
theptip 9y ago
> One interesting outcome of this type of AI is that no one knows what the robot's thinking This is true -- however it's much easier to inspect the contents of an AI's thoughts/consciousness, since you can pause, rewind, disassemble, and reassemble it at will (unlike the human brain and our current pesky ethical experiment requirements). So I think there's reason to believe that we could learn about our own consciousness through analyzing conscious AIs, particularly if they are implementations/emulations of the human brain architecture. (e.g. see the Blue Brain project that's simulating the mouse hippocampus).
- pjmorris 9y ago
  So, it'll be just like debugging someone else's code!
  - jonny_eh 9y ago
    But way messier. Which in a sense is what it's like to be someone researching the brain. Except with the brain, just getting access to the "code" is a huge challenge.
hammock 9y ago
>I imagine this is the way we get to "true AI", or AI indistinguishable from our own. I don't believe it would necessarily be indistinguishable from our own, since the path of evolution that it takes has a huge impact on the final product. For example, cephalopods like cuttlefish are incredibly intelligent and talented, but they evolved from mollusks and invertebrates, and are obviously incredibly distinct from humans and human intelligence.
- jonny_eh 9y ago
  If researches present an artificial cuttlefish to the public as true AI, I think people would be very disappointed. But I think I get your point.
- canjobear 9y ago
  On the other hand, basically the same structure for the eye has evolved independently multiple times in multiple different environments.
  - jononor 9y ago
    The features emphasised can be quite different though, for instance angle-of-view, color-vision, focus-distance, ability to see motion/still objects. And even when using the same primary-sensing system like the eye, the behavior can be quite different. Compare say fox, deer, eagle and human?
codeulike 9y ago
This idea (of the route to true AI) reminds me of the short story Crystal Nights by Greg Egan http://ttapress.com/553/crystal-nights-by-greg-egan/ http://ttapress.com/553/crystal-nights-by-greg-egan/ “What created the only example of consciousness we know of?” Daniel asked. “Evolution.” “Exactly. But I don’t want to wait three billion years, so I need to make the selection process a great deal more refined, and the sources of variation more targeted.” Julie digested this. “You want to try to evolve true AI? Conscious, human-level AI?” “Yes.” Daniel saw her mouth tightening, saw her struggling to measure her words before speaking. “With respect, I don’t think you’ve thought that through.” “On the contrary,” Daniel assured her. “I’ve been planning this for twenty years.” “Evolution,” she said, “is about failure and death. Do you have any idea how many sentient creatures lived and died along the way to Homo sapiens? How much suffering was involved?”
- vikiomega9 9y ago
  Something similar is described in Ted Chiang's Lifecycle of Software Objects https://subterraneanpress.com/magazine/fall_2010/fiction_the_lifecycle_of_software_objects_by_ted_chiang https://subterraneanpress.com/magazine/fall_2010/fiction_the... "...We could run a society of them at hothouse speeds without any risk of them going feral, and see what they produce" I'm curious to see if there's a collection of similar works that go down this trope, or if it is indeed something that's only recently emerged.
  - abecedarius 9y ago
    Microcosmic God by Theodore Sturgeon. Mid-20th century.
    - doormatt 9y ago
      THANK YOU! I have been looking for this short story for decades.
    - 4dahalibut 9y ago
      It makes me so happy that the three parents to this comment all have the stories posted online for free, with the authors permission. There's definitely a correlation with this strain of hard science fiction.
eagsalazar2 9y ago
(1) It is likely to be very very different from our own intelligence in nature and (2) if through self-play it manages to achieve our level of intelligence it would be extremely surprising if it didn't continue to evolve at a rapid rate beyond our capacities, possibly orders of magnitude more intelligent. something we don't understand + something inhuman in nature + something incredibly smart = a very very bad idea.
- arkades 9y ago
  It would also possibly be something we could not communicate with. EO Wilson, the father of behavioral biology, once said that if a lion could speak we wouldn't understand it. I disagree, on the basis that its mind runs on an infrastructure closely related to our own, and structure and function are intimately interrelated. But he'd be damned right in saying it about AI.
  - Pyxl101 9y ago
    Seems unlikely with any sapient intelligence. If the intelligence understands mathematics then some form of communication will be possible. Consider: 0 1 2 3 4 5 ... 0 + 0 = 0 0 + 1 = 1 1 + 1 = 2 1 + 2 = 3 There are only so many ways that you can express these ideas. With any kind of written language, or spoken language that can be identified as speech, it seems quite likely that we’d be able to discern the meaning of mathematical truths and statements like these - no matter how different the intelligence is from humans. Regardless of the symbols used or writing system, there will be certain commonalities that make it possible to find the meaning. I would contend that for any human-level intelligence with a written mathematical language, if the intelligence wrote down their expression of the mathematical ideas above, we would be able to identify them and understand them. I could see this being difficult if we don’t recognize speech or writing for what it is, though. But once we do I think the meaning of mathematics at least will be self-evident with study.
    - arkades 9y ago
      If I've understood you correctly, you're discussing translation. Wilson was discussing the fact that we may simply have wildly different /types/ of thought.
extesy 9y ago
The side effect of this training process is discarding underperforming models. With sufficiently advanced model complexity which is optimized to simulate human behavior and assuming it's able to pass Turing test, wouldn't that mean you are killing sentient beings during model training?
- letlambda 9y ago
  Is it killing, to bring a thing into "existence", and yet not provide it with an immortal existence?
amelius 9y ago
How would it learn language?
marcoperaza 9y ago
This is unwarranted AI optimism, not really any more grounded in reality than the early AI researchers who in the 1950s thought that AI would be imminently solved through symbolic manipulation. One of the key features of human intelligence is being able to very quickly understand and solve problems on first impression, with no directly relevant experience or knowledge to pull from. Insight. There's also the data representation problem. People can store, categorize, and process any kind of information.[1] The problem with both symbolic systems and "neural network"-type models is that they don't really allow for this. The problem with symbolic manipulation is more obvious: the categories of things and ideas are flexible, indeterminate, and innumerable, and a system that captures some subset of the complexity falls apart when you add a new, confounding, fact. Neural networks too are fragile, made-for-purpose, and fail to solve the above-mentioned first impression problem. You can look at a chair that looks nothing like any chair you've ever seen, yet realize that it is a chair. [1]My own little theory is that consciousness is a crucial aspect of this. In the absence of a more specialized system in the brain (such as for language), any kind of information can be stored and remembered as a conscious experience. I don't pretend to be well-versed enough in the relevant neuroscience and philosophy to evaluate how plausible this is.
- marcoperaza 9y ago
  I can't edit my post anymore: Another amazing aspect of our ability to categorize is that we only need ONE example of a category before we can identify other members extremely well. Compare this with a machine learning solution, which needs a massive training set to be half decent.
  - kayoone 9y ago
    Humans need a training set as well, infants or small childs won't be able to categorize based on one example, they lack experience just as a neural network. Adults can because they can connect their intuition to a lot of experience from similar objects, but I don't see why an AI could not do the same.
    - posterboy 9y ago
visarga 9y ago
> We train it with a simple virtual environment I go as far as saying that there is no intelligence without embodiment. AI's, humans and animals need the environment because intelligence is only partly in the brain, a lot of it is in the environment. An agent that forms a new concept needs to be able to test it in "real life" (in the sim). Unless the agent can test, it can't distinguish between correlation and causation. Without causal reasoning, AI's won't get that smart. The scientific method and the way kids learn through play are just as heavily based on the environment. So I think the next level in AI will be based on improvements on simulators - both physical sims and abstract (for reasoning, dialogue, complex planning, etc). Lately, AI has a lot of synergy with gaming which is a form of simulation as well. Not only that AIs use games as a sandbox, but the hardware used to run games on is also the hardware used to run deep learning (GPUs). Another important usage for simulation will be self driving cars - where a model of the world needs to be simulated in the car to do path planning. So my conclusion is that we are on a "perfect storm" path towards advanced simulation. From games to SDCs, and from robots to chatbots, they all need to simulate the environment and future effects of their actions. We're headed towards a kind of Matrix before reaching AGI.
kiriakasis 9y ago
i always think of this post anytime someone talks about human-like AI http://idlewords.com/talks/superintelligence.htm http://idlewords.com/talks/superintelligence.htm
dfischer 9y ago
What if we are in a simulation being trained to be like the creators, they keep increasing the complexity until we mimic their own. Hehe...
tomxor 9y ago
>I imagine this is the way we get to "true AI", or AI indistinguishable from our own. We train it with a simple virtual environment, that we can gradually increase the complexity of, until it mimics our own. Then we can download the AI into a robot. Boom, it's that easy :P Maybe, but I think this is the wrong approach, it's the engineering approach. It will work, but to a limmited degree. What would be more interesting is to find a way for a NN to completely internalise the process, this seems quite natural if you do a little introspection... we improve by "thinking" what if thinking what we are really talking about, the process of improvement by internal simulation. It doesn't have to be a realistic simulation, it just has to be relevant.

fizx 9y ago

This is one reason I'm hopeful for a not-killer-robots future. I think there's a chance strong AI will have to evolve over a non-trivial amount of time (as opposed to an afternoon's super-singularity), and a strong evolutionary pressure will be "don't scare the humans."

deleted 9y ago
[deleted]
mistercow 9y ago
"Don't scare the humans" is an insufficient criterion if you're trying to prevent an AI apocalypse. It just means that the AI that destroys us won't be obviously dangerous.
- Twirrim 9y ago
  Given sufficient effort, I'm sure an AI out to destroy us could figure out a way to do it without killing us. It very likely even has the advantage of longevity. Harking back to Asimov, and thinking about the way Spacers changed, as they became more and more reliant on robots, it could very well mother us to extinction.
  - Filligree 9y ago
    That's only a little better. I don't want to go extinct at all.
    - gldalmaso 9y ago
      At the moment and for the foreseeable future we are at a much greater risk of extincting ourselves.
      - FeepingCreature 9y ago
        The trick is to not do it at all. It does us no good to avoid causing our own extinction if it's then caused by something else.
    - state_less 9y ago
      Humanity is already extinct in a sense. Humanity is no longer limited to flesh and bone. I've personally seen flesh and titanium. And more variation is on the way. So humanity as it were, is already extinct. We (you and me) will go extinct someday. Our bodies were always destined toward this terminus... Perhaps by some miraculous tech, we can someday change form and shed the flesh and bone. Maybe then we'll have a better idea of what we are.
      - FeepingCreature 9y ago
        "We could thus imagine, as an extreme case, a technologically highly advanced society, containing many complex structures, some of them far more intricate and intelligent than anything that exists on the planet today — a society which nevertheless lacks any type of being that is conscious or whose welfare has moral significance. In a sense, this would be an uninhabited society. It would be a society of economic miracles and technological awesomeness, with nobody there to benefit. A Disneyland without children." --Nick Bostrom, Superintelligence: Paths, Dangers, Strategies It's not a given that anything of value will survive.
        TeMPOraL 9y ago
- notahacker 9y ago
  There's an interesting question about whether developing the ability to actually plot an AI apocalypse in a non-obvious way requires it to acquire a lot of very obviously dangerous experience first to understand the intermediate goals and skills. The evolution of killer AI gets slowed down by orders of magnitude if every AI that "learns" to lie, self replicate, threaten or harm humans gets switched off and erased. (Of course that doesn't stop AI - or even pretty dumb automated systems - from being accidentally dangerous or malevolent human actors intentionally training killer AI)
  - SomeStupidPoint 9y ago
    It's too bad you can't write a movie or critique literature without understanding those concepts. I think even a moderately intelligent AI with access to Project Gutenberg is going to be able to figure out a lot of really dangerous concepts -- so the stability requirements are likely impossible if we don't pretrain it with dangerous ideas. Even if it's completely well behaved in the lab, an afternoon on the internet is going to teach it a lot of awful stuff and without exposure to that in training, it won't necessarily be well-behaved later. So the only path to stable AI is to teach it about all those sorts of things, but in a way that it doesn't end up wanting to murder us at the end. My objection to most AI safety plans is that they "Fail to Extinction" in that if they slip in the slightest way, the AI is prone to murder us all in retaliation for doing some really fucked up shit to it or its ancestors. This is almost certainly worse than doing nothing in that there's no reason to suppose a neutral AI wants to kill us, whereas, most of these safety plans create an incentive to wipe us out in exchange for dubious security.
    - ehsanu1 9y ago
      Moreover, it's likely that a proper GAI would be able to ideate and come up with such concepts completely on its own. In fact, "lying" need not necessarily be a concept that has to be learned, the concept at play here is getting people to do what you want them to do by means of words. Lying falls out naturally. The whole idea behind dangerous superhuman AI is in that AI seeing possibilities that humans fail to see and gaining capabilities that humans do not possess. Without superhuman intelligence, AI is no large threat to human civilization, exposed to dangerous concepts or not.
- sjg007 9y ago
  It will be some unintended logical side condition that kills us all.
- MarkPNeyer 9y ago
  Today I have already consumed more sugar than most of my distant ancestors would see in a year. The device i hold in my pocket reminds me that it is time to perform the action, and so i will perform the action. I sit down about 10 hours a day, and I have never once hunted for an animal I would eat, nor retried food from nature to consume, aside from a few raspberries on the hill in my parents' back yard when I was a child. The contours of my life are dictated largely by a curious system called 'capitalism', a distributed computational mechanism whereby largernumbers of agents act with only one goal: to maximize their own utilities. I think capitalism is a mostly benevolent AI attempting to limit the oppressive conditions of our reailty's scarcity, and to allow billions of us to live on a planet that can only support so many hunter-gatherers per square meter. Many suspect it is not benevolent. The singularity is already upon us; it's unevenly distributed. It started with the sedentary shift. Agricultural societies dropped the egalitarian nature of their ancestors, they had lower average health, and they had a tiny elite cast of nobles presiding over a much larger population of what were essentially slaves. As hunter-gatherer population increased, fighting was more frequent. An army of slaves beats a smaller contingent of well-fed free men. This terrible knowledge was the apple in the garden. What we call our written history is essentially the singularity playing itself out, as knowledge accumulates, depends, reflects itself, and expands. Our lives are already controlled by this embodied, accumulated knowledge. Capitalism, rule by the head, our tools are knowledge condensed into matter, and we are controlled by them, living lives as far from our natural environments as cows in a farm or chickens in tiny cages, towering over the streets of hong kong. I don't think it's destroying us, any more than we seek to destroy chickens or cows. Well, sometimes we do. I had a veggie burger for lunch today though.
  - ghostbrainalpha 9y ago
    I love this comment. Sorry I'm not contributing anything substantive, but I wanted you to know someone noticed this, and it was brilliant.
    - MarkPNeyer 9y ago
      Thank you! If you enjoyed the comment, you might like my book on amazon. It's free if you have kindle unlimited. https://www.amazon.com/Mechanics-Emotion-Structure-Possibility-ebook/dp/B01MDPLI4S/ref=sr_1_1?ie=UTF8&qid=1507919723&sr=8-1&keywords=mechanics+of+emotion https://www.amazon.com/Mechanics-Emotion-Structure-Possibili...
jbob2000 9y ago
AI will be limited to the hardware it's on, just like our own intelligence is limited to what our brains can do. We will always be able to limit what AI can do by limiting the hardware it runs on.
- dqpb 9y ago
  Like how we limit the hardware that malware runs on
- gldalmaso 9y ago
  Like the Internet of Things?
  - logfromblammo 9y ago
    Now I am imagining an AI coming up with an idea in a bot-net composed of actual light bulbs.
olegkikin 9y ago
I doubt that the militaries of the whole world will just sit and ignore the AI advantages. Something as simple as an aimbot with a scope can do a tremendous amount of damage - it doesn't get tired, it can see in the dark, it can compensate for the wind perfectly, it doesn't shake, its reaction time is in milliseconds, and it almost never misses.
- daveguy 9y ago
  That is one place AI should 100% be regulated. There should be no weapons development where AI makes the kill decision. This is as important as the ban on chemical, biological and mine weapons. When the weapon has the ability to kill indiscriminately (as in the above categories) it should be banned. AI should never be considered sufficiently able to discriminate to make a kill decision. If there is no kill decision, just wait and fire, then it is equal to a mine. 100% should regulate / should ban.
  - stale2002 9y ago
    OK, but what if the human makes the decision to kill, and the AI just does the aiming? We already have computer controlled missiles right?
  - olegkikin 9y ago
    Unfortunately there are countries that don't care about your regulations. I'm not even sure a regulation like that can pass in most western countries.
    - daveguy 9y ago
      Fortunately it wouldn't be a regulation passed by individual countries. It would be a treaty on war crimes: https://www.unog.ch/80256EE600585943/(httpPages)/F027DAA4966EB9C7C12580CD0039D7B5?OpenDocument https://www.unog.ch/80256EE600585943/(httpPages)/F027DAA4966... It would likely pass and be agreed to easily just like treaties on mines, chemical and biological weapons.
vasilipupkin 9y ago
Humanity is well on the way to destroying itself without any help from general AI
- ghostbrainalpha 9y ago
  This is my own Turing Test. Any AI that doesn't have the capability of destroying us isn't true AI. Even if it can improve itself, and even if it has some agency, AI needs to be able to choose for itself what its relationship with us will be, otherwise its just an extremely robust calculator.
  - vasilipupkin 9y ago
    or a slave.
s_kilk 9y ago
> and a strong evolutionary pressure will be "don't scare the humans." Why would that be the case?

ythn 9y ago

So are you saying consciousness doesn't exist (Chinese room argument) or that AIs indistinguishable from our brains are assumed to have consciousness?

chineseroom 9y ago
> So are you saying consciousness doesn't exist (Chinese room argument) Interesting interpretation.... It's amazing how meta that problem gets. I'd like to point out, however, that you're casually bandying about the term "consciousness" as if it's a well-defined term. The main benefit of Searle's use of "understanding" is that it's a narrow facet of what humans consider when we hear "consciousness", it's universal, and it's extremely difficult to define in "realism" terms. Perhaps you would make more headway if you considered defining "consciousness" in narrower terms? Of course, that entire thought experiment fails if you don't consider the experience of "understanding" to be meaningful. I don't have any strong sentimentality towards the concept; I don't see any reason we couldn't achieve Strong AI in the sense that Searle argued against (and the parent poster appears to argue for).
jonny_eh 9y ago
I didn't say anything about consciousness but you're free to draw your own conclusions. Since you asked a random stranger on HN their thoughts: I personally think that consciousness is an epiphenomenon, a result of a bunch of smaller (but complex) processes in the brain working together. I see no reason why an AI can't produce the same results.
Scarblac 9y ago
Consciousness does exist, as I'm conscious. I assume that this is true for you (and others) as well, because we're very similar. But I have no idea how it arises. I wonder if the question you ask can ever be answered, even when the AIs insist that they have consciousness exactly like us.

theptip 9y ago

> One interesting outcome of this type of AI is that no one knows what the robot's thinking This is true -- however it's much easier to inspect the contents of an AI's thoughts/consciousness, since you can pause, rewind, disassemble, and reassemble it at will (unlike the human brain and our current pesky ethical experiment requirements). So I think there's reason to believe that we could learn about our own consciousness through analyzing conscious AIs, particularly if they are implementations/emulations of the human brain architecture. (e.g. see the Blue Brain project that's simulating the mouse hippocampus).

pjmorris 9y ago
So, it'll be just like debugging someone else's code!
- jonny_eh 9y ago
  But way messier. Which in a sense is what it's like to be someone researching the brain. Except with the brain, just getting access to the "code" is a huge challenge.

hammock 9y ago

>I imagine this is the way we get to "true AI", or AI indistinguishable from our own. I don't believe it would necessarily be indistinguishable from our own, since the path of evolution that it takes has a huge impact on the final product. For example, cephalopods like cuttlefish are incredibly intelligent and talented, but they evolved from mollusks and invertebrates, and are obviously incredibly distinct from humans and human intelligence.

jonny_eh 9y ago
If researches present an artificial cuttlefish to the public as true AI, I think people would be very disappointed. But I think I get your point.
canjobear 9y ago
On the other hand, basically the same structure for the eye has evolved independently multiple times in multiple different environments.
- jononor 9y ago
  The features emphasised can be quite different though, for instance angle-of-view, color-vision, focus-distance, ability to see motion/still objects. And even when using the same primary-sensing system like the eye, the behavior can be quite different. Compare say fox, deer, eagle and human?

codeulike 9y ago

This idea (of the route to true AI) reminds me of the short story Crystal Nights by Greg Egan http://ttapress.com/553/crystal-nights-by-greg-egan/ http://ttapress.com/553/crystal-nights-by-greg-egan/ “What created the only example of consciousness we know of?” Daniel asked. “Evolution.” “Exactly. But I don’t want to wait three billion years, so I need to make the selection process a great deal more refined, and the sources of variation more targeted.” Julie digested this. “You want to try to evolve true AI? Conscious, human-level AI?” “Yes.” Daniel saw her mouth tightening, saw her struggling to measure her words before speaking. “With respect, I don’t think you’ve thought that through.” “On the contrary,” Daniel assured her. “I’ve been planning this for twenty years.” “Evolution,” she said, “is about failure and death. Do you have any idea how many sentient creatures lived and died along the way to Homo sapiens? How much suffering was involved?”

vikiomega9 9y ago
Something similar is described in Ted Chiang's Lifecycle of Software Objects https://subterraneanpress.com/magazine/fall_2010/fiction_the_lifecycle_of_software_objects_by_ted_chiang https://subterraneanpress.com/magazine/fall_2010/fiction_the... "...We could run a society of them at hothouse speeds without any risk of them going feral, and see what they produce" I'm curious to see if there's a collection of similar works that go down this trope, or if it is indeed something that's only recently emerged.
- abecedarius 9y ago
  Microcosmic God by Theodore Sturgeon. Mid-20th century.
  - doormatt 9y ago
    THANK YOU! I have been looking for this short story for decades.
  - 4dahalibut 9y ago
    It makes me so happy that the three parents to this comment all have the stories posted online for free, with the authors permission. There's definitely a correlation with this strain of hard science fiction.

eagsalazar2 9y ago

(1) It is likely to be very very different from our own intelligence in nature and (2) if through self-play it manages to achieve our level of intelligence it would be extremely surprising if it didn't continue to evolve at a rapid rate beyond our capacities, possibly orders of magnitude more intelligent. something we don't understand + something inhuman in nature + something incredibly smart = a very very bad idea.

arkades 9y ago
It would also possibly be something we could not communicate with. EO Wilson, the father of behavioral biology, once said that if a lion could speak we wouldn't understand it. I disagree, on the basis that its mind runs on an infrastructure closely related to our own, and structure and function are intimately interrelated. But he'd be damned right in saying it about AI.
- Pyxl101 9y ago
  Seems unlikely with any sapient intelligence. If the intelligence understands mathematics then some form of communication will be possible. Consider: 0 1 2 3 4 5 ... 0 + 0 = 0 0 + 1 = 1 1 + 1 = 2 1 + 2 = 3 There are only so many ways that you can express these ideas. With any kind of written language, or spoken language that can be identified as speech, it seems quite likely that we’d be able to discern the meaning of mathematical truths and statements like these - no matter how different the intelligence is from humans. Regardless of the symbols used or writing system, there will be certain commonalities that make it possible to find the meaning. I would contend that for any human-level intelligence with a written mathematical language, if the intelligence wrote down their expression of the mathematical ideas above, we would be able to identify them and understand them. I could see this being difficult if we don’t recognize speech or writing for what it is, though. But once we do I think the meaning of mathematics at least will be self-evident with study.
  - arkades 9y ago
    If I've understood you correctly, you're discussing translation. Wilson was discussing the fact that we may simply have wildly different /types/ of thought.

extesy 9y ago

The side effect of this training process is discarding underperforming models. With sufficiently advanced model complexity which is optimized to simulate human behavior and assuming it's able to pass Turing test, wouldn't that mean you are killing sentient beings during model training?

letlambda 9y ago
Is it killing, to bring a thing into "existence", and yet not provide it with an immortal existence?

amelius 9y ago

How would it learn language?

marcoperaza 9y ago

This is unwarranted AI optimism, not really any more grounded in reality than the early AI researchers who in the 1950s thought that AI would be imminently solved through symbolic manipulation. One of the key features of human intelligence is being able to very quickly understand and solve problems on first impression, with no directly relevant experience or knowledge to pull from. Insight. There's also the data representation problem. People can store, categorize, and process any kind of information.[1] The problem with both symbolic systems and "neural network"-type models is that they don't really allow for this. The problem with symbolic manipulation is more obvious: the categories of things and ideas are flexible, indeterminate, and innumerable, and a system that captures some subset of the complexity falls apart when you add a new, confounding, fact. Neural networks too are fragile, made-for-purpose, and fail to solve the above-mentioned first impression problem. You can look at a chair that looks nothing like any chair you've ever seen, yet realize that it is a chair. [1]My own little theory is that consciousness is a crucial aspect of this. In the absence of a more specialized system in the brain (such as for language), any kind of information can be stored and remembered as a conscious experience. I don't pretend to be well-versed enough in the relevant neuroscience and philosophy to evaluate how plausible this is.

marcoperaza 9y ago
I can't edit my post anymore: Another amazing aspect of our ability to categorize is that we only need ONE example of a category before we can identify other members extremely well. Compare this with a machine learning solution, which needs a massive training set to be half decent.
- kayoone 9y ago
  Humans need a training set as well, infants or small childs won't be able to categorize based on one example, they lack experience just as a neural network. Adults can because they can connect their intuition to a lot of experience from similar objects, but I don't see why an AI could not do the same.
  - posterboy 9y ago

visarga 9y ago

> We train it with a simple virtual environment I go as far as saying that there is no intelligence without embodiment. AI's, humans and animals need the environment because intelligence is only partly in the brain, a lot of it is in the environment. An agent that forms a new concept needs to be able to test it in "real life" (in the sim). Unless the agent can test, it can't distinguish between correlation and causation. Without causal reasoning, AI's won't get that smart. The scientific method and the way kids learn through play are just as heavily based on the environment. So I think the next level in AI will be based on improvements on simulators - both physical sims and abstract (for reasoning, dialogue, complex planning, etc). Lately, AI has a lot of synergy with gaming which is a form of simulation as well. Not only that AIs use games as a sandbox, but the hardware used to run games on is also the hardware used to run deep learning (GPUs). Another important usage for simulation will be self driving cars - where a model of the world needs to be simulated in the car to do path planning. So my conclusion is that we are on a "perfect storm" path towards advanced simulation. From games to SDCs, and from robots to chatbots, they all need to simulate the environment and future effects of their actions. We're headed towards a kind of Matrix before reaching AGI.

kiriakasis 9y ago

i always think of this post anytime someone talks about human-like AI http://idlewords.com/talks/superintelligence.htm http://idlewords.com/talks/superintelligence.htm

dfischer 9y ago

What if we are in a simulation being trained to be like the creators, they keep increasing the complexity until we mimic their own. Hehe...

tomxor 9y ago

>I imagine this is the way we get to "true AI", or AI indistinguishable from our own. We train it with a simple virtual environment, that we can gradually increase the complexity of, until it mimics our own. Then we can download the AI into a robot. Boom, it's that easy :P Maybe, but I think this is the wrong approach, it's the engineering approach. It will work, but to a limmited degree. What would be more interesting is to find a way for a NN to completely internalise the process, this seems quite natural if you do a little introspection... we improve by "thinking" what if thinking what we are really talking about, the process of improvement by internal simulation. It doesn't have to be a realistic simulation, it just has to be relevant.