11 ms·
So, here's a random thought on this whole subject of "AI risk". Bostrom, Yudkowsky, etc. posit that an "artificial super-intelligence" will be many times smart
by mindcrime 11y ago
So, here's a random thought on this whole subject of "AI risk".
Bostrom, Yudkowsky, etc. posit that an "artificial super-intelligence" will be many times smarter than humans, and will represent a threat somewhat analogous to an atomic weapon. BUT... consider that the phrase "many times smarter than humans" may not even mean anything. Of course we don't know one way or the other, but it seems to me that it's possible that we're already roughly as intelligent as it's possible to be. Or close enough that being "smarter than human" does not represent anything analogous to an atomic bomb.
So this might be an interesting topic for research, or at least for the philosophers: "What's the limit of how 'smart' it's possible to be"? It may be that there's no possible way to determine that (you don't know what you don't know and all that) but if there is, it might be enlightening.
- Moshe_Silnorin 11y ago>It's possible that we're already roughly as intelligent as it's possible to be. Now that's more depressing than global annihilation.
- mindcrime 11y ago> Now that's more depressing than global annihilation. Fair point. But I honestly wonder if it's not true, or close to true. I mean, people have been talking about the "end of science" for a while now, even though we know we're not quite literally at "the end". But let's say that we unify relativity / gravity / QM in the next 100 years or so, and identify the exact nature of dark energy / dark matter. Presumably that would represent knowing close to everything about the physical world that its possible to know. And if we can discover that, it does lead you to wonder "what else would more intelligence represent"? Of course, maybe we're WAY more than 100 years away from those things. Or maybe we'll never get there. Or maybe if we do, it doesn't mean anything vis-a-vis the limits of intelligence. I'm just thinking out loud here, this isn't an argument that I've put a lot of time into developing...
- Kutta 11y ago> would represent knowing close to everything about the physical world that its possible to know. That's a severe misunderstanding of what it means to have a unified physical theory. The current standard model of physics allows us to theoretically predict the evolution of physical systems with an insanely high precision. A precision so great that there wouldn't be any meaningful error simulating macroscopic objects like human brains, given sufficient computing power. But we still have massive holes in our understanding of many things, including human biology or neurology. We don't have enough computing power to precisely simulate, but even if we had that power, it wouldn't necessarily let us understand anything. Elementary particles and humans are on hugely different scales. Observing a simulated human would be easier than observing a human simulated by physics itself, but it wouldn't just magically make everything clear. Having a TOE in physics brings us very little relative benefit compared to the Standard Model, when it comes to understanding biology, materials science, geology, psychology, computer science, etc.
- mindcrime 11y agoThat's a severe misunderstanding of what it means to have a unified physical theory. The current standard model of physics allows us to theoretically predict the evolution of physical systems with an insanely high precision. A precision so great that there wouldn't be any meaningful error simulating macroscopic objects like human brains, given sufficient computing power. Sure, like I said, I'm just thinking out loud here. Obviously TOE in and of itself doesn't automatically translate into full knowledge of everything in practice. But in principle, it would given sufficient computing power, allow us to simulate anything, which I think would yield additional understanding. But here's the thing, and where this all ties back together: sufficient computing resources may not be possible, even in principle, to allow use of that TOE to do $X. But an upper bound on the amount of computing resources should also reflect something of an upper bound on what our hypothetical AI can do as well. Or to put it all slightly differently.. if we had a TOE, we'd have shown, at least, that humans are smart enough to develop a TOE. Which I think at least raises the issue of "how much smarter can a machine be, or what would it mean for a machine to be much smarter than that?" Note to that I'm not necessarily arguing for this position. It's more of a thought experiment or discussion point than something I'm firmly convinced of.
- AnimalMuppet 11y agoIf you can't be more than you are now (or make something more), you'd rather not exist? That seems rather petulant of you.
- Moshe_Silnorin 11y agoLacking in principle the tools to solve the problems the universe gives you may be more depressing than non-existence. Though, I don't think it's actually possible to not exist. Name one person who's experienced nonexistence.
- AnimalMuppet 11y agoI'm not sure that "experiencing nonexistence" is theoretically possible. However, I can name lots of people who existed, and now don't. (Even if you're a theist or believe in reincarnation, I'm talking about this current, earthly existence.)
- isolate 11y agoI don't think a lack of intelligence of the IQ test variety is what's holding humanity back. I think it's distraction, greed, lack of empathy (or emotional intelligence), lack of information, and lack of communication. (Maybe there are more things.) Basically, I believe we have all the technology and intelligence we need to make this planet a much better place.
- steve19 11y agoWe had all the technology and intelligence we needed 2,000 years ago to make the world a better place. Yet our ancestors did not. For many 1000's of years different religions, some on opposite sides of the world from each other, have been saying the same thing. We have the ability to be better people, but all of us squander it with pettiness, jealousy, greed etc.
- mrob 11y agoEven if that's true, imagine an AI as smart as John von Neumann on modafinil, that never thinks about food/sex/art/etc., that never sleeps, that has access to Wikipedia etc. at the speed of thought, and no morals. That's not uncontrolled intelligence explosion level disaster, but it's still highly dangerous.
- murbard2 11y agoA virtual campus with thousands of hyperfocused John Von Neumanns collaborating telepathically and accomplish subjective years of works in seconds of real world time.
- vectorjohn 11y agoThe assumption being that these John Von Neumanns will only require cheap, modest hardware to run on.
- deleted 11y ago[deleted]
- cousin_it 11y agoIf we have a Von Neumann level intelligence running on a supercomputer, and we haven't solved friendliness by then, we've lost the future of humanity. As far as I can tell, all arguments against that conclusion are based on various kinds of wishful thinking.
- murbard2 11y agoYo, dawg, I heard ya like Von Neumann architecture.
- vectorjohn 11y agoAnd still I doubt that AI could talk its way out of a prison. It could be dangerous though.
- Kutta 11y agoEven if human intelligence was the pinnacle, AI could be still extremely dangerous just by running at accelerated simulation speed and using huge amounts of subjective time to invent faster hardware. See https://intelligence.org/files/IEM.pdf https://intelligence.org/files/IEM.pdf for discussion. The point is moot anyway though, since the hypothesis (that humans are the most intelligent possible) is just severely incompatible with our current understanding of science.
- vectorjohn 11y agoThis comes with the assumption that the first AI will be able to run at better than human levels on something like a raspbery pi. More realistically, it's going to have to run on an immense super computer, and even if it can marginally improve (marginal because a human level AI is no more likely to improve its own hardware than I am), it won't be able to just spread all over the world. It needs real physical hardware. That is, unless the reason we don't have AI is because we haven't put the pieces together correctly, and it really could work on minimal hardware.
- smegger001 11y agoJust because it requires real hardware does not mean it may be able to spread all over the world. It may just do it by setting up virtual machines in data centers all over the world, or as a botnet with a distrusted processing approach similar to folding at home.
- ctl 11y agoWell, if it's possible to build a human level intelligence, it's probably possible to build an intelligence that's much like a very smart human except it runs 100x faster. And in that case, somebody with sufficient resources could build an ensemble of 1000 superfast intelligences. That's a lower bound on the scariness of AI explosion, and it may already be enough to take over the world. Certainly it should be enough to take over the Internet circa 2015... To my mind it seems pretty clear that if AI exists, then scary AI is not far off. That said, I don't worry about this stuff too much, because I see AI as being much technically harder, and much less likely to materialize in our lifetimes, than articles like this suppose.
- MBlume 11y agoIf you look at the history of human evolution, this doesn't make sense. Evolution was very very slowly increasing human intelligence by, e.g., making our skulls bigger. Then we got to the point where we could use language/transmit knowledge between generations/etc. and got up to technological, spacefaring civilization in, evolutionarily speaking, no time whatsoever. This is not a story which suggests that human intelligence is some sort of maximum, or that evolution was running into diminishing returns and so stopped at human intelligence. It suggests that human intelligence is the minimum intelligence necessary to produce the kind of generational transfer that gets you up to technological civilization.
- alextgordon 11y agoIt's not like humans sit around all day thinking. We build tools to do the thinking for us. Humans ourselves are very bad at predicting the weather. So we build supercomputers, and weather models, and now we're a bit better at it. Now the question is: given the same budget, would a super-intelligent AI predict the weather in a substantially different way to humans? I cannot see how, but maybe that is my human stupidity talking.
- Houshalter 11y agoThis is a pretty random domain to pick. We don't know whether it's even possible to predict the weather better than we are. There are limits to that kind of prediction because weather is a chaotic system. However if it is possible to improve weather simulations, and if there are advancements to be made in meteorology, then it's very likely an AI would be able to find them.
- lmm 11y agoHumans are very bad at self improving. Imagine how smart we'd be now if our intelligence had increased at the same rate that computing power increases. Now imagine how fast computing power would've increased if we'd been that smart. Now imagine...
- tachyonbeam 11y agoClose to the limit of how smart it's possible to be? Don't be silly. The human brain is limited by its slow speed, by the amount of cortical mass you can fit inside a human skull, and by the length of human lifetimes. Computers will not have any of those limitations. In terms of speed: if you could build the exact silicon equivalent of a human brain, you may be able to run it several orders of magnitude faster, simply because it wouldn't be limited by the slow speeds of electrochemical processes in the human brain. Nerve impulses travel at speeds measured in meters per second. Neurons also need time to recharge between spike bursts, and they can physically damage themselves if they get too excited. In terms of volume: much of our intelligence is in perceiving patterns. That's limited by cortical mass. Pattern recognition is what all these "deep learning" systems excel at. The more depth they add, the better they get. Having deeper pattern recognizers, or simply having more of them, means you can see more patterns, more complex patterns, etc. Things that might be beyond the reach of any human. Then, in terms of data, machine have an advantage too. We're limited by our short lifetimes. How many people are expert musicians, genius mathematicians, rockstar programmers and great cooks? Very few. There's only so many hours in a day, and we only live so long. A machine could learn all those skills, and more. It could excel at everything. Speak every language, master every skill, be aware of so many more facts. And finally, I posit that maybe we, humans, are limited in our ability to grasp complex conceptual relationships. If you think about it, the average person can fit 7-8 items in their short-term memory, in their brain's "registers", so to speak. That probably limits our ability to reason by analogy. We can go "A is to B what C is to D", but maybe more complex relationships with 50 variables and several logical connectives will seem "intuitive" to a machine that can manipulate 200 items in its short-term memory.
- mindcrime 11y agoThe human brain is limited by its slow speed, by the amount of cortical mass you can fit inside a human skull, and by the length of human lifetimes. Computers will not have any of those limitations. Right, but the idea I'm playing around with is this: Suppose you had a hypothetical creature with a brain 20x as fast as the human brain and with twice the volume. How much smarter would that creature be in practice. It's kind of an abstract idea (and I probably don't fully understand it myself), but I'm getting at something like "is there a point where that additional raw computing power just doesn't buy you anything meaningful" at least in terms of "does it represent an existential threat?" or "does the nuclear analogy hold?"
- skybrian 11y agoAlternately, perhaps it is possible to be much smarter, but it's not as effective as we expect? If we think of intelligence as skill at solving problems, it might be that there are not many problems that are easily solved with increased intelligence alone, because the solutions we have are nearly optimal, or because other inputs are needed as well. This seems most likely to happen with mathematical proofs and scientific laws. Increased intelligence doesn't let you prove something that's false, and it doesn't let you violate scientific laws. But I don't find this particularly plausible. Consider computer security: hackers are finding new exploits all the time. We're far from fixing all the loopholes that could possibly be found.
- mindcrime 11y agoAlternately, perhaps it is possible to be much smarter, but it's not as effective as we expect? If we think of intelligence as skill at solving problems, it might be that there are not many problems that are easily solved with increased intelligence alone, because the solutions we have are nearly optimal, or because other inputs are needed as well. Yeah, I think that's a much better way of putting what I'm trying to get at. That is, maybe the machine is much "smarter" depending on how you define "smart". But maybe that doesn't enable it to do much more than we can do, because of other fundamental limits.
- AJ007 11y agoThat was the way I have been looking at it. It is not just an issue of finding solutions to impossible problems, or breaking scientific laws, but also problems where the solutions are along the lines of what George Soros would call reflexive. Computer security is like this, so is securities trading (no pun.) Secondly, what about problems which require the destruction of the problem solvers along the path to the optimal solution? I'm not sure about the correct word to describe this, or the best example but it could be seen in large systems. Where humans are right now is a result of this. We would not know many things if those things which came before were not destroyed (cities, 1000 year empires, etc.) Thirdly, is a uniform, singular AI the most optimal agent to solve these sorts of problems? Much the way we don't rely or use mainframes for computing today, perhaps there will be many AI agents each which may be really good at solving particular narrowly defined problem sets. This could be described perhaps as a swarm AI. Nick Bostrom's Superintelligence is a great book, but I don't recall much consideration along these lines. When a lot of AI agents are "live" the paths to solutions where AI compete against each other open up even more complex scenarios. There certainly are physical limitations to AI. Things like the speed of light can slow down processing. Consumption of energy. Physical elements that can be assembled for computational purposes. Between now and "super" AI, even really good AI could struggle to find solutions to the most difficult problems, especially if those are problems other AI are creating. The speed alone may be the largest challenge to humans. How do we measure this difficulty relative to human capabilities, I don't know. End of rant -- but the limits of not just AI but problem solving is quite interesting.
- ggreer 11y ago> Of course we don't know one way or the other, but it seems to me that it's possible that we're already roughly as intelligent as it's possible to be. I think Nick Bostrom had the perfect reply to that in Superintelligence: Paths, Dangers, Strategies: > Far from being the smartest possible biological species, we are probably better thought of as the stupidest possible biological species capable of starting a technological civilization—a niche we filled because we got there first, not because we are in any sense optimally adapted to it. It would be extremely strange if we were near the smartest possible minds. Just look at the evidence: Our fastest neurons send signals at 0.0000004c. Our working memory is smaller than a chimp's.[1] We need pencil and paper to do basic arithmetic. These are not attributes of the pinnacle of possible intelligences. Even if you think it's likely that we are near the smartest possible minds, consider the consequences of being wrong: The AI becomes much smarter than us and potentially destroys everyone and everything we care about. Unless you are supremely confident in humanity's intelligence, you should be concerned about AI risk. 1. https://www.youtube.com/watch?v=zsXP8qeFF6A https://www.youtube.com/watch?v=zsXP8qeFF6A
- mindcrime 11y agoThe AI becomes much smarter than us and potentially destroys everyone and everything we care about. Unless you are supremely confident in humanity's intelligence, you should be concerned about AI risk. I acknowledge it as a real risk, but it's not terribly high on my personal list of things to worry about right now. I like what Andrew Ng said about how worrying about this now is "like worrying about over-population on Mars".
- ggreer 11y agoPlease notice that your reply is a different argument than the one you first put forth. Originally, you weren't worried about AI because you thought it could never –even in principle– vastly exceed human abilities. Now you're basically saying, "I don't need to worry because it won't happen for a long time." That is a huge amount of ground to cede. I'm not so confident that human-level AI will take a long time. The timeline depends on algorithmic insights, which are notoriously difficult to predict. It could be a century. It could be a decade. Still, it seems like something worth worrying about.
- eli_gottlieb 11y ago> Of course we don't know one way or the other, but it seems to me that it's possible that we're already roughly as intelligent as it's possible to be. No, there's clearly at least one human smarter than you.
- mindcrime 11y agoWait, what? I'm sure there are a lot of humans smarter than me. I'm talking about the overall approximate intelligence of humans in a general sense, not any specific individual.
- eli_gottlieb 11y agoI certainly think there are individuals within our species whose cognitive algorithms are performing near-optimally. I just also have very good reason to think I can develop some algorithms that are better for certain tasks than the ones I consist in: I think there's sometimes more information in our sensory data than we take advantage of, and I think that better algorithms could tie together sensing and decision-making to obtain better data more often. Our brains "bite the Bayesian bullet" a bit too hard, accepting noisy data as-is and just making whatever inferences are feasible, rather than computing how to obtain clearer, less-noisy data (which is why we developed science as a form of domain knowledge rather than as an intuitive algorithm).
- sawwit 11y agoI think one of the best evidences we have is the level at which computers outperform the human mind in certain domains. An artificial general intelligence would have very low latency access to all these mathematical and computational tools we've invented (physical simulations, databases, theorem provers), and it would not need to mechanically enter program code on a keyboard, but it would be directly wired to the compiler. It could possibly learn to think in program code and execute it on the fly. The computation environment of neurons is also extremely noisy (axons are not well insulated) and neurons only fire at 7-200Hz. Assuming noise and low firing rate do not fulfill a certain task in mammalian brains, this is another way in which silicon-based minds could potentially be vastly superior. Thirdly, assuming sleep is not necessary for intelligence, artificial minds would never get exhausted. They could work 24 hours on a problem a day, which is possibly 5-10 time the amount of thinking time a human can do realistically. And lastly, an AI could easily make copies of itself. Doing so it could branch a certain problem to many computers which run copies of it and eventually collect the best result, or just shorten the time it takes to get a result. It could also evolve at a much faster rate than humans, assuming it has a genetic description: possibly hours to seconds instead of 20 years. Anyhow, it could easily perform experiments with slightly changed version of itself.
- Retra 11y agoI think the more relevant fact is that we don't have any ethical objections to shutting down computers, they're wholly dependent on our infrastructure, and the'll only evolve in ways that prove useful to us, because we wouldn't put a computer in charge of everything unless it were sufficiently compliant to our desires. I mean, are you going to put the same machine in charge of mineral extraction, weapon construction, transportation, and weapon deployment? When it hasn't proven to act correctly in a high-fidelity simulated environment? Probably not. We're also assuming that human ethics and intelligence are independent. I don't see many reasons to believe this. Social power and intelligence might be independent.
- je42 11y agoit doesn't work like that. you get into trouble when there is a trade-off to be made between switching off the AI and leaving it on.
- vectorjohn 11y agoI think most people didn't really understand the meaning of your comment. They seem to all equate intelligence and processing speed. I think it's legitimately an interesting question. As in, it could be something like Turing completeness. All Turing complete languages are capable of computing the same things, some are just faster. Maybe there's nothing beyond our level of understanding, just a more accelerated and accurate version of it. An AI will think on the same level as us, just faster. In that case, in that hypothetical, an AI 100x faster than a person is not much better than 100 people. It won't forget things (that's an assumption, actually), it's neuron firing or equivalent would be faster, but maybe it won't really be capable of anything fundamentally different than people. This is not the same as the difference between chimps and humans. We are fundamentally on another level. A chimp, or even a million chimps, can never accomplish what a person can. They will not discover abstract math, write a book, speak a language. Mind you, I suspect this is not the case. I suspect that a super intelligent AI will be able to think of things we can never hope to accomplish. But it is an interesting question that I think is worth thinking about, rather than inanely down voting the idea.
- mindcrime 11y agoAs in, it could be something like Turing completeness. All Turing complete languages are capable of computing the same things, some are just faster. Maybe there's nothing beyond our level of understanding, just a more accelerated and accurate version of it. An AI will think on the same level as us, just faster. In that case, in that hypothetical, an AI 100x faster than a person is not much better than 100 people. It won't forget things (that's an assumption, actually), it's neuron firing or equivalent would be faster, but maybe it won't really be capable of anything fundamentally different than people. Yeah, that's another really good way of putting it. That's probably closer to what I meant, than what I said above.
- derefr 11y ago> They seem to all equate intelligence and processing speed. It's helpful to remind people that every human brain that exists runs at the same processing speed†, yet we have greatly varying intelligence between us. (Also, IQ is an index, but people confuse it for a linear measure; human geniuses may be doing things that require many times the "intelligence" of the average person.) † Okay, I lied: there are some people whose neurons have abnormal firing rates; the visible result is Parkinsonism.
- vonnik 11y agoThis post is basically a repackaging of Nick Bostrom's book SuperIntelligence, a work suspended somewhere between the sci-fi and non-fiction aisles. As a philosopher of the future, Bostrom has successfully combined the obscurantism of Continental philosophy, the license of futurism and the jargon of technology to build a tower from which he foresees events that may or may not occur for centuries to come. Nostradamus in a hoody. Read this sentence: "It looks quite difficult to design a seed AI such that its preferences, if fully implemented, would be consistent with the survival of humans and the things we care about," Bostrom told Dylan Matthews, a reporter at Vox. Notice the mixture of pseudo-technical terms like “seed AI” and “fully implemented”, alongisde logical contructs such as “consistent with” -- all leading up to the phobic beacons radiating at the finale: “the survival of humans and the things we care about.” It's interesting, the technical challenges that he feels optimism and pessimism for. For reasons best known to himself, Bostrom has chosen to be optimistic that we can solve AI (some of the best researchers are not, and they are very conservative about the present state of research). It may perhaps the hardest problem in computer science. But he's pessimistic that we'll make it friendly. Bostrom’s tower is great for monologs. The parlor game of AI fearmongering has entertained, rattled and flattered a lot of people in Silicon Valley, because it is about us. It elevates one of our core, collective projects to apocalyptic status. But there is no dialog to enter, no opponent to grapple with, because no one can deny Bostrom's pronouncements any more than he can prove them. Superintelligence is like one of those books on chess strategy that walk you through one gambit after the other. Bostrom, too, walks us through gambits; for example, what are the possible consequences of developing hardware that allows us to upload or emulate a brain? Hint: It would make AI much easier, or in Bostrom’s words, reduce “recalcitrance.” But unlike the gambits of chess, which assume fixed rules and pieces, Bostrom’s gambits imagine new pieces and rules at each step, substituting dragons for knights and supersonic albatrosses for rooks, so that we are forced to consider the pros and cons of decreasingly likely scenarios painted brightly at the end of a line of mights and coulds. In science fiction, this can be intriguing; in a work of supposed non-fiction, it is tiresome. How can you possibly respond to someone positing a supersonic albatross? Maybe Bostrom thinks it will have two eyes, while I say three, and that might make all the difference, a few more speculative steps into the gambit. In the New Yorker article The Doomsday Invention, Bostrom noted that he was "learning how to code." http://www.newyorker.com/magazine/2015/11/23/doomsday-invention-artificial-intelligence-nick-bostrom http://www.newyorker.com/magazine/2015/11/23/doomsday-invent... We might have expected him to do that before he wrote a book about AI. In a way, it's the ultimate admission of a charlatan. He is writing about a discipline that he does not practice.
- Symmetry 11y agoIf you're concerned that humans are as smart as it's possible to be then I would recommend reading Thinking Fast and Slow or some other book on cognitive psychology. There's essentially a whole branch of academia studying topics isomorphic to figuring out those things we fail to realize we don't know on a day to day basis.
- mindcrime 11y agoIf you're concerned that humans are as smart as it's possible to be It's not about humans being as smart as possible though, it's more about being "smart enough" to where a hypothetical "smarter than human AI" is not analogous to a nuclear bomb. That is, are we smart enough to where a super-AGI can't come up anything fundamentally new, that humans aren't capable of coming up with, as bound by the fundamental laws of nature. then I would recommend reading Thinking Fast and Slow or some other book on cognitive psychology I'm reading Thinking, Fast and Slow right now, actually. And just to re-iterate this point: I'm not arguing for this position, just putting it out there as a thought experiment / discussion topic. I'm certainly not convinced this is true, it's just a possibility that occurred to me earlier while reading TFA.
- zupreme 11y ago"...will be many times smarter than humans." Stop there. We, as humans, don't even know how smart we actually are - and we probably never will. It's very unlikely that any species is equipped to accurately comprehend its own cognitive limits - if such limits even exist. It's even less likely that we can relegate the intelligence of a nonhuman entitity to a mathematically meaningful figure without restricting the testing material to concepts and topics meaningful to humans - which may have absolutely no relation to the intelligence or interest domains of a nonhuman entity.
- lmm 11y agoPlease. We refuse to talk about how smart we are for political reasons, that's all.
- deelowe 11y agoDumb AI will dominate the world well before "smart" AI even gets close to taking off. I think the more realistic scenario is something like the paperclip maximizer, but a little dumber. A world of highly interconnected, but somewhat stupid AIs could cause utter chaos in milliseconds by following just some very basic rules (e.g. maximize number of paperclips in collection). https://wiki.lesswrong.com/wiki/Paperclip_maximizer https://wiki.lesswrong.com/wiki/Paperclip_maximizer
- orionblastar 11y agoAs human beings we don't always reach our potential. As a child I had problems with other children picking on me, I was suspected of having autism due to social issues, I was given an IQ test and scored a 189. I was high functioning an in 1975 there was no such thing as high functioning autism, that wasn't discovered until 1994, so I got diagnosed with depression instead. Child Psychologist told my parents to put me into a school for gifted children, but they put me in public school instead where I struggled and my brain worked 10 times faster so I was always ahead of the class in learning and bored waiting for people to catch up. I was still bullied and picked on and this interfered with my learning. The same thing happened when I went to college and had a job, I was bullied and picked on. I never reached my potential and my mental illness was one of the reasons why and people picking on me was another reason, and had I been in a school for gifted children I'd be able to reach my potential better. I developed schizoaffective disorder in 2001 and it screws with my memory and focus and concentration. I ended up on disability in 2003. My career is basically over but I still have a potential I never met. What good is a high IQ if you can't reach your potential to use it? We keep hearing talk of an AI that is smarter than a human being, but we haven't seen one yet. Our current AI programs are not as smart as a human being yet, but they can do tasks that put human beings out of work. Just having a perfect memory, and being able to do fast math equations makes an AI in the "Rainman" category http://www.imdb.com/title/tt0095953/ http://www.imdb.com/title/tt0095953/ even if it is not as smart as a human being. I guess what I am trying to say is that an AI doesn't have to be as smart as a human being to be dangerous. Just like the Google Maps app that drives people off a cliff or into an ocean. An AI can make robocalls and sell a product and put people out of work. You can replace almost any line of work with an AI, and then it gets dangerous when a majority of people are unemployed by AIs that aren't even as smart as a human being. I'd like to see a personal AI that works for poor and disabled people to earn money for them, as they run it on a personal computer. Doing small tasks on an SEO marketplace using webbots for $5 each and 100 tasks a day for $500 in a Paypal account to help lift them out of poverty. I know there are people already doing this for their own personal gain, but if the AI that does that is open sourced so disabled and poor people can run it, it can help solve the problem of poverty.
- indrax 11y agoEven if there is a limit to the size of a mind, there is not a limit to the number or speed. An atomic scenario would be a billion human level intelligences running a hundred times faster.
- nshepperd 11y agoPeople have already studied this. You make it sound like an open question, but the answer is right there in Bostrom's Superintelligence, or any works on cognitive heuristics & biases, the physics of computing, or the mathematics of decision making. The answer is "no. we are nowhere near the smartest creatures possible". And there are multiple independent lines of argument and evidence that point directly to this conclusion.
- mindcrime 11y agoThe answer is "no. we are nowhere near the smartest creatures possible". Right, like I said, this whole thing was just "thinking out loud" more than any thoroughly researched and validated idea. And I didn't do a very good job of even explaining what I was getting at. It's not so much that the question is "are we the smartest creatures possible". It's more like: "as smart as we are, and given the constraints of the physical world, is there room for a hypothetical 'smarter than human AGI' to represent an existential threat, or something that justifies the analogy to nuclear weapons?" See also skybrian's comment which actually states it better than I did originally: https://news.ycombinator.com/item?id=10755109 https://news.ycombinator.com/item?id=10755109
- nshepperd 11y ago> "as smart as we are, and given the constraints of the physical world, is there room for a hypothetical 'smarter than human AGI' to represent an existential threat, or something that justifies the analogy to nuclear weapons?" The answer is still yes. Your "thinking out loud" is privileging an already refuted hypothesis.
- mindcrime 11y agoThe answer is still yes. Well that's a fine assertion, but how do you justify it? Your "thinking out loud" is privileging an already refuted hypothesis. WTH does that even mean? There is no hypothesis, there's a vague notion of an area to consider, which - if considered thoroughly - might or might not yield a hypothesis.
- avivo 11y agoWhy would you compare the limits on intelligence of an AI to the abilities of just one human? Why not compare it to 1000 people, all communicating and problem solving together? We know that this is possible because it happens all the time, and enables such groups to make lots of money in digital markets, and invest it in things like marketing, robotics, and policy. The intelligence of an AI is lower bounded by that of the most intelligent possible corporation. Potential corollary: Assuming one can make a human level AI, then if it is not sufficiently resource constrained (hard?) or somehow encoded with "human values" (very very hard), then it will be at least as dangerous as the most sociopathic human corporation.