8 ms·
What If the Singularity Does Not Happen? (2007) [video]
- tlb 7y agoAnother risk is that, as the singularity seems closer and closer, people are going to throw up their hands, stop learning math and science figuring the machines will do it all, and generally check out. If too many people do this too soon, sustainable recursive self-improvement won't happen and we'll be stuck with AI that doesn't quite work and nobody to fix it.
- conjectures 7y agoI don't think this is likely to happen. There is almost surely a class of interesting questions it takes AGI to answer but a math PhD to pose.
- cholmon 7y agoThe machine stops.
- gpderetta 7y agopeople still play chess even though machines are a class above.
- tlb 7y agoChess is fun, but will people enjoy fixing old security bugs? You could imagine an AI that's very good at adding features to itself, but just can't get security right because it doesn't think adversarially. Corporate software groups often have this same limitation.
- jeanvalmarc 7y agoThe video didn't load for me, but there was a similar talk on Youtube from Vinge: https://www.youtube.com/watch?v=RzRuPGnJxCs https://www.youtube.com/watch?v=RzRuPGnJxCs
- melling 7y agoYou can also read Vinge's essay from 1993: https://edoras.sdsu.edu/~vinge/misc/singularity.html https://edoras.sdsu.edu/~vinge/misc/singularity.html
- blacksqr 7y ago"Within thirty years, we will have the technological means to create superhuman intelligence. Shortly after, the human era will be ended." Four years to go... will we make it?
- Nuzzerino 7y agoI think the point was that we would have roughly the hardware needed to create superhuman AI. Then it would follow that "shortly after", someone would have figured out the rest.
- mcguire 7y agoI don't know about you, but I don't feel any smarter.
- melling 7y agoChina has made AI a priority. Hard to say what's going to happen but I would expect the AI arms race between the US and China to have quite an impact. https://www.technologyreview.com/s/609038/chinas-ai-awakening/ https://www.technologyreview.com/s/609038/chinas-ai-awakenin...
- katabasis 7y agoAt this point the most likely future looks like one where we use all of our resources, wreck the environment, and enter a new dark age. The potential benefits/dangers of AI seem distant in comparison.
- LifeLiverTransp 7y agoI never hear about nature catching up to us- its always defeated and in the corner, we remain dominant. But as soon as society crumbles, meet those creatures and plants we couldnt get rid off- hyper adapted to resist us. Plants to which we are extremely allergic, who are resilient to all herbizides and selective bred for maximum spread into our ecologic niche. Animals, who by virtue of surviving in information dense environments, are very stealthy- and very clever. Carrying out of knowledge or instinct diseases and allergic contagions, to drive us further back into domes. Most of our companion animals/plants unable to dwell in the open in the overheated world we created. All the resources we had for the first attempt gone- no easy to reach oil, the best mines old landfills and destroyed citys. To return from such a dark age, seems unlikely.
- chr1 7y agoIn the video the author specifically talks why this is not a realistic scenario, plants we can't get rid off in our fields are specifically adapted to the way we handle the fields, without us they will be outcompeted by other plants, the domesticated plant and animal species will give the new civilization a huge head start as most of the time our civilization existed went into creating the species that are able to support civilization, old landfills and cities would be very good mines, lack of oil will be compensated by the remnants of our libraries describing other ways of getting energy.
- primroot 7y ago> Plants ... who are resilient to all herbizides and selective bred for maximum spread into our ecologic niche. Ironically one of such plants is Cyperus Rotundus, very useful in times of famine. Perhaps we are living in hyperopic age, where the obsession with economies of scale and controlled environments makes the immediate localized complexity into a high entropy nuisance worthy of destruction.
- simplecomplex 7y agoWhat if the singularity already happened? How would we know? It’s unfalsifiable unmeasurable woo-woo. The singularity and AGI are apocalypse for atheists.
- TelmoMenezes 7y ago> It’s unfalsifiable unmeasurable woo-woo. An hypothesis that cannot be falsified does not count as a scientific theory (according to Popper, not everyone agrees). That does not mean that is woo. It just means that it's outside the epistemological limits of science. Another thing that is unmeasurable is science, and yet most people don't doubt it. > The singularity and AGI are apocalypse for atheists. I agree with that to a degree.
- the_af 7y agoAgreed. There is a really funny article by idlewords (who also regularly comments here) about this: https://idlewords.com/talks/superintelligence.htm https://idlewords.com/talks/superintelligence.htm Not only is the Singularity religion for atheists, it also sparked tremendously bizarre offshoot beliefs, like that "Roko's Basilisk" meme that the "Eliezer Yudkowsky" crowd was briefly obsessed with.
- burnte 7y ago"Not only is the Singularity religion for atheists" I also don't understand this statement. I asked the parent about why the singularity and AGI are "the apocalypse" for atheists, but I also don't follow the logic of this statement. Granted, I don't think there will ever be a "singularity", but I fail to see how it is in any way linked to atheism. Maybe a small subset of people, like you said, a "bizarre offshoot".
- the_af 7y agoAh, yes. Sorry, I'm an atheist myself and I can see how my sentence could be misconstrued. Let me rephrase: there's a bunch of Silicon Valley technologists and like-minded people, who consider themselves enlightened atheists, and who have fallen prey to a different kind of religion: an apocalyptic tech-religion with a set of distinct beliefs such as the Coming of the Singularity, the Evil AI, the Upload Your Mind to the Cloud and Live Forever cult, etc. Not everyone believes in the same subset, and not everyone believes in the more bizarre of these beliefs. As an atheist myself, I definitely didn't mean that all or even the majority of atheists will fall for this. I meant that a specific subset of tech-minded, nerd, and likely well-to-do atheists tend to fall for these techno-religions. It's a "religion for atheists" because it's a religion that tries to hide the trappings of a more traditional religion, for people who presumably reject them and who don't consider themselves open to new age pseudoscience either. But tell me it's not cute how Ray Kurzweil has faith in reaching immortality (salvation) in his own lifetime.
- JackFr 7y agoI really like Vernor Vinge's fiction, and he's got some really big ideas, but sometimes in contexts like these I'm surprised by how shallow and provincial the thinking is. He seems unable to step out the present western 20th/21st century worldview. The singularity (if it comes) should feature intelligences whose modes and motivations are so superior and advanced, they are literally incomprehensible to us. Questions like "Will the machines allow humans to live?" are ill-posed. As human philosophers can't agree on the meaning or the existence of free will, will the machines? Will they have will, free or otherwise? Or is free will itself the mere shadow on the wall of the a richer thing that we can't truly, ever comprehend? So talking about what the machines/intelligences 'want' is getting ahead of ourselves.
- DennisP 7y agoPossibly Vinge is well aware that a superior intelligence would be incomprehensible, but he just hasn't figured out how to write a compelling story about that.
- tlb 7y agoThe question might be impossible to answer until after the fact, but it's not ill-posed. Machines either will, or will not, kill all humans.
- pessimizer 7y agoNo, AI will desperately want sex, money, and lebensraum, just like human dictators. It won't think of either humanity or self-preservation as completely irrelevant and not interesting. Singularity talk is just projection and self-idealization. "AI will be like me, but perfect, and infinitely more capable of doing what I would do if I were omnipotent, which it will decide to do faster than I could ever make a decision."
- loup-vaillant 7y ago> Questions like "Will the machines allow humans to live?" are ill-posed. I don't think so. Even if the machine doesn't care about you (it may not even have a notion of what is a human), you are still made of atoms it could use for something else.
- jbob2000 7y agoOk, I have this theory and it's probably kinda ignorant, so I welcome some criticism. It looks like biological intelligence is purely a product of having an electrical network with tons of connections. That's the brain, right? And less intelligent species have less connections, so we can probably draw a straight line from low intellect - low connectivity to high intellect - high connectivity. My argument is that intelligence is a product of the complexity of an electrical network. So will machine intelligence be purely a product of creating an electrical network with tons of connections? If I took billions of computers, wrote some basic logic for how they communicated with each other, and then wired it all up, would I get an intelligent being? If those suppositions hold true, then all we need to do to reach AGI is to continue to invest in the internet (as it is the most advanced electrical network we've made to date). Continue to add more computers and continue to remove hindrances to the operation of the network (censorship, rate limiting, political logic). Then it will just magically wake up one day and say "hello world".
- tbenst 7y agoA neuron uses far more than electricity to communicate. The number of neurotransmitters and receptor types alone is staggering. There’s a reason why biology evolved past using gap junctions. The computer science view of the brain compares how the brain works to FLOPS, much like how previous generations compared the brain to horsepower from the steam engine. Better analogy, but still naive.
- ben_w 7y agoThis is an important thing that optimists often miss. If you naively apply simple estimates of how many flops it takes to simulate a mind… well, I have and it would be possible to put ~35,000 real-time human minds onto all the iPhones Apple sold this year: https://kitsunesoftware.wordpress.com/2018/10/01/pocket-brains/ https://kitsunesoftware.wordpress.com/2018/10/01/pocket-brai... There’s also the problem that we don’t yet know how ignorant we are about our minds. I’m sure neuroscientists do at least have a rough idea by now of how much there is left to learn, but I’d be surprised if they are confident enough to be able to estimate the cost of the remaining required research to less than a factor of 10, especially as we necessarily started with the easy-to-learn knowledge.
- Mediterraneo10 7y agoVisions of the Singularity in the near term failed to account for two things: 1) obtaining regulatory approval for innovations in various spheres (medicine, aviation) takes forever, 2) innovations that rely on the consumer market are slowed by people wanting to use their present purchases for some amount of time before shelling out again for the next generation of tech. An exponentially faster development of technology is limited by how fast the workings of government agencies and consumers' minds operate. I rolled my eyes when I read Kurzweil's predictions of medical advances in his early-millennium books. Even if we invent nanobots or whatever so quickly, the FDA and its counterparts are not going to allow applying those technologies the very moment they are invented.
- wiz21c 7y ago> the FDA and its counterparts are not going to allow applying those technologies the very moment they are invented. And probably for very good reasons.
- wetpaws 7y agoPeople act like they are immortal, but the sad truth is that we all gonna die and no amount of FDI regulation will save us from it. All our current medical technologies a way to late to the party, if anything else we should've have them 50 years ago.
- zdw 7y agoAnything that depends on current high tech or advanced manufacturing (a good portion of imaging technologies, many implants) would definitely not been around 30, let alone 50 years ago.
- PasserBy2024 7y agoWell, unless future generations will resurrect me then I would prefer the progress to stall completely. Otherwise it's kind of unfair.
- wetpaws 7y ago
- aj7 7y agoThis is ridiculous, a sudden singularity. The reason is that computation (brain) power required shows no signs of singular growth, just steady growth.
- lacker 7y agoTo me a singularity does not seem to be approaching. In particular it doesn’t seem like technology is “accelerating faster and faster”. The broadest metric for human progress is GDP growth, and in the U.S. GDP growth hasn’t really been accelerating, it’s been around 2.5% a year for the past decade or so. AI in particular has made some advances but most basic activities remain undoable. AI cannot fold my laundry or generate a believable sentence well enough for those features to make their way into a consumer product. Voice recognition like Alexa is the only big impact of AI so far, which places it below advances like “touch screen glass” in terms of its impact on society. It could certainly change. The biggest question right now is whether self driving cars will work, or whether it will fizzle. In the 50’s many car companies thought the next step was inevitably personal airplanes, and that fizzled. So it’s possible it just won’t happen.
- taneq 7y agoReally? How much progress was made on thinking machines in the 19th century? How much between 1900 and 1950? 1950 and 2000? 2000 and 2010? 2010 and now? I think each of those timespans is very roughly equivalent. In the past 10 years voice recognition has gone from useless to ubiquitous. Neural image labeling and segmentation likewise. Physical robots have gone from taking a few faltering steps to doing parkour. None of that counts as "AI" any more because it works and we know how, but the rate at which new capabilities are moving from the realm of "AI" to the realm of "normal engineering" is increasing exponentially.
- nwah1 7y agoI agree that technology isn't progressing "faster and faster," but GDP doesn't measure technological progress. That is absurd. It doesn't even measure economic growth very well, and even if it did that wouldn't mean technological progress would track it. Technological progress can happen even during a recession, and often has. GDP measures the value of monetized final goods in an economy. Linux and Wikipedia have thus reduced the GDP, because they displaced a huge chunk of existing markets that used to be monetized. GDP is useful for monetary policy conducted at central banks, primarily.
- samnwa 7y agoGreat episode of Peter Thiel and Eric Weinstein discussing this slowing of technological progress: https://www.youtube.com/watch?v=nM9f0W2KD5s https://www.youtube.com/watch?v=nM9f0W2KD5s