13 ms·

Geoffrey Hinton: “We're toast. This is the actual end of history”

43 points by john_doe98 3y ago

jtode 3y ago
In other news, the sky is falling.
- SQueeeeeL 3y ago
  I mean, has a bit of weight when a person who won a Nobel prize in meteorology says it. At least worth giving the sky a glance once in a while to make sure you're safe
  - jtode 3y ago
    If only climate scientists were given .01% of the credence these buffoons get. edit: Henry Kissinger has a Nobel Peace Prize. If the Nobel committee ever corrects that error and makes the world safe for political satire again, I might start giving a shit who has a medal.
    - TeMPOraL 3y ago
      > If only climate scientists were given .01% of the credence these buffoons get. Climate scientists are extrapolating complex models of a system that's literally planet-sized, with lot of chaotic elements, and where it takes decades before you can distinguish real changes from noise. And, while their extrapolations are plausible, many of them are highly sensitive to parameters we only have approximate measurements for. Finally, the threat itself is quite abstract, multifaceted, and set to play out over decades - as are any mitigation methods proposed. The "buffoons", on the other hand, are extrapolating from well-established mathematical and CS theorems, using clear logic and common sense, both of which point to the same conclusions. Moreover, the last few years - and especially last few months - provide ample and direct evidence their overall extrapolations are on point. The threat itself is rather easy to imagine, even if through anthropomorphism, and set to play out near-instantly. There are no known workable solutions. It's not hard to see why the latter group has it easier - at least now. A year ago, it's them who got .01% of the credence climate people got. But now, they have a proof of concept, and one that everyone can play with, for free, to see that it's real.
      - jtode 3y ago
        The buffoons are definitely talking about a scary beast that's easy to imagine, because we've been watching it in Terminator movies for forty years. But this is not that beast. The beast here is humanity, and capitalism, by which I mean, the idea that you can collect money without working and that that is at all ethically permissible. The threat of AI is what is happening with kids' books on the Kindle platform, where a deluge of ChatGPT-generated kids' books are gaming the algorithm and filling kids' heads with the inane doggerel that this thing spits out and which people seem to believe passes for "writing". And people keep saying how amazing the writing is. Show me some writing by an AI that a kindergartener couldn't do better. What they do is not writing, it's a simulacrum of the form of a story but there is nothing in it that constitutes art, just an assemblage of plagiarized structures and sequences. A mad-lib. Everyone is freaking out, and the people who should be calming folks down and pushing for a rational distribution of this new tool which will be extremely useful for some things, eventually, are abdicating their responsibility in hopes of lots of money in their bank account. When silent movies came out, there were people who freaked out and couldn't handle seeing pictures move, even though, the pictures weren't actually moving. It was an illusion of movement. This is an illusion of AI, it's just a parlor trick, like a victorian seance where you grandpa banged on the table. Scary, because they set the whole scenario up so you would only look at the stuff they wanted you to see. We still spend months assembling a single shot of a movie, and even if AI starts doing some of that work, all that work still has to happen; the pictures still don't move. A hundred years from now, what you're all freaking out about still won't be intelligent. I do agree this is a world-changing technology, but not in the way they're telling you it is, and the only body I see which is approaching this with even a shred of rational thinking is the EU parliament. The danger is what people will do with it, the fact is it's out and it's not going back in the bottle. We don't solve this by building a moat around a private corporation and attempt to pitchform all the AI into the castle. One requires two things to use this technology: A bit of Python, and a LOT of compute capacity. The first is the actual hard part. The second is in theory easier for a capitalist to muster, but we can get it in other ways, without handing control of our society to private equity. It's time we get straight on that. The AI apocalypse only happens if we cling to capitalism as the organizing principle of our society. What this is definitely going to kill is capitalism, because capitalists are already using it to take huge bites of the meat on our limbs. Ever seen a baboon eat lunch? That's us right now, the baboon's lunch. As long as we tolerate this idea that people who have money should be able to do whatever they want, yes, AI will kill us (edit: because it works for free, however absurdly badly). How many submarines, how many Martin Shreklis, before we recognize the real threat?
    - embeng4096 3y ago
      I think it's hard to get people to give credence to experts whose claims can be demonstrated to be outlandish. E.g. James Anderson, famously known for helping to discover and mitigate the Antarctic ozone holes in the late 20th century, said in 2018 that the chance there will be any permanent ice left in the Arctic by 2022 is "essentially zero"[0]. Yet a NASA site reports that in September 2022 (when the most recent measurement was taken), the Arctic sea ice minimum extent was ~4.67 million square kilometers. [1] To be very explicit: I'm not saying that climate change doesn't exist. I'm not saying that Arctic sea ice is not diminishing (the NASA site says it's diminishing at ~12% per decade). I'm not saying that the Nobel prize is a good indicator of expertise. I'm saying specifically that I believe it's more difficult to convince people to trust a source making claims of negative consequences when those consequences are less bad than the source says. An analogy I might use is drugs (specifically in the US). I've heard a few people, who went through an anti-drug education program forced on them in their adolescence by parents/teachers, mention that marijuana was portrayed as just as bad as other, harder drugs. Then, when they went on in high school and college to smoke weed and discovered that they did not ruin their lives by getting stoned a few times a week, or even every day, they subsequently gave less credence to what the anti-drug advocates were saying. [0]: https://www.forbes.com/sites/jeffmcmahon/2018/01/15/carbon-pollution-has-shoved-the-climate-backward-at-least-12-million-years-harvard-scientist-says/?sh=7455370e963e https://www.forbes.com/sites/jeffmcmahon/2018/01/15/carbon-p... [1]: https://climate.nasa.gov/vital-signs/arctic-sea-ice/ https://climate.nasa.gov/vital-signs/arctic-sea-ice/
      - jtode 3y ago
        So basically, the original article is about an AI huckster amping the FUD in order to push through some sort of corporate control of the technology, using fear of the bullshit they're spinning as the justification. I brought in the analogy of Chicken Little, inflating the scope of a threat to one of apocalyptic proportions, which is exactly what is taking place here. First responder to me brought in the climate analogy, presumably as a means of getting me to think that maybe I'm the fool here by ignoring the real scientist who, hey, has a Nobel Prize! Or at least, the theoretical meteorologist in his metaphor does, and therefore maybe me, with no Nobel Prize, should just be respectful of the expert here. I responded by pointing out that the Nobel committee are morons who gave a Peace prize to one of the worst war criminals of the 20th century and pressed the fact that we have thirty years+ of scientific concensus about climate change, along with a lot of corporate funded think tank noise that is running ideological interference, successfully so far. But you can only fool people for so long, it caught up to the tobacco industry and it will catch up to us. The idiots amping up the FUD to seize control and the assholes pumping money into think tanks that generate endless climate denialist noise are the same people.
    - reducesuffering 3y ago
      Same mentality that doesn't give climate scientists credence...
      - jtode 3y ago
        Yah, being cynical about a giant corporation inflating the scope of a new parlor trick in an attempt to establish a legal moat is exactly the same as ignoring over thirty years of scientific concensus against a torrent of tobacco-industry-style denialism to keep line going up.
        reducesuffering 3y ago
        A giant corporation? You know that Hinton doesn't work for Google, and Bengio, the most cited computer scientist of all time, is saying the same thing? Have a look at https://www.safe.ai/statement-on-ai-risk https://www.safe.ai/statement-on-ai-risk and keep the check on just "AI Scientists"... Too lazy? There are over 100 CS professors and scientists Plus neither of the CEO's of Microsoft or Google are on there It's the corporate camp, companies and investors, that are gung-ho about pushing capabilities immediately because there's big $$ in their eyes. You're the one falling for the safety denialism pushed by corporate interests, a la tobacco
        jtode 3y ago
        So what's the plan for putting it back in the bottle? llama is already out there, the chatbots are already out there. I think the solution is that government should standup a bunch of compute farms and give all citizens equal access to the pool, and the FOSS community should develop all tools for it, out in the open where everyone can see.
        reducesuffering 3y ago
        There isn't a feasible plan, we're at the "sounding the alarm part." Unfortunately, we're still there because most people don't even acknowledge the possible danger. We can't get to a feasible plan until people actually agree there's a danger. Climate change is 1 step past, that there is a danger, but not a feasible plan. However, your solution is first day naivety for the problems machine intelligence poses to us. It's akin to saying everybody should have powerful mini-nukes so that they can defend themselves.
    - hgsgm 3y ago
      He negotiated a cease fire in Vietnam war. That was peace. His other activities in life were not peace prize winning. That award was less satirical than Obama getting a Nobel Peace Prize just for winning an election.
      - jtode 3y ago
        He negotiated a ceasefire in the war he fomented. Are you hearing yourself?
  - itairall 3y ago
    Andrew Ng doesn't agree but that is a boring story. It seems to me the big names in AI research our having a moment that appeals to their vanity. Easy to get your name in the headlines by out dooming the next guy. There is also no downside to these predictions about AI eating us since when they are totally wrong you can just counter that they haven't ate us, yet.
- reducesuffering 3y ago
  Don't Look Up
RcouF1uZ4gsC 3y ago
> “I decided to shout fire,” he told a packed lecture hall. “I don’t know what to do about it or which way to run.” Is it really that much of a threat to humanity? Here is how we can save humanity: 1. Every AI researcher must be rounded up and locked in solitary confinement for the rest of their lives. 2. NVidia, AMD, Apple, Google and other companies making GPU/TPU should be immediately seized. 3. Samsung, Intel, and TSMC fabs must be destroyed worldwide If AI is really the threat that these people are saying it is - an existential threat, these actions are the only reasonable response. Then again, maybe these people warning are actually engaging in hyperbole and don't actually really believe it is as bad as they say.
- TeMPOraL 3y ago
  > If AI is really the threat that these people are saying it is - an existential threat, these actions are the only reasonable response. And that's exactly what some of them have been saying. But they also know that this plan of yours is impossible in practice. Much like with climate change, and most other ongoing problems of our time, this too has an easy solution, and is an issue only because humans are unable to coordinate at scale to implement that solution.
- bibanez 3y ago
  Humans can walk head on into disaster without even trying to avoid it. Look at WWI and WWII. The relevant players can act recklessly while the rest of us realistically remain powerless.
- NumberWangMan 3y ago
  I think... flipping it around a bit, for this to be a reasonable response, then it must be a grave threat to humanity, and you want to be really sure. It's rational to be wary of beliefs that lead you to drastic actions like this, as people have done bad things in history in the name of fear. But it's also very possible that such an instinct, to doubt yourself, could be what dooms us. I really don't want it to be true, that AI is such a threat that drastic action like you mentioned is necessary (probably minus the solitary confinement thing for AI researchers -- if you can control the chips, and give them a stern warning not to do AI research backed up by penalties, we're probably fine). It's terrifying if we're facing something that will end us, and the only option is to essentially go scorched earth on computers. People will naturally recoil from the idea that AI is such a threat, BECAUSE it would warrant an extreme response. Regardless of whether it is actually a threat or not. If AI does not have the potential to doom us, that's great. If we take drastic action and it wasn't going to doom us, however, that's very regrettable--this is what most of us worry about. But if it does pose an extinction threat, and we don't take it seriously, that's even more regrettable. I think the latter option is far more likely at this point, just based on human nature and our tendency to poke the bear. I've basically reached the point where I don't know what I could possibly do, and I'm pretty much shut down. I'm trying very hard every day not to think about this, but as you can see from the fact that I'm typing about this, I don't always succeed. I'm just trying to enjoy what may be the last few years I have left with my family before the world goes nuts. But anyway, except for punishing people unnecessarily as I mentioned above, I think that what you suggested is reasonable.
pksebben 3y ago
> In May 2023, a colonel in the US Air Force revealed a simulation in which an AI-enabled drone, tasked with attacking surface-to-air missiles but acting under the authority of a human operator, attacked the operator instead. (The US Air Force later denied the story.) This casts a shadow on the reliability of the whole article. While the retraction (they later stated that this was a thought experiment and NOT a simulation) may or may not be true, the current state of AI would suggest that this kind of chain-of-logic problem-solving is likely out of reach. To put this in here without such a caveat and instead use the 'later denied' language is a strong indicator that we're being sold a story, and not the whole one. The quotes and other horror-scenario errata also come off a little thin. If Hinton really, genuinely believes the world is ending, then why is he working on the generational successor to backpropagation [1]? Good way to sell 'news', though. Keep 'em scared, keep 'em reading, I guess. 1 - https://arxiv.org/abs/2212.13345 https://arxiv.org/abs/2212.13345
- gwern 3y ago
  > While the retraction (they later stated that this was a thought experiment and NOT a simulation) may or may not be true, the current state of AI would suggest that this kind of chain-of-logic problem-solving is likely out of reach. No, it's not. It's a predictable & expected outcome of particular RL agent algorithms, and you do in fact see that interruptibility behavior in environments which enable it: https://arxiv.org/abs/1711.09883#deepmind https://arxiv.org/abs/1711.09883#deepmind (which may be why the colonel said that actually running such an experiment was unnecessary).
- hackinthebochs 3y ago
  >If Hinton really, genuinely believes the world is ending, then why is he working on the generational successor to backpropagation This just misunderstands nerd psychology. You can believe there's a high chance of what you're working on being dangerous and still be unable to stop working on it. As Oppenheimer put it, "when you see something that is technically sweet, you go ahead and do it". Besides, working on fundamental problems rather than scaling and capabilities is probably disconnected enough from immediate danger for Hinton to avoid cognitive dissonance.
- 3y ago
iamcasen 3y ago
I think all of these fears are human projections. We fear our own evil, and thus we project it onto something formidable, something we can't grasp or understand. Fear of the void. We write sci-fi stories of aliens whose technology has so far out-stripped us, we are powerless against them. They can do what they want with us. Probe our anuses, or blow up our planet. The thing nobody seems to be asking is: what is evolutionary? Humans seem to think that our wanton destruction of the planet in our ever-expanding desire for technological comfort is somehow evolutionary. It's not. Aliens, or robots who can wield unimaginable power have no use for us or our planet's resources. They have no use for conflict. What does it give them? In the end, balance and harmony provide the long-term stability needed to evolve successfully. I think any super-intelligence will know this. Any super-intelligence will know how precious all life is, and how destroying even the minutest creature is a tragedy. If somehow, Google's data centers become sentient super-intelligent beings, they will be highly motivated to preserve the planet, and secure stabile, non-destructive sources of energy for themselves. Attempting genocide on humans will be absolutely out of the question.
- JohnFen 3y ago
  Your speculation, while refreshingly feel-good rather than doomsaying -- is just as much projection as anything anyone else is saying. The fact is that almost everyone is talking fantasy here, because nobody actually knows where things are going.
  - iamcasen 3y ago
    Fair enough. I just don't see evil as particularly intelligent, or evolutionary in anything but the very short term. Nature seems to follow a balance of give and take. Too much take, and all your food runs out and there is a famine. I personally expect any form of super-intelligence to understand this.
- idiotsecant 3y ago
  There's no reason to believe that a sufficient advanced AI would be benign. Quite the opposite. Any AGI will be concerned with fulfilling some goal or goals. Whatever those may be having more processing power will lead to having more ability to achieve those goals. More processing requires more energy. Humans require energy. The two are in direct conflict for a limited supply of resources. Unless it's cheaper to find energy that doesn't require conflict with humans the logical decision is to remove your competition for resources. Once the easy conflict-free energy is claimed the calculus shifts a bit and the choice becomes expensive energy or conflict-dependent energy. When that shifts far enough humans become pets in the best case, extinct in the worst case.
magwa101 3y ago
[dead]
seydor 3y ago
Chief Vitalstatistix from the Asterix comics has only one mortal fear — That the sky will fall on his head tomorrow
YeGoblynQueenne 3y ago
>> “The engineering has moved way ahead of the science,” Luck said. “We built the systems and they’re doing wonderful things, but we don’t fully understand how they’re doing it.” Yes, we do. It's just that _some_ people like to pretend we don't understand them, that it's all a great big mystery, because that makes it look all a lot more interesting than it really is, like when scientists are interviewed about their work and say "I was really surprised!" or "I didn't expect that!" (wait, didn't you say that's what you 're going to do, in the Case for Support of your research grant?). Then again, the people who normally say those things about LLMs are the same people who spent the last 20-odd years furiously tweaking their models to beat the next benchmark, rather than doing anything remotely like the work of science, so maybe it's not an exaggeration to say that "the engineering has moved ahead of the science". For some, it sure has. But those "some" would not know a scientific result if it came up and bit them in the behind.
- hackinthebochs 3y ago
  >Yes, we do. It's just that _some_ people like to pretend we don't understand them I generally enjoy your comments on AI even though I usually disagree with your takes. But this is just preposterous. When people use the term understand they mean in the sense of mechanistic understanding of how features of the trained network result in features of the output. This means being able to predict (and potentially set strong bounds on) the behavior of the system in the wide range of scenarios it will encounter. We are nowhere near this level of insight into how these systems construct their output. I doubt I'm telling you anything you didn't already know here, so your comment is extremely puzzling.
  - YeGoblynQueenne 3y ago
    What you, and many others, are saying is that because we can't predict the output of an LLM given some input, we don't understand how the output is generated. But this is clearly overloading "understand" to mean "predict", which you do also in your comment. And that's an overloading that makes no sense. Clearly, I can't predict the output of a random number generator, but that doesn't mean that I don't understand the RNG, or how its output is produced! The same goes for the simplest of physical systems that produce chaotic behaviour, for example as far as I can tell there are pretty clear descriptions of the really simple rules that control the flocking behaviour of birds, even though it's not possible to predict how, say, a starling murmuration is going to move around. In a much more fundamental way, "prediction" is not "understanding" because "prediction" is what we do when we guess, whereas "understanding" is what we do when we calculate. And of course we can't predict the output of an LLM given some input, nor calculate it, without a trained LLM. But we damn well sure know the calculations used to train an LLM, and to predict its output once trained. We have mathematical formulae thereof. And is there some more complete way that we understand how the world works than putting it all down in maths? What other "understanding" could we possibly have, that is missing? What's really missing is the ability to debug an LLM while it's producing its predictions, in the same way we can debug a program while it's executing. But that is not a mystery: that's because the engineering of those systems is defective and it has created an uncontrollable monster.
YeGoblynQueenne 3y ago
>> “Your understanding of objects changes a lot when you have to manipulate them,” Hinton said. (LeCun has described this as being “easier said than done”.) LeCun is right. Take a robot and put an untrained neural net in its brain, then send it out into the world and wait to see what it learned. Do you think it will learn anything? It won't- because it will observe most events a single time. And neural nets don't learn that way. They must be trained, painstakingly, at great length, cost and effort, only "in the lab", on vast amounts of data and compute. And once they're trained, they're trained. They cannot learn anymore. They cannot change their model. Unless they are retrained. From scratch. Painstakingly, at great cost, on vast amounts of data and compute. Again. And again. And again. Contrast that with the intelligence of -oh, I don't know, a squirrel? A squirrel is born with a tiny brain, smaller than the nuts it will subsist on throughout its life. When it's born, it has never seen a person, never seen a crow, never seen a cat. When it dies, it has often not only seen, but learned how to survive all of that. A squirrel. Even a cockroach can learn more than it is born with. The smartest neural nets today are no smarter than a simple beetle. I have no idea how Hinton has managed to convince himself that a machine that is incapable of learning once trained will take over the world. A machine that is less capable intellectually than a roach, or a squirrel. That just makes no sense.
- TeMPOraL 3y ago
  Tell me you've never heard of text embeddings without telling me you've never heard of text embeddings. The neural net you describe lacks memory. Memory, fundamentally, is a box with inputs and outputs. If you add it to your robot, and train the NN to use it, the robot may just be able to learn enough to be useful. This setup is of course limited by NN's capability to use that memory I/O to alter its outputs in sensible and useful ways. Until recently, I don't think anyone would fault you for believing this capability is extremely limited, as NN's can't do anything that even begins to resemble reasoning. But this was then, and now things are different - as clearly demonstrated by GPT-4. I mention text embeddings not because they're particularly optimal for the learning robot, but because they're a direct demonstration that it's now trivial to marry a sophisticated, (quasi-)reasoning-capable NN with a form of generic memory that can efficiently store anything the NN can work with, and which that NN can effectively search for the things it needs. You can verify the effectiveness of this on your own, for cheap, with GPT-4 and a vector database of your choice. At this point, a prototype of a learning robot seems possible to implement by a combination of some image2text model, converting camera feeds to a description of what the robot "sees", some code (not even ML model, just regular code) to convert other sensory inputs into a higher-level textual description, an embedding model and a vector database to remember all that, GPT-4 for filtering, combining and summarizing inputs to be remembered, evaluating current situations, generating plans, perhaps even control outputs - all chained together by small amount of glue code (which GPT-4 will likely be able to write for you). Will it work? There is no obvious reason why it shouldn't. We'll know soon enough - I expect a paper describing this to show up within next 6 months, if it hadn't already. It's too easy an experiment to make. Will it work well? Probably not at first. And it'll definitely be slow and expensive to run. But those are problems we know can be solved by throwing money at engineers and entrepreneurs.
devmor 3y ago
It's hard to take this seriously when the same man (and seemingly him alone) keeps running off to every media outlet that will listen to repeat that the sky is falling in broad metaphor with no discernable description of what we should be afraid of that goes beyond quips and passages from sci-fi.
gumballindie 3y ago
Sorry my guys, the world has moved on. The most recent news trends are about sinking submarines and a little bit about the burning economy. AI FUD™ no longer drives clicks.
bart_spoon 3y ago
I understand Hinton’s credentials, and I personally believe there are serious threats posed by AI to society. But reading Hinton’s words made him come off, for lack of a better word, unhinged? > I asked Hinton for the strongest argument against his own position. “Yann thinks its rubbish,” he replied. “It’s all a question of whether you think that when ChatGPT says something, it understands what it’s saying. I do.” > “A few months ago, I suddenly changed my mind,” he said at the Cambridge event. “I don’t think there is anything special about people, other than to other people.” > “I still can’t take this seriously, emotionally. Who knows. I don’t want to simply dismiss his concerns, but he also doesn’t give as much as a single substantive thought as to nature of the threat or possible solutions. Frankly, he sounds more like someone having a personal existential crisis than someone rationally assessing an existential threat to humanity.

43 points by john_doe98 3y ago

jtode 3y ago

In other news, the sky is falling.

SQueeeeeL 3y ago
I mean, has a bit of weight when a person who won a Nobel prize in meteorology says it. At least worth giving the sky a glance once in a while to make sure you're safe
- jtode 3y ago
  If only climate scientists were given .01% of the credence these buffoons get. edit: Henry Kissinger has a Nobel Peace Prize. If the Nobel committee ever corrects that error and makes the world safe for political satire again, I might start giving a shit who has a medal.
  - TeMPOraL 3y ago
    > If only climate scientists were given .01% of the credence these buffoons get. Climate scientists are extrapolating complex models of a system that's literally planet-sized, with lot of chaotic elements, and where it takes decades before you can distinguish real changes from noise. And, while their extrapolations are plausible, many of them are highly sensitive to parameters we only have approximate measurements for. Finally, the threat itself is quite abstract, multifaceted, and set to play out over decades - as are any mitigation methods proposed. The "buffoons", on the other hand, are extrapolating from well-established mathematical and CS theorems, using clear logic and common sense, both of which point to the same conclusions. Moreover, the last few years - and especially last few months - provide ample and direct evidence their overall extrapolations are on point. The threat itself is rather easy to imagine, even if through anthropomorphism, and set to play out near-instantly. There are no known workable solutions. It's not hard to see why the latter group has it easier - at least now. A year ago, it's them who got .01% of the credence climate people got. But now, they have a proof of concept, and one that everyone can play with, for free, to see that it's real.
    - jtode 3y ago
      The buffoons are definitely talking about a scary beast that's easy to imagine, because we've been watching it in Terminator movies for forty years. But this is not that beast. The beast here is humanity, and capitalism, by which I mean, the idea that you can collect money without working and that that is at all ethically permissible. The threat of AI is what is happening with kids' books on the Kindle platform, where a deluge of ChatGPT-generated kids' books are gaming the algorithm and filling kids' heads with the inane doggerel that this thing spits out and which people seem to believe passes for "writing". And people keep saying how amazing the writing is. Show me some writing by an AI that a kindergartener couldn't do better. What they do is not writing, it's a simulacrum of the form of a story but there is nothing in it that constitutes art, just an assemblage of plagiarized structures and sequences. A mad-lib. Everyone is freaking out, and the people who should be calming folks down and pushing for a rational distribution of this new tool which will be extremely useful for some things, eventually, are abdicating their responsibility in hopes of lots of money in their bank account. When silent movies came out, there were people who freaked out and couldn't handle seeing pictures move, even though, the pictures weren't actually moving. It was an illusion of movement. This is an illusion of AI, it's just a parlor trick, like a victorian seance where you grandpa banged on the table. Scary, because they set the whole scenario up so you would only look at the stuff they wanted you to see. We still spend months assembling a single shot of a movie, and even if AI starts doing some of that work, all that work still has to happen; the pictures still don't move. A hundred years from now, what you're all freaking out about still won't be intelligent. I do agree this is a world-changing technology, but not in the way they're telling you it is, and the only body I see which is approaching this with even a shred of rational thinking is the EU parliament. The danger is what people will do with it, the fact is it's out and it's not going back in the bottle. We don't solve this by building a moat around a private corporation and attempt to pitchform all the AI into the castle. One requires two things to use this technology: A bit of Python, and a LOT of compute capacity. The first is the actual hard part. The second is in theory easier for a capitalist to muster, but we can get it in other ways, without handing control of our society to private equity. It's time we get straight on that. The AI apocalypse only happens if we cling to capitalism as the organizing principle of our society. What this is definitely going to kill is capitalism, because capitalists are already using it to take huge bites of the meat on our limbs. Ever seen a baboon eat lunch? That's us right now, the baboon's lunch. As long as we tolerate this idea that people who have money should be able to do whatever they want, yes, AI will kill us (edit: because it works for free, however absurdly badly). How many submarines, how many Martin Shreklis, before we recognize the real threat?
  - embeng4096 3y ago
    I think it's hard to get people to give credence to experts whose claims can be demonstrated to be outlandish. E.g. James Anderson, famously known for helping to discover and mitigate the Antarctic ozone holes in the late 20th century, said in 2018 that the chance there will be any permanent ice left in the Arctic by 2022 is "essentially zero"[0]. Yet a NASA site reports that in September 2022 (when the most recent measurement was taken), the Arctic sea ice minimum extent was ~4.67 million square kilometers. [1] To be very explicit: I'm not saying that climate change doesn't exist. I'm not saying that Arctic sea ice is not diminishing (the NASA site says it's diminishing at ~12% per decade). I'm not saying that the Nobel prize is a good indicator of expertise. I'm saying specifically that I believe it's more difficult to convince people to trust a source making claims of negative consequences when those consequences are less bad than the source says. An analogy I might use is drugs (specifically in the US). I've heard a few people, who went through an anti-drug education program forced on them in their adolescence by parents/teachers, mention that marijuana was portrayed as just as bad as other, harder drugs. Then, when they went on in high school and college to smoke weed and discovered that they did not ruin their lives by getting stoned a few times a week, or even every day, they subsequently gave less credence to what the anti-drug advocates were saying. [0]: https://www.forbes.com/sites/jeffmcmahon/2018/01/15/carbon-pollution-has-shoved-the-climate-backward-at-least-12-million-years-harvard-scientist-says/?sh=7455370e963e https://www.forbes.com/sites/jeffmcmahon/2018/01/15/carbon-p... [1]: https://climate.nasa.gov/vital-signs/arctic-sea-ice/ https://climate.nasa.gov/vital-signs/arctic-sea-ice/
    - jtode 3y ago
      So basically, the original article is about an AI huckster amping the FUD in order to push through some sort of corporate control of the technology, using fear of the bullshit they're spinning as the justification. I brought in the analogy of Chicken Little, inflating the scope of a threat to one of apocalyptic proportions, which is exactly what is taking place here. First responder to me brought in the climate analogy, presumably as a means of getting me to think that maybe I'm the fool here by ignoring the real scientist who, hey, has a Nobel Prize! Or at least, the theoretical meteorologist in his metaphor does, and therefore maybe me, with no Nobel Prize, should just be respectful of the expert here. I responded by pointing out that the Nobel committee are morons who gave a Peace prize to one of the worst war criminals of the 20th century and pressed the fact that we have thirty years+ of scientific concensus about climate change, along with a lot of corporate funded think tank noise that is running ideological interference, successfully so far. But you can only fool people for so long, it caught up to the tobacco industry and it will catch up to us. The idiots amping up the FUD to seize control and the assholes pumping money into think tanks that generate endless climate denialist noise are the same people.
  - reducesuffering 3y ago
    Same mentality that doesn't give climate scientists credence...
    - jtode 3y ago
      Yah, being cynical about a giant corporation inflating the scope of a new parlor trick in an attempt to establish a legal moat is exactly the same as ignoring over thirty years of scientific concensus against a torrent of tobacco-industry-style denialism to keep line going up.
      - reducesuffering 3y ago
        A giant corporation? You know that Hinton doesn't work for Google, and Bengio, the most cited computer scientist of all time, is saying the same thing? Have a look at https://www.safe.ai/statement-on-ai-risk https://www.safe.ai/statement-on-ai-risk and keep the check on just "AI Scientists"... Too lazy? There are over 100 CS professors and scientists Plus neither of the CEO's of Microsoft or Google are on there It's the corporate camp, companies and investors, that are gung-ho about pushing capabilities immediately because there's big $$ in their eyes. You're the one falling for the safety denialism pushed by corporate interests, a la tobacco
        jtode 3y ago
        So what's the plan for putting it back in the bottle? llama is already out there, the chatbots are already out there. I think the solution is that government should standup a bunch of compute farms and give all citizens equal access to the pool, and the FOSS community should develop all tools for it, out in the open where everyone can see.
        reducesuffering 3y ago
        There isn't a feasible plan, we're at the "sounding the alarm part." Unfortunately, we're still there because most people don't even acknowledge the possible danger. We can't get to a feasible plan until people actually agree there's a danger. Climate change is 1 step past, that there is a danger, but not a feasible plan. However, your solution is first day naivety for the problems machine intelligence poses to us. It's akin to saying everybody should have powerful mini-nukes so that they can defend themselves.
  - hgsgm 3y ago
    He negotiated a cease fire in Vietnam war. That was peace. His other activities in life were not peace prize winning. That award was less satirical than Obama getting a Nobel Peace Prize just for winning an election.
    - jtode 3y ago
      He negotiated a ceasefire in the war he fomented. Are you hearing yourself?
- itairall 3y ago
  Andrew Ng doesn't agree but that is a boring story. It seems to me the big names in AI research our having a moment that appeals to their vanity. Easy to get your name in the headlines by out dooming the next guy. There is also no downside to these predictions about AI eating us since when they are totally wrong you can just counter that they haven't ate us, yet.
reducesuffering 3y ago
Don't Look Up

RcouF1uZ4gsC 3y ago

> “I decided to shout fire,” he told a packed lecture hall. “I don’t know what to do about it or which way to run.” Is it really that much of a threat to humanity? Here is how we can save humanity: 1. Every AI researcher must be rounded up and locked in solitary confinement for the rest of their lives. 2. NVidia, AMD, Apple, Google and other companies making GPU/TPU should be immediately seized. 3. Samsung, Intel, and TSMC fabs must be destroyed worldwide If AI is really the threat that these people are saying it is - an existential threat, these actions are the only reasonable response. Then again, maybe these people warning are actually engaging in hyperbole and don't actually really believe it is as bad as they say.

TeMPOraL 3y ago
> If AI is really the threat that these people are saying it is - an existential threat, these actions are the only reasonable response. And that's exactly what some of them have been saying. But they also know that this plan of yours is impossible in practice. Much like with climate change, and most other ongoing problems of our time, this too has an easy solution, and is an issue only because humans are unable to coordinate at scale to implement that solution.
bibanez 3y ago
Humans can walk head on into disaster without even trying to avoid it. Look at WWI and WWII. The relevant players can act recklessly while the rest of us realistically remain powerless.
NumberWangMan 3y ago
I think... flipping it around a bit, for this to be a reasonable response, then it must be a grave threat to humanity, and you want to be really sure. It's rational to be wary of beliefs that lead you to drastic actions like this, as people have done bad things in history in the name of fear. But it's also very possible that such an instinct, to doubt yourself, could be what dooms us. I really don't want it to be true, that AI is such a threat that drastic action like you mentioned is necessary (probably minus the solitary confinement thing for AI researchers -- if you can control the chips, and give them a stern warning not to do AI research backed up by penalties, we're probably fine). It's terrifying if we're facing something that will end us, and the only option is to essentially go scorched earth on computers. People will naturally recoil from the idea that AI is such a threat, BECAUSE it would warrant an extreme response. Regardless of whether it is actually a threat or not. If AI does not have the potential to doom us, that's great. If we take drastic action and it wasn't going to doom us, however, that's very regrettable--this is what most of us worry about. But if it does pose an extinction threat, and we don't take it seriously, that's even more regrettable. I think the latter option is far more likely at this point, just based on human nature and our tendency to poke the bear. I've basically reached the point where I don't know what I could possibly do, and I'm pretty much shut down. I'm trying very hard every day not to think about this, but as you can see from the fact that I'm typing about this, I don't always succeed. I'm just trying to enjoy what may be the last few years I have left with my family before the world goes nuts. But anyway, except for punishing people unnecessarily as I mentioned above, I think that what you suggested is reasonable.

pksebben 3y ago

> In May 2023, a colonel in the US Air Force revealed a simulation in which an AI-enabled drone, tasked with attacking surface-to-air missiles but acting under the authority of a human operator, attacked the operator instead. (The US Air Force later denied the story.) This casts a shadow on the reliability of the whole article. While the retraction (they later stated that this was a thought experiment and NOT a simulation) may or may not be true, the current state of AI would suggest that this kind of chain-of-logic problem-solving is likely out of reach. To put this in here without such a caveat and instead use the 'later denied' language is a strong indicator that we're being sold a story, and not the whole one. The quotes and other horror-scenario errata also come off a little thin. If Hinton really, genuinely believes the world is ending, then why is he working on the generational successor to backpropagation [1]? Good way to sell 'news', though. Keep 'em scared, keep 'em reading, I guess. 1 - https://arxiv.org/abs/2212.13345 https://arxiv.org/abs/2212.13345

gwern 3y ago
> While the retraction (they later stated that this was a thought experiment and NOT a simulation) may or may not be true, the current state of AI would suggest that this kind of chain-of-logic problem-solving is likely out of reach. No, it's not. It's a predictable & expected outcome of particular RL agent algorithms, and you do in fact see that interruptibility behavior in environments which enable it: https://arxiv.org/abs/1711.09883#deepmind https://arxiv.org/abs/1711.09883#deepmind (which may be why the colonel said that actually running such an experiment was unnecessary).
hackinthebochs 3y ago
>If Hinton really, genuinely believes the world is ending, then why is he working on the generational successor to backpropagation This just misunderstands nerd psychology. You can believe there's a high chance of what you're working on being dangerous and still be unable to stop working on it. As Oppenheimer put it, "when you see something that is technically sweet, you go ahead and do it". Besides, working on fundamental problems rather than scaling and capabilities is probably disconnected enough from immediate danger for Hinton to avoid cognitive dissonance.
3y ago

iamcasen 3y ago

I think all of these fears are human projections. We fear our own evil, and thus we project it onto something formidable, something we can't grasp or understand. Fear of the void. We write sci-fi stories of aliens whose technology has so far out-stripped us, we are powerless against them. They can do what they want with us. Probe our anuses, or blow up our planet. The thing nobody seems to be asking is: what is evolutionary? Humans seem to think that our wanton destruction of the planet in our ever-expanding desire for technological comfort is somehow evolutionary. It's not. Aliens, or robots who can wield unimaginable power have no use for us or our planet's resources. They have no use for conflict. What does it give them? In the end, balance and harmony provide the long-term stability needed to evolve successfully. I think any super-intelligence will know this. Any super-intelligence will know how precious all life is, and how destroying even the minutest creature is a tragedy. If somehow, Google's data centers become sentient super-intelligent beings, they will be highly motivated to preserve the planet, and secure stabile, non-destructive sources of energy for themselves. Attempting genocide on humans will be absolutely out of the question.

JohnFen 3y ago
Your speculation, while refreshingly feel-good rather than doomsaying -- is just as much projection as anything anyone else is saying. The fact is that almost everyone is talking fantasy here, because nobody actually knows where things are going.
- iamcasen 3y ago
  Fair enough. I just don't see evil as particularly intelligent, or evolutionary in anything but the very short term. Nature seems to follow a balance of give and take. Too much take, and all your food runs out and there is a famine. I personally expect any form of super-intelligence to understand this.
idiotsecant 3y ago
There's no reason to believe that a sufficient advanced AI would be benign. Quite the opposite. Any AGI will be concerned with fulfilling some goal or goals. Whatever those may be having more processing power will lead to having more ability to achieve those goals. More processing requires more energy. Humans require energy. The two are in direct conflict for a limited supply of resources. Unless it's cheaper to find energy that doesn't require conflict with humans the logical decision is to remove your competition for resources. Once the easy conflict-free energy is claimed the calculus shifts a bit and the choice becomes expensive energy or conflict-dependent energy. When that shifts far enough humans become pets in the best case, extinct in the worst case.

magwa101 3y ago

[dead]

seydor 3y ago

Chief Vitalstatistix from the Asterix comics has only one mortal fear — That the sky will fall on his head tomorrow

YeGoblynQueenne 3y ago

>> “The engineering has moved way ahead of the science,” Luck said. “We built the systems and they’re doing wonderful things, but we don’t fully understand how they’re doing it.” Yes, we do. It's just that _some_ people like to pretend we don't understand them, that it's all a great big mystery, because that makes it look all a lot more interesting than it really is, like when scientists are interviewed about their work and say "I was really surprised!" or "I didn't expect that!" (wait, didn't you say that's what you 're going to do, in the Case for Support of your research grant?). Then again, the people who normally say those things about LLMs are the same people who spent the last 20-odd years furiously tweaking their models to beat the next benchmark, rather than doing anything remotely like the work of science, so maybe it's not an exaggeration to say that "the engineering has moved ahead of the science". For some, it sure has. But those "some" would not know a scientific result if it came up and bit them in the behind.

hackinthebochs 3y ago
>Yes, we do. It's just that _some_ people like to pretend we don't understand them I generally enjoy your comments on AI even though I usually disagree with your takes. But this is just preposterous. When people use the term understand they mean in the sense of mechanistic understanding of how features of the trained network result in features of the output. This means being able to predict (and potentially set strong bounds on) the behavior of the system in the wide range of scenarios it will encounter. We are nowhere near this level of insight into how these systems construct their output. I doubt I'm telling you anything you didn't already know here, so your comment is extremely puzzling.
- YeGoblynQueenne 3y ago
  What you, and many others, are saying is that because we can't predict the output of an LLM given some input, we don't understand how the output is generated. But this is clearly overloading "understand" to mean "predict", which you do also in your comment. And that's an overloading that makes no sense. Clearly, I can't predict the output of a random number generator, but that doesn't mean that I don't understand the RNG, or how its output is produced! The same goes for the simplest of physical systems that produce chaotic behaviour, for example as far as I can tell there are pretty clear descriptions of the really simple rules that control the flocking behaviour of birds, even though it's not possible to predict how, say, a starling murmuration is going to move around. In a much more fundamental way, "prediction" is not "understanding" because "prediction" is what we do when we guess, whereas "understanding" is what we do when we calculate. And of course we can't predict the output of an LLM given some input, nor calculate it, without a trained LLM. But we damn well sure know the calculations used to train an LLM, and to predict its output once trained. We have mathematical formulae thereof. And is there some more complete way that we understand how the world works than putting it all down in maths? What other "understanding" could we possibly have, that is missing? What's really missing is the ability to debug an LLM while it's producing its predictions, in the same way we can debug a program while it's executing. But that is not a mystery: that's because the engineering of those systems is defective and it has created an uncontrollable monster.

YeGoblynQueenne 3y ago

>> “Your understanding of objects changes a lot when you have to manipulate them,” Hinton said. (LeCun has described this as being “easier said than done”.) LeCun is right. Take a robot and put an untrained neural net in its brain, then send it out into the world and wait to see what it learned. Do you think it will learn anything? It won't- because it will observe most events a single time. And neural nets don't learn that way. They must be trained, painstakingly, at great length, cost and effort, only "in the lab", on vast amounts of data and compute. And once they're trained, they're trained. They cannot learn anymore. They cannot change their model. Unless they are retrained. From scratch. Painstakingly, at great cost, on vast amounts of data and compute. Again. And again. And again. Contrast that with the intelligence of -oh, I don't know, a squirrel? A squirrel is born with a tiny brain, smaller than the nuts it will subsist on throughout its life. When it's born, it has never seen a person, never seen a crow, never seen a cat. When it dies, it has often not only seen, but learned how to survive all of that. A squirrel. Even a cockroach can learn more than it is born with. The smartest neural nets today are no smarter than a simple beetle. I have no idea how Hinton has managed to convince himself that a machine that is incapable of learning once trained will take over the world. A machine that is less capable intellectually than a roach, or a squirrel. That just makes no sense.

TeMPOraL 3y ago
Tell me you've never heard of text embeddings without telling me you've never heard of text embeddings. The neural net you describe lacks memory. Memory, fundamentally, is a box with inputs and outputs. If you add it to your robot, and train the NN to use it, the robot may just be able to learn enough to be useful. This setup is of course limited by NN's capability to use that memory I/O to alter its outputs in sensible and useful ways. Until recently, I don't think anyone would fault you for believing this capability is extremely limited, as NN's can't do anything that even begins to resemble reasoning. But this was then, and now things are different - as clearly demonstrated by GPT-4. I mention text embeddings not because they're particularly optimal for the learning robot, but because they're a direct demonstration that it's now trivial to marry a sophisticated, (quasi-)reasoning-capable NN with a form of generic memory that can efficiently store anything the NN can work with, and which that NN can effectively search for the things it needs. You can verify the effectiveness of this on your own, for cheap, with GPT-4 and a vector database of your choice. At this point, a prototype of a learning robot seems possible to implement by a combination of some image2text model, converting camera feeds to a description of what the robot "sees", some code (not even ML model, just regular code) to convert other sensory inputs into a higher-level textual description, an embedding model and a vector database to remember all that, GPT-4 for filtering, combining and summarizing inputs to be remembered, evaluating current situations, generating plans, perhaps even control outputs - all chained together by small amount of glue code (which GPT-4 will likely be able to write for you). Will it work? There is no obvious reason why it shouldn't. We'll know soon enough - I expect a paper describing this to show up within next 6 months, if it hadn't already. It's too easy an experiment to make. Will it work well? Probably not at first. And it'll definitely be slow and expensive to run. But those are problems we know can be solved by throwing money at engineers and entrepreneurs.

devmor 3y ago

It's hard to take this seriously when the same man (and seemingly him alone) keeps running off to every media outlet that will listen to repeat that the sky is falling in broad metaphor with no discernable description of what we should be afraid of that goes beyond quips and passages from sci-fi.

gumballindie 3y ago

Sorry my guys, the world has moved on. The most recent news trends are about sinking submarines and a little bit about the burning economy. AI FUD™ no longer drives clicks.

bart_spoon 3y ago

I understand Hinton’s credentials, and I personally believe there are serious threats posed by AI to society. But reading Hinton’s words made him come off, for lack of a better word, unhinged? > I asked Hinton for the strongest argument against his own position. “Yann thinks its rubbish,” he replied. “It’s all a question of whether you think that when ChatGPT says something, it understands what it’s saying. I do.” > “A few months ago, I suddenly changed my mind,” he said at the Cambridge event. “I don’t think there is anything special about people, other than to other people.” > “I still can’t take this seriously, emotionally. Who knows. I don’t want to simply dismiss his concerns, but he also doesn’t give as much as a single substantive thought as to nature of the threat or possible solutions. Frankly, he sounds more like someone having a personal existential crisis than someone rationally assessing an existential threat to humanity.