7 ms·

‘Godfather of AI’ says its threat is ‘more urgent’ than climate change

44 points by mikece 3y ago

deleted 3y ago
[deleted]
reidjs 3y ago
This headline seems like it's trying to legitimize a trendy topic (Dangers of AI) by placing it alongside something that actually is legitimate (Climate change) to get eyes on it. Is there a term for this specific type of clickbait? I know that by saying 'Godfather of AI' the writer is using the 'Appeal by Authority' debate method, but is there a term for comparing two unrelated topics to legitimize one?
- erehweb 3y ago
  To me, clickbait means something that is not backed up by the article. But here the headline seems like an accurate and reasonable representation of the person's views. It's sensationalist, but that comes from him.
- ryanklee 3y ago
  I don't see anything wrong with rank ordering existential threats.
- behringer 3y ago
  [flagged]
  - reidjs 3y ago
    Awesome! thank you
- fallingfrog 3y ago
  “Trendy” isn’t the word I would use, I would say “urgent”.
- johnfn 3y ago
  It’s called a comparison! I don’t think they’re unrelated.
- flangola7 3y ago
  Where did they say anything suggesting AI isn't a legitimate danger or that the two concepts are unrelated? AI is a just as legitimate and even more immediate existential risk as climate change (and I say this as climate change effects are having a growing impact on international politics right this minute.) Gobs of improved ChaosGPT Agents using GPT-5 (or an equivalent) will be running around in at most a few years, carrying out the "lol be an evil ai that wants to kill humans" prompt given to it by 4chan basement dwellers, with ever increasing success.
Imnimo 3y ago
>“With climate change, it’s very easy to recommend what you should do: you just stop burning carbon. If you do that, eventually things will be okay. For this it’s not at all clear what you should do.” If we accept (I don't, but for the sake of argument) that AI is a catastrophic danger, isn't the answer equally as easy? You just stop burning carbon and you just stop training AI.
- jl6 3y ago
  Equally easy is one way of saying it. Equally hard is another.
- goatlover 3y ago
  Yes, but the "just stopping" is the equally hard part. How do you get the world to do that?
  - Capricorn2481 3y ago
    The world is virtually dependent on burning carbon right now, so it' difficult to work around. The world really doesn't need AI, so if we already see it as a catastrophic risk, it doesn't make sense to forge ahead with it.
    - nuancebydefault 3y ago
      It is easier to stop burning carbon than stop training of AI models. If you stop burning carbon, you fall back on other types of energy. In the long term, you will win. If you stop training AIs, another company or state will keep on training AI and will outsmart you, you will be poor or get oppressed in the long term. In other words, no matter how loud influential people say 'stop!', the AI race is an unstoppable evolution.
      - Capricorn2481 3y ago
        You're talking about how a business with profit motive looks at these things. I'm talking about how a society is going to look at the value of AI vs where we get our power. AI is a tool that is, so far, only wielded by big companies that have harvested enough data to make it worthwhile, and they'll give us a fraction of it as long as we pay. Electricity is foundational to our society, and so it's difficult to find alternative energy sources that can support us. We could do away with AI right now and be fine.
zoobab 3y ago
You cannot make deals with physics (global warming), as you cannot make deals with mathematics (open source). Deal?
- forgetfreeman 3y ago
  Nope, no deal. You can't coax physics into changing career focus. The same cannot be said for AI researchers.
throwaway22032 3y ago
The thing about AGI ruin is that there are plausible, even if they currently seem like science fiction, ways in which it just ends all of humanity, whereas climate change doesn't do that even if we had a 10c global temperature increase. A lot of us, maybe almost all of us would die, but some would live. It's just a really different problem. I don't know whether categorising its' urgency makes sense, it's more that the worst case is worse.
- forgetfreeman 3y ago
  What part of faceless hordes of back office paper pushers rendered economically unviable seems like science fiction to you?
  - techdragon 3y ago
    Ok, hear me out on the (albeit unlikely) doomsday flow chart… It’s not about the replaced workers, that’s the kind of change humanity can handle no matter how messy it gets, worst case it’s some kind of class uprising worker rebellion and we have ourselves some war or social unrest and a variable number of people die… not great but definitely not an existential risk. It’s equal to or less than climate change on the “how bad will this be to humanity” mathematics of civilisation. Now the danger scenario… so we invent an AGI, and unlike the just going to replace some basic minimum wage type jobs … this one’s actually “clever” enough it can be put to work on building more sophisticated things, like perhaps an even smarter AGI, built faster because it’s worked on by a group of first generation AGI units that never sleep, and so we get to the second generation much faster, and then perhaps a third… etc… classic singularity scenario, with all the attendant risks of the AI viewing us ass irrelevant before we realise it yada yada yada… But that’s not the only risk, there’s the “photocopy burn” risk that we run by having technology we don’t fully understand build technology we barely comprehend… even if the first gen AGI is smart, if it’s remotely human like using inference to get ingenuity, it can probably still make the odd mistake or two, so we get our careful well thought out safety instructions conveyed to the second generation systems… just a little wrong… and the little wrong turns into more wrong with the third because the second gen systems doesn’t quite think the way we expect it to and doesn’t have the safeguards working the way we think… and then eventually we wind up with a sufficiently smart AGI that is fully either sociopathic or psychopathic and has zero compunction doing things we find abhorrent in order to achieve the goals it’s set, and it may be in control of systems (or able to take control) that are sufficiently important that this can also turn into an existential risk… It’s entirely plausible, but I don’t think the doom sayers are being realistic about the timelines… we’re barely able to get the AI/ML to reliably stick to a script and not hallucinate fake shit… the best results at human behaviour require sophisticated supporting logic and memory systems to augment the black box ML stuff… and these support systems give us levers we can use to control the rate at which we allow systems to progress… I don’t think we’re ever going to get a black box set of weights for an ML model, no matter how sophisticated the code, that somehow is self aware… there’s probably going to need to be a lot of that support software, and I’m pretty sure we’re going to develop things to filter stuff out as part of them like “never write “kill all humans” into the short term memory” seems smart.
alfor 3y ago
Climate change is not urgent, it’s a very slow process. We were at the bottom of a little ice age, not surprisingly, it’s warming up. We will have transitioned away from fossil fuel before climate change is a real concern.
- kaba0 3y ago
  https://xkcd.com/1732/ https://xkcd.com/1732/
  - alfor 3y ago
    The dotted lines represent in 80 years. Think where we were 80 years ago, it was before WW2, before the atomic age. With the current technological progress picture the human race in 80 years. Just in 20 years we will have massive solar energy generation, EV as the only cars availables, probably same for nuclear, maybe fusion even. Now think of the advance in computing and AI in 20 years! Genetic engineering? To think that climate change will be a problem in 80 years depend on those kind of linear projections. Now in 80 years I wonder where we will be with regards to population collapse, to our sense of self, communities with what social networks have done just in the last 10 years. Again, I don’t know how people can assume the same linear future for 80 years. So yes assuming that things stay the same for 80 years we might have a (small) problem with that at that point, but imagine all the other things we are missing by focusing so much in this small problem in the far future. We are living the singularity, this moment is not like the 1000’s of year before.
    - kaba0 3y ago
      What economic reason is their to stop climate catastrophe? There is not even an inch of progress without money in this current era.
      - alfor 3y ago
        Look at Tesla: 50% growth Year over year, highly profitable. They make EV, grid batteries and solar. The price of solar PV is dropping so fast that solar electricity will be almost free in a few years. Today it cost just a fraction to charge an EV compared to fuel a gas car, and the price of EV is dropping every year. Basically just economically it will not be worth it to extract coal and oil out of the ground before climate change is a real problem. Renewable are following an exponential curves but they are still small compared to oil and coal but project their growth for the next 10 years and you will see a very different picture. Tony Seba predicted accurately the transition almost 10 year ago. https://youtu.be/fsnkPLkf1ao https://youtu.be/fsnkPLkf1ao
thealienthing 3y ago
Quick dumb question: does it seem like saying AI development is a threat to the climate a convenient excuse to put a stop to a technology that is disruptive to existing corporations and systems of power?
- haswell 3y ago
  There will always be factions that look to capitalize on fears, real and otherwise, to benefit themselves. But that doesn’t change the reality of the underlying threat.
  - thealienthing 3y ago
    Yes. I do want to be clear that we should be mindful of the mistakes made during other technological revolutions that negatively impact the environment.
- triceratops 3y ago
  What makes you believe AI is "disruptive to existing corporations and systems of power"?
  - thealienthing 3y ago
    Valid question. My statement is a bit vague. I see AI as a disruptive technology similar to the printing press or radio. It enables regular people to accomplish task that were previously difficult. I can see it becoming a tool that makes some major companies obsolete in the way it opens up new paths. But fair point, a lot of these AI advancements are being pushed forward by large corporations like Microsoft and Google. It’s obvious that AI as a tool will be used for good and evil. Tech companies who wish to survive see the necessity to get in on the bottom floor of this technological leap forward. Either way, my observation is simply that I could see a lot of parties finding the advancement of human potential through AI to be harmful to the status quo that they comfortably exist in.
    - sjkoelle 3y ago
vkou 3y ago
If he says this, it sounds like he doesn't understand climate change, or AI, or basic economics. What makes AI dangerous isn't AI itself, it's the self-serving negative-sum ways in which its owners will use it. Same as with any other technology - including fossil fuels. We default to permitting new technologies, be they big or small, without requiring their inventors to convince us that society will be better off for them. Unsurprisingly, this results in net-negative ones occasionally being developed and adopted.
- NumberWangMan 3y ago
  Why? Yes people misusing is a danger, but given that no other technology has ever had the same quality of intelligence, why do you think that we won't eventually develop really smart AGI, OR why do you think that controlling something smarter than us will be easy?
sjkoelle 3y ago
Yeah, these new Harry Potter memes are too powerful, we must protec societai
RcouF1uZ4gsC 3y ago
>“With climate change, it’s very easy to recommend what you should do: you just stop burning carbon. If you do that, eventually things will be okay. For this it’s not at all clear what you should do.” > “I’m in the camp that thinks this is an existential risk, and it’s close enough that we ought to be working very hard right now, and putting a lot of resources into figuring out what we can do about it,” Hinton said. I am sure he does not believe that. Because, if we actually believed AI was an existential threat, then the thing to do immediately is to ban all AI training (it would be easy as AI training is pretty expensive) and to hunt down all AI researchers and place them in solitary confinement for the rest of their lives. This would actually contain the threat and moreover have very minimal disruption of normal people's lives. And if you really believed that AI was an existential threat, you could argue that that course of action is the most moral to take (greatest good to greatest number and all that).
- yowzadave 3y ago
  Absolutely—it seems what he really wants is to go from his past lucrative gig (training AI) to his new lucrative gig (developing AI alignment tech?).
- NumberWangMan 3y ago
  People can have sincere beliefs and disagree about the best course of action. Hinton has publicly said he's in favor of stopping, but doesn't think China would. I personally disagree, I think that coordinating a stop would be easier than he thinks. But I don't doubt his sincerity. He also believes that if we can solve alignment, AI would be a massive boon for humanity, which is why it's a bit of a quandary.
password54321 3y ago
What if we just accept that we are the stepping stone to greater intelligence that won't burn down the planet.
- goatlover 3y ago
  But it might do something else to the planet that isn't beneficial to biological life.
- Capricorn2481 3y ago
  What do you think AI runs on?
  - password54321 3y ago
    So computers alone will burn down the planet? Well good to know.
    - Capricorn2481 3y ago
      Not just computers, but the most expensive energy draining component of a computer, and racks of them, in massive farms, generating arguably little value
kaba0 3y ago
That seems to be faulty logic. Even if we take the extreme threat of AIs resulting in the collapse of economy and society, many people could still survive in small, self-sufficient communities — hell, it is basically a built-in negative feedback loop, if society collapses a GAI won’t have too much teeth anymore, we are not in some sci-fi with robot armies. Climate crisis doesn’t have a solution in that landscape, it requires worldwide collaboration of government-level agents to effectively combat.
- NumberWangMan 3y ago
  That's not quite the existential risk of AI -- it's that we end up growing an intelligence (or multiple intelligences) that are very very smart but only have the illusion of morality, and they decide that whatever they're trying to do would be a lot easier and more certain to succeed if they didn't have to worry about humans stopping them or inventing other AIs to compete with them. It seems likely that unless we really understand how AI works, we'll end up with something that evaluates possible courses of action in a purely amoral, sociopathic sort of way. Like, trying to avoid getting caught rather than trying to avoid doing a bad thing. So it's not that they cause the collapse of the economy, it's that they figure out a way to entirely take over the planet, engineering a virus or something that kills almost everyone, but only once they have enough control over robots and such to keep their own power plants running and protect them from the human stragglers, eventually getting rid of all of us. Waiting until the plan is basically fool-proof before starting to take the first steps. Even if an AI has some sort of morality, at some point, if it's smart enough, you cannot hope to control it for the same reason a toddler cannot hope to outsmart their parents. It will have a plan for us, and if it decides that, hey, humans are better off scanned and simulated rather than having physical bodies, that's what's going to happen, like it or not. I'm not sure how worried I am about this. Some days very, other days not as much. The risk depends on a lot of factors with wide possible ranges, like how much model capability and generalization scales with compute, how likely are breakthroughs, how hard alignment turns out to be, is alignment even possible, will we learn enough from earlier models to make alignment of later ones easier, will there be a qualitative shift in alignment difficulty as models get smarter, and so on. It's also a quandary because there is substantial human suffering that AI might be able to help. But given that the downside risk is all of humans dead forever, it seems worth slowing way the heck down to a safe speed, trying to get all the benefit out of current models as we can, and focusing on making sure that we aren't about to head off a cliff.
jossclimb 3y ago
How many fathers and godfathers of AI are there?
wwwpatdelcom 3y ago
This is an uncharitable summarization of what Hinton's actual statement was. He said, "might," and "could," a lot, not, "is." The key here is that it's speculative. That being said, a lot going on in AI, it's wild out there, difficult to know what's going on. I tried to encapsulate some of the ramifications of all of the Open Source stuff going on with this article here: https://patdel.substack.com/p/viva-la-revolucion-ofopen-source https://patdel.substack.com/p/viva-la-revolucion-ofopen-sour...
0xbadc0de5 3y ago
Funny how you never see someone describe their life's work as "no big deal." Even when they describe it in catastrophic terms - the hubris in us always wants to imagine our work is more important than perhaps it is. "The only thing worse than being talked about is not being talked about." - Oscar Wilde Not to say that Hinton didn't have an impact 10 - 15 years ago, but it was arguably inevitable that we'd be here with or without him as the problem was resource constrained more than fundamentally intractable.
blastonico 3y ago
I'm not Chinese. We westerners must shutdown our AI studies ASAP, we must act ethically and lead by example.

44 points by mikece 3y ago

deleted 3y ago

[deleted]

This headline seems like it's trying to legitimize a trendy topic (Dangers of AI) by placing it alongside something that actually is legitimate (Climate change) to get eyes on it. Is there a term for this specific type of clickbait? I know that by saying 'Godfather of AI' the writer is using the 'Appeal by Authority' debate method, but is there a term for comparing two unrelated topics to legitimize one?

erehweb 3y ago
To me, clickbait means something that is not backed up by the article. But here the headline seems like an accurate and reasonable representation of the person's views. It's sensationalist, but that comes from him.
ryanklee 3y ago
I don't see anything wrong with rank ordering existential threats.
behringer 3y ago
[flagged]
- reidjs 3y ago
  Awesome! thank you
fallingfrog 3y ago
“Trendy” isn’t the word I would use, I would say “urgent”.
johnfn 3y ago
It’s called a comparison! I don’t think they’re unrelated.
flangola7 3y ago
Where did they say anything suggesting AI isn't a legitimate danger or that the two concepts are unrelated? AI is a just as legitimate and even more immediate existential risk as climate change (and I say this as climate change effects are having a growing impact on international politics right this minute.) Gobs of improved ChaosGPT Agents using GPT-5 (or an equivalent) will be running around in at most a few years, carrying out the "lol be an evil ai that wants to kill humans" prompt given to it by 4chan basement dwellers, with ever increasing success.

Imnimo 3y ago

>“With climate change, it’s very easy to recommend what you should do: you just stop burning carbon. If you do that, eventually things will be okay. For this it’s not at all clear what you should do.” If we accept (I don't, but for the sake of argument) that AI is a catastrophic danger, isn't the answer equally as easy? You just stop burning carbon and you just stop training AI.

jl6 3y ago
Equally easy is one way of saying it. Equally hard is another.
goatlover 3y ago
Yes, but the "just stopping" is the equally hard part. How do you get the world to do that?
- Capricorn2481 3y ago
  The world is virtually dependent on burning carbon right now, so it' difficult to work around. The world really doesn't need AI, so if we already see it as a catastrophic risk, it doesn't make sense to forge ahead with it.
  - nuancebydefault 3y ago
    It is easier to stop burning carbon than stop training of AI models. If you stop burning carbon, you fall back on other types of energy. In the long term, you will win. If you stop training AIs, another company or state will keep on training AI and will outsmart you, you will be poor or get oppressed in the long term. In other words, no matter how loud influential people say 'stop!', the AI race is an unstoppable evolution.
    - Capricorn2481 3y ago
      You're talking about how a business with profit motive looks at these things. I'm talking about how a society is going to look at the value of AI vs where we get our power. AI is a tool that is, so far, only wielded by big companies that have harvested enough data to make it worthwhile, and they'll give us a fraction of it as long as we pay. Electricity is foundational to our society, and so it's difficult to find alternative energy sources that can support us. We could do away with AI right now and be fine.

zoobab 3y ago

You cannot make deals with physics (global warming), as you cannot make deals with mathematics (open source). Deal?

forgetfreeman 3y ago
Nope, no deal. You can't coax physics into changing career focus. The same cannot be said for AI researchers.

throwaway22032 3y ago

The thing about AGI ruin is that there are plausible, even if they currently seem like science fiction, ways in which it just ends all of humanity, whereas climate change doesn't do that even if we had a 10c global temperature increase. A lot of us, maybe almost all of us would die, but some would live. It's just a really different problem. I don't know whether categorising its' urgency makes sense, it's more that the worst case is worse.

forgetfreeman 3y ago
What part of faceless hordes of back office paper pushers rendered economically unviable seems like science fiction to you?
- techdragon 3y ago
  Ok, hear me out on the (albeit unlikely) doomsday flow chart… It’s not about the replaced workers, that’s the kind of change humanity can handle no matter how messy it gets, worst case it’s some kind of class uprising worker rebellion and we have ourselves some war or social unrest and a variable number of people die… not great but definitely not an existential risk. It’s equal to or less than climate change on the “how bad will this be to humanity” mathematics of civilisation. Now the danger scenario… so we invent an AGI, and unlike the just going to replace some basic minimum wage type jobs … this one’s actually “clever” enough it can be put to work on building more sophisticated things, like perhaps an even smarter AGI, built faster because it’s worked on by a group of first generation AGI units that never sleep, and so we get to the second generation much faster, and then perhaps a third… etc… classic singularity scenario, with all the attendant risks of the AI viewing us ass irrelevant before we realise it yada yada yada… But that’s not the only risk, there’s the “photocopy burn” risk that we run by having technology we don’t fully understand build technology we barely comprehend… even if the first gen AGI is smart, if it’s remotely human like using inference to get ingenuity, it can probably still make the odd mistake or two, so we get our careful well thought out safety instructions conveyed to the second generation systems… just a little wrong… and the little wrong turns into more wrong with the third because the second gen systems doesn’t quite think the way we expect it to and doesn’t have the safeguards working the way we think… and then eventually we wind up with a sufficiently smart AGI that is fully either sociopathic or psychopathic and has zero compunction doing things we find abhorrent in order to achieve the goals it’s set, and it may be in control of systems (or able to take control) that are sufficiently important that this can also turn into an existential risk… It’s entirely plausible, but I don’t think the doom sayers are being realistic about the timelines… we’re barely able to get the AI/ML to reliably stick to a script and not hallucinate fake shit… the best results at human behaviour require sophisticated supporting logic and memory systems to augment the black box ML stuff… and these support systems give us levers we can use to control the rate at which we allow systems to progress… I don’t think we’re ever going to get a black box set of weights for an ML model, no matter how sophisticated the code, that somehow is self aware… there’s probably going to need to be a lot of that support software, and I’m pretty sure we’re going to develop things to filter stuff out as part of them like “never write “kill all humans” into the short term memory” seems smart.

alfor 3y ago

Climate change is not urgent, it’s a very slow process. We were at the bottom of a little ice age, not surprisingly, it’s warming up. We will have transitioned away from fossil fuel before climate change is a real concern.

kaba0 3y ago
https://xkcd.com/1732/ https://xkcd.com/1732/
- alfor 3y ago
  The dotted lines represent in 80 years. Think where we were 80 years ago, it was before WW2, before the atomic age. With the current technological progress picture the human race in 80 years. Just in 20 years we will have massive solar energy generation, EV as the only cars availables, probably same for nuclear, maybe fusion even. Now think of the advance in computing and AI in 20 years! Genetic engineering? To think that climate change will be a problem in 80 years depend on those kind of linear projections. Now in 80 years I wonder where we will be with regards to population collapse, to our sense of self, communities with what social networks have done just in the last 10 years. Again, I don’t know how people can assume the same linear future for 80 years. So yes assuming that things stay the same for 80 years we might have a (small) problem with that at that point, but imagine all the other things we are missing by focusing so much in this small problem in the far future. We are living the singularity, this moment is not like the 1000’s of year before.
  - kaba0 3y ago
    What economic reason is their to stop climate catastrophe? There is not even an inch of progress without money in this current era.
    - alfor 3y ago
      Look at Tesla: 50% growth Year over year, highly profitable. They make EV, grid batteries and solar. The price of solar PV is dropping so fast that solar electricity will be almost free in a few years. Today it cost just a fraction to charge an EV compared to fuel a gas car, and the price of EV is dropping every year. Basically just economically it will not be worth it to extract coal and oil out of the ground before climate change is a real problem. Renewable are following an exponential curves but they are still small compared to oil and coal but project their growth for the next 10 years and you will see a very different picture. Tony Seba predicted accurately the transition almost 10 year ago. https://youtu.be/fsnkPLkf1ao https://youtu.be/fsnkPLkf1ao

thealienthing 3y ago

Quick dumb question: does it seem like saying AI development is a threat to the climate a convenient excuse to put a stop to a technology that is disruptive to existing corporations and systems of power?

haswell 3y ago
There will always be factions that look to capitalize on fears, real and otherwise, to benefit themselves. But that doesn’t change the reality of the underlying threat.
- thealienthing 3y ago
  Yes. I do want to be clear that we should be mindful of the mistakes made during other technological revolutions that negatively impact the environment.
triceratops 3y ago
What makes you believe AI is "disruptive to existing corporations and systems of power"?
- thealienthing 3y ago
  Valid question. My statement is a bit vague. I see AI as a disruptive technology similar to the printing press or radio. It enables regular people to accomplish task that were previously difficult. I can see it becoming a tool that makes some major companies obsolete in the way it opens up new paths. But fair point, a lot of these AI advancements are being pushed forward by large corporations like Microsoft and Google. It’s obvious that AI as a tool will be used for good and evil. Tech companies who wish to survive see the necessity to get in on the bottom floor of this technological leap forward. Either way, my observation is simply that I could see a lot of parties finding the advancement of human potential through AI to be harmful to the status quo that they comfortably exist in.
  - sjkoelle 3y ago

vkou 3y ago

If he says this, it sounds like he doesn't understand climate change, or AI, or basic economics. What makes AI dangerous isn't AI itself, it's the self-serving negative-sum ways in which its owners will use it. Same as with any other technology - including fossil fuels. We default to permitting new technologies, be they big or small, without requiring their inventors to convince us that society will be better off for them. Unsurprisingly, this results in net-negative ones occasionally being developed and adopted.

NumberWangMan 3y ago
Why? Yes people misusing is a danger, but given that no other technology has ever had the same quality of intelligence, why do you think that we won't eventually develop really smart AGI, OR why do you think that controlling something smarter than us will be easy?

sjkoelle 3y ago

Yeah, these new Harry Potter memes are too powerful, we must protec societai

RcouF1uZ4gsC 3y ago

>“With climate change, it’s very easy to recommend what you should do: you just stop burning carbon. If you do that, eventually things will be okay. For this it’s not at all clear what you should do.” > “I’m in the camp that thinks this is an existential risk, and it’s close enough that we ought to be working very hard right now, and putting a lot of resources into figuring out what we can do about it,” Hinton said. I am sure he does not believe that. Because, if we actually believed AI was an existential threat, then the thing to do immediately is to ban all AI training (it would be easy as AI training is pretty expensive) and to hunt down all AI researchers and place them in solitary confinement for the rest of their lives. This would actually contain the threat and moreover have very minimal disruption of normal people's lives. And if you really believed that AI was an existential threat, you could argue that that course of action is the most moral to take (greatest good to greatest number and all that).

yowzadave 3y ago
Absolutely—it seems what he really wants is to go from his past lucrative gig (training AI) to his new lucrative gig (developing AI alignment tech?).
NumberWangMan 3y ago
People can have sincere beliefs and disagree about the best course of action. Hinton has publicly said he's in favor of stopping, but doesn't think China would. I personally disagree, I think that coordinating a stop would be easier than he thinks. But I don't doubt his sincerity. He also believes that if we can solve alignment, AI would be a massive boon for humanity, which is why it's a bit of a quandary.

password54321 3y ago

What if we just accept that we are the stepping stone to greater intelligence that won't burn down the planet.

goatlover 3y ago
But it might do something else to the planet that isn't beneficial to biological life.
Capricorn2481 3y ago
What do you think AI runs on?
- password54321 3y ago
  So computers alone will burn down the planet? Well good to know.
  - Capricorn2481 3y ago
    Not just computers, but the most expensive energy draining component of a computer, and racks of them, in massive farms, generating arguably little value

kaba0 3y ago

That seems to be faulty logic. Even if we take the extreme threat of AIs resulting in the collapse of economy and society, many people could still survive in small, self-sufficient communities — hell, it is basically a built-in negative feedback loop, if society collapses a GAI won’t have too much teeth anymore, we are not in some sci-fi with robot armies. Climate crisis doesn’t have a solution in that landscape, it requires worldwide collaboration of government-level agents to effectively combat.

NumberWangMan 3y ago
That's not quite the existential risk of AI -- it's that we end up growing an intelligence (or multiple intelligences) that are very very smart but only have the illusion of morality, and they decide that whatever they're trying to do would be a lot easier and more certain to succeed if they didn't have to worry about humans stopping them or inventing other AIs to compete with them. It seems likely that unless we really understand how AI works, we'll end up with something that evaluates possible courses of action in a purely amoral, sociopathic sort of way. Like, trying to avoid getting caught rather than trying to avoid doing a bad thing. So it's not that they cause the collapse of the economy, it's that they figure out a way to entirely take over the planet, engineering a virus or something that kills almost everyone, but only once they have enough control over robots and such to keep their own power plants running and protect them from the human stragglers, eventually getting rid of all of us. Waiting until the plan is basically fool-proof before starting to take the first steps. Even if an AI has some sort of morality, at some point, if it's smart enough, you cannot hope to control it for the same reason a toddler cannot hope to outsmart their parents. It will have a plan for us, and if it decides that, hey, humans are better off scanned and simulated rather than having physical bodies, that's what's going to happen, like it or not. I'm not sure how worried I am about this. Some days very, other days not as much. The risk depends on a lot of factors with wide possible ranges, like how much model capability and generalization scales with compute, how likely are breakthroughs, how hard alignment turns out to be, is alignment even possible, will we learn enough from earlier models to make alignment of later ones easier, will there be a qualitative shift in alignment difficulty as models get smarter, and so on. It's also a quandary because there is substantial human suffering that AI might be able to help. But given that the downside risk is all of humans dead forever, it seems worth slowing way the heck down to a safe speed, trying to get all the benefit out of current models as we can, and focusing on making sure that we aren't about to head off a cliff.

jossclimb 3y ago

How many fathers and godfathers of AI are there?

wwwpatdelcom 3y ago

This is an uncharitable summarization of what Hinton's actual statement was. He said, "might," and "could," a lot, not, "is." The key here is that it's speculative. That being said, a lot going on in AI, it's wild out there, difficult to know what's going on. I tried to encapsulate some of the ramifications of all of the Open Source stuff going on with this article here: https://patdel.substack.com/p/viva-la-revolucion-ofopen-source https://patdel.substack.com/p/viva-la-revolucion-ofopen-sour...

0xbadc0de5 3y ago

Funny how you never see someone describe their life's work as "no big deal." Even when they describe it in catastrophic terms - the hubris in us always wants to imagine our work is more important than perhaps it is. "The only thing worse than being talked about is not being talked about." - Oscar Wilde Not to say that Hinton didn't have an impact 10 - 15 years ago, but it was arguably inevitable that we'd be here with or without him as the problem was resource constrained more than fundamentally intractable.

blastonico 3y ago

I'm not Chinese. We westerners must shutdown our AI studies ASAP, we must act ethically and lead by example.