4 ms·

While I'm no expert on AI, everything I've read from LeCunn on AI risk so far strikes me as directionally correct. I keep revisting the best examples I can find

While I'm no expert on AI, everything I've read from LeCunn on AI risk so far strikes me as directionally correct. I keep revisting the best examples I can find of the 'Foom' hypothesis and it just doesn't seem likely. Not to say that AI won't be both very useful and disruptive, just that existential fears like Skynet scenarios don't strike me as plausible.

Elucalidavah 2y ago
> it just doesn't seem likely It is likely conditional on the price of compute dropping the way it has been. If you can basically simulate a human brain on a $1000 machine, you don't really need to employ any AI researchers. Of course, there has been some fear that the current models are a year away from FOOMing, but that does seem to be just the hype talking.
- threeseed 2y ago
  If you can simulate a human brain and it required a $100b machine you would still get funding in a weekend. Because you could easily find ways to print money e.g. curing types of cancers or inventing a better Ozempic. But the fact is that there is no path to simulating a human brain.
  - llamaimperative 2y ago
    There is no path to it? That's a bold claim. Are brains imbued with special brain-magic that makes them more than, at rock bottom, a bunch of bog-standard chemical and electrical and thermal reactions? It seems very obviously fundamentally solvable, though I agree it is nowhere in the near future.
    - bob1029 2y ago
      > Are brains imbued with special brain-magic that makes them more than, at rock bottom, a bunch of bog-standard chemical and electrical and thermal reactions? Some have made this argument (quantum effects, external fields, etc.). If any of these are proven to be true then we are looking at a completely different roadmap.
      - llamaimperative 2y ago
        Uh yeah, but we have no evidence for any of them (aside from quantum effects, which are "engineerable" to the extent they exist in brains anyway).
        deleted 2y ago
        [deleted]
        threeseed 2y ago
        > "engineerable" to the extent they exist in brains anyway Can you please enlighten us then since you clearly know to what extent quantum effects exist in the brain.
        llamaimperative 2y ago
        I’m saying to whatever extent they occur, they are just quantum interactions. There’s a path to reproducing them with engineering. It’s odd to say “reproduce quantum interactions” but remember to the extent they exist in the brain, they also behave as finicky/noisy quantum interactions. They’re not special brain quantum things.
    - aithrowawaycomm 2y ago
      This seems like a misreading - there's also no real path to P-NP or to disentangling the true chemical origins of life. OP didn't say it was impossible. The problem is we don't know very much about intelligence in animals generally, and even less about intelligence in humans. In particular, we know far less about intelligence than we do computational complexity or early forms of life.
      - CooCooCaCha 2y ago
        Those seem like silly analogies. There are billions of brains on the planet, humans can grow them inside themselves (pregnancy). Don’t get me wrong, it’s a hard problem, they just seem like different classes of problems. I could see P=NP being impossible to prove but I find it hard to believe intelligence is impossible to figure out. Heck if you said it’d take us 100 years I would still think that’s a bit much.
        RandomLensman 2y ago
        We have not even figured out single cell organisms, let alone slightly more complex organisms - why would intelligence be such an easy target?
        CooCooCaCha 2y ago
        I didn’t say easy.
        RandomLensman 2y ago
        Yes, sorry, but it still could be impossibly hard for a long time. A lot of things happen in nature all the time we cannot do and have no (practical) path to doing them.
        threeseed 2y ago
        > Don’t get me wrong, it’s a hard problem, they just seem like different classes of problems Time travel. Teleportation through quantum entanglement. Intergalactic travel through wormholes. And don't get me wrong they are hard. But just another class of problems. Right ?
    - deleted 2y ago
      [deleted]
- mrandish 2y ago
  > If you can basically simulate a human brain Based on the evidence I've seen to date, doing this part at the scale of human intelligence (regardless of cost) is highly unlikely to be possible for at least decades. (a note to clarify: the goal "simulate a human brain" is substantially harder than other goals usually discussed around AI, like "exceed domain expert human ability on tests measuring problem solving in certain domain(s).)
llamaimperative 2y ago
> just that existential fears like Skynet scenarios don't strike me as plausible. What's the most plausible (even if you find it implausible) disaster scenario you came across in your research? It's a little surprising to see someone who has seriously looked into these ideas describe the bundle of them as "like Skynet."
- trescenzi 2y ago
  I think the risk is much higher with regards to how people use it and much less that it becomes some sudden super intelligent monster. AI doesn’t have to be rational or intelligent to cause massive amounts of damage it just has to be put in charge of dangerous enough systems. Or more pernicious you give it the power to make health care or employment decisions. It seems silly to me the idea of risk is all concentrated around the runaway intelligence scenario. While that might be possible there is real risk today in how we use these systems.
  - mrandish 2y ago
    I agree with what you've said. Personally, I have no doubt that, like any powerful new technology, AI will be used for all kinds of negative and annoying things as well as beneficial things. This is what I meant by "disruptive" in my GP. However, I also think that society will adapt to address these disruptions just like we have in the past.
  - nradov 2y ago
    We have had software making healthcare decisions for decades in areas like ECG pattern analysis, clinical decision support, medication administration, insurance claims processing, etc. Occasionally software defects or usability flaws lead to patient harm but mostly they work pretty well. There's no evidence that using AI to supplement existing deterministic algorithms will make things worse, it's just a lot of uninformed and unscientific fearmongering.
    - trescenzi 2y ago
      I’m not trying to say it is a problem, more that it could be a problem. The fear mongering around an AI apocalypse is silly. Concern around bugs in real software making minor but real decisions however are warranted. Just as we try to reduce human error so should we try to reduce algorithmic error.
habitue 2y ago
Statements like "It doesnt seem plausible", "it doesn't seem likely" aren't the strongest arguments. How things seem to us is based on what we've seen happen before. None of us has witnessed humanity replace itself with something that we dont understand before. Our intuition isn't a good guide here. Intuitions are honed through repeated exposure and feedback, and we clearly don't have that in this domain. Even though it doesn't feel dangerous, we can navigate this by reasoning through it. We understand that intelligence trumps brawn (e.g. Humans don't out-claw a tiger, we master it with intelligence). We understand that advances in AI have been very rapid, and that even though current AI doesnt feel dangerous, current AI turns into much more advanced future AI very quickly. And we understand that we dont really understand how these things work. We "control them safely" through mechanisms similar to how evolution controls us: theough the objective function. That shouldn't fill us with confidence because we find loopholes in evolution's objective function left and right like contraception, hyper-palatable foods, tiktok, etc. All these lines of evidence converge on the conclusion that what we're building is dangerous to us.
- mrandish 2y ago
  > ... "it doesn't seem likely" aren't the strongest arguments. Since we're talking about the future, it would be incorrect to talk in absolutes so speaking in probabilities and priors is appropriate. > Our intuition isn't a good guide here. I'm not just using intuition. I've done as extensive an evaluation of the technology, trends, predictions and, most importantly, history as I'm personally willing to do on this topic. Your post is an excellent summary of basically the precautionary principle approach but, as I'm sure you know, the precautionary principle can be over-applied to justify almost any level of response to almost any conceivable risk. If the argument construes the risk as probably existential, then almost any degree of draconian response could be justified. Hence my caution when the precautionary principle is invoked to argue for disruptive levels of response (and to be clear, you didn't). So the question really comes down to which scenarios at which level of probability and then what levels of response those bell-curve probabilities justify. Since I put 'foom-like' scenarios at low probability (sub-5%) and truly existential risk at sub-1%, I don't find extreme prevention measures justified due to their significant costs, burdens and disruptions. At the same time, I'm not arguing we shouldn't pay close attention as the technology develops while expending some reasonable level of resources on researching ways to detect, manage and mitigate possible serious AI risks, if and when they materialize. In particular, I find the current proposed legislative responses to regulate a still-nascent emerging technology to be ill-advised. It's still far too early and at this point I find such proposals by (mostly) grandstanding politicians and bureaucrats more akin to crafting potions to ward off an unseen bogeyman. They're as likely to hurt as to help while imposing substantial costs and burdens either way. I see the current AI giants embracing such proposals as simply them seeing these laws as an opportunity to raise the drawbridge behind themselves since they have the size and funds to comply while new startups don't - and those startups may be the most likely source of whatever 'solutions' we actually need to the problems which have yet to make themselves evident.
Yoric 2y ago
On the other hand, we can wipe our civilization (with or without AI) without needing anything as sophisticated as Skynet.

Elucalidavah 2y ago

> it just doesn't seem likely It is likely conditional on the price of compute dropping the way it has been. If you can basically simulate a human brain on a $1000 machine, you don't really need to employ any AI researchers. Of course, there has been some fear that the current models are a year away from FOOMing, but that does seem to be just the hype talking.

threeseed 2y ago
If you can simulate a human brain and it required a $100b machine you would still get funding in a weekend. Because you could easily find ways to print money e.g. curing types of cancers or inventing a better Ozempic. But the fact is that there is no path to simulating a human brain.
- llamaimperative 2y ago
  There is no path to it? That's a bold claim. Are brains imbued with special brain-magic that makes them more than, at rock bottom, a bunch of bog-standard chemical and electrical and thermal reactions? It seems very obviously fundamentally solvable, though I agree it is nowhere in the near future.
  - bob1029 2y ago
    > Are brains imbued with special brain-magic that makes them more than, at rock bottom, a bunch of bog-standard chemical and electrical and thermal reactions? Some have made this argument (quantum effects, external fields, etc.). If any of these are proven to be true then we are looking at a completely different roadmap.
    - llamaimperative 2y ago
      Uh yeah, but we have no evidence for any of them (aside from quantum effects, which are "engineerable" to the extent they exist in brains anyway).
      - deleted 2y ago
        [deleted]
      - threeseed 2y ago
        > "engineerable" to the extent they exist in brains anyway Can you please enlighten us then since you clearly know to what extent quantum effects exist in the brain.
        llamaimperative 2y ago
        I’m saying to whatever extent they occur, they are just quantum interactions. There’s a path to reproducing them with engineering. It’s odd to say “reproduce quantum interactions” but remember to the extent they exist in the brain, they also behave as finicky/noisy quantum interactions. They’re not special brain quantum things.
  - aithrowawaycomm 2y ago
    This seems like a misreading - there's also no real path to P-NP or to disentangling the true chemical origins of life. OP didn't say it was impossible. The problem is we don't know very much about intelligence in animals generally, and even less about intelligence in humans. In particular, we know far less about intelligence than we do computational complexity or early forms of life.
    - CooCooCaCha 2y ago
      Those seem like silly analogies. There are billions of brains on the planet, humans can grow them inside themselves (pregnancy). Don’t get me wrong, it’s a hard problem, they just seem like different classes of problems. I could see P=NP being impossible to prove but I find it hard to believe intelligence is impossible to figure out. Heck if you said it’d take us 100 years I would still think that’s a bit much.
      - RandomLensman 2y ago
        We have not even figured out single cell organisms, let alone slightly more complex organisms - why would intelligence be such an easy target?
        CooCooCaCha 2y ago
        I didn’t say easy.
        RandomLensman 2y ago
        Yes, sorry, but it still could be impossibly hard for a long time. A lot of things happen in nature all the time we cannot do and have no (practical) path to doing them.
      - threeseed 2y ago
        > Don’t get me wrong, it’s a hard problem, they just seem like different classes of problems Time travel. Teleportation through quantum entanglement. Intergalactic travel through wormholes. And don't get me wrong they are hard. But just another class of problems. Right ?
  - deleted 2y ago
    [deleted]
mrandish 2y ago
> If you can basically simulate a human brain Based on the evidence I've seen to date, doing this part at the scale of human intelligence (regardless of cost) is highly unlikely to be possible for at least decades. (a note to clarify: the goal "simulate a human brain" is substantially harder than other goals usually discussed around AI, like "exceed domain expert human ability on tests measuring problem solving in certain domain(s).)

llamaimperative 2y ago

> just that existential fears like Skynet scenarios don't strike me as plausible. What's the most plausible (even if you find it implausible) disaster scenario you came across in your research? It's a little surprising to see someone who has seriously looked into these ideas describe the bundle of them as "like Skynet."

trescenzi 2y ago
I think the risk is much higher with regards to how people use it and much less that it becomes some sudden super intelligent monster. AI doesn’t have to be rational or intelligent to cause massive amounts of damage it just has to be put in charge of dangerous enough systems. Or more pernicious you give it the power to make health care or employment decisions. It seems silly to me the idea of risk is all concentrated around the runaway intelligence scenario. While that might be possible there is real risk today in how we use these systems.
- mrandish 2y ago
  I agree with what you've said. Personally, I have no doubt that, like any powerful new technology, AI will be used for all kinds of negative and annoying things as well as beneficial things. This is what I meant by "disruptive" in my GP. However, I also think that society will adapt to address these disruptions just like we have in the past.
- nradov 2y ago
  We have had software making healthcare decisions for decades in areas like ECG pattern analysis, clinical decision support, medication administration, insurance claims processing, etc. Occasionally software defects or usability flaws lead to patient harm but mostly they work pretty well. There's no evidence that using AI to supplement existing deterministic algorithms will make things worse, it's just a lot of uninformed and unscientific fearmongering.
  - trescenzi 2y ago
    I’m not trying to say it is a problem, more that it could be a problem. The fear mongering around an AI apocalypse is silly. Concern around bugs in real software making minor but real decisions however are warranted. Just as we try to reduce human error so should we try to reduce algorithmic error.

habitue 2y ago

Statements like "It doesnt seem plausible", "it doesn't seem likely" aren't the strongest arguments. How things seem to us is based on what we've seen happen before. None of us has witnessed humanity replace itself with something that we dont understand before. Our intuition isn't a good guide here. Intuitions are honed through repeated exposure and feedback, and we clearly don't have that in this domain. Even though it doesn't feel dangerous, we can navigate this by reasoning through it. We understand that intelligence trumps brawn (e.g. Humans don't out-claw a tiger, we master it with intelligence). We understand that advances in AI have been very rapid, and that even though current AI doesnt feel dangerous, current AI turns into much more advanced future AI very quickly. And we understand that we dont really understand how these things work. We "control them safely" through mechanisms similar to how evolution controls us: theough the objective function. That shouldn't fill us with confidence because we find loopholes in evolution's objective function left and right like contraception, hyper-palatable foods, tiktok, etc. All these lines of evidence converge on the conclusion that what we're building is dangerous to us.

mrandish 2y ago
> ... "it doesn't seem likely" aren't the strongest arguments. Since we're talking about the future, it would be incorrect to talk in absolutes so speaking in probabilities and priors is appropriate. > Our intuition isn't a good guide here. I'm not just using intuition. I've done as extensive an evaluation of the technology, trends, predictions and, most importantly, history as I'm personally willing to do on this topic. Your post is an excellent summary of basically the precautionary principle approach but, as I'm sure you know, the precautionary principle can be over-applied to justify almost any level of response to almost any conceivable risk. If the argument construes the risk as probably existential, then almost any degree of draconian response could be justified. Hence my caution when the precautionary principle is invoked to argue for disruptive levels of response (and to be clear, you didn't). So the question really comes down to which scenarios at which level of probability and then what levels of response those bell-curve probabilities justify. Since I put 'foom-like' scenarios at low probability (sub-5%) and truly existential risk at sub-1%, I don't find extreme prevention measures justified due to their significant costs, burdens and disruptions. At the same time, I'm not arguing we shouldn't pay close attention as the technology develops while expending some reasonable level of resources on researching ways to detect, manage and mitigate possible serious AI risks, if and when they materialize. In particular, I find the current proposed legislative responses to regulate a still-nascent emerging technology to be ill-advised. It's still far too early and at this point I find such proposals by (mostly) grandstanding politicians and bureaucrats more akin to crafting potions to ward off an unseen bogeyman. They're as likely to hurt as to help while imposing substantial costs and burdens either way. I see the current AI giants embracing such proposals as simply them seeing these laws as an opportunity to raise the drawbridge behind themselves since they have the size and funds to comply while new startups don't - and those startups may be the most likely source of whatever 'solutions' we actually need to the problems which have yet to make themselves evident.

Yoric 2y ago

On the other hand, we can wipe our civilization (with or without AI) without needing anything as sophisticated as Skynet.