3 ms·

It is both incredibly bad faith nonsense and extremely ignorant. The ethics debates around AI are filled with military jargon dropping morons. It is extremely h

It is both incredibly bad faith nonsense and extremely ignorant. The ethics debates around AI are filled with military jargon dropping morons. It is extremely hard to find a rational anyone not pushing some corporate-wealth-scheme or crank-conspiracy nonsense. I work in AI, and the non-technically specific conversations, such as ethics, are extremely shallow.

sicp-enjoyer 4y ago
It's a form of marketing that appeals to people in tech.
hutzlibu 4y ago
The main problem with non technical people talking about AI, is that they mainly have no understanding about AI, but everyone has still a strong opinion (or thinks he has to have one)
- Sakos 4y ago
  The main problem is that even in the field among experts, ethics has extremely low priority. It's shallow not because of laypersons, but because experts don't think ethics are worth thinking about.
  - visarga 4y ago
    Up until a few years ago AI systems were not sophisticated enough to have an impact. Papers only recently started having an ethics section, and even now it's mostly low effort, just write something and get over it.
  - jameshart 4y ago
    Right. If you are concerned that the only writing on ethics of AI is by people who lack the technical understanding of the field to meaningfully comment, then it is incumbent on the people who are technically competent in the field to meaningfully engage in discussing ethics. As it is I see AI researchers engaging in a curious doublethink: both assuming that someone else is thinking about the ethical questions, and at the same time whenever someone raises an ethical critique, dismissing it as technically unsophisticated.
- astrange 4y ago
  Technical people, or tech-adjacent people at least, seem to be even worse. They seem to assume that since they know how computers work that AGI “obviously” works the same way and will obviously instantly self-reproduce and take over the world for… some reason. Of course, they were equally confident logic and expert systems could build an AI and despite none of that working out at all, it doesn’t seem to have stopped them.
  - jimrandomh 4y ago
    The keyword you're looking for is "instrumental convergence" (https://www.lesswrong.com/tag/instrumental-convergence https://www.lesswrong.com/tag/instrumental-convergence). It's not an intuition or obviousness based argument; there's significant prior work, if someone calls it obvious and skips over the argument they're being lazy by not providing links but the links do exist. The expert-system/GOFAI people are mostly not the same people.
    - astrange 4y ago
      The argument that an imaginary AGI is going to take over the world because of an imaginary principle is not actually very convincing though. It would be better if you said they'd do something humans already did; at least we know that'd be possible. Most of this "instrumental convergence" assumes that it has an infinite attention span, infinite energy, all its plans work properly… the only reason it all goes that well is you're imagining it did. I think last week someone on twitter told me an AI could be bad because it'd "kill us all by releasing viruses into the atmosphere", but if you had a baby it could also grow up to do that, and (correctly) nobody worries about this. Not clear why these are supposed to be different. More importantly, it mostly relies on thinking AGIs behave like computer programs in that 1. they don't get bored 2. they can be copied or copy themselves. Is that necessarily true? If someone copied you could you take over the world easier? Maybe you'd just fight over who gets the bank account.
- walnutclosefarm 4y ago
  When it comes to the impact and ethics of AI, I don't think technically informed people are much, if any, better than the non technically informed at understanding AI and forming judgments. They don't really understand fully how it works, either, and they most definitely are not social-systems aware enough to understand how it actually impacts and functions in use.
  - hutzlibu 4y ago
    I mean yes, the nerd deep in his field, is mainly focused on getting it to work and ethics would only be a potential hurdle, so they are clearly biased and the decision should not be left to them. But I am not sure if it helps, when people talk about it, who do not have a clue at all. Which is sadly most people. (computers are a black box to them) This will lead to situations, when activists and politicians lobby for automatic AI image control to fight child porn, not understanding false positives and that this means that lots of legitimate and very private pictures would be send of away for some human to check.
- bsenftner 4y ago
  The biggest misconception is people believe AI "thinks", which would require AI possessing comprehension, which it does not. That is the big hurdle laypeople can't seem to grasp: without comprehension, AI is literally an idiot savant, it can answer but it cannot tell if the question is nonsense or valid. This lack of comprehension is exactly why a human is required in AI loops which impact lives.
nonrandomstring 4y ago
I am hoping to address that, as best I can. One must remember that ethics is philosophy that has only so much traction where it touches necessity and survival.

sicp-enjoyer 4y ago

It's a form of marketing that appeals to people in tech.

hutzlibu 4y ago

The main problem with non technical people talking about AI, is that they mainly have no understanding about AI, but everyone has still a strong opinion (or thinks he has to have one)

Sakos 4y ago
The main problem is that even in the field among experts, ethics has extremely low priority. It's shallow not because of laypersons, but because experts don't think ethics are worth thinking about.
- visarga 4y ago
  Up until a few years ago AI systems were not sophisticated enough to have an impact. Papers only recently started having an ethics section, and even now it's mostly low effort, just write something and get over it.
- jameshart 4y ago
  Right. If you are concerned that the only writing on ethics of AI is by people who lack the technical understanding of the field to meaningfully comment, then it is incumbent on the people who are technically competent in the field to meaningfully engage in discussing ethics. As it is I see AI researchers engaging in a curious doublethink: both assuming that someone else is thinking about the ethical questions, and at the same time whenever someone raises an ethical critique, dismissing it as technically unsophisticated.
astrange 4y ago
Technical people, or tech-adjacent people at least, seem to be even worse. They seem to assume that since they know how computers work that AGI “obviously” works the same way and will obviously instantly self-reproduce and take over the world for… some reason. Of course, they were equally confident logic and expert systems could build an AI and despite none of that working out at all, it doesn’t seem to have stopped them.
- jimrandomh 4y ago
  The keyword you're looking for is "instrumental convergence" (https://www.lesswrong.com/tag/instrumental-convergence https://www.lesswrong.com/tag/instrumental-convergence). It's not an intuition or obviousness based argument; there's significant prior work, if someone calls it obvious and skips over the argument they're being lazy by not providing links but the links do exist. The expert-system/GOFAI people are mostly not the same people.
  - astrange 4y ago
    The argument that an imaginary AGI is going to take over the world because of an imaginary principle is not actually very convincing though. It would be better if you said they'd do something humans already did; at least we know that'd be possible. Most of this "instrumental convergence" assumes that it has an infinite attention span, infinite energy, all its plans work properly… the only reason it all goes that well is you're imagining it did. I think last week someone on twitter told me an AI could be bad because it'd "kill us all by releasing viruses into the atmosphere", but if you had a baby it could also grow up to do that, and (correctly) nobody worries about this. Not clear why these are supposed to be different. More importantly, it mostly relies on thinking AGIs behave like computer programs in that 1. they don't get bored 2. they can be copied or copy themselves. Is that necessarily true? If someone copied you could you take over the world easier? Maybe you'd just fight over who gets the bank account.
walnutclosefarm 4y ago
When it comes to the impact and ethics of AI, I don't think technically informed people are much, if any, better than the non technically informed at understanding AI and forming judgments. They don't really understand fully how it works, either, and they most definitely are not social-systems aware enough to understand how it actually impacts and functions in use.
- hutzlibu 4y ago
  I mean yes, the nerd deep in his field, is mainly focused on getting it to work and ethics would only be a potential hurdle, so they are clearly biased and the decision should not be left to them. But I am not sure if it helps, when people talk about it, who do not have a clue at all. Which is sadly most people. (computers are a black box to them) This will lead to situations, when activists and politicians lobby for automatic AI image control to fight child porn, not understanding false positives and that this means that lots of legitimate and very private pictures would be send of away for some human to check.
bsenftner 4y ago
The biggest misconception is people believe AI "thinks", which would require AI possessing comprehension, which it does not. That is the big hurdle laypeople can't seem to grasp: without comprehension, AI is literally an idiot savant, it can answer but it cannot tell if the question is nonsense or valid. This lack of comprehension is exactly why a human is required in AI loops which impact lives.

nonrandomstring 4y ago

I am hoping to address that, as best I can. One must remember that ethics is philosophy that has only so much traction where it touches necessity and survival.