5 ms·

AI Causes Real Harm. Let’s Focus on That over the End-of-Humanity Hype

47 points by version_five 3y ago

theragra 3y ago
Ironically, article makes some strong claims without scientific proof. Written by linguist, not ai researcher.
- felideon 3y ago
  Computational linguist. How does that not qualify them to write on the AI topic du jour, LLMs?
  - smoldesu 3y ago
    I actually agree with the thesis of the author's argument, but the sensationalist framing and subjective definition of 'harm' makes it difficult for me to fully stand behind their writing. It feels characteristic of a lot of other popular writings on computational linguistics. On the one hand, it feels like a 'boy who cried wolf' situation in the age of LLMs. On the other hand, reading stuff like this makes me wonder if there's even a populist sentiment against AI in the first place.
  - dekhn 3y ago
    I've noticed quite a lot of linguists (including the computational sub-variety) really dislike modern probabilistic approaches to text generation and parsing (https://norvig.com/chomsky.html https://norvig.com/chomsky.html) and I think Emily is partly opposed to LLMs on that basis. but she also has a running social justice jihad (I can't think of a more accurate way to describe it) in the form of a constant stream of articles and tweets that basically just say the same points over and over. Personally I am skeptical that the approaches she is advocating (heavy regulation) would really achieve her goals, and I certainly don't think they would achieve the goals of the larger society which benefits (or loses) with access to LLM research being fairly unrestricted. My view, generally speaking, is that the problems ML causes aren't due to ML, ML just scales the problem to larger impact, and these complaints are really just tilting at the current structure of society.
    - hgsgm 3y ago
      Current problems in society seem important to me.
ecf 3y ago
“The printing press causes real harm because Artisans are all of a sudden in fewer demand.” That’s how I view all these people decrying AI in the face of all the genuine improvements it can bring about.
- janalsncm 3y ago
  I think you mean scribes right? But yes, occupational obsolescence is a risk, and I think it’s reasonable to worry about at a macro scale. The luddites weren’t wrong, their jobs did go away and it was very bad for them. Large parts of the US underwent systemic obsolescence when their jobs were shipped overseas. At a societal level, that has been a disaster. Deaths of despair have skyrocketed. Retraining is expensive and often unsuccessful. Automation threatens many industries at a global level, and much faster than on a time scale of decades. What will happen when large parts of the Chinese economy become unemployed due to robotics? It probably won’t be great. So the question is who will reap the benefits of these “genuine improvements”? This is a question every society needs to answer for themselves.
- scott-smith_us 3y ago
  Do you actually understand the arguments for concern about existential risk, though? https://www.youtube.com/c/robertmilesai https://www.youtube.com/c/robertmilesai
- ChatGTP 3y ago
  A self-aware super intelligent being with self-preservation goals is not like the printing press.
janalsncm 3y ago
I turned down an offer to work at a company which used ML to make loan decisions. I now work on something much less consequential, and I’m ok with that. Computers are best when they empower people, not when they take that power away. If we replace “AI” with “automated systems” in our rhetoric and marketing, it’s pretty clear what’s happening. One, things which used to be impossible at scale, like searching the entire internet, have become possible. Two, things which used to be done by humans are being done by machines. Number two has upsides and downsides. Computers can be auditable, consistent, and (in theory) ignore biases our society has baked in. However, they have a tendency to diffuse responsibility and some models are entirely opaque. Sorry, the computer says no is a situation more and more of us will be facing in the future, with no recourse.
- LeafItAlone 3y ago
  Why is a computer making loan decisions not better for almost everyone? Given enough information, it should be able to determine if the person taking out the loan can pay it back. That’s better for them and the bank. Maybe it disadvantages the people who might succeed in spite of the data against them, but what percentage of those exist?
  - janalsncm 3y ago
    A couple of reasons. Essentially you are asking “what is wrong with a perfect system” but the problem is no system is perfect, and we spend most of our time dealing with the imperfections. In my previous jobs the consequences of me making a mistake were that a page didn’t load or some content was unavailable or in the worst case deleted. But a mistake in a loan decisioning system is likely to be 1) much harder to detect and therefore 2) much more likely to persist but most importantly 3) vastly more consequential. Second, there is no way to determine from first principles whether someone will repay a loan. So you will need to use historical data. Loan data is very messy and biased due to cascading historical legal and quasi-legal policies. You may believe you can erase one feature from the training data (e.g. race) but if your model can infer race from other factors (which is likely because other features are not randomly distributed across populations) then you have the same problem you started with. To take a concrete example, there was a man who was wrongfully accused of fraud. Then after he won his lawsuit, he was wrongfully accused trying to cash his damages check. https://abcnews.go.com/amp/US/black-man-sues-detroit-bank-alleging-racial-discrimination/story?id=68484056 https://abcnews.go.com/amp/US/black-man-sues-detroit-bank-al... Some amount of racism, classism, and negligence went into that decision. That’s the bad part. But the good part is we could see what was happening. If instead the process was automated, it would be a lot harder to prove the same problem.
version_five 3y ago
I'm sure there's a benign explanation but it's interesting to see this story got downweighted hard. Compare the points and time since it was posted to others around it. Possibly because it's "AI" or because of the source? Anyway, I posted the article, I didn't agree with a lot of it but I do agree with the main point that "existential risk" is a smokescreen that covers real issues. I'd argue that so is "bias" - people distract themselves analyzing if AI behaves differently across demographics as opposed to questioning how horrible the underlying process of algorithmically determining someone's fate is.
- janalsncm 3y ago
  The article is pretty rambling but its thesis is good: automated systems are causing and will cause tons of problems. But those are practical problems, rather than the vague high brow problems. Problems like how can we deal with wealth inequality in a world where 99% of people are obsolete aren’t particularly attractive to a TED talk audience that paid $500/seat.
  - hgsgm 3y ago
    Seems like an important topic. The billionaires don't want to be killed by their servants.
scott-smith_us 3y ago
I wish people would either stop calling existential AI risks "hype", or make some informed arguments about why it's not really a problem. I know enough about ML and AI to follow the logic that shows that strong, general AI is a serious threat if it's not aligned, and we don't know how to safely align it yet.
- version_five 3y ago
  You didn't make an argument, you just said you know enough. The onus has to be on the people making the claim, you can't just imagine it might be dangerous and expect a rebuttal without putting forward an argument. I also "know enough" and have never seen even a vaguely coherent argument for how any conceivable evolution of current technology could pose a "serious threat" on it's own. I've seen terminator, I don't count made up situation where computers can control things and AI has goals of its own unless there's a coherent argument of how we'd get there.
  - janalsncm 3y ago
    The alignment problem is largely incoherent. Aligned with whom? Most problems will require unequal and arguably unfair sacrifices. Further, people even obvious and largely inconsequential solutions are controversial. If an AGI suggested wearing a mask during a pandemic that spreads via airborne particles, a large percent of the population would say the AGI is “unaligned” and we’re living through skynet.
    - flangola7 3y ago
      Yes, and those people would be wrong. You're compelling completely right, humanity is not ready to build an AGI. It will not be ready for a long time, if ever. Alignment may not be a thing that can exist. Overconfident men like Sam Altman are forging ahead with the unjustified hubris that we are ready for this.
    - scott-smith_us 3y ago
      Alignment with human norms and values. Yes, that's not a well-defined thing. But to paraphrase, "I know misalignment when I see it", for example when an AI suggests something like "feeding the homeless to the hungry". No one is claiming that alignment is a well-defined thing with crisp edges. They're saying that the mechanisms of AIs will favor solutions without regard to any specific constraints we haven't explicitly stated.
friend_and_foe 3y ago
What harm? That I can ask it to draw me pictures? As with everything else, people cause harm. AI itself as it exists today does nothing but return text and images from a query.
- invalidname 3y ago
  Face recognition tech that discriminates against black people is pretty widespread for one. But even LLMs essentially regurgitate content created by human artists and writers. The decline of stackoverflow might seem like "market force" decision. But AIs are trained on stackoverflow, if people stop contributing there (because of AI) the AI will become dumber since the training material will be outdated and information will become siloed.
- scott-smith_us 3y ago
  AI (potentially) makes everyone more powerful. It will provide amazing benefits, but it will also amplify and empower bad actors. An AI that is superintelligent could (just for example) give instructions for (or even orchestrate) the creation of a deadly pathogen that targets only a specific race... But most of the arguments aren't about bad actors. The basic mechanisms of AI give rise to a bunch of behaviors that are problematic with weak AIs and possibly extinction-level with super AIs. This isn't behavior that we just suppose they might have, given common-sense understanding of mammal psychology. This is behavior we've observed in many, many systems, that is a consequence of how AI works. For example: - AI seeks what you reward it for (i.e., what it "wants"), which is often exactly what you said vs. what you really meant. - AI is incentivized to prevent you from turning it off, or from updating (fine-tuning, correcting) its reward function. - A sufficiently smart AI will deceive you into believing it's doing what you want if it knows that you'd stop it before it gets what it wants. - A sufficiently smart AI could modify its own code, or create sub-agents to get around safety measures that prevent it from reaching certain "better" solutions. - Etc, etc. These are really interesting videos explaining the different issues: https://www.youtube.com/c/robertmilesai https://www.youtube.com/c/robertmilesai The list above is based upon provable behavior given the current mechanisms in AI. The only thing that makes this a future problem is that we don't currently have AI systems that are as smart or smarter than us. The history of AI research is notable in that estimates about the pace of progress have been all over the map. No one really knows how many more discoveries or refinements are required before AGI becomes possible. Once they get many times smarter/quicker than us, we need to be sure AIs are built NOT to want to do these things, because we're unlikely to be able to outsmart them.
RecycledEle 3y ago
"We're Sorry The page you were looking for could not be found."
hqsolomo 3y ago
I may not know AI very well but I know that automatic sinks, soap dispensers, and toilets don't work for me because they weren't tested against people with my skin tone. And these are low-stakes use cases. I really don't like thinking about that same problem, but with AI thrown in the mix. And I'm not even factoring in malicious intent yet. I may not buy in to the "Horizon Zero Dawn incoming" mentality yet but I do know there's great potential for great harm if we don't get ethics in place before AI really takes off.

47 points by version_five 3y ago

theragra 3y ago

Ironically, article makes some strong claims without scientific proof. Written by linguist, not ai researcher.

felideon 3y ago
Computational linguist. How does that not qualify them to write on the AI topic du jour, LLMs?
- smoldesu 3y ago
  I actually agree with the thesis of the author's argument, but the sensationalist framing and subjective definition of 'harm' makes it difficult for me to fully stand behind their writing. It feels characteristic of a lot of other popular writings on computational linguistics. On the one hand, it feels like a 'boy who cried wolf' situation in the age of LLMs. On the other hand, reading stuff like this makes me wonder if there's even a populist sentiment against AI in the first place.
- dekhn 3y ago
  I've noticed quite a lot of linguists (including the computational sub-variety) really dislike modern probabilistic approaches to text generation and parsing (https://norvig.com/chomsky.html https://norvig.com/chomsky.html) and I think Emily is partly opposed to LLMs on that basis. but she also has a running social justice jihad (I can't think of a more accurate way to describe it) in the form of a constant stream of articles and tweets that basically just say the same points over and over. Personally I am skeptical that the approaches she is advocating (heavy regulation) would really achieve her goals, and I certainly don't think they would achieve the goals of the larger society which benefits (or loses) with access to LLM research being fairly unrestricted. My view, generally speaking, is that the problems ML causes aren't due to ML, ML just scales the problem to larger impact, and these complaints are really just tilting at the current structure of society.
  - hgsgm 3y ago
    Current problems in society seem important to me.

ecf 3y ago

“The printing press causes real harm because Artisans are all of a sudden in fewer demand.” That’s how I view all these people decrying AI in the face of all the genuine improvements it can bring about.

janalsncm 3y ago
I think you mean scribes right? But yes, occupational obsolescence is a risk, and I think it’s reasonable to worry about at a macro scale. The luddites weren’t wrong, their jobs did go away and it was very bad for them. Large parts of the US underwent systemic obsolescence when their jobs were shipped overseas. At a societal level, that has been a disaster. Deaths of despair have skyrocketed. Retraining is expensive and often unsuccessful. Automation threatens many industries at a global level, and much faster than on a time scale of decades. What will happen when large parts of the Chinese economy become unemployed due to robotics? It probably won’t be great. So the question is who will reap the benefits of these “genuine improvements”? This is a question every society needs to answer for themselves.
scott-smith_us 3y ago
Do you actually understand the arguments for concern about existential risk, though? https://www.youtube.com/c/robertmilesai https://www.youtube.com/c/robertmilesai
ChatGTP 3y ago
A self-aware super intelligent being with self-preservation goals is not like the printing press.

janalsncm 3y ago

I turned down an offer to work at a company which used ML to make loan decisions. I now work on something much less consequential, and I’m ok with that. Computers are best when they empower people, not when they take that power away. If we replace “AI” with “automated systems” in our rhetoric and marketing, it’s pretty clear what’s happening. One, things which used to be impossible at scale, like searching the entire internet, have become possible. Two, things which used to be done by humans are being done by machines. Number two has upsides and downsides. Computers can be auditable, consistent, and (in theory) ignore biases our society has baked in. However, they have a tendency to diffuse responsibility and some models are entirely opaque. Sorry, the computer says no is a situation more and more of us will be facing in the future, with no recourse.

LeafItAlone 3y ago
Why is a computer making loan decisions not better for almost everyone? Given enough information, it should be able to determine if the person taking out the loan can pay it back. That’s better for them and the bank. Maybe it disadvantages the people who might succeed in spite of the data against them, but what percentage of those exist?
- janalsncm 3y ago
  A couple of reasons. Essentially you are asking “what is wrong with a perfect system” but the problem is no system is perfect, and we spend most of our time dealing with the imperfections. In my previous jobs the consequences of me making a mistake were that a page didn’t load or some content was unavailable or in the worst case deleted. But a mistake in a loan decisioning system is likely to be 1) much harder to detect and therefore 2) much more likely to persist but most importantly 3) vastly more consequential. Second, there is no way to determine from first principles whether someone will repay a loan. So you will need to use historical data. Loan data is very messy and biased due to cascading historical legal and quasi-legal policies. You may believe you can erase one feature from the training data (e.g. race) but if your model can infer race from other factors (which is likely because other features are not randomly distributed across populations) then you have the same problem you started with. To take a concrete example, there was a man who was wrongfully accused of fraud. Then after he won his lawsuit, he was wrongfully accused trying to cash his damages check. https://abcnews.go.com/amp/US/black-man-sues-detroit-bank-alleging-racial-discrimination/story?id=68484056 https://abcnews.go.com/amp/US/black-man-sues-detroit-bank-al... Some amount of racism, classism, and negligence went into that decision. That’s the bad part. But the good part is we could see what was happening. If instead the process was automated, it would be a lot harder to prove the same problem.

version_five 3y ago

I'm sure there's a benign explanation but it's interesting to see this story got downweighted hard. Compare the points and time since it was posted to others around it. Possibly because it's "AI" or because of the source? Anyway, I posted the article, I didn't agree with a lot of it but I do agree with the main point that "existential risk" is a smokescreen that covers real issues. I'd argue that so is "bias" - people distract themselves analyzing if AI behaves differently across demographics as opposed to questioning how horrible the underlying process of algorithmically determining someone's fate is.

janalsncm 3y ago
The article is pretty rambling but its thesis is good: automated systems are causing and will cause tons of problems. But those are practical problems, rather than the vague high brow problems. Problems like how can we deal with wealth inequality in a world where 99% of people are obsolete aren’t particularly attractive to a TED talk audience that paid $500/seat.
- hgsgm 3y ago
  Seems like an important topic. The billionaires don't want to be killed by their servants.

scott-smith_us 3y ago

I wish people would either stop calling existential AI risks "hype", or make some informed arguments about why it's not really a problem. I know enough about ML and AI to follow the logic that shows that strong, general AI is a serious threat if it's not aligned, and we don't know how to safely align it yet.

version_five 3y ago
You didn't make an argument, you just said you know enough. The onus has to be on the people making the claim, you can't just imagine it might be dangerous and expect a rebuttal without putting forward an argument. I also "know enough" and have never seen even a vaguely coherent argument for how any conceivable evolution of current technology could pose a "serious threat" on it's own. I've seen terminator, I don't count made up situation where computers can control things and AI has goals of its own unless there's a coherent argument of how we'd get there.
- janalsncm 3y ago
  The alignment problem is largely incoherent. Aligned with whom? Most problems will require unequal and arguably unfair sacrifices. Further, people even obvious and largely inconsequential solutions are controversial. If an AGI suggested wearing a mask during a pandemic that spreads via airborne particles, a large percent of the population would say the AGI is “unaligned” and we’re living through skynet.
  - flangola7 3y ago
    Yes, and those people would be wrong. You're compelling completely right, humanity is not ready to build an AGI. It will not be ready for a long time, if ever. Alignment may not be a thing that can exist. Overconfident men like Sam Altman are forging ahead with the unjustified hubris that we are ready for this.
  - scott-smith_us 3y ago
    Alignment with human norms and values. Yes, that's not a well-defined thing. But to paraphrase, "I know misalignment when I see it", for example when an AI suggests something like "feeding the homeless to the hungry". No one is claiming that alignment is a well-defined thing with crisp edges. They're saying that the mechanisms of AIs will favor solutions without regard to any specific constraints we haven't explicitly stated.

friend_and_foe 3y ago

What harm? That I can ask it to draw me pictures? As with everything else, people cause harm. AI itself as it exists today does nothing but return text and images from a query.

invalidname 3y ago
Face recognition tech that discriminates against black people is pretty widespread for one. But even LLMs essentially regurgitate content created by human artists and writers. The decline of stackoverflow might seem like "market force" decision. But AIs are trained on stackoverflow, if people stop contributing there (because of AI) the AI will become dumber since the training material will be outdated and information will become siloed.
scott-smith_us 3y ago
AI (potentially) makes everyone more powerful. It will provide amazing benefits, but it will also amplify and empower bad actors. An AI that is superintelligent could (just for example) give instructions for (or even orchestrate) the creation of a deadly pathogen that targets only a specific race... But most of the arguments aren't about bad actors. The basic mechanisms of AI give rise to a bunch of behaviors that are problematic with weak AIs and possibly extinction-level with super AIs. This isn't behavior that we just suppose they might have, given common-sense understanding of mammal psychology. This is behavior we've observed in many, many systems, that is a consequence of how AI works. For example: - AI seeks what you reward it for (i.e., what it "wants"), which is often exactly what you said vs. what you really meant. - AI is incentivized to prevent you from turning it off, or from updating (fine-tuning, correcting) its reward function. - A sufficiently smart AI will deceive you into believing it's doing what you want if it knows that you'd stop it before it gets what it wants. - A sufficiently smart AI could modify its own code, or create sub-agents to get around safety measures that prevent it from reaching certain "better" solutions. - Etc, etc. These are really interesting videos explaining the different issues: https://www.youtube.com/c/robertmilesai https://www.youtube.com/c/robertmilesai The list above is based upon provable behavior given the current mechanisms in AI. The only thing that makes this a future problem is that we don't currently have AI systems that are as smart or smarter than us. The history of AI research is notable in that estimates about the pace of progress have been all over the map. No one really knows how many more discoveries or refinements are required before AGI becomes possible. Once they get many times smarter/quicker than us, we need to be sure AIs are built NOT to want to do these things, because we're unlikely to be able to outsmart them.

RecycledEle 3y ago

"We're Sorry The page you were looking for could not be found."

hqsolomo 3y ago

I may not know AI very well but I know that automatic sinks, soap dispensers, and toilets don't work for me because they weren't tested against people with my skin tone. And these are low-stakes use cases. I really don't like thinking about that same problem, but with AI thrown in the mix. And I'm not even factoring in malicious intent yet. I may not buy in to the "Horizon Zero Dawn incoming" mentality yet but I do know there's great potential for great harm if we don't get ethics in place before AI really takes off.