3 ms·

The “Humans on average are bad at X, therefore it isn’t a problem that AI makes mistakes.” argument is getting tired. No, I’ve never personally fallen for a phi

The “Humans on average are bad at X, therefore it isn’t a problem that AI makes mistakes.” argument is getting tired. No, I’ve never personally fallen for a phishing scam. If my email marked a phishing email as “Priority”, I would be much more likely to fall for it. This seems bad.

acdha 2y ago
I wasn’t sure whether they intended that as exoneration or explanation but I think it should be the latter. The right way to think about LLMs is as a credulous hire with no job history: McDonald’s will hire a random 16 year old but only to work in a controlled environment with oversight, not to open the CEO’s mail and tell them what’s important.
- barrell 2y ago
  I would shy away from any comparisons to a human. The right way to think about LLMs is like generative fill for text or reverse text summarization imho
  - acdha 2y ago
    Fair, I’m really thinking about it in response to people pushing products with terminology we normally use for people but the comparison really is tricky since this is our first collective experience with something which can sound authoritative without any deeper understanding and historically many people have used one as the proxy for the other.
  - consteval 2y ago
    I agree. I think as soon as you refer to AIs as what they are, computer programs, a lot of problems and solutions prevent themselves. For example, why are some people trying to give rights to computer programs? Since when have computer programs had rights? Fair use doctrine, for example, is a right for human beings.
SkyBelow 2y ago
The argument, in isolate, seems fine to me. My problem is that it conflicts with how it is being deployed and being trusted. People trust computer systems far beyond what trust they deserve, because they are use to some critical systems being made significantly resistant and most of the others as not having any significant problems. This logic is already a threat when it applied to standard applications built where a programmer, in theory, should have understood each part, at least while building it. This logic works much worse when applied to AI, yet AI are being sold using the common faith that people have in computer systems to give it more responsibility than it can rightly claim given its error rates and the faith people have. I think the solution is to teach people to doubt expert systems, which will greatly harm their usefulness, but trust should be earned by these systems, on a system by system basis, and they don't deserve the level of trust they currently enjoy.
potatoman22 2y ago
We'd be most likely to fall for phishing scams if there was no system filtering out bad emails. No model is perfect, but some are useful.
- elicksaur 2y ago
  Also a false equivalency. Current state-of-the-art is not “no spam filter”. When AI is useful, it won’t be a debate.
  - potatoman22 2y ago
    I don't think I made a false equivalency. What I'm trying to say is AI can't be perfect or always produce state of the art results. Bad outcomes can always occur, so we should remain vigilant, but not let perfection be the enemy of useful.
ryandrake 2y ago
It is bad. Like it or not, the general public considers computers to be deterministic calculating machines that produce the correct result. And when this doesn't happen, we expect the software developer to treat the case as a defect and work to correct it. Now, we have people telling us, no, computers are not calculation tools, they are more like an overconfident 14-year-old Redditor, and any mistakes they make are not defects, but unavoidable limitations of AI and you should expect them.
II2II 2y ago
It's not so much that it is acceptable for filters to misclassify email as we should keep our expectations realistic. Then again, I've never been one to treat the priority email folder with credulity. What it classifies as priority is often quite different from what I would. Never mind treating those emails as inherently legitimate.
bradknowles 2y ago
There are people who have fallen for a phishing e-mail, and those who have not yet knowingly fallen for a phishing e-mail. No one is perfect. No one is invulnerable. All human beings are fallible. Given the current state of LLMs and Generative AI systems, I submit that we should not be surprised that the same is true of them.

acdha 2y ago

I wasn’t sure whether they intended that as exoneration or explanation but I think it should be the latter. The right way to think about LLMs is as a credulous hire with no job history: McDonald’s will hire a random 16 year old but only to work in a controlled environment with oversight, not to open the CEO’s mail and tell them what’s important.

barrell 2y ago
I would shy away from any comparisons to a human. The right way to think about LLMs is like generative fill for text or reverse text summarization imho
- acdha 2y ago
  Fair, I’m really thinking about it in response to people pushing products with terminology we normally use for people but the comparison really is tricky since this is our first collective experience with something which can sound authoritative without any deeper understanding and historically many people have used one as the proxy for the other.
- consteval 2y ago
  I agree. I think as soon as you refer to AIs as what they are, computer programs, a lot of problems and solutions prevent themselves. For example, why are some people trying to give rights to computer programs? Since when have computer programs had rights? Fair use doctrine, for example, is a right for human beings.

SkyBelow 2y ago

The argument, in isolate, seems fine to me. My problem is that it conflicts with how it is being deployed and being trusted. People trust computer systems far beyond what trust they deserve, because they are use to some critical systems being made significantly resistant and most of the others as not having any significant problems. This logic is already a threat when it applied to standard applications built where a programmer, in theory, should have understood each part, at least while building it. This logic works much worse when applied to AI, yet AI are being sold using the common faith that people have in computer systems to give it more responsibility than it can rightly claim given its error rates and the faith people have. I think the solution is to teach people to doubt expert systems, which will greatly harm their usefulness, but trust should be earned by these systems, on a system by system basis, and they don't deserve the level of trust they currently enjoy.

potatoman22 2y ago

We'd be most likely to fall for phishing scams if there was no system filtering out bad emails. No model is perfect, but some are useful.

elicksaur 2y ago
Also a false equivalency. Current state-of-the-art is not “no spam filter”. When AI is useful, it won’t be a debate.
- potatoman22 2y ago
  I don't think I made a false equivalency. What I'm trying to say is AI can't be perfect or always produce state of the art results. Bad outcomes can always occur, so we should remain vigilant, but not let perfection be the enemy of useful.

ryandrake 2y ago

It is bad. Like it or not, the general public considers computers to be deterministic calculating machines that produce the correct result. And when this doesn't happen, we expect the software developer to treat the case as a defect and work to correct it. Now, we have people telling us, no, computers are not calculation tools, they are more like an overconfident 14-year-old Redditor, and any mistakes they make are not defects, but unavoidable limitations of AI and you should expect them.

II2II 2y ago

It's not so much that it is acceptable for filters to misclassify email as we should keep our expectations realistic. Then again, I've never been one to treat the priority email folder with credulity. What it classifies as priority is often quite different from what I would. Never mind treating those emails as inherently legitimate.

bradknowles 2y ago

There are people who have fallen for a phishing e-mail, and those who have not yet knowingly fallen for a phishing e-mail. No one is perfect. No one is invulnerable. All human beings are fallible. Given the current state of LLMs and Generative AI systems, I submit that we should not be surprised that the same is true of them.