4 ms·

> Worse, they were putting an untrustworthy AI summary in the exact place that users expect to see an email subject, with no mention of it being AI-generated T

> Worse, they were putting an untrustworthy AI summary in the exact place that users expect to see an email subject, with no mention of it being AI-generated This seems like one of the greater sins here. Why in the world would you ever replace the actual subject that people have been expecting to see in that location for older than I've been alive?

bbor 2y ago
I think that’s the one and only sin, and it’s a serious one! Just mind boggling stuff. This is like the product launch to summarize headlines: why??? Certainly an interesting wrinkle to keep an eye on as AI takes up progressively more of the news; this kind of shortsighted tomfoolery with important info is helping to grow the already-burgeoning anti-AI-purists group
Terr_ 2y ago
I am continually surprised and depressed by how much blind trust is being placed in the accuracy and security of these LLM-mad-libs. I worry they'll do a lot of damage before attitudes catch up. In some cases it might not be blind trust, but a command from above: "It's good enough when I tried it, and we need to demonstrate that We Are An AI Company to investors..."
- Hamuko 2y ago
  My favourite is when companies replace their customer support with LLMs and then act like the LLMs are not their representatives and how they can't be held responsible for the shit they say. https://www.bbc.com/travel/article/20240222-air-canada-chatbot-misinformation-what-travellers-should-know https://www.bbc.com/travel/article/20240222-air-canada-chatb...
  - layer8 2y ago
    Maybe they replaced their legal department with LLMs as well.
PeterStuer 2y ago
"with no mention of it being AI-generated" is the problem. I use AI sumerization and translation in my products. It is not a problem as users are aware of that and the consequences that might have in terms of errors, and behave accordingly. Still saves them boatloads of time each day.
- pavel_lishin 2y ago
  But the folks affected here weren't aware of it. And in any case, replacing the subject line with a summary, is an absolutely insane thing to do. If it were an addition to the subject line? Sure. But replacing it is madness.
- consp 2y ago
  > and behave accordingly That is a big, a very big, assumption.
  - PeterStuer 2y ago
    In my case it works as the users are experts in their fields and thus (a) can spot errors quickly and (b) use the system as an information shortcut but never act without extensive verification with the full source.
    - eqvinox 2y ago
      If they can spot errors quickly (which implies they're reading the original anyway, since how do you spot errors otherwise), what is the benefit of the summaries to begin with?
      - PeterStuer 2y ago
        It answers the question 'which of these 100's of new articles and publications produced in the last few hours in 24 languages should I look into today. This is in very niche domains, so a 'that is probably not realy correct' stands out like a sore thumb.
lloeki 2y ago
> Subject: Your medical results are in! Test is negative > We regret to inform you that the test for grave condition X is positive.

bbor 2y ago

I think that’s the one and only sin, and it’s a serious one! Just mind boggling stuff. This is like the product launch to summarize headlines: why??? Certainly an interesting wrinkle to keep an eye on as AI takes up progressively more of the news; this kind of shortsighted tomfoolery with important info is helping to grow the already-burgeoning anti-AI-purists group

Terr_ 2y ago

I am continually surprised and depressed by how much blind trust is being placed in the accuracy and security of these LLM-mad-libs. I worry they'll do a lot of damage before attitudes catch up. In some cases it might not be blind trust, but a command from above: "It's good enough when I tried it, and we need to demonstrate that We Are An AI Company to investors..."

Hamuko 2y ago
My favourite is when companies replace their customer support with LLMs and then act like the LLMs are not their representatives and how they can't be held responsible for the shit they say. https://www.bbc.com/travel/article/20240222-air-canada-chatbot-misinformation-what-travellers-should-know https://www.bbc.com/travel/article/20240222-air-canada-chatb...
- layer8 2y ago
  Maybe they replaced their legal department with LLMs as well.

PeterStuer 2y ago

"with no mention of it being AI-generated" is the problem. I use AI sumerization and translation in my products. It is not a problem as users are aware of that and the consequences that might have in terms of errors, and behave accordingly. Still saves them boatloads of time each day.

pavel_lishin 2y ago
But the folks affected here weren't aware of it. And in any case, replacing the subject line with a summary, is an absolutely insane thing to do. If it were an addition to the subject line? Sure. But replacing it is madness.
consp 2y ago
> and behave accordingly That is a big, a very big, assumption.
- PeterStuer 2y ago
  In my case it works as the users are experts in their fields and thus (a) can spot errors quickly and (b) use the system as an information shortcut but never act without extensive verification with the full source.
  - eqvinox 2y ago
    If they can spot errors quickly (which implies they're reading the original anyway, since how do you spot errors otherwise), what is the benefit of the summaries to begin with?
    - PeterStuer 2y ago
      It answers the question 'which of these 100's of new articles and publications produced in the last few hours in 24 languages should I look into today. This is in very niche domains, so a 'that is probably not realy correct' stands out like a sore thumb.

lloeki 2y ago

> Subject: Your medical results are in! Test is negative > We regret to inform you that the test for grave condition X is positive.