3 ms·

Google used AI to summarize the ~200 questions its employees posed ahead of this meeting regarding its DEI and AI policies. The final two paragraphs of the art

Google used AI to summarize the ~200 questions its employees posed ahead of this meeting regarding its DEI and AI policies. The final two paragraphs of the article are telling: The third-most-popular question employees asked was why the AI summaries were so bad. “The AI summaries of questions on Ask are terrible. Can we go back to answering the questions people actually asked?” it read. I thought summarization was a task AI was considered good at. If their own tech sucks at this, what makes us think it's safe to use in the context of war?

kotaKat 2y ago
What, Googlers don't like AI summaries shoved down their throat? Too bad, it's a feature, and features can't be disabled! :)
- amarcheschi 2y ago
  You will eat your slop and you will enjoy it
funkychicken 2y ago
I actually felt bad for the poor bot :(
zombiwoof 2y ago
AI can’t do summaries but let’s unleash it on our military. Makes sense
whimsicalism 2y ago
AI has become political and Google is a very political company nowadays, i wouldn’t take this at face value
weikju 2y ago
> I thought summarization was a task AI was considered good at. I found this article was a good explanation why this isn’t the case: https://ea.rna.nl/2024/05/27/when-chatgpt-summarises-it-actually-does-nothing-of-the-kind/ https://ea.rna.nl/2024/05/27/when-chatgpt-summarises-it-actu... TLDR: it all comes back to the same principle: LLMs are predicting words based on their training. Which means they are not actually summarizing ideas from documents (or whatever other task we give them).
Aachen 2y ago
About half the casualties in any war are civilian (https://en.wikipedia.org/wiki/Civilian_casualty_ratio https://en.wikipedia.org/wiki/Civilian_casualty_ratio) > what makes us think it's safe to use in the context of war? Those in charge probably found that it performs on par but without paid time off or medical bills Frankly, I hadn't put this together yet until thinking on your comment. This is hecking scary if the country I live in ends up in a war now
Aurornis 2y ago
Summarization can be okay when letting you know the general content of a longer document or conversation. Summarization is universally disappointing when it tries to reduce a lot of different pieces of information (like answers to a lot of different questions) into a much smaller number of bullet points. It’s simply not possible to compress a lot of disparate pieces of information into a much smaller set of sentences.
sensanaty 2y ago
They're okay at best for getting the most generic possible vibe from a large piece of text. Actually relying on it to accurately summarize many points, it's completely useless. We have it enabled in our meetings and it's hilariously bad at summarizing even the simplest and shortest of meetings.

kotaKat 2y ago

What, Googlers don't like AI summaries shoved down their throat? Too bad, it's a feature, and features can't be disabled! :)

amarcheschi 2y ago
You will eat your slop and you will enjoy it

funkychicken 2y ago

I actually felt bad for the poor bot :(

zombiwoof 2y ago

AI can’t do summaries but let’s unleash it on our military. Makes sense

whimsicalism 2y ago

AI has become political and Google is a very political company nowadays, i wouldn’t take this at face value

weikju 2y ago

> I thought summarization was a task AI was considered good at. I found this article was a good explanation why this isn’t the case: https://ea.rna.nl/2024/05/27/when-chatgpt-summarises-it-actually-does-nothing-of-the-kind/ https://ea.rna.nl/2024/05/27/when-chatgpt-summarises-it-actu... TLDR: it all comes back to the same principle: LLMs are predicting words based on their training. Which means they are not actually summarizing ideas from documents (or whatever other task we give them).

Aachen 2y ago

About half the casualties in any war are civilian (https://en.wikipedia.org/wiki/Civilian_casualty_ratio https://en.wikipedia.org/wiki/Civilian_casualty_ratio) > what makes us think it's safe to use in the context of war? Those in charge probably found that it performs on par but without paid time off or medical bills Frankly, I hadn't put this together yet until thinking on your comment. This is hecking scary if the country I live in ends up in a war now

Aurornis 2y ago

Summarization can be okay when letting you know the general content of a longer document or conversation. Summarization is universally disappointing when it tries to reduce a lot of different pieces of information (like answers to a lot of different questions) into a much smaller number of bullet points. It’s simply not possible to compress a lot of disparate pieces of information into a much smaller set of sentences.

sensanaty 2y ago

They're okay at best for getting the most generic possible vibe from a large piece of text. Actually relying on it to accurately summarize many points, it's completely useless. We have it enabled in our meetings and it's hilariously bad at summarizing even the simplest and shortest of meetings.