3 ms·

I read through your entire article and the three main points I took away from it were also contained in the gpt4o summary I then generated to compare afterwards

I read through your entire article and the three main points I took away from it were also contained in the gpt4o summary I then generated to compare afterwards. So here's some empirical counter evidence. I would suggest a less strong but more plausible claim that GPT4o has trouble summarizing for longer form content outside the bounds of it's context window or something like a lossier attention mechanism is being used as a compromise for resource usage. Summary: https://chatgpt.com/share/21d81811-db45-4ac5-b3c7-b25a79b2bafe https://chatgpt.com/share/21d81811-db45-4ac5-b3c7-b25a79b2ba...

richardreeze 2y ago
> the three main points I took away from it were also contained in the gpt4o summary I think here's the key why some people like llm summaries while others don't. Two people can read an article and take away different points from it. If the llm summary contains the points you would have taken, you like it. If it didn't, you don't. (e.g. if I look at your kindle highlights for a book and compare them to mine, they'll be very different - this is why I find it hard to use a service like Blinkist - but I think a good llm like gpt-4o or claude 3.5 sonnet does as good a job as a wikipedia article would... Not sure what else people expect)
Swizec 2y ago
> I would suggest a less strong but more plausible claim that GPT4o has trouble summarizing for longer form content outside the bounds of it's context window or something like a lossier attention mechanism is being used as a compromise for resource usage. This extends to your human readers. One of the more useful AI tools I’ve built for myself is a little thingy that looks at a piece of writing and answers ”What point is this article making?”. If the AI gets it wrong, I know my readers will also misunderstand what I’m saying. Back to the drawing board. The problem with making a subtle point that hinges on 1 detail in a vast sea of text is that 80% of humans will also miss that 1 detail.
- timeon 2y ago
  How do you know that most readers will also miss it? Isn't this just optimising for the tool?
  - Swizec 2y ago
    15 years of publishing things online :)
nostrademons 2y ago
Interesting. I read through the ChatGPT summary first, it seemed very plausible, then I read the original and I kinda see the author's point. The ChatGPT summary basically did gloss over every important detail in the original - but then, the details weren't important for generating a summary. I think that one of the author's key premises is false: > To summarise, you need to understand what the paper is saying. A summary is not about the author or about the summarizer, it's about the reader. It's about picking up on the portions of the original work that will matter to the reader's estimation of whether it's important to read the full work. And that actually depends much more on context and how the work relates to other works than it does about the specific details contained in the work. For example, Betteridge's Law of Headlines [1] basically provides a summary of any article whose title ends in a question, that summary is just "No", and it does so by making an observation about the authors rather than the content of the articles (about which it's completely agnostic). It reminds me of the problems that plague AI sentiment analysis. Machine learning is actually very good at that task, but you top out at about 70% precision because humans top out at 70% precision on sentiment analysis. At best, people only agree with each other ~70% of the time when judging the sentiment of a piece. [1] https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...
- sebastiennight 2y ago
  - You're assuming that the goal of a summary is to decide whether one needs to read the full work. That's certainly one niche usage of a summary, but is definitely not the only one and is likely a minority of usage. Most summaries I've needed, from school to my career, were about not having to read the full work. - I would say Betteridge's Law of Headlines does not provide "No" as a summary to all of these articles. The accurate summary would be "[Title]? No.", since it seems fairly obvious that the word "No" is very light in information conveyed. On a personal note, I have to say I can't see the point in this current craze about summarizing everything. I never saw the point of those subscription programs which promise you'll "read" a book a week/a month because they'll send you a 5-min audio about the book. I think you're better off choosing one really awesome book a year and actually reading that. So I can't see how a (flawed or not) ChatGPT summary will provide any epiphany on the level you'd get from consuming fewer works, thoroughly.
Terr_ 2y ago
> I read through your entire article and the three main points I took away from it were also contained in the gpt4o summary I then generated to compare afterwards. So here's some empirical counter evidence. But evidence of what, precisely? How do we measure failure and what failure rate is sufficient? As is, would you be comfortable with doctors applying it to all of your medical records?
- Closi 2y ago
  The failure rate that is “sufficient” will obviously change based on the use case. No, I might not want AI summarising my medical records right now, but I might be quite happy for it to summarise a blog post for me.
- ianbutler 2y ago
  Evidence it can summarize something correctly. Medical records are not my bar though, nor would I be comfortable giving my medical records to 99% of the human race. The bar these tools need to meet for me is far more innocuous (code and apparently summarizing fluff articles) and two things can happen at the same time: 1. Models, and not to be understated, the engineering around them can improve until they pass the point we are comfortable using them in high stakes situations like medical records. 2. People's expectations come down until we settle on the agreed upon tasks where LLMs provide real value. My perspective is 1 will happen in the long term and 2 is what we should be focusing on right now to provide real immediate value with an eye on 1 so we reap the continued improvements. For 2 calibrating people's expectations are going to be messy and people rarely estimate the utility of something correctly in the early days, either over or under which is the reason for the mess. LLMs have been in the works for almost a decade (building on research that's been around since at least the 70s), but their utility has only been apparent for about two years. We're still super early days for that settling period.
__loam 2y ago
Doing one run of a stochastic system and holding it up like "see it works fine" is basically just citing an anecdote.
- ianbutler 2y ago
  I’d argue their evidence was just as weak. I also didn't say hey it works fine I said here is counter evidence. We probably need a fully rigorous study to conclude anything. Which is not what the author or myself did.

richardreeze 2y ago

> the three main points I took away from it were also contained in the gpt4o summary I think here's the key why some people like llm summaries while others don't. Two people can read an article and take away different points from it. If the llm summary contains the points you would have taken, you like it. If it didn't, you don't. (e.g. if I look at your kindle highlights for a book and compare them to mine, they'll be very different - this is why I find it hard to use a service like Blinkist - but I think a good llm like gpt-4o or claude 3.5 sonnet does as good a job as a wikipedia article would... Not sure what else people expect)

Swizec 2y ago

> I would suggest a less strong but more plausible claim that GPT4o has trouble summarizing for longer form content outside the bounds of it's context window or something like a lossier attention mechanism is being used as a compromise for resource usage. This extends to your human readers. One of the more useful AI tools I’ve built for myself is a little thingy that looks at a piece of writing and answers ”What point is this article making?”. If the AI gets it wrong, I know my readers will also misunderstand what I’m saying. Back to the drawing board. The problem with making a subtle point that hinges on 1 detail in a vast sea of text is that 80% of humans will also miss that 1 detail.

timeon 2y ago
How do you know that most readers will also miss it? Isn't this just optimising for the tool?
- Swizec 2y ago
  15 years of publishing things online :)

nostrademons 2y ago

Interesting. I read through the ChatGPT summary first, it seemed very plausible, then I read the original and I kinda see the author's point. The ChatGPT summary basically did gloss over every important detail in the original - but then, the details weren't important for generating a summary. I think that one of the author's key premises is false: > To summarise, you need to understand what the paper is saying. A summary is not about the author or about the summarizer, it's about the reader. It's about picking up on the portions of the original work that will matter to the reader's estimation of whether it's important to read the full work. And that actually depends much more on context and how the work relates to other works than it does about the specific details contained in the work. For example, Betteridge's Law of Headlines [1] basically provides a summary of any article whose title ends in a question, that summary is just "No", and it does so by making an observation about the authors rather than the content of the articles (about which it's completely agnostic). It reminds me of the problems that plague AI sentiment analysis. Machine learning is actually very good at that task, but you top out at about 70% precision because humans top out at 70% precision on sentiment analysis. At best, people only agree with each other ~70% of the time when judging the sentiment of a piece. [1] https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...

sebastiennight 2y ago
- You're assuming that the goal of a summary is to decide whether one needs to read the full work. That's certainly one niche usage of a summary, but is definitely not the only one and is likely a minority of usage. Most summaries I've needed, from school to my career, were about not having to read the full work. - I would say Betteridge's Law of Headlines does not provide "No" as a summary to all of these articles. The accurate summary would be "[Title]? No.", since it seems fairly obvious that the word "No" is very light in information conveyed. On a personal note, I have to say I can't see the point in this current craze about summarizing everything. I never saw the point of those subscription programs which promise you'll "read" a book a week/a month because they'll send you a 5-min audio about the book. I think you're better off choosing one really awesome book a year and actually reading that. So I can't see how a (flawed or not) ChatGPT summary will provide any epiphany on the level you'd get from consuming fewer works, thoroughly.

Terr_ 2y ago

> I read through your entire article and the three main points I took away from it were also contained in the gpt4o summary I then generated to compare afterwards. So here's some empirical counter evidence. But evidence of what, precisely? How do we measure failure and what failure rate is sufficient? As is, would you be comfortable with doctors applying it to all of your medical records?

Closi 2y ago
The failure rate that is “sufficient” will obviously change based on the use case. No, I might not want AI summarising my medical records right now, but I might be quite happy for it to summarise a blog post for me.
ianbutler 2y ago
Evidence it can summarize something correctly. Medical records are not my bar though, nor would I be comfortable giving my medical records to 99% of the human race. The bar these tools need to meet for me is far more innocuous (code and apparently summarizing fluff articles) and two things can happen at the same time: 1. Models, and not to be understated, the engineering around them can improve until they pass the point we are comfortable using them in high stakes situations like medical records. 2. People's expectations come down until we settle on the agreed upon tasks where LLMs provide real value. My perspective is 1 will happen in the long term and 2 is what we should be focusing on right now to provide real immediate value with an eye on 1 so we reap the continued improvements. For 2 calibrating people's expectations are going to be messy and people rarely estimate the utility of something correctly in the early days, either over or under which is the reason for the mess. LLMs have been in the works for almost a decade (building on research that's been around since at least the 70s), but their utility has only been apparent for about two years. We're still super early days for that settling period.

__loam 2y ago

Doing one run of a stochastic system and holding it up like "see it works fine" is basically just citing an anecdote.

ianbutler 2y ago
I’d argue their evidence was just as weak. I also didn't say hey it works fine I said here is counter evidence. We probably need a fully rigorous study to conclude anything. Which is not what the author or myself did.