3 ms·

Interesting. I used to think that having an article summarization tool, like GPT-4 (provided the above is done correctly), would be a really cool thing. But no

by js8 4y ago

Interesting. I used to think that having an article summarization tool, like GPT-4 (provided the above is done correctly), would be a really cool thing. But now I have some doubts. I am reading through Wolfram's article on LLMs, and I am enjoying it. However, if GPT-4 wrote a summary, it would probably just also said (or imply) that the main points of the article are pretty boring standard stuff. I would conclude it's not worth reading from the summary. So the summary doesn't really tell me whether it's worth a read or not. Even if those main points have been presented before, maybe Wolfram is making some new arguments, or interesting analogies, or just some good joke or observation. Taken to the extreme, AI could probably summarize many stories as "hero's journey". But that doesn't distinguish between a really well-written story and a really bad one.

nothrowaways 4y ago
The article on LLM has misconceptions
ElFitz 4y ago
That’s one of the issues with summaries. Not only are some written by writers who couldn’t write as well as the original author, but you also lose information, no matter what you do. Sometimes it’s just the form. Sometimes it’s fluff. But sometimes it’s actual information. And sometimes the form is (or is a part of) the point.
- maegul 4y ago
  My simple take on this is that summaries are for consumption or extraction in pursuit of a task essentially extraneous to the source material. Reading the original (properly) is for participating in the discourse, which entails the potential of writing a response in essentially the same medium and so continuing the discourse. Unfortunately much of our education with its emphasis on assessment leans hard on extraction and habituates us the same way. Shallow and attention seeking content online hasn’t helped at all, obviously. With universities, I always found it confusing the cognitive dissonance evinced by the attempts at signalling the importance of discourse through seminars/tutorials and Socratic methods while pretending we all weren’t reading the assessment schemes on day 1. When you read and pay attention to the original source, the extra information you get are contextual clues, biases and emphases … essentially insights into the cognitive structure behind the piece and how it might play out at more general levels or in response to future conditions. You capture more of the “thinking”, not just “the thought”, and so can adapt, continue and extend it.
  - ElFitz 4y ago
    I also see them as a way of filtering content. Sure, it could lead one to miss out on some relevant things. But there is just so much out there. Most of it mostly noise. Most of the rest, irrelevant to us individually. Being able to navigate through the fluff, to focus on what matters to you, no matter what it is, seems more valuable now than ever.
chrisco255 4y ago
I've seen it work well in Discord chats I've participated in, for summarizing the previous 30 minutes worth of dozens to hundreds of posts, and it works great for that purpose. The above is not a great summary of Wolfram's article, though, and not sure what params were provided to it.
- ziml77 4y ago
  Don't feed anything you wouldn't share on the open internet into ChatGPT! https://help.openai.com/en/articles/6783457-chatgpt-general-faq https://help.openai.com/en/articles/6783457-chatgpt-general-... > 6. Will you use my conversations for training? > Yes. Your conversations may be reviewed by our AI trainers to improve our systems.
SanderNL 4y ago
Using a raw "unprompted" GPT is a bit unsophicated and the result will be bland. It's basically - in these cases - limited to your creativity. Ask it to summarize only the most interesting arguments or analogies. You need to hone it into what it is that you are trying to get out of it. It's not a human, it doesn't adapt without you telling it to. You can turn it positive, negative, whatever you want. Ask it to make the article look silly and proceed to debunk its arguments and it will - probably successfully.
mejutoco 4y ago
I feel like we are asking too much of a summary. If a summary is a lossless jpeg, do we expect it to show the image more or less or do we expect it to tell us if the image is worth seeing? The summary is a tool. You can decide independently if the article is worth reading, based on other factors. For instance, some people might read it just because of who the author is. A historian might read it just based on the date. > So the summary doesn't really tell me whether it's worth a read or not. I do not see the problem. Maybe ask gpt4 if you should read the article. Or ask a fellow human. This thinking is paralyzing: is it worth listening to this album, or I better watch this series? I do not think that is a problem with gpt4 summarization ability.
- kuhewa 4y ago
  Personally, what I want from a summary is to know that the story follows the standard heros journey arc and maybe a couple details to know the setting etc, but then I want to know what sets it apart.

by js8 4y ago

nothrowaways 4y ago

The article on LLM has misconceptions

ElFitz 4y ago

That’s one of the issues with summaries. Not only are some written by writers who couldn’t write as well as the original author, but you also lose information, no matter what you do. Sometimes it’s just the form. Sometimes it’s fluff. But sometimes it’s actual information. And sometimes the form is (or is a part of) the point.

maegul 4y ago
My simple take on this is that summaries are for consumption or extraction in pursuit of a task essentially extraneous to the source material. Reading the original (properly) is for participating in the discourse, which entails the potential of writing a response in essentially the same medium and so continuing the discourse. Unfortunately much of our education with its emphasis on assessment leans hard on extraction and habituates us the same way. Shallow and attention seeking content online hasn’t helped at all, obviously. With universities, I always found it confusing the cognitive dissonance evinced by the attempts at signalling the importance of discourse through seminars/tutorials and Socratic methods while pretending we all weren’t reading the assessment schemes on day 1. When you read and pay attention to the original source, the extra information you get are contextual clues, biases and emphases … essentially insights into the cognitive structure behind the piece and how it might play out at more general levels or in response to future conditions. You capture more of the “thinking”, not just “the thought”, and so can adapt, continue and extend it.
- ElFitz 4y ago
  I also see them as a way of filtering content. Sure, it could lead one to miss out on some relevant things. But there is just so much out there. Most of it mostly noise. Most of the rest, irrelevant to us individually. Being able to navigate through the fluff, to focus on what matters to you, no matter what it is, seems more valuable now than ever.

chrisco255 4y ago

I've seen it work well in Discord chats I've participated in, for summarizing the previous 30 minutes worth of dozens to hundreds of posts, and it works great for that purpose. The above is not a great summary of Wolfram's article, though, and not sure what params were provided to it.

ziml77 4y ago
Don't feed anything you wouldn't share on the open internet into ChatGPT! https://help.openai.com/en/articles/6783457-chatgpt-general-faq https://help.openai.com/en/articles/6783457-chatgpt-general-... > 6. Will you use my conversations for training? > Yes. Your conversations may be reviewed by our AI trainers to improve our systems.

SanderNL 4y ago

Using a raw "unprompted" GPT is a bit unsophicated and the result will be bland. It's basically - in these cases - limited to your creativity. Ask it to summarize only the most interesting arguments or analogies. You need to hone it into what it is that you are trying to get out of it. It's not a human, it doesn't adapt without you telling it to. You can turn it positive, negative, whatever you want. Ask it to make the article look silly and proceed to debunk its arguments and it will - probably successfully.

mejutoco 4y ago

I feel like we are asking too much of a summary. If a summary is a lossless jpeg, do we expect it to show the image more or less or do we expect it to tell us if the image is worth seeing? The summary is a tool. You can decide independently if the article is worth reading, based on other factors. For instance, some people might read it just because of who the author is. A historian might read it just based on the date. > So the summary doesn't really tell me whether it's worth a read or not. I do not see the problem. Maybe ask gpt4 if you should read the article. Or ask a fellow human. This thinking is paralyzing: is it worth listening to this album, or I better watch this series? I do not think that is a problem with gpt4 summarization ability.

kuhewa 4y ago
Personally, what I want from a summary is to know that the story follows the standard heros journey arc and maybe a couple details to know the setting etc, but then I want to know what sets it apart.