5 ms·
There's a fundamental problem with all these "summary" tasks, and it's obvious from the disclaimer that's on all these AI products: "AI can be wrong, you must v
by ADeerAppeared 2y ago
There's a fundamental problem with all these "summary" tasks, and it's obvious from the disclaimer that's on all these AI products: "AI can be wrong, you must verify this".
A summary for which you must always read the un-summarized text is useless as a summary, this should be obvious to literally everyone, yet AI developers stick their heads in the sand about it because RAG lets them pretend AI is more useful than it actually is.
RAG is useless, just fucking let it go and have AI stay in it's lane.
- probably_wrong 2y agoThe solution is to feed both the text and the summary back to ChatGPT and ask it to identify any inconsistencies. If you repeat these steps over and over for enough iterations eventually you will run out of money and the problem will be moot anyway
- jasonsb 2y agoFair enough, but what if I have access to infinite capital? Will I reach AGI?
- bigcat12345678 2y agoYou are asking an meaningless question: If I have everything, will I have this one thing If you have everything, you have anything If you have infinite capital, then you have AGI
- drdeca 2y agoNo? They are asking if in the limit as amount of money spent in a particular way goes to infinity, whether AGI would thereby be achieved. They aren’t just asking “If I had infinite capital, would that be sufficient to achieve AGI by some method?”.
- Culonavirus 2y agoYour post is genuinely hilarious. Like perfect dry humour. > The solution is to feed both the text and the summary back to ChatGPT and ask it to identify any inconsistencies. Hmm... Sounds interesting, ... > If you repeat these steps over and over for enough iterations Oookay, continue... > eventually you will run out of money and the problem will be moot anyway Oh. :D
- moffkalast 2y agoThat's why you use a local model instead, that way you're out of money after buying the GPUs already and don't have to bother with the implementation taps temple
- deleted 2y ago[deleted]
- qeternity 2y agoThis is just a trust issue, which applies to pretty much any task where there is delegation. If you ask an intern to summarize some text, you trust them to do a half decent job. You're not going to re-read the original text. The hiring process is meant to filter out bad interns.
- worble 2y agoIf an intern gets it wrong, then you can sit down with them and teach them the correct process. Hopefully over time they get better, and once the trust is built you then stop being so involved. If they don't get any better, you find someone else to summarize for you. You can't go through this process with an AI, every time it's just a shot in the dark.
- bmicraft 2y agoWith a specific model (just like with an intern), you only need to evaluate their work a certain amount of times to decide whether they're doing their job good enough to leave them alone and continue without further supervision
- ADeerAppeared 2y ago> If you ask an intern to summarize some text, you trust them to do a half decent job Yes. And if I were to propose that we filter all information we consume through completel unqualified interns summarizing it, I'd be laughed out of the room. Yet that is the future all these AI firms seek to build.
- stuven 2y agoWhat would you say to the standard counterargument that most existing processes that AI might aim to augment or replace _already_ have a non-zero error rate? For example if I had a secretary, his summaries _could_ be wrong. Doesn't mean he's not a useful employee!
- __loam 2y agoThe classic humans do it too fallacy.
- signatoremo 2y agoAny specific rebuke in this case? Because it’s especially true for summarization, if someone doesn’t care enough to do a thorough job.
- CooCooCaCha 2y agoYeah it's asinine if you think about it for more than a few seconds. The implication is that there is no nuance. Humans are imperfect and AI is imperfect so therefore they are equivalent.
- j5155 2y agoIf that secretary’s summaries were as consistently wrong and unhelpful as those ChatGPT generates, they would be fired.
- jiggawatts 2y agoNo, they wouldn’t. I regularly work with a wide variety of project managers, product owners, secretaries, etc… I swear that most of them willfully misunderstand everything they’re told or sent in writing, invariably refusing to simply forward emails and instead insisting on rephrasing everything in terms they understand, also known as gibberish that only vaguely resembles English. All of them are still “gainfully” employed.
- Sharlin 2y agoSimple, that's a false equivalence argument that ignores not only error rates, but the quality of the errors made. https://en.wikipedia.org/wiki/False_equivalence https://en.wikipedia.org/wiki/False_equivalence
- deleted 2y ago[deleted]
- efilife 2y agoits*
- slibhb 2y ago> There's a fundamental problem with all these "summary" tasks, and it's obvious from the disclaimer that's on all these AI products: "AI can be wrong, you must verify this". > A summary for which you must always read the un-summarized text is useless as a summary, this should be obvious to literally everyone Nah, it's still useful if the summary is usually right or mostly right. At the limit, it's not even clear that something can be summarized perfectly. Consider that the alternative to reading the summary often isn't reading the entire text yourself. It's reading nothing. Also, in my experience, these tools often fail when it comes to questions with a definitive answer. E.g. if you pass them a lot of text and ask a detailed question with a very clear answer, they often get it wrong. But when your question is vague like "summarize the text," they're very useful.
- thwarted 2y agoit's still useful if the summary is usually right or mostly right. And you'd confirm that by having to read the unsummarized content. Thus useless. Consider that the alternative to reading the summary often isn't reading the entire text yourself. It's reading nothing. Reading an inaccurate summary is actually less useful than reading nothing. It's not like the utility of the summary is to exercise a reading muscle.
- jimkleiber 2y agoI agree. Heck, even reading clickbait headlines can cause adverse effects, even if they've summarized the text properly.
- temporarely 2y agoThis is correct. There is a social context to this matter. The advocates of 'almost right is most cases is ok no harm done' are ignoring the (likely) operational and utility context of these tools.
- slibhb 2y agoYou can just assume it's right or mostly right and move on with your life. We're not talking about designing spaceships here. Being wrong is allowed. > Reading an inaccurate summary is actually less useful than reading nothing. It's not like the utility of the summary is to exercise a reading muscle. No it's not. You're acting like these tools generately wildly inaccurate text. Even when they're wrong, they're mostly accurate. Almost everything people read is a mostly accurate summary, whether it's from some guy's article or wikipedia. We rarely go to the primary source. edit - as a concrete example, I did my taxes this year with help from ChatGPT. It was a big improvement over using Google or reading through the instructions myself. And if it was wrong, well, maybe I'll get a bill or a check in the mail, but that was always a possibility and making a mistake on your taxes isn't illegal.
- dhon_ 2y agoWhat would be an acceptable error rate for your use case? There are situations where AI is good enough, other cases where you need more accuracy, and others still where you should be reading the reference directly. AI is improving quickly though, and context windows will allow for summaries to be tailored to each end user.
- simonmysun 2y agoI am more curious why this disclaimer is missing on human backed products: "Human can be wrong, you must verify this". How do they overcome the fact that the information they provided can be wrong?
- brookst 2y agoCan human-generated summaries ever be wrong?
- sweca 2y agoAgreed that RAG, especially undeclared RAG, needs to go. It's misleading and causes hallucinations for the majority of tasks.
- rldjbpin 2y ago> RAG is useless, just fucking let it go and have AI stay in it's lane. imho a side effect of promoting RAG is that the vector search by itself (on chunks of documents) might be a good-enough thing for most people. if we create a system without the LLM-summarization part, it might be the best of both worlds. Alas, people actually don't care about that stuff.