4 ms·
This is dangerous because people that has no knowledge of the science would blindly trust whatever it summarizes. There is no way to verify, an example is if y
by m3kw9 4y ago
This is dangerous because people that has no knowledge of the science would blindly trust whatever it summarizes. There is no way to verify, an example is if you ask to summarize a book that you have understanding of the subject, at least you can sense some bs or open up the book at verify a few points. Here you would be at GPT3 mercy
- dekken_ 4y ago> This is dangerous because people that has no knowledge of the science would blindly trust whatever it summarizes. This says more about you than the hypothetical "people" you are talking about.
- m3kw9 4y agoRealy? The “You are projecting” replies. That’s cute
- dekken_ 4y agoYou're assuming, not projecting.
- kurisufag 4y agoI don't think so. The average person already thinks (Chat)GPT is an all-knowing AGI homework-solver, and the problem only worsens if you add the airs of "science" to the situation.
- dekken_ 4y agoI think we're in uncharted waters to some extent and should have some restraint about predictions.
- m3kw9 4y agoIf you have used the summary function of Gpt, it can be out right wrong but sound very plausible. With the amount of disinformation out there people that are interested in science but wants it the easy way is could make things worse. Imagine the summarized states the result show that certain meds give good results but without the right statistic context, it could be just marginally good, or even not statistically significant to the trained reader. Now they pass it off and start validating their own biases
- newswasboring 4y agoI'm so tired of all discussions on LLMs starting with "this is dangerous" or forms of this. First because at some point this discourages people from even attempting cool things with LLMs and second because it really stalls the discussion. We are all aware that there is a hallucination issue in LLMs, so what do we do about this? What's your proposal? If it's just "don't do it", I don't think that's useful. I think if we were following the true spirit of HN, we would be giving suggestions on what to change. Just a suggestion to add disclaimers everywhere would be better than these "it's dangerous" comments. Not everything needs to be perfect, for a hobby project not everything even needs to be working. These comments are just discouraging for no real reason. Edit: [*] 'we' in my comment here is indicating the HN community, not entirety of humanity.
- rat9988 4y agoWhile I agree with the sentiment, I don't think there is a "true" spirit of HN. His sentiment as much as yours are in its spirit.
- newswasboring 4y agoI don't know man, maybe I'm wrong. I've been using HN for over a decade now, I've found constructive criticism to be a huge part of the positive culture here. It can of course be selection bias as I try to read more of that stuff and collapse a thread as soon as it gets into less constructive direction. But on the other hand, this is what the guidelines say > Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something. I don't think these type of comments teach us anything. But these are guidelines and not rules for a reason.
- swatcoder 4y ago> I don't think these type of comments teach us anything. Comments like the one at the root of this thread teach people that the current/approximate generation of LLM’s are poorly suited for certain kinds of tasks and remind us that many people don’t yet seem to understand what their systemic limitations are. LLM’s are statistical models over their training data and aim responses mostly towards the most dense, data-rich, and redundant centers of their corpus. Summarizing novel, esoteric or or expert material is something they’re poorly suited for because it inherently has poor representation in that data. Scoping is very constructive feedback.
- bilsbie 4y agoThis already happens when the popular press reports on papers. Can’t be any worse.