4 ms·

I know hearing this gets old, however, please review sources outside of LLMs for accuracy. LLMs take a whole bunch off stuff from all over the internet and dist

by eek2121 10mo ago

I know hearing this gets old, however, please review sources outside of LLMs for accuracy. LLMs take a whole bunch off stuff from all over the internet and distill it down to something you can consume. Those sources include everything from reddit to a certain de-wormer that folks still think treats COVID (side note: I've a few long COVID victims in a support group I am in, and they are not happy about the disinfo that was spread, at any rate)...LLMs/"AI" does not and cannot innovate, it can only take all existing information it knows, mash it all together, and present you with a result according to what the model is trained on. I'm not against AI summaries being on HN, however, users should verify and cite sources so others can verify. However, I'm just a normal nerd that wants to fact check stuff. Perhaps I'm wrong in wanting to do this. We'll see.

brailsafe 10mo ago
> I'm not against AI summaries being on HN, however, users should verify and cite sources so others can verify. I don't see how they contribute anything to a discussion. Even a speculative comment organically produced is more worthwhile than feeding a slop machine back into itself. I don't go out for coffee to discuss LLM summaries with friends, and I can't imagine why anyone would want to do that here. Earlier today I asked Gemini Pro to find information on a person's death that was turning up nothing for me otherwise, and it just imagined finding verbatim Obituary quotes in every source, cobbled together vaguely related names, plausible bits and pieces from wherever, almost like it was 2023 again. It ain't search, and it ain't worthwhile; I'd much rather someone ask an llm the question and then post a question out of curiosity based on it, but without the summary itself
- xupybd 10mo ago
  It is search if you ask it to produce a list of links. It does well at filtering information for you. Going to primary sources is required to verify what it says but it can reduce the leg work rather a lot.
  - roarcher 10mo ago
    > It is search if you ask it to produce a list of links. Unfortunately it can hallucinate those too. I've had ChatGPT cite countless nonexistent academic papers, complete with links that go nowhere.
    - tharkun__ 10mo ago
      Which is "fine" so to speak. We do this with using AIs for coding all the time, don't we? As in, we ask it to do things or tell us things about our code base (which we might be new to as well) but essentially use it as a "search engine+" so to speak. Hopefully it's faster and can provide some sort of understanding faster than we could with searching ourselves and building a mental model while doing it. But we still need to ask it for and then follow file and line number references (aka "links") and verify it's true and it got the references right and build enough of a mental model ourselves. With code (at least for our code base) it usually does get that right (the references) and I can verify. I might be biased because I both know our code base very well already (but not everything in detail) and I'm a very suspicious person, questioning everything. With humans it sometimes "drives them crazy" but the LLM doesn't mind when I call its BS over and over. I'm always "right" :P The problem is when you just trust anything it says. I think we need to treat it like a super junior that's trained to very convincingly BS you if it's out of its depth. But it's still great to have said junior do your bidding while you do other things and faster than an actual junior and this junior is available 24/7 (barring any outages ;)).
  - anjel 10mo ago
    Ask it to solve a tough Euler Math puzzle with the search button on and it just copies the answer from the web. Turn search off and it actually computes the answer. Funny how the search button is taken away though.
  - duskdozer 10mo ago
    The issue as I see it is just straight copy/pasting its output. You want to use it as a search tool to give you pointers on things to look up and links to read? Great. Then use that as a basis to read the sources and write your own response. If you aren't familiar enough with the subject area to do that, then you also shouldn't be pasting LLM output on it.
DivingForGold 10mo ago
I have significant experience in polymer chemistry, as an experiment, I decided to ask gemini some very specific questions to try and back it into a corner, so to speak. It blew me away with the answer, discussing quite a bit of info I was not even aware of.
- jijijijij 10mo ago
  Cool story, bro.
justinc8687 10mo ago
I've had quite good luck asking Gemini and ChatGPT to include links to research papers for every claim they make. Not only can I review at least the abstracts but I find when I do this, they'll retract some of the hallucinations they've have made in prior messages. It almost seems (and maybe they do) in their web searching tools, reread the content they include. Thus, greatly reducing errors, with minimal extra effort on my part.

by eek2121 10mo ago

brailsafe 10mo ago

> I'm not against AI summaries being on HN, however, users should verify and cite sources so others can verify. I don't see how they contribute anything to a discussion. Even a speculative comment organically produced is more worthwhile than feeding a slop machine back into itself. I don't go out for coffee to discuss LLM summaries with friends, and I can't imagine why anyone would want to do that here. Earlier today I asked Gemini Pro to find information on a person's death that was turning up nothing for me otherwise, and it just imagined finding verbatim Obituary quotes in every source, cobbled together vaguely related names, plausible bits and pieces from wherever, almost like it was 2023 again. It ain't search, and it ain't worthwhile; I'd much rather someone ask an llm the question and then post a question out of curiosity based on it, but without the summary itself

xupybd 10mo ago
It is search if you ask it to produce a list of links. It does well at filtering information for you. Going to primary sources is required to verify what it says but it can reduce the leg work rather a lot.
- roarcher 10mo ago
  > It is search if you ask it to produce a list of links. Unfortunately it can hallucinate those too. I've had ChatGPT cite countless nonexistent academic papers, complete with links that go nowhere.
  - tharkun__ 10mo ago
    Which is "fine" so to speak. We do this with using AIs for coding all the time, don't we? As in, we ask it to do things or tell us things about our code base (which we might be new to as well) but essentially use it as a "search engine+" so to speak. Hopefully it's faster and can provide some sort of understanding faster than we could with searching ourselves and building a mental model while doing it. But we still need to ask it for and then follow file and line number references (aka "links") and verify it's true and it got the references right and build enough of a mental model ourselves. With code (at least for our code base) it usually does get that right (the references) and I can verify. I might be biased because I both know our code base very well already (but not everything in detail) and I'm a very suspicious person, questioning everything. With humans it sometimes "drives them crazy" but the LLM doesn't mind when I call its BS over and over. I'm always "right" :P The problem is when you just trust anything it says. I think we need to treat it like a super junior that's trained to very convincingly BS you if it's out of its depth. But it's still great to have said junior do your bidding while you do other things and faster than an actual junior and this junior is available 24/7 (barring any outages ;)).
- anjel 10mo ago
  Ask it to solve a tough Euler Math puzzle with the search button on and it just copies the answer from the web. Turn search off and it actually computes the answer. Funny how the search button is taken away though.
- duskdozer 10mo ago
  The issue as I see it is just straight copy/pasting its output. You want to use it as a search tool to give you pointers on things to look up and links to read? Great. Then use that as a basis to read the sources and write your own response. If you aren't familiar enough with the subject area to do that, then you also shouldn't be pasting LLM output on it.

DivingForGold 10mo ago

I have significant experience in polymer chemistry, as an experiment, I decided to ask gemini some very specific questions to try and back it into a corner, so to speak. It blew me away with the answer, discussing quite a bit of info I was not even aware of.

jijijijij 10mo ago
Cool story, bro.

justinc8687 10mo ago

I've had quite good luck asking Gemini and ChatGPT to include links to research papers for every claim they make. Not only can I review at least the abstracts but I find when I do this, they'll retract some of the hallucinations they've have made in prior messages. It almost seems (and maybe they do) in their web searching tools, reread the content they include. Thus, greatly reducing errors, with minimal extra effort on my part.