3 ms·

This is all stuff I would expect an LLM to "hallucinate" about. Every bit of it.

steveklabnik 1y ago
These tools can search the web to find this kind of data, and show you what they searched. Double checking is essential because hallucinations are still possible, but it's not like in the past where it would just try to make up the data from its training set. That said, it also may find bad data and give you a summary of that, which isn't a direct hallucination, but can still be inaccurate. This is why checking the sources is helpful too.
- majormajor 1y ago
  I wouldn't expect it to hallucinate but how do you evaluate it's ability to distinguish spam from good info? I.e. the "the first four pages of google results is all crap nowdays" problem.
  - steveklabnik 1y ago
    By looking at the pages it looked at and deciding for yourself, just like you would with a web search you invoked yourself. I’ve generally found it to use trustworthy stuff like Stack Overflow, Wikipedia, and university websites. But I also haven’t used it in this way that much or for very serious things. I’d imagine more obscure questions are more likely to end up involving less trustworthy sites.
    - leptons 1y ago
      >By looking at the pages it looked at and deciding for yourself, just like you would with a web search you invoked yourself. Or you could just cut out the middleman(bot) and just do the search yourself, since you're going to have to anyway to verify what the "AI" wrote. It's just all so stupid that society is rushing towards this iffy-at-best technology when we still need to do the same work anyway to verify it isn't bullshitting us. Ugh, I hate this timeline.
      - steveklabnik 1y ago
        You could. What's correct depends on your needs. Search engines aren't useless because you have to make sure that the pages they return to you are quality, and so neither is using an agent in this way.
- leptons 1y ago
  This is why I won't use these "AI" tools, because they obfuscate the bullshit. Since I have to check what it said anyway, it's easier if I just do a search and see where the info is coming from, and I can tell good from bad sources really easily. Putting an LLM in front off it is like polishing a turd - garbage in, garbage out.
thelock85 1y ago
I recently tried a version of this landscape analysis within a space I understand very well (CA college access nonprofits) and was shocked at how few organizations were named, let alone described in detail. Even worse, the scope and reach of the named orgs were pretty off the mark. My best guess is that they were the SEO winners of the past.
vessenes 1y ago
Well you’d be wrong in this case: Deep research will trigger a series of web searches first then reach out to tooling for follow ups as needed; most of the facts will be grounded in the sources it finds. With no deep research - agreed; too recent to believe info is accurately stored in the model weights.

These tools can search the web to find this kind of data, and show you what they searched. Double checking is essential because hallucinations are still possible, but it's not like in the past where it would just try to make up the data from its training set. That said, it also may find bad data and give you a summary of that, which isn't a direct hallucination, but can still be inaccurate. This is why checking the sources is helpful too.

majormajor 1y ago
I wouldn't expect it to hallucinate but how do you evaluate it's ability to distinguish spam from good info? I.e. the "the first four pages of google results is all crap nowdays" problem.
- steveklabnik 1y ago
  By looking at the pages it looked at and deciding for yourself, just like you would with a web search you invoked yourself. I’ve generally found it to use trustworthy stuff like Stack Overflow, Wikipedia, and university websites. But I also haven’t used it in this way that much or for very serious things. I’d imagine more obscure questions are more likely to end up involving less trustworthy sites.
  - leptons 1y ago
    >By looking at the pages it looked at and deciding for yourself, just like you would with a web search you invoked yourself. Or you could just cut out the middleman(bot) and just do the search yourself, since you're going to have to anyway to verify what the "AI" wrote. It's just all so stupid that society is rushing towards this iffy-at-best technology when we still need to do the same work anyway to verify it isn't bullshitting us. Ugh, I hate this timeline.
    - steveklabnik 1y ago
      You could. What's correct depends on your needs. Search engines aren't useless because you have to make sure that the pages they return to you are quality, and so neither is using an agent in this way.
leptons 1y ago
This is why I won't use these "AI" tools, because they obfuscate the bullshit. Since I have to check what it said anyway, it's easier if I just do a search and see where the info is coming from, and I can tell good from bad sources really easily. Putting an LLM in front off it is like polishing a turd - garbage in, garbage out.

thelock85 1y ago

I recently tried a version of this landscape analysis within a space I understand very well (CA college access nonprofits) and was shocked at how few organizations were named, let alone described in detail. Even worse, the scope and reach of the named orgs were pretty off the mark. My best guess is that they were the SEO winners of the past.

vessenes 1y ago

Well you’d be wrong in this case: Deep research will trigger a series of web searches first then reach out to tooling for follow ups as needed; most of the facts will be grounded in the sources it finds. With no deep research - agreed; too recent to believe info is accurately stored in the model weights.