4 ms·

For “deep research” I’m also reading “getting the answers right”. Most people I talk to are at the point now where getting completely incorrect answers 10% of

For “deep research” I’m also reading “getting the answers right”. Most people I talk to are at the point now where getting completely incorrect answers 10% of the time — either obviously wrong from common sense, or because the answers are self contradictory — undermines a lot of trust in any kind of interaction. Other than double checking something you already know, language models aren’t large enough to actually know everything. They can only sound like they do. What I’m looking for is therefore not just the correct answer, but the correct answer in an amount of time that’s faster than it would take me to research the answer myself, and also faster than it takes me to verify the answer given by the machine. It’s one thing to ask a pupil to answer an exam paper to which you know the answers. It’s a whole next level to have it answer questions to which you don’t know the answers, and on whose answers you are relying to be correct.

sandos 2y ago
I mean this all falls down due to the need of verification: "Limitations Deep research unlocks significant new capabilities, but it’s still early and has limitations. It can sometimes hallucinate facts in responses or make incorrect inferences" How do I know which parts are false? It will take as long to verify as to research!
- wnurmi 2y ago
  [dead]
igleria 2y ago
It's really worrying to me, even as a self proclaimed "LLM <-> AI" skeptic, to see what kind of stuff people pretend to get out from an LLM. Typewriter monkeys as a service almost. Still useful for the odd task here and there, but not as useful as all the money being invested in this (except for the companies getting that money, that is). edit: actual example of something I'd expect a real AI to be able to solve by itself, but currently LLMs fail miserably https://x.com/RadishHarmers/status/1885884032220643587 https://x.com/RadishHarmers/status/1885884032220643587
- mdp2021 2y ago
  > Typewriter monkeys as a service almost. // Still useful for the odd task here and there 1) Paramount task: searching in naturally structured language, as opposed to keywords. Odd tasks: oh yes, several tasks of fuzzy sophisticated text processing previously unsolved. 2) They translate NN encodings in natural language! The issue remains about the quality of /what/ they translate in natural language, but one important chunk of the problem* is in a way solved... Now, I've been probably one of the most vocal here, shouting "That's the opposite of intelligence!" - even in the past 24 hours -, but be objective: there are also progresses ... (* Around five years ago we were still stuck with Hinton's problem of interpreting pronouns as pointers in "the item won't fit in the case: it's too big" vs "the item won't fit in the case: it's too small" - look at it now...)
  - igleria 2y ago
    Of course I see progress, but I feel like the bridge of "thinks" versus "regurgitates" is still far off, if it is still in the horizon with the current approach. IMHO. edit: furthermore, LLMs probably tackle very little "real state" in the "make machines THINK" land. But a crucial piece on the overall puzzle.
chombier 2y ago
> and also faster than it takes me to verify the answer given by the machine. I always thought there was a kind of NP-flavor to the problems for which LLMs-like AI are helpful in practice, in the sense that solving the problem may be hard but checking the solution must be fast. Unless the domain can accomodate errors/hallucination, checking the solution (by a human) should be exponentially faster than finding it (by some AI) otherwise there's little practical gain.
jeswin 2y ago
> Most people I talk to are at the point now where getting completely incorrect answers 10% of the time A year back that number was 30%, and a couple of years back it was 60%. There will be a point where it'll be good enough. There are also better and better ways to verify answers these days. It'll never be a solution for everything, but that's similar to many engineering problems we have: for example, ORMs aren't great for all types of queries, but they're sufficient for a good part of them.
- dimitri-vs 2y ago
  It contributes little to discuss a hypothetical future. Maybe we'll have fusion energy, delivery drones, everyone using VR, etc. Maybe we will go into a deep recession due to trade wars, or maybe not. The meaningful discussion is about how they perform NOW and the edge cases that have persisted since GPT-2 which no one has yet found a good solution for.
  - infecto 2y ago
    We already have delivery drones though. I disagree though, it is useful as this problem has been whittled down and I think there is expectation that there will be continued effort. Its of course worth discussing but I find that for my workflows, I rarely encounter issues with hallucinations, they certainly exist but its gotten to a point that I don't have major issue with it.
    - skywhopper 2y ago
      At best, a proof of concept of experimental delivery drones exist, but only for small, lightweight items, and only in a few places, only in the right weather, and only if you place a target on your driveway and are there to receive the item in person, and all at the cost of a very high noise level. That's not exactly a real service.
      - infecto 2y ago
        You are sort of moving the goal post. The fact remains, drone delivery exists and is a solved problem. Major metro areas like Dallas, Texas have it through retailers like Walmart. Just because it does not meet your specific goal post does not mean it's a proof of concept.
herculity275 2y ago
My worry is that all these recent capabilities attempt to minimize hallucinations by relying on extensive web search, however web itself is being actively degraded by unfiltered LLM output. After a certain point running your research agent against a ~5-year-old snapshot of the web will be strictly more accurate (for non-current affairs queries) than querying live web.
shakes_mcjunkie 2y ago
> What I’m looking for is therefore not just the correct answer, but the correct answer in an amount of time that’s faster than it would take me to research the answer myself, and also faster than it takes me to verify the answer given by the machine. This is why I haven't found AI tools very useful. I find my self spending more time verifying and fixing it's answers than I would have just doing or learning the darn thing myself.
- 7thpower 2y ago
  It is added cognitive load, but there is a lot of value in async tasks if you can trust the output or if the opportunity cost of validating is low. The challenge with something like this for research, in its current state, is you’ll need to go double check it because you don’t trust it and it will end up effectively being a list of links. It’s progress though and evidently good enough to find a sweet NSX in Japan, which is all some really need.
squigz 2y ago
> They can only sound like they do. More importantly, I think, is that they are incapable of not doing so. Have we figured out how to make an LLM realize and answer that it doesn't know an answer?
HarHarVeryFunny 2y ago
> language models aren’t large enough to actually know everything I'd say they don't know anything. An LLM base model, before it is post-trained with RL, just has access to a sliced and diced corpus of human output. Take the contents of 4chan and WikiPedia, put in blender and mix and chop into "training sample" sized bites, then learn the statistical regularities of this blended mess. It is what it is - not exactly what I'd call a knowledge base, even though there are bits of knowledge in there. When you add RL-based post-training for reasoning, all you are doing is trying get the model to be more selective when you are sampling from it - encouraging it to suppress some statistics, and emphazise others, such that when you sample from it the output looks more like valid reasoning steps and/or conclusions, per the verified reasoning examples you train it on. I'm well aware of how useful RL-tuned models (whatever the goal) can be, but at the end of the day all they are doing is taking a statistical babbler and saying "try to output patterns more like this". It's not exactly a recipe for factuality or rationality - we've just gone from hallucination-prone base models, to gaslighting-prone RL-tuned "reasoning" models that output stuff that sounds like coherent reasoning. What missing from all of this - what makes it different from how animals learn - it that the model has no experience of it's own, no autonomy or motivation to explore, learn and verify, and hence no episodic memories of how it learnt something (tried it and ran controlled experiments, or just overheard it on the bus), and what that implies about it's trustworthiness. It's amazing that LLMs work as well as they do, a reflection of how much of what we do can be accomplished by reactive pattern matching, but if you want to go beyond that to something that can learn and discern the truth for itself, this seems the wrong paradigm altogether.
- southernplaces7 2y ago
  The amusing thing is that despite all you describe, and which many other people have described on this site and in others in great detail about the nature of LLMs, there are still many people who believe that these models essentially possess intelligence, and that it's little different from how a human mind expresses cognition. I've seen this thinking to be especially prevalent among tech types (especially among many comments on this site), and more so than among average non-tech people I know. It seems to be a reflection of a certain forced, almost ideological techno-reductionist thinking against the honestly complex and largely mysterious nature of consciousness. Many non-tech people on the other hand accept this mystery of consciousness and paradoxically are thus less likely to consider an LLM to be anything deeper than the clever but mindless pattern-matching trick that it is

sandos 2y ago

I mean this all falls down due to the need of verification: "Limitations Deep research unlocks significant new capabilities, but it’s still early and has limitations. It can sometimes hallucinate facts in responses or make incorrect inferences" How do I know which parts are false? It will take as long to verify as to research!

wnurmi 2y ago
[dead]

igleria 2y ago

It's really worrying to me, even as a self proclaimed "LLM <-> AI" skeptic, to see what kind of stuff people pretend to get out from an LLM. Typewriter monkeys as a service almost. Still useful for the odd task here and there, but not as useful as all the money being invested in this (except for the companies getting that money, that is). edit: actual example of something I'd expect a real AI to be able to solve by itself, but currently LLMs fail miserably https://x.com/RadishHarmers/status/1885884032220643587 https://x.com/RadishHarmers/status/1885884032220643587

mdp2021 2y ago
> Typewriter monkeys as a service almost. // Still useful for the odd task here and there 1) Paramount task: searching in naturally structured language, as opposed to keywords. Odd tasks: oh yes, several tasks of fuzzy sophisticated text processing previously unsolved. 2) They translate NN encodings in natural language! The issue remains about the quality of /what/ they translate in natural language, but one important chunk of the problem* is in a way solved... Now, I've been probably one of the most vocal here, shouting "That's the opposite of intelligence!" - even in the past 24 hours -, but be objective: there are also progresses ... (* Around five years ago we were still stuck with Hinton's problem of interpreting pronouns as pointers in "the item won't fit in the case: it's too big" vs "the item won't fit in the case: it's too small" - look at it now...)
- igleria 2y ago
  Of course I see progress, but I feel like the bridge of "thinks" versus "regurgitates" is still far off, if it is still in the horizon with the current approach. IMHO. edit: furthermore, LLMs probably tackle very little "real state" in the "make machines THINK" land. But a crucial piece on the overall puzzle.

chombier 2y ago

> and also faster than it takes me to verify the answer given by the machine. I always thought there was a kind of NP-flavor to the problems for which LLMs-like AI are helpful in practice, in the sense that solving the problem may be hard but checking the solution must be fast. Unless the domain can accomodate errors/hallucination, checking the solution (by a human) should be exponentially faster than finding it (by some AI) otherwise there's little practical gain.

jeswin 2y ago

> Most people I talk to are at the point now where getting completely incorrect answers 10% of the time A year back that number was 30%, and a couple of years back it was 60%. There will be a point where it'll be good enough. There are also better and better ways to verify answers these days. It'll never be a solution for everything, but that's similar to many engineering problems we have: for example, ORMs aren't great for all types of queries, but they're sufficient for a good part of them.

dimitri-vs 2y ago
It contributes little to discuss a hypothetical future. Maybe we'll have fusion energy, delivery drones, everyone using VR, etc. Maybe we will go into a deep recession due to trade wars, or maybe not. The meaningful discussion is about how they perform NOW and the edge cases that have persisted since GPT-2 which no one has yet found a good solution for.
- infecto 2y ago
  We already have delivery drones though. I disagree though, it is useful as this problem has been whittled down and I think there is expectation that there will be continued effort. Its of course worth discussing but I find that for my workflows, I rarely encounter issues with hallucinations, they certainly exist but its gotten to a point that I don't have major issue with it.
  - skywhopper 2y ago
    At best, a proof of concept of experimental delivery drones exist, but only for small, lightweight items, and only in a few places, only in the right weather, and only if you place a target on your driveway and are there to receive the item in person, and all at the cost of a very high noise level. That's not exactly a real service.
    - infecto 2y ago
      You are sort of moving the goal post. The fact remains, drone delivery exists and is a solved problem. Major metro areas like Dallas, Texas have it through retailers like Walmart. Just because it does not meet your specific goal post does not mean it's a proof of concept.

herculity275 2y ago

My worry is that all these recent capabilities attempt to minimize hallucinations by relying on extensive web search, however web itself is being actively degraded by unfiltered LLM output. After a certain point running your research agent against a ~5-year-old snapshot of the web will be strictly more accurate (for non-current affairs queries) than querying live web.

shakes_mcjunkie 2y ago

> What I’m looking for is therefore not just the correct answer, but the correct answer in an amount of time that’s faster than it would take me to research the answer myself, and also faster than it takes me to verify the answer given by the machine. This is why I haven't found AI tools very useful. I find my self spending more time verifying and fixing it's answers than I would have just doing or learning the darn thing myself.

7thpower 2y ago
It is added cognitive load, but there is a lot of value in async tasks if you can trust the output or if the opportunity cost of validating is low. The challenge with something like this for research, in its current state, is you’ll need to go double check it because you don’t trust it and it will end up effectively being a list of links. It’s progress though and evidently good enough to find a sweet NSX in Japan, which is all some really need.

squigz 2y ago

> They can only sound like they do. More importantly, I think, is that they are incapable of not doing so. Have we figured out how to make an LLM realize and answer that it doesn't know an answer?

HarHarVeryFunny 2y ago

> language models aren’t large enough to actually know everything I'd say they don't know anything. An LLM base model, before it is post-trained with RL, just has access to a sliced and diced corpus of human output. Take the contents of 4chan and WikiPedia, put in blender and mix and chop into "training sample" sized bites, then learn the statistical regularities of this blended mess. It is what it is - not exactly what I'd call a knowledge base, even though there are bits of knowledge in there. When you add RL-based post-training for reasoning, all you are doing is trying get the model to be more selective when you are sampling from it - encouraging it to suppress some statistics, and emphazise others, such that when you sample from it the output looks more like valid reasoning steps and/or conclusions, per the verified reasoning examples you train it on. I'm well aware of how useful RL-tuned models (whatever the goal) can be, but at the end of the day all they are doing is taking a statistical babbler and saying "try to output patterns more like this". It's not exactly a recipe for factuality or rationality - we've just gone from hallucination-prone base models, to gaslighting-prone RL-tuned "reasoning" models that output stuff that sounds like coherent reasoning. What missing from all of this - what makes it different from how animals learn - it that the model has no experience of it's own, no autonomy or motivation to explore, learn and verify, and hence no episodic memories of how it learnt something (tried it and ran controlled experiments, or just overheard it on the bus), and what that implies about it's trustworthiness. It's amazing that LLMs work as well as they do, a reflection of how much of what we do can be accomplished by reactive pattern matching, but if you want to go beyond that to something that can learn and discern the truth for itself, this seems the wrong paradigm altogether.

southernplaces7 2y ago
The amusing thing is that despite all you describe, and which many other people have described on this site and in others in great detail about the nature of LLMs, there are still many people who believe that these models essentially possess intelligence, and that it's little different from how a human mind expresses cognition. I've seen this thinking to be especially prevalent among tech types (especially among many comments on this site), and more so than among average non-tech people I know. It seems to be a reflection of a certain forced, almost ideological techno-reductionist thinking against the honestly complex and largely mysterious nature of consciousness. Many non-tech people on the other hand accept this mystery of consciousness and paradoxically are thus less likely to consider an LLM to be anything deeper than the clever but mindless pattern-matching trick that it is