6 ms·

I'm just going to put this out there: chatbots for search seem to be barking up the wrong tree, at least for the next few years. People expect searches for fac

I'm just going to put this out there: chatbots for search seem to be barking up the wrong tree, at least for the next few years. People expect searches for factual information to be inaccurate probably 1 in 10,000 times. A chatbot literally cannot guarantee anything better than about 1 in 50. Keyword matching plus authority scoring was unreasonably effective at this. There are so many other applications, but execs at Google and Microsoft are obsessed with re-creating "the computer" from Star Trek.

gonzo41 4y ago
I think going on an exciting journey with a really intelligent chatbot that is suffering from Dunning-Kruger effects is going spice up things on the web. Imagine searching for symptoms! What an emotional rollercoaster awaits.
- dahart 4y ago
  Dunning & Krueger’s paper showed that self-assessment correlates positively with performance, at least for the small number of tasks and undergrads they measured, so this joke doesn’t land the way you hoped. Apologies for the nit, but DK is almost universally misunderstood (and the authors are complicit in encouraging the widespread misunderstanding). That paper and its aftermath represents a lot of what’s wrong with social science and popular conceptions of it, so I take every opportunity I can to dissuade people from citing (or mis-citing) DK.
  - vba616 4y ago
    It's not really a citation. "Dunning-Krueger" has become shorthand for people not knowing what they don't know. Kleenex and Xerox sympathize.
    - dahart 4y ago
      It certainly is a citation. What makes you say that referring to a specific paper via the name of the authors isn’t citing it? That’s the definition of the word. It’s not a formal academic citation, but it is a citation. I’m well aware of the shorthand, it was the subtitle for the paper (“unskilled and unaware of it”), and that is more or less exactly what I’m talking about, because it’s not generally true. People also tend to claim that excessive confidence signals low competence in the name of DK, which may be implied in the comment above. Both of those concepts are basically incorrect, DK didn’t broadly measure what people know or don’t know, they used 4 narrow tests, one of which was subjective(!). Nor did they measure confidence, and when it comes to normal topics that people get college degrees in, followup research has shown the effect disappears and sometimes reverses [1]. The DK paper essentially measured how well people can guess their rank amongst a small set of peers (Cornell undergrads), which unsurprisingly isn’t perfect because it’s a guess. This is why DK is probably only poor methodology measuring a regression to the mean [1][2], and not uncovering anything interesting about human behavior. [1] https://www.talyarkoni.org/blog/2010/07/07/what-the-dunning-kruger-effect-is-and-isnt/ https://www.talyarkoni.org/blog/2010/07/07/what-the-dunning-... [2] https://en.wikipedia.org/wiki/Dunning–Kruger_effect#Statistical https://en.wikipedia.org/wiki/Dunning–Kruger_effect#Statisti... It’s just better to not mention DK in the first place, it’s not something you can observe on a single speaker outside of a statistical study, and it simply doesn’t apply to all topics and all people, even when one does understand what the paper did show.
      - vba616 4y ago
        >It certainly is a citation That, to me, is a statement about someone else's intent.
        dahart 4y ago
        I have no idea what you mean, would you like to elaborate?
        vba616 4y ago
        I believe a formal citation (APA style) looks vaguely like this: gonzo41. (2023, March 22). Bard uses a Hacker News comment as source to say that Bard has shut down (By cryptoz). Hacker News. Retrieved March 26, 2023, from https://news.ycombinator.com/item?id=35256223 Sorry if I screwed it up; that is just for the overall look. I'd agree that a citation doesn't have to be formal, but if it isn't, then the meaning of a passing mention of names has to be more inferred from context.
        dahart 4y ago
        Right, we’re not talking about a formal academic citation, so the format is irrelevant in this context. Why does it matter if the meaning has to be inferred? An informal citation or reference is still a citation or reference to something by speaker intent, and is not defined as such by whether or not the audience understands the reference, right? In any case, citations, references, mentions, and uses of the DK effect are almost always wrong. I’ve seen a lot of them on HN, almost always to imply someone is incompetent, and I’ve yet to see one that’s correct.
carrolldunham 4y ago
>People expect searches for factual information to be inaccurate probably 1 in 10,000 times. It's okay they've recently cooled this off with google's 'answers' or whatever they are called that are placed on top of the results which are almost never right
MBCook 4y ago
> People expect searches for factual information to be inaccurate probably 1 in 10,000 times. Do they? I saw someone recommend using ChatGPT to learn how something worked earlier today on HN. And there were replies that agreed. A ton of people seem excited about AI search. Google is bad enough without it at this point. But to me it feels like those helper chatbots that appeared everywhere a few years ago, redux. Sure they have a more realistic tone and can understand questions far better than basic pattern matching. But it still doesn’t work. I switched to DuckDuckGo about 2 months ago. The results seem about 95% as good as Google for most things, without all the ads and nonsense. I have a hard time seeing today’s AI, impressive as it is in many ways, making search results more accurate. If they wanted to do that I feel like Google has a ton of low hanging fruit to fix already instead of hasty “features” to try and not look “behind”.
- pclmulqdq 4y ago
  I just want to point out that ChatGPT is not an AI search product. It is a little like a "rubber duck" in my usage, and that is pretty helpful, but not something you lean on for factual research. Some other people use it as a copywriter or editor, which is another good use.
  - MBCook 4y ago
    I agree. But I seem to keep seeing people trying to use it like a search engine or a Q&A site where you can just ask it whatever you want to know. But it can’t do that. It can do a lot of other things, useful or just fun. But it’s not a virtual professor that knows the correct answer to any question.
    - codefreakxff 4y ago
      In some areas it does a good job of giving me a better answer than me checking 5-10 random stack overflow and blog posts with different answers over 20 years... But, it is completely incapable of saying "I don't know the answer to that" and will spew the wrong answer to anything. For fun I have arguments with ChatGPT that it is just making things up and it should just tell me "I don't know" and it will happily say "I'm sorry, you are right. Here is the right answer <insert made up fact>" and it will do so generating random garbage indefinitely. The problem is that I can't trust it to provide facts any better than asking my bull-shitting co-worker who is convinced he is right about everything and when called out says "yeah, that's what I was saying, <insert contradictory thing>"
TylerE 4y ago
I think people are way more skeptical than that - even non-technical users are onto content farms - but you are being generous by not mentioning the time horizon problem. A search engine which doesn't not know anything that's happened in the past 18 months is not useful.
Barrin92 4y ago
while that's true the bigger issue is that the conversational interface of chatbots is uniform so you as the user have no ability to at first glance get any indication of whether the source is trustworthy. When you Google and you get two links, one from the YT comment section and another one from the Stanford Encyclopaedia of Philosophy most people who want factual information have a decent idea what to click on. Even when these chatbots provide sources it seems like half the time it's just made up. Putting a game of telephone between you and the source with the guy in the middle being completely indifferent to reality by design just seems bad.

gonzo41 4y ago

I think going on an exciting journey with a really intelligent chatbot that is suffering from Dunning-Kruger effects is going spice up things on the web. Imagine searching for symptoms! What an emotional rollercoaster awaits.

dahart 4y ago
Dunning & Krueger’s paper showed that self-assessment correlates positively with performance, at least for the small number of tasks and undergrads they measured, so this joke doesn’t land the way you hoped. Apologies for the nit, but DK is almost universally misunderstood (and the authors are complicit in encouraging the widespread misunderstanding). That paper and its aftermath represents a lot of what’s wrong with social science and popular conceptions of it, so I take every opportunity I can to dissuade people from citing (or mis-citing) DK.
- vba616 4y ago
  It's not really a citation. "Dunning-Krueger" has become shorthand for people not knowing what they don't know. Kleenex and Xerox sympathize.
  - dahart 4y ago
    It certainly is a citation. What makes you say that referring to a specific paper via the name of the authors isn’t citing it? That’s the definition of the word. It’s not a formal academic citation, but it is a citation. I’m well aware of the shorthand, it was the subtitle for the paper (“unskilled and unaware of it”), and that is more or less exactly what I’m talking about, because it’s not generally true. People also tend to claim that excessive confidence signals low competence in the name of DK, which may be implied in the comment above. Both of those concepts are basically incorrect, DK didn’t broadly measure what people know or don’t know, they used 4 narrow tests, one of which was subjective(!). Nor did they measure confidence, and when it comes to normal topics that people get college degrees in, followup research has shown the effect disappears and sometimes reverses [1]. The DK paper essentially measured how well people can guess their rank amongst a small set of peers (Cornell undergrads), which unsurprisingly isn’t perfect because it’s a guess. This is why DK is probably only poor methodology measuring a regression to the mean [1][2], and not uncovering anything interesting about human behavior. [1] https://www.talyarkoni.org/blog/2010/07/07/what-the-dunning-kruger-effect-is-and-isnt/ https://www.talyarkoni.org/blog/2010/07/07/what-the-dunning-... [2] https://en.wikipedia.org/wiki/Dunning–Kruger_effect#Statistical https://en.wikipedia.org/wiki/Dunning–Kruger_effect#Statisti... It’s just better to not mention DK in the first place, it’s not something you can observe on a single speaker outside of a statistical study, and it simply doesn’t apply to all topics and all people, even when one does understand what the paper did show.
    - vba616 4y ago
      >It certainly is a citation That, to me, is a statement about someone else's intent.
      - dahart 4y ago
        I have no idea what you mean, would you like to elaborate?
        vba616 4y ago
        I believe a formal citation (APA style) looks vaguely like this: gonzo41. (2023, March 22). Bard uses a Hacker News comment as source to say that Bard has shut down (By cryptoz). Hacker News. Retrieved March 26, 2023, from https://news.ycombinator.com/item?id=35256223 Sorry if I screwed it up; that is just for the overall look. I'd agree that a citation doesn't have to be formal, but if it isn't, then the meaning of a passing mention of names has to be more inferred from context.
        dahart 4y ago
        Right, we’re not talking about a formal academic citation, so the format is irrelevant in this context. Why does it matter if the meaning has to be inferred? An informal citation or reference is still a citation or reference to something by speaker intent, and is not defined as such by whether or not the audience understands the reference, right? In any case, citations, references, mentions, and uses of the DK effect are almost always wrong. I’ve seen a lot of them on HN, almost always to imply someone is incompetent, and I’ve yet to see one that’s correct.

carrolldunham 4y ago

>People expect searches for factual information to be inaccurate probably 1 in 10,000 times. It's okay they've recently cooled this off with google's 'answers' or whatever they are called that are placed on top of the results which are almost never right

MBCook 4y ago

> People expect searches for factual information to be inaccurate probably 1 in 10,000 times. Do they? I saw someone recommend using ChatGPT to learn how something worked earlier today on HN. And there were replies that agreed. A ton of people seem excited about AI search. Google is bad enough without it at this point. But to me it feels like those helper chatbots that appeared everywhere a few years ago, redux. Sure they have a more realistic tone and can understand questions far better than basic pattern matching. But it still doesn’t work. I switched to DuckDuckGo about 2 months ago. The results seem about 95% as good as Google for most things, without all the ads and nonsense. I have a hard time seeing today’s AI, impressive as it is in many ways, making search results more accurate. If they wanted to do that I feel like Google has a ton of low hanging fruit to fix already instead of hasty “features” to try and not look “behind”.

pclmulqdq 4y ago
I just want to point out that ChatGPT is not an AI search product. It is a little like a "rubber duck" in my usage, and that is pretty helpful, but not something you lean on for factual research. Some other people use it as a copywriter or editor, which is another good use.
- MBCook 4y ago
  I agree. But I seem to keep seeing people trying to use it like a search engine or a Q&A site where you can just ask it whatever you want to know. But it can’t do that. It can do a lot of other things, useful or just fun. But it’s not a virtual professor that knows the correct answer to any question.
  - codefreakxff 4y ago
    In some areas it does a good job of giving me a better answer than me checking 5-10 random stack overflow and blog posts with different answers over 20 years... But, it is completely incapable of saying "I don't know the answer to that" and will spew the wrong answer to anything. For fun I have arguments with ChatGPT that it is just making things up and it should just tell me "I don't know" and it will happily say "I'm sorry, you are right. Here is the right answer <insert made up fact>" and it will do so generating random garbage indefinitely. The problem is that I can't trust it to provide facts any better than asking my bull-shitting co-worker who is convinced he is right about everything and when called out says "yeah, that's what I was saying, <insert contradictory thing>"

TylerE 4y ago

I think people are way more skeptical than that - even non-technical users are onto content farms - but you are being generous by not mentioning the time horizon problem. A search engine which doesn't not know anything that's happened in the past 18 months is not useful.

Barrin92 4y ago

while that's true the bigger issue is that the conversational interface of chatbots is uniform so you as the user have no ability to at first glance get any indication of whether the source is trustworthy. When you Google and you get two links, one from the YT comment section and another one from the Stanford Encyclopaedia of Philosophy most people who want factual information have a decent idea what to click on. Even when these chatbots provide sources it seems like half the time it's just made up. Putting a game of telephone between you and the source with the guy in the middle being completely indifferent to reality by design just seems bad.