9 ms·

If that's true, why then did it take me 10 seconds yesterday to find an extremely obscure fact of 18th century history with ChatGPT 3.5? I tried Google queries

by lqet 3y ago

If that's true, why then did it take me 10 seconds yesterday to find an extremely obscure fact of 18th century history with ChatGPT 3.5? I tried Google queries for over an hour, without success. ChatGPT first gave a wrong answer (because my question actually wasn't accurate), but after some clarification answered something that appeared correct, and after 1-2 minutes of further research and validation, I was 100% that it was indeed the correct answer. > AI systems like ChatGPT are trained with text from Twitter, Facebook, Reddit, and other huge archives of bullshit, alongside plenty of actual facts (including Wikipedia and text ripped off from professional writers). But there is no algorithm in ChatGPT to check which parts are true. The output is literally bullshit. Well. Any philosopher, mathematician, scientist, or novelist was trained on plenty of actual facts (established works of other people, school learning, experiments) and huge archives of bullshit (individual day-to-day experiences that cannot be called scientific, casual conversations just like Twitter conversations, propaganda, lies). Their output then would also be bullshit, and by extension, everything humanity ever created.

mnd999 3y ago
> ChatGPT first gave a wrong answer I think you answered your own question there. It doesn’t say your question makes no sense, or that’s incorrect. It just makes up a bullshit answer.
- exitb 3y ago
  Most tools require some preparation, knowledge and understanding before you get good results. Would he be better off not getting the right answer in the subsequent question?
  - falserum 3y ago
    It’s not all or nothing. Most of negative sentiment is not “world would be better without chatgpt“, but “one must be very cautious and always validate the answer”. It’s valuable tool, but it has a LOT of hype, which led to things like people trusting chatgpt output in a court case.
- empath-nirvana 3y ago
  GPT4 will definitely tell you that your question is wrong and doesn't make sense, and a lot of times it is right about that.
chrisvalleybay 3y ago
Exactly. It's great for these use cases. I often use it to find things of which I barely remember some obsure detail and mostly I'm able to find it. Would never have been able to Google my way to it.
goda90 3y ago
What if you didn't know enough to know the first answer was wrong? Because a lot of people are going to ask a lot of questions where that will be the case. They won't know to tweak their question and then do further research. And if you try to use GPT for automatic systems, well how is that automatic system going to recognize what's wrong vs right?
- capableweb 3y ago
  Not knowing what you don't know, is really hard to verify. But if you get any sort of answer, it's usually pretty trivial to verify. Where I found ChatGPT to be really helpful is when I don't know anything regarding something, and ChatGPT can give me something I could at least verify as true or false. Compared to before, where I just got stuck on some things, as I couldn't find out what I was actually looking for at all.
- ben_w 3y ago
  P vs. NP, answers are often easier to verify than to reach in the first place. That said, there are many people who don't bother to verify anything, as can be observed in comments sections where one can in many cases find people who clearly only read the headline and not the article.
- osigurdson 3y ago
  >> What if you didn't know enough to know the first answer was wrong? Anyone who uses ChatGPT for a while will learn to take everything with a grain of salt.
  - falserum 3y ago
    s/Grain/huge rock/
  - petemir 3y ago
    Are we sure? (Some) People are still going to vote for Donald Trump in 2024, one of the "bullshit generators" (such as chatGPT) used as example in the article. (n.b. I am not trying to make any type of political judgement here, I am just using the example in TFA).
gostsamo 3y ago
Because things that sound good sometimes are indeed good. So, the author disregards this fact, while everyone using ChatGPT knows that verification is part of the proccess. The final evaluation is if it is cheaper to dig the info ourselves or we can use the AI to narrow down the search space. I think the second, but only if people do the verification part.
- lcnPylGDnU4H9OF 3y ago
  > So, the author disregards this fact, while everyone using ChatGPT knows that verification is part of the proccess. Why is verification part of the process if not because ChatGPT generates bullshit? I don’t think the author is disregarding that it is sometimes correct.
  - gostsamo 3y ago
    Because chat gpt is yet another authority and like any other authority, its claims are not perfect. Don't you check when you find random info on the internet? The difference is that the ai gives answers faster and with much vaguer prompts than a handcoded system is usually able to.
    - lcnPylGDnU4H9OF 3y ago
      > Because chat gpt is yet another authority and like any other authority, its claims are not perfect. Don't you check when you find random info on the internet? What authority does random information on the internet have? How do I verify the information without looking at random information on the internet?
- 2muchcoffeeman 3y ago
  You don’t think that it’s a concern that verification must be done? People don’t do that now and we just made it easier to get confident answers that no one will check.
- proto_lambda 3y ago
  > everyone using ChatGPT knows that verification is part of the proccess lol. lmao
lcnPylGDnU4H9OF 3y ago
Why did you undergo the 1-2 minutes of validation?
ben_w 3y ago
That which sounds true is loosely correlated with that which is true. On the other hand: "Things that try to look like things often do look more like things than things. Well-known fact," said Granny. "But I don’t hold with encouraging it." - Terry Pratchett, Wyrd Sisters This makes LLMs much better than claimed by those who dismiss it as BS or stochastic parrot or whatever, and also not as good as domain experts.
krapp 3y ago
Given that ChatGPT generates responses based on the same data set that Google has access to, I find it difficult to believe you tried Google for hours and never recieved a single useful result. It also seems odd that you had the materials available to research and validate the response ChatGPT gave you, but somehow no ability to glean the necessary answer from those materials to begin with. Obviously, because Google was completely useless to you, you didn't simply use Google to do that validation, nor could you have used another search engine, because then you could have found the result you were looking for to begin with. Did you just use ChatGPT to validate its own responses?
rvz 3y ago
> ChatGPT first gave a wrong answer (because my question actually wasn't accurate). but after some clarification answered something that appeared correct, and after 1-2 minutes of further research and validation, I was 100% that it was indeed the correct answer Exactly. You answered your own question and that is the author's point. Do you even know why it gave the incorrect answer at first? Not even ChatGPT can explain to you why it did that or keeps doing this. How can you trust it to give the right answer (without you Googling to check) if you really don't know the answer? It already means you don't trust it an you know it bullshits as the author has claimed. It can confidently convince someone outside of one's own expertise that it can give a soundly correct answer but can easily be bullshitting nonsense. > Well. Any philosopher, mathematician, scientist, or novelist was trained on plenty of actual facts (established works of other people, school learning, experiments) and huge archives of bullshit (individual day-to-day experiences that cannot be called scientific, casual conversations just like Twitter conversations, propaganda, lies). Their output then would also be bullshit, and by extension, everything humanity ever created. The difference is with humans, they can be transparently held to account on whatever they say. An AI system cannot. So this actual whataboutism towards humans is incredibly weak here and a common excuse by AI proponents supporting the nonsense that AI systems like ChatGPT can uncontrollably generate in a black box system.
mhb 3y ago
> ChatGPT 3.5 It's a little surprising how hard it is to convince people that it's worth $20 to use v4.0. It is very apparent how much better it is.
- Filligree 3y ago
  Especially lately. The distilled version we’re getting today actually seems to be smarter and less likely to make things up, from my tests; I suppose it’s generalising better.
galdosdi 3y ago
> If that's true, why then did it take me 10 seconds yesterday to find an extremely obscure fact of 18th century history with ChatGPT 3.5? I tried Google queries for over an hour, without success. This is true, yet I wonder how much of this "ChatGPT is way better than Google at Googling" effect is due simply to how bad Google has gotten over the last decade. ChatGPT and similar models seem to be uncontroversially best at this kind of task -- just being a replacement for Google Search.
roflyear 3y ago
You know, your post reads like "if that's true, why did this one thing happen once to me?" It's not extremely convincing.
latexr 3y ago
> Any philosopher, mathematician, scientist, or novelist was trained on (…). Their output then would also be bullshit, and by extension, everything humanity ever created. Humans don’t learn the same way AIs are trained. Equating the two is a fallacy.
falserum 3y ago
> why then did it take me 10 seconds yesterday It definitely took you more than 10 seconds, though I guess a bit of hyperbole does not hurt. You validated findings against other source, which is obligatory step with chat gpt.
em500 3y ago
> If that's true, why then did it take me 10 seconds yesterday to find an extremely obscure fact of 18th century history with ChatGPT 3.5? I tried Google queries for over an hour, without success. Can you share which obscure 18th century historical fact you were looking for? Maybe you happen to be exceptionally bad at google queries?
amatheus 3y ago
> If that's true, why then did it take me 10 seconds yesterday to find an extremely obscure fact of 18th century history with ChatGPT 3.5? Would you mind telling what's the fact you were searching for? (and how did you search for it?)
marcosdumay 3y ago
You found it because LLMs are incredibly good search engines. But anything they give you that isn't a search result is bullshit. So, they have those two modes of operation, and most of them are built in a way that won't let you know which mode they are working on. That can be useful depending on what you want to do, but it doesn't strike me as the most effective architecture for them. (They are also reasonably good pattern fitters, what is kinda nice for code autocompletion on some languages. But they are way too inefficient here. Yet, nobody created any good high-efficiency autocomplete that merges searching and pattern fitting, so people use them.)
thenoblesunfish 3y ago
I think the answer is that bullshitters are great inspiration. They're fun, they get the smart people pumped up. The combination of a bullshitter and a more precise but less gregarious person is powerful. See a lot of successful companies.
al_be_back 3y ago
>> If that's true, why then did it take me 10 seconds yesterday to... Google vs cGPT it's neither here nor there in this case, after all Google research drives cGPT, their market focus differs. The OP is directly mapping Frankfurt's depiction of Bullshit atop the current AI phenomena, mainly because how generative AI approaches "truth". Natural languages (NL) differ greatly from computer languages (CL), especially on grammar (NL tolerates vagueness) and semantics (meaning, truth). Loosely-speaking, when AI generates NL text it relies on LLMs for context, structure, length etc, a kinda "the model suggests so". However, where content is computer code, even computers can test & verify the output. With NL content on the other hand, the end-user has to essentially test & verify the output to establish truth, meaning. In my view, the main difference between a Human Bullshitter and a Automated Bullshitter, is that AI can potentially improve exponentially (as smart tech does), thus leaving the end-user (average person) severely overwhelmed.

by lqet 3y ago

mnd999 3y ago

> ChatGPT first gave a wrong answer I think you answered your own question there. It doesn’t say your question makes no sense, or that’s incorrect. It just makes up a bullshit answer.

exitb 3y ago
Most tools require some preparation, knowledge and understanding before you get good results. Would he be better off not getting the right answer in the subsequent question?
- falserum 3y ago
  It’s not all or nothing. Most of negative sentiment is not “world would be better without chatgpt“, but “one must be very cautious and always validate the answer”. It’s valuable tool, but it has a LOT of hype, which led to things like people trusting chatgpt output in a court case.
empath-nirvana 3y ago
GPT4 will definitely tell you that your question is wrong and doesn't make sense, and a lot of times it is right about that.

chrisvalleybay 3y ago

Exactly. It's great for these use cases. I often use it to find things of which I barely remember some obsure detail and mostly I'm able to find it. Would never have been able to Google my way to it.

goda90 3y ago

What if you didn't know enough to know the first answer was wrong? Because a lot of people are going to ask a lot of questions where that will be the case. They won't know to tweak their question and then do further research. And if you try to use GPT for automatic systems, well how is that automatic system going to recognize what's wrong vs right?

capableweb 3y ago
Not knowing what you don't know, is really hard to verify. But if you get any sort of answer, it's usually pretty trivial to verify. Where I found ChatGPT to be really helpful is when I don't know anything regarding something, and ChatGPT can give me something I could at least verify as true or false. Compared to before, where I just got stuck on some things, as I couldn't find out what I was actually looking for at all.
ben_w 3y ago
P vs. NP, answers are often easier to verify than to reach in the first place. That said, there are many people who don't bother to verify anything, as can be observed in comments sections where one can in many cases find people who clearly only read the headline and not the article.
osigurdson 3y ago
>> What if you didn't know enough to know the first answer was wrong? Anyone who uses ChatGPT for a while will learn to take everything with a grain of salt.
- falserum 3y ago
  s/Grain/huge rock/
- petemir 3y ago
  Are we sure? (Some) People are still going to vote for Donald Trump in 2024, one of the "bullshit generators" (such as chatGPT) used as example in the article. (n.b. I am not trying to make any type of political judgement here, I am just using the example in TFA).

gostsamo 3y ago

Because things that sound good sometimes are indeed good. So, the author disregards this fact, while everyone using ChatGPT knows that verification is part of the proccess. The final evaluation is if it is cheaper to dig the info ourselves or we can use the AI to narrow down the search space. I think the second, but only if people do the verification part.

lcnPylGDnU4H9OF 3y ago
> So, the author disregards this fact, while everyone using ChatGPT knows that verification is part of the proccess. Why is verification part of the process if not because ChatGPT generates bullshit? I don’t think the author is disregarding that it is sometimes correct.
- gostsamo 3y ago
  Because chat gpt is yet another authority and like any other authority, its claims are not perfect. Don't you check when you find random info on the internet? The difference is that the ai gives answers faster and with much vaguer prompts than a handcoded system is usually able to.
  - lcnPylGDnU4H9OF 3y ago
    > Because chat gpt is yet another authority and like any other authority, its claims are not perfect. Don't you check when you find random info on the internet? What authority does random information on the internet have? How do I verify the information without looking at random information on the internet?
2muchcoffeeman 3y ago
You don’t think that it’s a concern that verification must be done? People don’t do that now and we just made it easier to get confident answers that no one will check.
proto_lambda 3y ago
> everyone using ChatGPT knows that verification is part of the proccess lol. lmao

lcnPylGDnU4H9OF 3y ago

Why did you undergo the 1-2 minutes of validation?

ben_w 3y ago

That which sounds true is loosely correlated with that which is true. On the other hand: "Things that try to look like things often do look more like things than things. Well-known fact," said Granny. "But I don’t hold with encouraging it." - Terry Pratchett, Wyrd Sisters This makes LLMs much better than claimed by those who dismiss it as BS or stochastic parrot or whatever, and also not as good as domain experts.

krapp 3y ago

Given that ChatGPT generates responses based on the same data set that Google has access to, I find it difficult to believe you tried Google for hours and never recieved a single useful result. It also seems odd that you had the materials available to research and validate the response ChatGPT gave you, but somehow no ability to glean the necessary answer from those materials to begin with. Obviously, because Google was completely useless to you, you didn't simply use Google to do that validation, nor could you have used another search engine, because then you could have found the result you were looking for to begin with. Did you just use ChatGPT to validate its own responses?

rvz 3y ago

> ChatGPT first gave a wrong answer (because my question actually wasn't accurate). but after some clarification answered something that appeared correct, and after 1-2 minutes of further research and validation, I was 100% that it was indeed the correct answer Exactly. You answered your own question and that is the author's point. Do you even know why it gave the incorrect answer at first? Not even ChatGPT can explain to you why it did that or keeps doing this. How can you trust it to give the right answer (without you Googling to check) if you really don't know the answer? It already means you don't trust it an you know it bullshits as the author has claimed. It can confidently convince someone outside of one's own expertise that it can give a soundly correct answer but can easily be bullshitting nonsense. > Well. Any philosopher, mathematician, scientist, or novelist was trained on plenty of actual facts (established works of other people, school learning, experiments) and huge archives of bullshit (individual day-to-day experiences that cannot be called scientific, casual conversations just like Twitter conversations, propaganda, lies). Their output then would also be bullshit, and by extension, everything humanity ever created. The difference is with humans, they can be transparently held to account on whatever they say. An AI system cannot. So this actual whataboutism towards humans is incredibly weak here and a common excuse by AI proponents supporting the nonsense that AI systems like ChatGPT can uncontrollably generate in a black box system.

mhb 3y ago

> ChatGPT 3.5 It's a little surprising how hard it is to convince people that it's worth $20 to use v4.0. It is very apparent how much better it is.

Filligree 3y ago
Especially lately. The distilled version we’re getting today actually seems to be smarter and less likely to make things up, from my tests; I suppose it’s generalising better.

galdosdi 3y ago

> If that's true, why then did it take me 10 seconds yesterday to find an extremely obscure fact of 18th century history with ChatGPT 3.5? I tried Google queries for over an hour, without success. This is true, yet I wonder how much of this "ChatGPT is way better than Google at Googling" effect is due simply to how bad Google has gotten over the last decade. ChatGPT and similar models seem to be uncontroversially best at this kind of task -- just being a replacement for Google Search.

roflyear 3y ago

You know, your post reads like "if that's true, why did this one thing happen once to me?" It's not extremely convincing.

latexr 3y ago

> Any philosopher, mathematician, scientist, or novelist was trained on (…). Their output then would also be bullshit, and by extension, everything humanity ever created. Humans don’t learn the same way AIs are trained. Equating the two is a fallacy.

falserum 3y ago

> why then did it take me 10 seconds yesterday It definitely took you more than 10 seconds, though I guess a bit of hyperbole does not hurt. You validated findings against other source, which is obligatory step with chat gpt.

em500 3y ago

> If that's true, why then did it take me 10 seconds yesterday to find an extremely obscure fact of 18th century history with ChatGPT 3.5? I tried Google queries for over an hour, without success. Can you share which obscure 18th century historical fact you were looking for? Maybe you happen to be exceptionally bad at google queries?

amatheus 3y ago

> If that's true, why then did it take me 10 seconds yesterday to find an extremely obscure fact of 18th century history with ChatGPT 3.5? Would you mind telling what's the fact you were searching for? (and how did you search for it?)

marcosdumay 3y ago

You found it because LLMs are incredibly good search engines. But anything they give you that isn't a search result is bullshit. So, they have those two modes of operation, and most of them are built in a way that won't let you know which mode they are working on. That can be useful depending on what you want to do, but it doesn't strike me as the most effective architecture for them. (They are also reasonably good pattern fitters, what is kinda nice for code autocompletion on some languages. But they are way too inefficient here. Yet, nobody created any good high-efficiency autocomplete that merges searching and pattern fitting, so people use them.)

thenoblesunfish 3y ago

I think the answer is that bullshitters are great inspiration. They're fun, they get the smart people pumped up. The combination of a bullshitter and a more precise but less gregarious person is powerful. See a lot of successful companies.

al_be_back 3y ago

>> If that's true, why then did it take me 10 seconds yesterday to... Google vs cGPT it's neither here nor there in this case, after all Google research drives cGPT, their market focus differs. The OP is directly mapping Frankfurt's depiction of Bullshit atop the current AI phenomena, mainly because how generative AI approaches "truth". Natural languages (NL) differ greatly from computer languages (CL), especially on grammar (NL tolerates vagueness) and semantics (meaning, truth). Loosely-speaking, when AI generates NL text it relies on LLMs for context, structure, length etc, a kinda "the model suggests so". However, where content is computer code, even computers can test & verify the output. With NL content on the other hand, the end-user has to essentially test & verify the output to establish truth, meaning. In my view, the main difference between a Human Bullshitter and a Automated Bullshitter, is that AI can potentially improve exponentially (as smart tech does), thus leaving the end-user (average person) severely overwhelmed.