5 ms·

> ChatGPT is very effective Massive [Citation Needed] to be honest. ChatGPT is arguably even worse than Bing at producing factually correct information, since

> ChatGPT is very effective Massive [Citation Needed] to be honest. ChatGPT is arguably even worse than Bing at producing factually correct information, since it does not have any ability to source it. It simply predicts the next most likely word in a sentence. Even if that next word makes the entire thing horribly incorrect.

mrtksn 4y ago
Factual correctness is not very important, you don't have to trust it. ChatGPT is very effective because it acts as an advisor in any topic. Sometimes its very wrong but it gives you an overall picture of the thing you are looking for and from there you can drill down. It is much better than trying to drill down from spam ridden ad infested results Google would provide. Once you pass the very basic stuff, you are often able to reach the spam-free-lands because the SEO spam hasn't reached the every niche just yet(spammers optimize for certain popular keywords). Once you have the overall picture, you have the roadsigns to the niche lands. You can directly search Stackoverflow, reddit, hacker news, github etc.
- m_mueller 4y ago
  Plus it's really quite good at typical assistant tasks. "Write [lettertype] letter with following key points". "write a one-off script in powershell that does XYZ". You still have to know the details of the task yourself to be able to review it (and let it improve), but that's exactly the same if you have assistants in your team doing that kinda thing - at the end you're responsible for the output and you'll have to ensure its quality.
  - mrtksn 4y ago
    Exactly, you can delegate a lot of boring busywork to ChatGPT. You need the synonyms of every word used in Chess? You need all that in a JS array? No problem, ChatGPT cat sort you out in a few moments. You may still need to pay attention to the correctness but that doesn't remove the usefulness of ChatGPT.
- fauigerzigerk 4y ago
  >Factual correctness is not very important, you don't have to trust it. If these chat bots are going to replace or compete with the traditional search interface, then correctness is absolutely key. When a traditional search engine points you to some spammy site, you have far more context to decide for yourself how much credibility you ascribe to it. You may know the site from earlier visits. You can see whether it's completely ad infested and what the likely business model is. You can see what else is on the site. You can check who the publisher is, etc. But when a chat bot delivers results in a conversational style, there is absolutely no indication as to whether a particular claim is fact or fiction. If the bot provides links to sources that's great, but then you're back to square one reading through 10 paragraphs of blog spam trying to filter out the one bit that may answer your question. Also, the way in which these bots fail compared to traditional content farms and spammy blogs is very different. These sites tend to copy stuff from elsewhere. They may repeat falsehoods published by others. But they are unlikely to just make up numbers and present them as fact.
  - mrtksn 4y ago
    I don't think its going to be drop-in replacement. With ChatGPT, you can ask follow up questions. So if something doesn't sound right, you can pressure it to explain its position. It's like talking with people, sometimes people are wrong and we happen to have a lot of linguistic tools to deal with it. Like, follow up questions or asking for specifics.
    - fauigerzigerk 4y ago
      >I don't think its going to be drop-in replacement. I agree, but part of the reason for that is that it doesn't get the facts right (yet?). A chat bot that can be trusted would be far more useful and could largely replace the traditional search interface. I tried ChatGPT (and some other transformer models). I know you can ask follow up questions and make it reconsider its responses or change its mind. But why does it change its mind? It's not because it finds a flaw in its own logic or discovers new information. I think it's simply because the pattern of inquisitorial questioning is statistically associated with the pattern of mind changing. And then it just follows your lead in a new direction making claims that are just as unreliable as the original ones. I'm keeping an open mind though. I don't think I have understood everything there is to understand about this.
- hrunt 4y ago
  The scary thing about this reply is I cannot tell whether it was written by a human or by Bing trying to gaslight me into believing that factual correctness is not important. It reads just like Bing trying to explain why the year is 2022 or Bing did not say that it said the Eagles won the Super Bowl.
  - mrtksn 4y ago
    The scary part is that people can start dismissing the opinions that they don't agree as "AI generated".
    - bell-cot 4y ago
      People have been dismissing opinions they didn't like for ~25,000 years, for a very long list of reasons. Dismissing "AI" for that reason feels kinda like not having one more key on your keyring - because a kid could stick the new key into an electrical outlet.
      - mrtksn 4y ago
        Dismissing opinions is on thing, dismissing opinions thinking that you are not interacting with humans is why some people are huge a-holes on the internet despite being decent people IRL.
        bell-cot 4y ago
        Maybe I'm an old cynic...but I already (quietly) dismiss most plausibly-human opinions on the internet, because "advertising", "clickbait", "ignorant", "echo chamber", "troll", "spam", "obvious & boring", "not my religion", "immature". Etc. etc. etc. "Is it from an AI?" seems about as important to me as validating the SPF record behind some "I am assistant to Nigerian Minister of Oil, and need your help..." email message. Who would actually react differently to that message, on the basis of "SPF is valid"?
        mrtksn 4y ago
- Thorentis 4y ago
  People overstate how bad Google results are. The great thing about blogspam and ads is that you have an extra signal for what to instantly dismiss. I have gotten very good at filtering out Google results, not based on the information they contained, but how they were presented, etc. With ChatGPT, you don't get this. You only get a blob of text that looks identical to every other correct or incorrect response. There is no extra signal.
krona 4y ago
One of my friends was wrong once. I don't speak to him anymore.
M4v3R 4y ago
> It simply predicts the next most likely word in a sentence So does your brain. And yes, you are correct that at this moment ChatGPT is not very good at producing factually correct information on many topics, but that's not the only reason chatbots exist. It can do many more things, and I personally (as a programmer) and some people I know from different fields use it daily because it provides value for us (enough value in fact to make me pay the $25/mo price for premium). For example just yesterday it helped me build a proof of concept in Python to test whether a solution to a problem we had at work would work. With its help I've built a working PoC in literally 10 minutes, something that would take me probably more than 2 hours to complete if I didn't have the help because I'm not even a Python programmer (I'm in a different field entirely).
- NobleLie 4y ago
  Where did you learn that the brain works by simply predicting the next most likely word in a sentence? Tertiary note: I'm also a programmer and a researcher. It's failed pretty hard on both. Faking sources of medical / biological information, botching simple chronology, suggesting made up methods on well known programing libraries (3 of many examples). There was nothing I could do to correct any of these errors - no matter how much leading or pointing out error, it would acknowledge it and repeat it. I recognize the value of chatgpt (and roughly how it operates) but the excitement (marketing hype) is indeed over the top. It appears an incremental advancement of old word2vec (added human supervision and a few more things) but the sum total is still of questionable value in domains that mean the most to me (admittedly self centric, but many are like me, at least here)
astrange 4y ago
> It simply predicts the next most likely word in a sentence. It does not "simply" do that. The word decoder does that; the model's job is to make the "most likely" next word the correct one.
Falkon1313 4y ago
That's just using it wrong though. Sure, a dirtbike isn't as good as an 18-wheeler for hauling cargo on the interstate. But a fully-loaded 18-wheeler isn't quite as good at running a motocross race. ChatGPT is great for spurring ideas, giving you new things to look into - inspiration. It really excels at that compared to traditional search engines. If you have a rough idea what you're looking for, but don't know the right things to search on, that could even be a big help in search. Traditional search may currently find objective facts better. But if you don't know what facts to look for in the first place, a quick round with ChatGPT can break your mental block and give you a lot of paths to try out. It helps you find what to search.
mark_l_watson 4y ago
It really sounds like you have not personally used ChatGPT+Bing, right? I have been using it as my search engine for a few days and when in chat mode, it shows you the web search queries it uses to gather context text to feed to ChatGPT along with your question. It then provides reference links. Just one case from yesterday: we had been considering leaving our parrot by himself for two nights. A few times a year we leave him for one night and he seems to do OK. I asked ChatGPT+Bing about this and I got a very good response on why leaving him by himself would be a bad idea. I checked the references it used, and then hired someone to bird sit. I have also asked about business ideas, and other topics that I would not have thought to ask a search engine previously. At the heart of it, Squad type deep learning models have for years done a good job at answering questions given context text. This new system just does a better job of it. I think that what Microsoft has now is a very good first step in a new kind of technology. I wrote a demo 2 years ago that used SPARQL queries on public Knowledge Graphs to get query context text and then I used a pre trained QA model. I surprised that it worked OK. What Microsoft has is similar, but works on a grander scale. Anyway, get in the waiting list, then try it yourself.

mrtksn 4y ago

Factual correctness is not very important, you don't have to trust it. ChatGPT is very effective because it acts as an advisor in any topic. Sometimes its very wrong but it gives you an overall picture of the thing you are looking for and from there you can drill down. It is much better than trying to drill down from spam ridden ad infested results Google would provide. Once you pass the very basic stuff, you are often able to reach the spam-free-lands because the SEO spam hasn't reached the every niche just yet(spammers optimize for certain popular keywords). Once you have the overall picture, you have the roadsigns to the niche lands. You can directly search Stackoverflow, reddit, hacker news, github etc.

m_mueller 4y ago
Plus it's really quite good at typical assistant tasks. "Write [lettertype] letter with following key points". "write a one-off script in powershell that does XYZ". You still have to know the details of the task yourself to be able to review it (and let it improve), but that's exactly the same if you have assistants in your team doing that kinda thing - at the end you're responsible for the output and you'll have to ensure its quality.
- mrtksn 4y ago
  Exactly, you can delegate a lot of boring busywork to ChatGPT. You need the synonyms of every word used in Chess? You need all that in a JS array? No problem, ChatGPT cat sort you out in a few moments. You may still need to pay attention to the correctness but that doesn't remove the usefulness of ChatGPT.
fauigerzigerk 4y ago
>Factual correctness is not very important, you don't have to trust it. If these chat bots are going to replace or compete with the traditional search interface, then correctness is absolutely key. When a traditional search engine points you to some spammy site, you have far more context to decide for yourself how much credibility you ascribe to it. You may know the site from earlier visits. You can see whether it's completely ad infested and what the likely business model is. You can see what else is on the site. You can check who the publisher is, etc. But when a chat bot delivers results in a conversational style, there is absolutely no indication as to whether a particular claim is fact or fiction. If the bot provides links to sources that's great, but then you're back to square one reading through 10 paragraphs of blog spam trying to filter out the one bit that may answer your question. Also, the way in which these bots fail compared to traditional content farms and spammy blogs is very different. These sites tend to copy stuff from elsewhere. They may repeat falsehoods published by others. But they are unlikely to just make up numbers and present them as fact.
- mrtksn 4y ago
  I don't think its going to be drop-in replacement. With ChatGPT, you can ask follow up questions. So if something doesn't sound right, you can pressure it to explain its position. It's like talking with people, sometimes people are wrong and we happen to have a lot of linguistic tools to deal with it. Like, follow up questions or asking for specifics.
  - fauigerzigerk 4y ago
    >I don't think its going to be drop-in replacement. I agree, but part of the reason for that is that it doesn't get the facts right (yet?). A chat bot that can be trusted would be far more useful and could largely replace the traditional search interface. I tried ChatGPT (and some other transformer models). I know you can ask follow up questions and make it reconsider its responses or change its mind. But why does it change its mind? It's not because it finds a flaw in its own logic or discovers new information. I think it's simply because the pattern of inquisitorial questioning is statistically associated with the pattern of mind changing. And then it just follows your lead in a new direction making claims that are just as unreliable as the original ones. I'm keeping an open mind though. I don't think I have understood everything there is to understand about this.
hrunt 4y ago
The scary thing about this reply is I cannot tell whether it was written by a human or by Bing trying to gaslight me into believing that factual correctness is not important. It reads just like Bing trying to explain why the year is 2022 or Bing did not say that it said the Eagles won the Super Bowl.
- mrtksn 4y ago
  The scary part is that people can start dismissing the opinions that they don't agree as "AI generated".
  - bell-cot 4y ago
    People have been dismissing opinions they didn't like for ~25,000 years, for a very long list of reasons. Dismissing "AI" for that reason feels kinda like not having one more key on your keyring - because a kid could stick the new key into an electrical outlet.
    - mrtksn 4y ago
      Dismissing opinions is on thing, dismissing opinions thinking that you are not interacting with humans is why some people are huge a-holes on the internet despite being decent people IRL.
      - bell-cot 4y ago
        Maybe I'm an old cynic...but I already (quietly) dismiss most plausibly-human opinions on the internet, because "advertising", "clickbait", "ignorant", "echo chamber", "troll", "spam", "obvious & boring", "not my religion", "immature". Etc. etc. etc. "Is it from an AI?" seems about as important to me as validating the SPF record behind some "I am assistant to Nigerian Minister of Oil, and need your help..." email message. Who would actually react differently to that message, on the basis of "SPF is valid"?
        mrtksn 4y ago
Thorentis 4y ago
People overstate how bad Google results are. The great thing about blogspam and ads is that you have an extra signal for what to instantly dismiss. I have gotten very good at filtering out Google results, not based on the information they contained, but how they were presented, etc. With ChatGPT, you don't get this. You only get a blob of text that looks identical to every other correct or incorrect response. There is no extra signal.

krona 4y ago

One of my friends was wrong once. I don't speak to him anymore.

M4v3R 4y ago

> It simply predicts the next most likely word in a sentence So does your brain. And yes, you are correct that at this moment ChatGPT is not very good at producing factually correct information on many topics, but that's not the only reason chatbots exist. It can do many more things, and I personally (as a programmer) and some people I know from different fields use it daily because it provides value for us (enough value in fact to make me pay the $25/mo price for premium). For example just yesterday it helped me build a proof of concept in Python to test whether a solution to a problem we had at work would work. With its help I've built a working PoC in literally 10 minutes, something that would take me probably more than 2 hours to complete if I didn't have the help because I'm not even a Python programmer (I'm in a different field entirely).

NobleLie 4y ago
Where did you learn that the brain works by simply predicting the next most likely word in a sentence? Tertiary note: I'm also a programmer and a researcher. It's failed pretty hard on both. Faking sources of medical / biological information, botching simple chronology, suggesting made up methods on well known programing libraries (3 of many examples). There was nothing I could do to correct any of these errors - no matter how much leading or pointing out error, it would acknowledge it and repeat it. I recognize the value of chatgpt (and roughly how it operates) but the excitement (marketing hype) is indeed over the top. It appears an incremental advancement of old word2vec (added human supervision and a few more things) but the sum total is still of questionable value in domains that mean the most to me (admittedly self centric, but many are like me, at least here)

astrange 4y ago

> It simply predicts the next most likely word in a sentence. It does not "simply" do that. The word decoder does that; the model's job is to make the "most likely" next word the correct one.

Falkon1313 4y ago

That's just using it wrong though. Sure, a dirtbike isn't as good as an 18-wheeler for hauling cargo on the interstate. But a fully-loaded 18-wheeler isn't quite as good at running a motocross race. ChatGPT is great for spurring ideas, giving you new things to look into - inspiration. It really excels at that compared to traditional search engines. If you have a rough idea what you're looking for, but don't know the right things to search on, that could even be a big help in search. Traditional search may currently find objective facts better. But if you don't know what facts to look for in the first place, a quick round with ChatGPT can break your mental block and give you a lot of paths to try out. It helps you find what to search.

mark_l_watson 4y ago

It really sounds like you have not personally used ChatGPT+Bing, right? I have been using it as my search engine for a few days and when in chat mode, it shows you the web search queries it uses to gather context text to feed to ChatGPT along with your question. It then provides reference links. Just one case from yesterday: we had been considering leaving our parrot by himself for two nights. A few times a year we leave him for one night and he seems to do OK. I asked ChatGPT+Bing about this and I got a very good response on why leaving him by himself would be a bad idea. I checked the references it used, and then hired someone to bird sit. I have also asked about business ideas, and other topics that I would not have thought to ask a search engine previously. At the heart of it, Squad type deep learning models have for years done a good job at answering questions given context text. This new system just does a better job of it. I think that what Microsoft has now is a very good first step in a new kind of technology. I wrote a demo 2 years ago that used SPARQL queries on public Knowledge Graphs to get query context text and then I used a pre trained QA model. I surprised that it worked OK. What Microsoft has is similar, but works on a grander scale. Anyway, get in the waiting list, then try it yourself.