4 ms·

The next problem is verifying all of that information. It will require multiple dozen auxillary Google searches and reading through countless articles and mess

The next problem is verifying all of that information. It will require multiple dozen auxillary Google searches and reading through countless articles and message boards. GPT-3 is a great research tool as a brainstorming engine, but sadly cannot currently be formally trusted to output truthful information, even with conditioning. It takes someone with prior knowledge or subsequent research in order to make use of the result. I'd say this isn't what the author was intending.

drc500free 4y ago
Google's answer box also can't be trusted to output truthful information.
- soulofmischief 4y ago
  I believe this is the point that @drivebycomment was making, that answering such a query is not possible right now. The query needs to be reframed by using keywords like the GPT-3 prompt did, and even further refined than that, and you still have to finish the last mile of research.
- ceejayoz 4y ago
  Fun one today: https://www.google.com/search?q=is+fortigate+open+source https://www.google.com/search?q=is+fortigate+open+source Prompt: "Is Fortigate open source?" Article: "Based on the parameters we compared, it seems that pfSense is the more favorable solution because it is open source and also offers great features." Google: "It is open source!"
  - hammyhavoc 4y ago
    Yes, these are forever spreading disinformation on various topics in my social group. I am forever telling people to actually click and read the article rather than trust the automatic tl;dr by Google.
- summerlight 4y ago
  Google simply returns human written results, so delegating the trust problem. You can track author's record and judge if they're trustworthy yourself. Doing the same thing for GPT-3 is much harder. How are you going to debug the model? Probably there is only a handful amount of people who is capable of debugging the model even with all necessary access granted. You can think it as autonomous driving which already has reached to a good level for 99% of use cases (at least for Waymo and Cruise), but no one think it's going to mass deploy robotaxi anytime soon.
spullara 4y ago
Even if you got the info straight from another human being I can't imagine anyone would take it at face value and just schedule their trip.
- mattkrause 4y ago
  I wouldn't buy the tickets sight unseen, but I would trust you that it's actually a real place, actually in Croatia, and not being conflated with similar-sounding places. Humans, unless they're disturbed or malicious, tend not to hallucinate things out of whole cloth.
  - mrcartmeneses 4y ago
    So... we’ve got ML models to the point of being able to hallucinate?
    - yarg 4y ago
      Hallucination in context means something rather different. Something like coming to accept some data-set local optima as a global truth and deriving beliefs without underlying basis.
    - hnfong 4y ago
      "Yes"? The whole point of these models is to make up something that seems "realistic", but not regurgitate the original contents. If that's not hallucination I don't know what is.
    - mattkrause 4y ago
      Yup: prompt a large language model to tell you about itself and it will sometimes weave weird tales. For example, I just had the following interaction with GTP2large[0]: Me: What is your name? Model: I'm Mark A. <continue> Dreyer, and I'm with the <continue> Federal Communications Commission (FCC) Obviously, it's not actually a guy named Mark A. Dreyer and, as far as I can tell, there is no Mark. A. Dreyer in a prominent role at the FCC. That, as far as I'm concerned, is essentially a hallucination. If you think about how they work, this isn't totally surprising. The model has been programmed to take the input, turn it into a series of tokens, and then produce another series of tokens that are especially likely to follow the input ones. That's it. To the extent that this corresponds to reality, it's only because reality is overrepresented in the training data, and so token sequences corresponding to reality are often more likely under its model. This also explains how it goes off the rails: Mark is a very common name (in the top 25 for men for decades) and once you generate a stereotypical middle-aged white guy name, it's not a huge leap to him being a bureaucrat, etc. I forgot to delete the huggingface boilerplate, which talks about advances in technology (etc), which probably sent it down the FCC route too.
gmt2027 4y ago
Could this be addressed by an ensemble of GPT-3s fine tuned with different data?

drc500free 4y ago

Google's answer box also can't be trusted to output truthful information.

soulofmischief 4y ago
I believe this is the point that @drivebycomment was making, that answering such a query is not possible right now. The query needs to be reframed by using keywords like the GPT-3 prompt did, and even further refined than that, and you still have to finish the last mile of research.
ceejayoz 4y ago
Fun one today: https://www.google.com/search?q=is+fortigate+open+source https://www.google.com/search?q=is+fortigate+open+source Prompt: "Is Fortigate open source?" Article: "Based on the parameters we compared, it seems that pfSense is the more favorable solution because it is open source and also offers great features." Google: "It is open source!"
- hammyhavoc 4y ago
  Yes, these are forever spreading disinformation on various topics in my social group. I am forever telling people to actually click and read the article rather than trust the automatic tl;dr by Google.
summerlight 4y ago
Google simply returns human written results, so delegating the trust problem. You can track author's record and judge if they're trustworthy yourself. Doing the same thing for GPT-3 is much harder. How are you going to debug the model? Probably there is only a handful amount of people who is capable of debugging the model even with all necessary access granted. You can think it as autonomous driving which already has reached to a good level for 99% of use cases (at least for Waymo and Cruise), but no one think it's going to mass deploy robotaxi anytime soon.

spullara 4y ago

Even if you got the info straight from another human being I can't imagine anyone would take it at face value and just schedule their trip.

mattkrause 4y ago
I wouldn't buy the tickets sight unseen, but I would trust you that it's actually a real place, actually in Croatia, and not being conflated with similar-sounding places. Humans, unless they're disturbed or malicious, tend not to hallucinate things out of whole cloth.
- mrcartmeneses 4y ago
  So... we’ve got ML models to the point of being able to hallucinate?
  - yarg 4y ago
    Hallucination in context means something rather different. Something like coming to accept some data-set local optima as a global truth and deriving beliefs without underlying basis.
  - hnfong 4y ago
    "Yes"? The whole point of these models is to make up something that seems "realistic", but not regurgitate the original contents. If that's not hallucination I don't know what is.
  - mattkrause 4y ago
    Yup: prompt a large language model to tell you about itself and it will sometimes weave weird tales. For example, I just had the following interaction with GTP2large[0]: Me: What is your name? Model: I'm Mark A. <continue> Dreyer, and I'm with the <continue> Federal Communications Commission (FCC) Obviously, it's not actually a guy named Mark A. Dreyer and, as far as I can tell, there is no Mark. A. Dreyer in a prominent role at the FCC. That, as far as I'm concerned, is essentially a hallucination. If you think about how they work, this isn't totally surprising. The model has been programmed to take the input, turn it into a series of tokens, and then produce another series of tokens that are especially likely to follow the input ones. That's it. To the extent that this corresponds to reality, it's only because reality is overrepresented in the training data, and so token sequences corresponding to reality are often more likely under its model. This also explains how it goes off the rails: Mark is a very common name (in the top 25 for men for decades) and once you generate a stereotypical middle-aged white guy name, it's not a huge leap to him being a bureaucrat, etc. I forgot to delete the huggingface boilerplate, which talks about advances in technology (etc), which probably sent it down the FCC route too.

gmt2027 4y ago

Could this be addressed by an ensemble of GPT-3s fine tuned with different data?