3 ms·

> some of these gaps can be tackled Seems like the major gap is in facts. It'll often make up completely plausible specific details that are flat wrong, or ref

> some of these gaps can be tackled Seems like the major gap is in facts. It'll often make up completely plausible specific details that are flat wrong, or refuse to proceed if it's "not allowed" to do so. Coupling it (handwave) with a massive structured data repository like Wolfram Alpha and Wikidata would be really something.

fnordpiglet 4y ago
I find the guard rails they put in maddening and weirdly implemented. I wanted it to make up jokes for me and it’s jokes are the same few over and over. (I can never remember jokes so this is a super useful tool for me!) so I wanted it to tell me jokes in the style of Jerry Seinfeld. The guard rails prevented it from doing that because he’s a real person and it tries to not bend its personality pretty hard. However by just resetting the convo a few times I randomly got it to agree to tell me a joke. I can’t remember it but it was actually funny and in the style of Seinfeld. shrug It almost certainly has indexed Wikipedia, fwiw.
- Oxidation 4y ago
  Indexing Wikipedia (or any) text is a statistical thing though. When you ask a specific question like the size of a car, say, it will often come back with a compelling number in context, but if you actually look it up, it's usually not quite right. Another example is that it can just "invent" new functions when coming up with code snippets. Syntactically usually correct and completely plausible in the context, but simply doesn't exist. I also am pretty baffled by the limitations. I just assume they're trying to avoid "scammers use AI to construct false financial documents" type of furores in the early days (once the market is saturated with competition, blame will be too diffuse to stick) and convincing customers of their proprietary systems that they won't end up with a customer support bot that starts reciting racial slurs.
Miraste 4y ago
It insisted to me for a while that the sun was red. The "guardrails" do nothing to stop it from lying confidently about almost everything.
- mwigdahl 4y ago
  A lot of people say that the model is "lying" when it returns incorrect facts. Generally when friends or family confidently assert facts that they are sure but also incorrect about, we call them "misinformed" rather than connoting malice with the word "lying". Have they heard the right facts but associated them incorrectly? Heard the wrong facts from whatever source they obtained them from? Either or both could also be true of ChatGPT.
  - Miraste 4y ago
    Yes, and either word is anthropomorphising. I stick with "lying" nonetheless, because if you start a conversation with "What color is the sun?" it returns a correct answer. It already has the association between "sun" and "white" in the model, but for whatever reason the context of my conversation made it say the sun is red. "Lying" seems to describe this behavior better than "misinformed."
  - Oxidation 4y ago
    It's more like you can't actually read but you've seen a lot of media about cars that contain text saying they have a thing called a "width" around the 1.7-2m mark and usually aren't round numbers. So when someone asks you how wide a 2011[1] Ford Focus is[2], you say "1,769mm" and look confident. Syntactically right and by any probabilistic model, pretty good and plausible. [1] and if you ask for 2012, it tells you "I'm sorry, but I am not able to browse the internet to provide you with the specific dimensions of the 2012 Ford Focus." [2] according to Google results, it's 1823 mm to 1842 mm.
- fnordpiglet 4y ago
  I’ve only seen it do stuff like that if I assert a false premise and ask it questions off the false premise.
  - Oxidation 4y ago
    It seems also to do it if you ask about something that's not documented in detail. It'll get details of the 1976 Tangshan earthquake dead on, but ask for the 1982 Bonin Island earthquake and apparently it was at a depth of 40km (it was actually 22.2), but it gets the magnitude right. The Nicobar Islands earthquake on 26 December the same year apparently killed 4000 with a magnitude of 7.1. It didn't, it injured 52, and there were two: magnitude 6.2 and 6.3. And it was in January. 26th December was indeed an earthquake that affected those islands. In 2004. Which is not that surprising, as how would it know, considering it probably didn't have much input about it. What is more amazing is the seamless blending of correctly regurgitated details with very plausible wholesale fabrication and, even more convincingly, blending of fact with fiction.

fnordpiglet 4y ago

I find the guard rails they put in maddening and weirdly implemented. I wanted it to make up jokes for me and it’s jokes are the same few over and over. (I can never remember jokes so this is a super useful tool for me!) so I wanted it to tell me jokes in the style of Jerry Seinfeld. The guard rails prevented it from doing that because he’s a real person and it tries to not bend its personality pretty hard. However by just resetting the convo a few times I randomly got it to agree to tell me a joke. I can’t remember it but it was actually funny and in the style of Seinfeld. shrug It almost certainly has indexed Wikipedia, fwiw.

Oxidation 4y ago
Indexing Wikipedia (or any) text is a statistical thing though. When you ask a specific question like the size of a car, say, it will often come back with a compelling number in context, but if you actually look it up, it's usually not quite right. Another example is that it can just "invent" new functions when coming up with code snippets. Syntactically usually correct and completely plausible in the context, but simply doesn't exist. I also am pretty baffled by the limitations. I just assume they're trying to avoid "scammers use AI to construct false financial documents" type of furores in the early days (once the market is saturated with competition, blame will be too diffuse to stick) and convincing customers of their proprietary systems that they won't end up with a customer support bot that starts reciting racial slurs.

Miraste 4y ago

It insisted to me for a while that the sun was red. The "guardrails" do nothing to stop it from lying confidently about almost everything.

mwigdahl 4y ago
A lot of people say that the model is "lying" when it returns incorrect facts. Generally when friends or family confidently assert facts that they are sure but also incorrect about, we call them "misinformed" rather than connoting malice with the word "lying". Have they heard the right facts but associated them incorrectly? Heard the wrong facts from whatever source they obtained them from? Either or both could also be true of ChatGPT.
- Miraste 4y ago
  Yes, and either word is anthropomorphising. I stick with "lying" nonetheless, because if you start a conversation with "What color is the sun?" it returns a correct answer. It already has the association between "sun" and "white" in the model, but for whatever reason the context of my conversation made it say the sun is red. "Lying" seems to describe this behavior better than "misinformed."
- Oxidation 4y ago
  It's more like you can't actually read but you've seen a lot of media about cars that contain text saying they have a thing called a "width" around the 1.7-2m mark and usually aren't round numbers. So when someone asks you how wide a 2011[1] Ford Focus is[2], you say "1,769mm" and look confident. Syntactically right and by any probabilistic model, pretty good and plausible. [1] and if you ask for 2012, it tells you "I'm sorry, but I am not able to browse the internet to provide you with the specific dimensions of the 2012 Ford Focus." [2] according to Google results, it's 1823 mm to 1842 mm.
fnordpiglet 4y ago
I’ve only seen it do stuff like that if I assert a false premise and ask it questions off the false premise.
- Oxidation 4y ago
  It seems also to do it if you ask about something that's not documented in detail. It'll get details of the 1976 Tangshan earthquake dead on, but ask for the 1982 Bonin Island earthquake and apparently it was at a depth of 40km (it was actually 22.2), but it gets the magnitude right. The Nicobar Islands earthquake on 26 December the same year apparently killed 4000 with a magnitude of 7.1. It didn't, it injured 52, and there were two: magnitude 6.2 and 6.3. And it was in January. 26th December was indeed an earthquake that affected those islands. In 2004. Which is not that surprising, as how would it know, considering it probably didn't have much input about it. What is more amazing is the seamless blending of correctly regurgitated details with very plausible wholesale fabrication and, even more convincingly, blending of fact with fiction.