4 ms·

Computers are the ultimate appeal to authority in the style of "Computer says no". >If you make a mistake and honestly ask ChatGPT to produce the answer to som

by ux-app 4y ago

Computers are the ultimate appeal to authority in the style of "Computer says no". >If you make a mistake and honestly ask ChatGPT to produce the answer to something that doesn't exist, it will happily oblige. Just like asking people right? We're already accepting "truths" which we don't ourselves verify. I imagine that the vast majority of people accept the answers google provides on their results page. Is this any different? Schools teach young people to critically evaluate information sources whether they're books, TV, webpages etc. This is no different.

worble 4y ago
>Just like asking people right? Not really? I mean if you ask people to explain the 10 endings to Inside they'd probably just go "there we 10 endings? I had no idea" or something to that effect (if they weren't going "What's Inside?" that is). ChatGPT has no concept of being wrong, all it wants to do is a respond in a way that sounds human, as accurately as it can according to it's training data, and that is it's major flaw.
- kaetemi 4y ago
  More like asking students in an exam question to summarize the book they were supposed to read.
- ux-app 4y ago
  >Not really? I mean if you ask people to explain the 10 endings to Inside that's just the computer doing what computers do. the point is that humans routinely and confidently state their opinion as fact, any online forum will demonstrate that. >as accurately as it can according to it's training data, and that is it's major flaw. so we teach people that they need to be discerning with the way they use the tool. Kids in elementary school are taught that generally trust runs from : personal blog -> .com -> .org -> .gov -> .edu At this point ChatGPT sits around the "personal blog" level of trust. As long as people use the tool with this in mind then it's just business as usual.
- PeterisP 4y ago
  The thing to understand about ChatGPT is that at its core it isn't a question-answering machine, it's a text-continuation machine which by pure coincidence often continues a question with its answer. And it's learning primarily from data from people posting stuff on the internet (and only very rarely seeing actual conversations) so it has learned that "human language" pretty much never involves people saying that they don't know stuff, but does involve a LOT of assertive speculating and convinced 'preaching' even when saying untrue things. Like, imagine seeing a human-written a blog post for some click-bait site titled "Venus Williams eighth grand slam victory" or "The 10 secret endings for Inside" - ChatGPT will write pretty much the same thing as that human would. That blog post won't say "ah, I have no idea about these endings, I haven't seen them in my games", it would confidently repeat various rumors and if the author would invent some new fan theories, then they would often be stated in an assertive manner as fact. Furthermore, when doing its final reinforcement learning fine-tuning on top of the core GPT "3.5" model, it almost certainly was asked about some stuff which was simply more recent than the data used in training the core model - I mean, it can't know what happened last month - but it would be penalized if it would assert that things are the exact same as they were when the training data ended and reject things the user explicitly states that contradict its (inevitably outdated) knowledge. For all it knows, Williams has won 8 grand slams, simply the eighth one happened in the year after ChatGPT "stopped reading newspapers", so if a discussion mentions the eighth victory it has to assume that it is real, otherwise it would fail in so many other scenarios - e.g. refusing to acknowledge that some country has elected a different government or that a war has started.
  - jack_h 4y ago
    To drive the point home, look at all the rampant speculation about the recent high altitude balloon news. People play armchair ___ and invent plausible sounding explanations all the time, and people believe them.
    - PeterisP 4y ago
      Yes, and for the other part, if you asked ChatGPT "Hey, why did USA shoot down that Chinese balloon" and it answered "as far as I know, USA has never ever shot down a Chinese balloon" - which is technically true - people would consider it a bad answer.
- gremlinsinc 4y ago
  If you ask google this same thing, you could still get the wrong answer, what if there's two games called 'inside', or it mistakes 'inside' the game for the Pixar movie of the same name. Are you telling me you've never googled and gotten shit results, aka unusable stuff? Everything comes back with 100% accuracy? I mean, I know we need to be more skeptical of the results, but this is also early days. All we need are fact-checking engines that can give some sort of confidence score to results, and that should at least start moving in the right direction. 10 years from now, this will be moot, as the models they will have will be so far advanced we, probably won't even think it's possible or can fathom what it does to disrupt society. To say generative X hasn't disrupted industry already is insane, whether it's art, music, video, text - generative ai is going to only get better and more insane as text to video and voice cloning tech becomes mainstream. Imagine being self-conscous and wanting to do a youtube channel, so you have a virtual avatar with your own voice do it for you - you just write the script. (This is where I really want the ai tech to shine, as I want to do just this. :))
  - the_af 4y ago
    > If you ask google this same thing, you could still get the wrong answer, what if there's two games called 'inside', or it mistakes 'inside' the game for the Pixar movie of the same name. To me the difference is the quality of the hallucinations. If I ask Google about the secret Inside endings, and it gives me something about a space adventure, I know something is off. There's no "space" in the Inside I know. But ChatGPT when asked about 10 secret endings hallucinates completely sensible endings that make sense for Inside... it's just that none of them actually exist. That's a major difference between Google's incorrect search results and what ChatGPT provides.

by ux-app 4y ago

worble 4y ago

>Just like asking people right? Not really? I mean if you ask people to explain the 10 endings to Inside they'd probably just go "there we 10 endings? I had no idea" or something to that effect (if they weren't going "What's Inside?" that is). ChatGPT has no concept of being wrong, all it wants to do is a respond in a way that sounds human, as accurately as it can according to it's training data, and that is it's major flaw.

kaetemi 4y ago
More like asking students in an exam question to summarize the book they were supposed to read.
ux-app 4y ago
>Not really? I mean if you ask people to explain the 10 endings to Inside that's just the computer doing what computers do. the point is that humans routinely and confidently state their opinion as fact, any online forum will demonstrate that. >as accurately as it can according to it's training data, and that is it's major flaw. so we teach people that they need to be discerning with the way they use the tool. Kids in elementary school are taught that generally trust runs from : personal blog -> .com -> .org -> .gov -> .edu At this point ChatGPT sits around the "personal blog" level of trust. As long as people use the tool with this in mind then it's just business as usual.
PeterisP 4y ago
The thing to understand about ChatGPT is that at its core it isn't a question-answering machine, it's a text-continuation machine which by pure coincidence often continues a question with its answer. And it's learning primarily from data from people posting stuff on the internet (and only very rarely seeing actual conversations) so it has learned that "human language" pretty much never involves people saying that they don't know stuff, but does involve a LOT of assertive speculating and convinced 'preaching' even when saying untrue things. Like, imagine seeing a human-written a blog post for some click-bait site titled "Venus Williams eighth grand slam victory" or "The 10 secret endings for Inside" - ChatGPT will write pretty much the same thing as that human would. That blog post won't say "ah, I have no idea about these endings, I haven't seen them in my games", it would confidently repeat various rumors and if the author would invent some new fan theories, then they would often be stated in an assertive manner as fact. Furthermore, when doing its final reinforcement learning fine-tuning on top of the core GPT "3.5" model, it almost certainly was asked about some stuff which was simply more recent than the data used in training the core model - I mean, it can't know what happened last month - but it would be penalized if it would assert that things are the exact same as they were when the training data ended and reject things the user explicitly states that contradict its (inevitably outdated) knowledge. For all it knows, Williams has won 8 grand slams, simply the eighth one happened in the year after ChatGPT "stopped reading newspapers", so if a discussion mentions the eighth victory it has to assume that it is real, otherwise it would fail in so many other scenarios - e.g. refusing to acknowledge that some country has elected a different government or that a war has started.
- jack_h 4y ago
  To drive the point home, look at all the rampant speculation about the recent high altitude balloon news. People play armchair ___ and invent plausible sounding explanations all the time, and people believe them.
  - PeterisP 4y ago
    Yes, and for the other part, if you asked ChatGPT "Hey, why did USA shoot down that Chinese balloon" and it answered "as far as I know, USA has never ever shot down a Chinese balloon" - which is technically true - people would consider it a bad answer.
gremlinsinc 4y ago
If you ask google this same thing, you could still get the wrong answer, what if there's two games called 'inside', or it mistakes 'inside' the game for the Pixar movie of the same name. Are you telling me you've never googled and gotten shit results, aka unusable stuff? Everything comes back with 100% accuracy? I mean, I know we need to be more skeptical of the results, but this is also early days. All we need are fact-checking engines that can give some sort of confidence score to results, and that should at least start moving in the right direction. 10 years from now, this will be moot, as the models they will have will be so far advanced we, probably won't even think it's possible or can fathom what it does to disrupt society. To say generative X hasn't disrupted industry already is insane, whether it's art, music, video, text - generative ai is going to only get better and more insane as text to video and voice cloning tech becomes mainstream. Imagine being self-conscous and wanting to do a youtube channel, so you have a virtual avatar with your own voice do it for you - you just write the script. (This is where I really want the ai tech to shine, as I want to do just this. :))
- the_af 4y ago
  > If you ask google this same thing, you could still get the wrong answer, what if there's two games called 'inside', or it mistakes 'inside' the game for the Pixar movie of the same name. To me the difference is the quality of the hallucinations. If I ask Google about the secret Inside endings, and it gives me something about a space adventure, I know something is off. There's no "space" in the Inside I know. But ChatGPT when asked about 10 secret endings hallucinates completely sensible endings that make sense for Inside... it's just that none of them actually exist. That's a major difference between Google's incorrect search results and what ChatGPT provides.