4 ms·

From TFA: > For instance, if you ask it to define "At what tournament did Venus Williams win her eighth grand slam?" it will make up an answer for you, even th

by the_af 4y ago

From TFA: > For instance, if you ask it to define "At what tournament did Venus Williams win her eighth grand slam?" it will make up an answer for you, even though Venus Williams has won seven grand slams. She wanted to win her eighth, as many outlets reported, but she didn't. And the model is confusing the two notions of "wanted to win" vs. "won." I find even this is giving ChatGPT too much credit. You need way less to make it bullshit its way through an answer: when asking it to explain the plot of the videogame "Inside" (as an aside: an amazing game from the makers of "Limbo"), ChatGPT did reasonably well. As expected. Then I was curious what it thought about the secret alternate ending and... it made something up, completely unrelated to the real ending. This is surprising, since plenty of articles speculate about the meaning of this ending. Then I told ChatGPT, "now that I think about it, there were 10 endings, could you describe them?"... and it happily produced 10 endings for Inside, all in keeping with the theme of the game, all completely made up. If you make a mistake and honestly ask ChatGPT to produce the answer to something that doesn't exist, it will happily oblige.

ux-app 4y ago
Computers are the ultimate appeal to authority in the style of "Computer says no". >If you make a mistake and honestly ask ChatGPT to produce the answer to something that doesn't exist, it will happily oblige. Just like asking people right? We're already accepting "truths" which we don't ourselves verify. I imagine that the vast majority of people accept the answers google provides on their results page. Is this any different? Schools teach young people to critically evaluate information sources whether they're books, TV, webpages etc. This is no different.
- worble 4y ago
  >Just like asking people right? Not really? I mean if you ask people to explain the 10 endings to Inside they'd probably just go "there we 10 endings? I had no idea" or something to that effect (if they weren't going "What's Inside?" that is). ChatGPT has no concept of being wrong, all it wants to do is a respond in a way that sounds human, as accurately as it can according to it's training data, and that is it's major flaw.
  - kaetemi 4y ago
    More like asking students in an exam question to summarize the book they were supposed to read.
  - ux-app 4y ago
    >Not really? I mean if you ask people to explain the 10 endings to Inside that's just the computer doing what computers do. the point is that humans routinely and confidently state their opinion as fact, any online forum will demonstrate that. >as accurately as it can according to it's training data, and that is it's major flaw. so we teach people that they need to be discerning with the way they use the tool. Kids in elementary school are taught that generally trust runs from : personal blog -> .com -> .org -> .gov -> .edu At this point ChatGPT sits around the "personal blog" level of trust. As long as people use the tool with this in mind then it's just business as usual.
  - PeterisP 4y ago
    The thing to understand about ChatGPT is that at its core it isn't a question-answering machine, it's a text-continuation machine which by pure coincidence often continues a question with its answer. And it's learning primarily from data from people posting stuff on the internet (and only very rarely seeing actual conversations) so it has learned that "human language" pretty much never involves people saying that they don't know stuff, but does involve a LOT of assertive speculating and convinced 'preaching' even when saying untrue things. Like, imagine seeing a human-written a blog post for some click-bait site titled "Venus Williams eighth grand slam victory" or "The 10 secret endings for Inside" - ChatGPT will write pretty much the same thing as that human would. That blog post won't say "ah, I have no idea about these endings, I haven't seen them in my games", it would confidently repeat various rumors and if the author would invent some new fan theories, then they would often be stated in an assertive manner as fact. Furthermore, when doing its final reinforcement learning fine-tuning on top of the core GPT "3.5" model, it almost certainly was asked about some stuff which was simply more recent than the data used in training the core model - I mean, it can't know what happened last month - but it would be penalized if it would assert that things are the exact same as they were when the training data ended and reject things the user explicitly states that contradict its (inevitably outdated) knowledge. For all it knows, Williams has won 8 grand slams, simply the eighth one happened in the year after ChatGPT "stopped reading newspapers", so if a discussion mentions the eighth victory it has to assume that it is real, otherwise it would fail in so many other scenarios - e.g. refusing to acknowledge that some country has elected a different government or that a war has started.
totetsu 4y ago
It's great that you can just use the prompt "just make it up" when chatGPT gives I can't answer that response.

by the_af 4y ago

ux-app 4y ago

Computers are the ultimate appeal to authority in the style of "Computer says no". >If you make a mistake and honestly ask ChatGPT to produce the answer to something that doesn't exist, it will happily oblige. Just like asking people right? We're already accepting "truths" which we don't ourselves verify. I imagine that the vast majority of people accept the answers google provides on their results page. Is this any different? Schools teach young people to critically evaluate information sources whether they're books, TV, webpages etc. This is no different.

worble 4y ago
>Just like asking people right? Not really? I mean if you ask people to explain the 10 endings to Inside they'd probably just go "there we 10 endings? I had no idea" or something to that effect (if they weren't going "What's Inside?" that is). ChatGPT has no concept of being wrong, all it wants to do is a respond in a way that sounds human, as accurately as it can according to it's training data, and that is it's major flaw.
- kaetemi 4y ago
  More like asking students in an exam question to summarize the book they were supposed to read.
- ux-app 4y ago
  >Not really? I mean if you ask people to explain the 10 endings to Inside that's just the computer doing what computers do. the point is that humans routinely and confidently state their opinion as fact, any online forum will demonstrate that. >as accurately as it can according to it's training data, and that is it's major flaw. so we teach people that they need to be discerning with the way they use the tool. Kids in elementary school are taught that generally trust runs from : personal blog -> .com -> .org -> .gov -> .edu At this point ChatGPT sits around the "personal blog" level of trust. As long as people use the tool with this in mind then it's just business as usual.
- PeterisP 4y ago
  The thing to understand about ChatGPT is that at its core it isn't a question-answering machine, it's a text-continuation machine which by pure coincidence often continues a question with its answer. And it's learning primarily from data from people posting stuff on the internet (and only very rarely seeing actual conversations) so it has learned that "human language" pretty much never involves people saying that they don't know stuff, but does involve a LOT of assertive speculating and convinced 'preaching' even when saying untrue things. Like, imagine seeing a human-written a blog post for some click-bait site titled "Venus Williams eighth grand slam victory" or "The 10 secret endings for Inside" - ChatGPT will write pretty much the same thing as that human would. That blog post won't say "ah, I have no idea about these endings, I haven't seen them in my games", it would confidently repeat various rumors and if the author would invent some new fan theories, then they would often be stated in an assertive manner as fact. Furthermore, when doing its final reinforcement learning fine-tuning on top of the core GPT "3.5" model, it almost certainly was asked about some stuff which was simply more recent than the data used in training the core model - I mean, it can't know what happened last month - but it would be penalized if it would assert that things are the exact same as they were when the training data ended and reject things the user explicitly states that contradict its (inevitably outdated) knowledge. For all it knows, Williams has won 8 grand slams, simply the eighth one happened in the year after ChatGPT "stopped reading newspapers", so if a discussion mentions the eighth victory it has to assume that it is real, otherwise it would fail in so many other scenarios - e.g. refusing to acknowledge that some country has elected a different government or that a war has started.

totetsu 4y ago

It's great that you can just use the prompt "just make it up" when chatGPT gives I can't answer that response.