3 ms·

> The cynic in me says that there will be Chat-GPT generation of kids that won't be able to think rationally on their own This is, unfortunately, already happe

> The cynic in me says that there will be Chat-GPT generation of kids that won't be able to think rationally on their own This is, unfortunately, already happening. I've seen a terrifying number of users on Reddit uncritically using ChatGPT as a "source", or expecting (largely unsuccessfully) to have it give them expert guidance in performing tasks like writing software. Making ChatGPT available to the general public, even as a "test", was a mistake. It's simply too good at sounding convincingly authoritative while being completely wrong.

philistine 4y ago
They simply should have charged for it.
blibble 4y ago
hopefully the lesson that enters our culture is "AI produces garbage", which then kills off the AI revolution permanently
- tempodox 4y ago
  Nah, it will be just another “AI winter”, if even that much, until the next hype cycle begins. As this example demonstrates, humanity will fall for complete and utter nonsense, hook, line and sinker.
  - mercer 4y ago
    Yeah let's just chuck it all in the bin unless it's 100% useful!
zelphirkalt 4y ago
> It's simply too good at sounding convincingly authoritative while being completely wrong. In a way that is what we humans do way too often. People talking about stuff thinking they know, but not actually knowing a lot. I guess that is where it learned that from.
- ryanar 4y ago
  yep, my dad would always say I have an answer for everything, even topics I know nothing about, I was/am just really good at stringing together bits of knowledge in a way that sounds very plausible, combine that with cis/white/het male speaker and you have HumanGPT
  - peepeepoopoo3 4y ago
    Way to throw identity politics into it.
  - ryanar_subhuman 4y ago
    [dead]
- cheald 4y ago
  In humans, though, there's a degree of matching between knowledge of a topic and ability to speak on that topic, which can betray the uninformed. People often speak on a topic too coarsely (in a way that betrays their lack of understanding beyond a headline level), or more often, they try to disguise their ignorance with postdoc-level vocabulary and wordy dissertations that try to overwhelm the listener to cause them to conclude that since they don't understand all the terms being thrown about, the speaker must know something they don't. But, after you've been exposed to this a few times, you start to get a sense for it. Most people have a "bullshit meter" which gets calibrated over time, and it's remarkably good at sniffing out the people who are out of their depth. ChatGPT is somewhat different in that its level of command of the English language doesn't change in response to its level of knowledge about the topic it's asked about. It most likely has the most thorough command of the English language of any entity ever born or created; that is exactly and specifically what LLMs are supposed to be good at. Because of that, it can speak "perfectly" no matter the topic; it can be prompted to explain things in "simple terms", but remain eloquent. It doesn't fall back on jargon to mask its ignorance, and it doesn't breeze past important concepts without bothering to explain them. As a result, it communicates exactly like an actual domain expert would - carefully, precisely, and simply. This is wonderful for clear, effective communication, but it completely subverts the bullshit meter. In some ways, maybe this could be a blessing. Media and marketing organizations have gotten very, very good at sitting in the "happy language zone" as they straight-up lie to our faces. Perhaps just flooding the zone with language which doesn't trip the bullshit meter and still manages to be empirically wrong could cause people to readjust and redevelop the heuristics they use to evaluate the trustworthiness of information.
pessimizer 4y ago
No, it's actually good, because it's one of the rare huge advances that the public is being let in on at the outset, and we'll have a lot more ability to judge LLM potential than if it had been sprung on us out of the blue (with inevitable marketing implying that it was actual strong AI, and restrictions on interaction with it carefully designed to maintain that falsehood.) > This is, unfortunately, already happening. Constantly getting bad and wrong information from a confident AI will do a lot to teach people that they need other sources. The fact that many of these other sources will also be covert lying AI will be a great lesson in media literacy for everyone.
- chowells 4y ago
  Why do you think people will respond rationally when they have an irrational problem?
- layer8 4y ago
  > The fact that many of these other sources will also be covert lying AI will be a great lesson in media literacy for everyone. Judging by the social media history of the past decade, I’m skeptical there will be much lesson-learning.
klyrs 4y ago
> I've seen a terrifying number of users on Reddit uncritically using ChatGPT as a "source"... I see this occurring on HN, too. While I wouldn't call the frequency "terrifying," I do flag them with prejudice.

philistine 4y ago

They simply should have charged for it.

blibble 4y ago

hopefully the lesson that enters our culture is "AI produces garbage", which then kills off the AI revolution permanently

tempodox 4y ago
Nah, it will be just another “AI winter”, if even that much, until the next hype cycle begins. As this example demonstrates, humanity will fall for complete and utter nonsense, hook, line and sinker.
- mercer 4y ago
  Yeah let's just chuck it all in the bin unless it's 100% useful!

zelphirkalt 4y ago

> It's simply too good at sounding convincingly authoritative while being completely wrong. In a way that is what we humans do way too often. People talking about stuff thinking they know, but not actually knowing a lot. I guess that is where it learned that from.

ryanar 4y ago
yep, my dad would always say I have an answer for everything, even topics I know nothing about, I was/am just really good at stringing together bits of knowledge in a way that sounds very plausible, combine that with cis/white/het male speaker and you have HumanGPT
- peepeepoopoo3 4y ago
  Way to throw identity politics into it.
- ryanar_subhuman 4y ago
  [dead]
cheald 4y ago
In humans, though, there's a degree of matching between knowledge of a topic and ability to speak on that topic, which can betray the uninformed. People often speak on a topic too coarsely (in a way that betrays their lack of understanding beyond a headline level), or more often, they try to disguise their ignorance with postdoc-level vocabulary and wordy dissertations that try to overwhelm the listener to cause them to conclude that since they don't understand all the terms being thrown about, the speaker must know something they don't. But, after you've been exposed to this a few times, you start to get a sense for it. Most people have a "bullshit meter" which gets calibrated over time, and it's remarkably good at sniffing out the people who are out of their depth. ChatGPT is somewhat different in that its level of command of the English language doesn't change in response to its level of knowledge about the topic it's asked about. It most likely has the most thorough command of the English language of any entity ever born or created; that is exactly and specifically what LLMs are supposed to be good at. Because of that, it can speak "perfectly" no matter the topic; it can be prompted to explain things in "simple terms", but remain eloquent. It doesn't fall back on jargon to mask its ignorance, and it doesn't breeze past important concepts without bothering to explain them. As a result, it communicates exactly like an actual domain expert would - carefully, precisely, and simply. This is wonderful for clear, effective communication, but it completely subverts the bullshit meter. In some ways, maybe this could be a blessing. Media and marketing organizations have gotten very, very good at sitting in the "happy language zone" as they straight-up lie to our faces. Perhaps just flooding the zone with language which doesn't trip the bullshit meter and still manages to be empirically wrong could cause people to readjust and redevelop the heuristics they use to evaluate the trustworthiness of information.

pessimizer 4y ago

No, it's actually good, because it's one of the rare huge advances that the public is being let in on at the outset, and we'll have a lot more ability to judge LLM potential than if it had been sprung on us out of the blue (with inevitable marketing implying that it was actual strong AI, and restrictions on interaction with it carefully designed to maintain that falsehood.) > This is, unfortunately, already happening. Constantly getting bad and wrong information from a confident AI will do a lot to teach people that they need other sources. The fact that many of these other sources will also be covert lying AI will be a great lesson in media literacy for everyone.

chowells 4y ago
Why do you think people will respond rationally when they have an irrational problem?
layer8 4y ago
> The fact that many of these other sources will also be covert lying AI will be a great lesson in media literacy for everyone. Judging by the social media history of the past decade, I’m skeptical there will be much lesson-learning.

klyrs 4y ago

> I've seen a terrifying number of users on Reddit uncritically using ChatGPT as a "source"... I see this occurring on HN, too. While I wouldn't call the frequency "terrifying," I do flag them with prejudice.