6 ms·

I've been playing with the combination of Chat GPT and Wolfram, and while there are gold nuggets in the output sometimes, it is mostly terrifying non-sense for

by dsign 4y ago

I've been playing with the combination of Chat GPT and Wolfram, and while there are gold nuggets in the output sometimes, it is mostly terrifying non-sense for anybody who couldn't already verify those outputs on their own. The cynic in me says that there will be Chat-GPT generation of kids that won't be able to think rationally on their own, the same way that kids and young people these days are completely unable to have a normal sleep schedule.

s3p 4y ago
I'm not sure the last claim you make follows from your line of reasoning. If you are sure tech(?) is ruining kids' sleep schedules, but only kids, I would love to see evidence of it.
duskwuff 4y ago
> The cynic in me says that there will be Chat-GPT generation of kids that won't be able to think rationally on their own This is, unfortunately, already happening. I've seen a terrifying number of users on Reddit uncritically using ChatGPT as a "source", or expecting (largely unsuccessfully) to have it give them expert guidance in performing tasks like writing software. Making ChatGPT available to the general public, even as a "test", was a mistake. It's simply too good at sounding convincingly authoritative while being completely wrong.
- philistine 4y ago
  They simply should have charged for it.
- blibble 4y ago
  hopefully the lesson that enters our culture is "AI produces garbage", which then kills off the AI revolution permanently
  - tempodox 4y ago
    Nah, it will be just another “AI winter”, if even that much, until the next hype cycle begins. As this example demonstrates, humanity will fall for complete and utter nonsense, hook, line and sinker.
    - mercer 4y ago
      Yeah let's just chuck it all in the bin unless it's 100% useful!
- zelphirkalt 4y ago
  > It's simply too good at sounding convincingly authoritative while being completely wrong. In a way that is what we humans do way too often. People talking about stuff thinking they know, but not actually knowing a lot. I guess that is where it learned that from.
  - ryanar 4y ago
    yep, my dad would always say I have an answer for everything, even topics I know nothing about, I was/am just really good at stringing together bits of knowledge in a way that sounds very plausible, combine that with cis/white/het male speaker and you have HumanGPT
    - peepeepoopoo3 4y ago
      Way to throw identity politics into it.
    - ryanar_subhuman 4y ago
      [dead]
  - cheald 4y ago
    In humans, though, there's a degree of matching between knowledge of a topic and ability to speak on that topic, which can betray the uninformed. People often speak on a topic too coarsely (in a way that betrays their lack of understanding beyond a headline level), or more often, they try to disguise their ignorance with postdoc-level vocabulary and wordy dissertations that try to overwhelm the listener to cause them to conclude that since they don't understand all the terms being thrown about, the speaker must know something they don't. But, after you've been exposed to this a few times, you start to get a sense for it. Most people have a "bullshit meter" which gets calibrated over time, and it's remarkably good at sniffing out the people who are out of their depth. ChatGPT is somewhat different in that its level of command of the English language doesn't change in response to its level of knowledge about the topic it's asked about. It most likely has the most thorough command of the English language of any entity ever born or created; that is exactly and specifically what LLMs are supposed to be good at. Because of that, it can speak "perfectly" no matter the topic; it can be prompted to explain things in "simple terms", but remain eloquent. It doesn't fall back on jargon to mask its ignorance, and it doesn't breeze past important concepts without bothering to explain them. As a result, it communicates exactly like an actual domain expert would - carefully, precisely, and simply. This is wonderful for clear, effective communication, but it completely subverts the bullshit meter. In some ways, maybe this could be a blessing. Media and marketing organizations have gotten very, very good at sitting in the "happy language zone" as they straight-up lie to our faces. Perhaps just flooding the zone with language which doesn't trip the bullshit meter and still manages to be empirically wrong could cause people to readjust and redevelop the heuristics they use to evaluate the trustworthiness of information.
- pessimizer 4y ago
  No, it's actually good, because it's one of the rare huge advances that the public is being let in on at the outset, and we'll have a lot more ability to judge LLM potential than if it had been sprung on us out of the blue (with inevitable marketing implying that it was actual strong AI, and restrictions on interaction with it carefully designed to maintain that falsehood.) > This is, unfortunately, already happening. Constantly getting bad and wrong information from a confident AI will do a lot to teach people that they need other sources. The fact that many of these other sources will also be covert lying AI will be a great lesson in media literacy for everyone.
  - chowells 4y ago
    Why do you think people will respond rationally when they have an irrational problem?
  - layer8 4y ago
    > The fact that many of these other sources will also be covert lying AI will be a great lesson in media literacy for everyone. Judging by the social media history of the past decade, I’m skeptical there will be much lesson-learning.
- klyrs 4y ago
  > I've seen a terrifying number of users on Reddit uncritically using ChatGPT as a "source"... I see this occurring on HN, too. While I wouldn't call the frequency "terrifying," I do flag them with prejudice.
mrcwinn 4y ago
It's a fair point. I actually slept 8 hours last night (made sure not to eat after 7 or so, set my phone down early on). What's noteworthy is that... I noticed and was so excited I'd slept 8 hours. That used to not be noteworthy!
ToValueFunfetti 4y ago
I would expect an unreliable source to have the opposite effect. When I use ChatGPT, I find myself thinking critically and questioning everything in a way that I really should be applying to everything else I read.
- Sharlin 4y ago
  First you have to realize that ChatGPT is unreliable in the first place. I guess many will have to learn their lesson the hard way.
  - pessimizer 4y ago
    All people know stuff about stuff, will talk (have already talked) to AI about that stuff that they know, and will figure out that AI is lying about stuff they know. Maybe this can reduce the number of people who believe everything they read to a less dangerous level.
    - BenjiWiebe 4y ago
      Gell-Mann amnesia...
  - mercer 4y ago
    Everyone I know already knows this, because ChatGPT makes such obvious mistakes in between brilliant takes. Hell, my grandmother knows that this 'ai' thing does something called 'hallucinate' because it was on the news a few times.
    - Sharlin 4y ago
      Everyone you know may be a less representative sample of the general population than you think.
- thrashh 4y ago
  I agree. You only need to be done dirty once, then you'll remember.
random3 4y ago
Let's call them gpt-cyborgs. Some will be worse, some better off. Leveraging the "real" world would be in their advantage. Here's an example - can't prove it's correct, but it could be measured. Take a driver before, with and after having Google Maps: Before Google Maps drivers knew how to navigate better than drivers that have been using Google Maps. I.e. After removing Google Maps some drivers will be lost. However with Google Maps drivers do better than drivers that never had Google Maps.
RC_ITR 4y ago
>The cynic in me says that there will be Chat-GPT generation of kids that won't be able to think rationally on their own It’s interesting to me that your implication is “ChatGPT” will give reasonable but wrong answers to things, therefore people will accept those answers. It reminds me quite a bit of “no one will know math because calculators.” There’s plenty of feedback loops that exist for exactly this problem. Kids submit GPT homework and it’s either correct (rendering the skill the homework was testing worthless), or it’s not, and those kids will be punished with bad grades. If anything it will teach a whole new generation how to analyze semi-unreliable text, in the same way Google taught millennials how to search though disparate sources and synthesize answers. >the same way that kids and young people these days are completely unable to have a normal sleep schedule. Conclusions: A lack of empirical evidence for sleep recommendations was universally acknowledged. Inadequate sleep was seen as a consequence of "modern life," associated with technologies of the time. No matter how much sleep children are getting, it has always been assumed that they need more. https://pubmed.ncbi.nlm.nih.gov/22331340/ https://pubmed.ncbi.nlm.nih.gov/22331340/
- asimpleusecase 4y ago
  On the kids and calculators… in the past week I have had two times that it was clear the person at the till could not make change. In one case the total was £4.90 I gave them a £10 note. Quickly I added a 20p coin so they could give me £5.10 back and save them having to give me so many coins. The person was in the 20’s and I could see immediately they did not know what to do. The phone came out of the pocket and calculator function stated and some tapping proceeded but they did not look comforted and then there was tapping on the till and then I was handed £5 and they rushed to the back room. I would have chalked it up a one off but it happened again a few days later. Admittedly it dealt with an antiquated form of money. However, I would guess that level of innumeracy has to impact them in many way in normal life.
  - snozolli 4y ago
    In one case the total was £4.90 I gave them a £10 note. Quickly I added a 20p coin so they could give me £5.10 10 - 4.90 = 5.10. Why was another 20 involved? Edit: I'm guessing the total was 5.10, so you gave 10.20 to get 5.10 back.
ryanar 4y ago
Can you elaborate more on “kids and young people these days are completely unable to have a normal sleep schedule”? I have seen cases of this in young people, but I didn’t know the issue was widespread and would be interested to learn more.
barrysteve 4y ago
The second most terrifying thing would be an expert university professor on the warpath because they have to correct every single passing remark ever coughed up. We are giving critical experts way too much responsibility to hold the untrained chatgpt-er masses to account. Open-standards fact checking, or a global heirarchy of reputation could be plausible solutions. I really don't want to listen to another set of public academics, so a global resource like wikipedia, with a secure (or conceptually secure-ish) blockchain-alike technology to ensure the untampered communication and authenticity of the original fact source, to relay "base factual" information on the web, would be preferable to my mind.
bondarchuk 4y ago
On the contrary, I think there will be a chatgpt generation of kids who understand exactly to take these outputs with a huge grain of salt, while there will be a cohort of currently-adult people who are so used to believing almost anything written in text that they will do the same with ai outputs.
- sterlind 4y ago
  100%. Most of the people I see posting fake news (as in, rumors from bot tweets/generated pages that I've determined is false from easily accessible, primary sources) are older. Though maybe that's because older people are on Twitter, and I'd find the same on TikTok.
- wruza 4y ago
  They’ll have to take something without salt to learn anyway. It seems pretty inevitable that AIs will replace generic search engines and change the way “blogs” work. Like it doesn’t matter to google today that you wanted ["foo" "bar"]. You may be smart, but a completely irrelevant “S”E“R”P is all you can have.

by dsign 4y ago

s3p 4y ago

I'm not sure the last claim you make follows from your line of reasoning. If you are sure tech(?) is ruining kids' sleep schedules, but only kids, I would love to see evidence of it.

duskwuff 4y ago

> The cynic in me says that there will be Chat-GPT generation of kids that won't be able to think rationally on their own This is, unfortunately, already happening. I've seen a terrifying number of users on Reddit uncritically using ChatGPT as a "source", or expecting (largely unsuccessfully) to have it give them expert guidance in performing tasks like writing software. Making ChatGPT available to the general public, even as a "test", was a mistake. It's simply too good at sounding convincingly authoritative while being completely wrong.

philistine 4y ago
They simply should have charged for it.
blibble 4y ago
hopefully the lesson that enters our culture is "AI produces garbage", which then kills off the AI revolution permanently
- tempodox 4y ago
  Nah, it will be just another “AI winter”, if even that much, until the next hype cycle begins. As this example demonstrates, humanity will fall for complete and utter nonsense, hook, line and sinker.
  - mercer 4y ago
    Yeah let's just chuck it all in the bin unless it's 100% useful!
zelphirkalt 4y ago
> It's simply too good at sounding convincingly authoritative while being completely wrong. In a way that is what we humans do way too often. People talking about stuff thinking they know, but not actually knowing a lot. I guess that is where it learned that from.
- ryanar 4y ago
  yep, my dad would always say I have an answer for everything, even topics I know nothing about, I was/am just really good at stringing together bits of knowledge in a way that sounds very plausible, combine that with cis/white/het male speaker and you have HumanGPT
  - peepeepoopoo3 4y ago
    Way to throw identity politics into it.
  - ryanar_subhuman 4y ago
    [dead]
- cheald 4y ago
  In humans, though, there's a degree of matching between knowledge of a topic and ability to speak on that topic, which can betray the uninformed. People often speak on a topic too coarsely (in a way that betrays their lack of understanding beyond a headline level), or more often, they try to disguise their ignorance with postdoc-level vocabulary and wordy dissertations that try to overwhelm the listener to cause them to conclude that since they don't understand all the terms being thrown about, the speaker must know something they don't. But, after you've been exposed to this a few times, you start to get a sense for it. Most people have a "bullshit meter" which gets calibrated over time, and it's remarkably good at sniffing out the people who are out of their depth. ChatGPT is somewhat different in that its level of command of the English language doesn't change in response to its level of knowledge about the topic it's asked about. It most likely has the most thorough command of the English language of any entity ever born or created; that is exactly and specifically what LLMs are supposed to be good at. Because of that, it can speak "perfectly" no matter the topic; it can be prompted to explain things in "simple terms", but remain eloquent. It doesn't fall back on jargon to mask its ignorance, and it doesn't breeze past important concepts without bothering to explain them. As a result, it communicates exactly like an actual domain expert would - carefully, precisely, and simply. This is wonderful for clear, effective communication, but it completely subverts the bullshit meter. In some ways, maybe this could be a blessing. Media and marketing organizations have gotten very, very good at sitting in the "happy language zone" as they straight-up lie to our faces. Perhaps just flooding the zone with language which doesn't trip the bullshit meter and still manages to be empirically wrong could cause people to readjust and redevelop the heuristics they use to evaluate the trustworthiness of information.
pessimizer 4y ago
No, it's actually good, because it's one of the rare huge advances that the public is being let in on at the outset, and we'll have a lot more ability to judge LLM potential than if it had been sprung on us out of the blue (with inevitable marketing implying that it was actual strong AI, and restrictions on interaction with it carefully designed to maintain that falsehood.) > This is, unfortunately, already happening. Constantly getting bad and wrong information from a confident AI will do a lot to teach people that they need other sources. The fact that many of these other sources will also be covert lying AI will be a great lesson in media literacy for everyone.
- chowells 4y ago
  Why do you think people will respond rationally when they have an irrational problem?
- layer8 4y ago
  > The fact that many of these other sources will also be covert lying AI will be a great lesson in media literacy for everyone. Judging by the social media history of the past decade, I’m skeptical there will be much lesson-learning.
klyrs 4y ago
> I've seen a terrifying number of users on Reddit uncritically using ChatGPT as a "source"... I see this occurring on HN, too. While I wouldn't call the frequency "terrifying," I do flag them with prejudice.

mrcwinn 4y ago

It's a fair point. I actually slept 8 hours last night (made sure not to eat after 7 or so, set my phone down early on). What's noteworthy is that... I noticed and was so excited I'd slept 8 hours. That used to not be noteworthy!

ToValueFunfetti 4y ago

I would expect an unreliable source to have the opposite effect. When I use ChatGPT, I find myself thinking critically and questioning everything in a way that I really should be applying to everything else I read.

Sharlin 4y ago
First you have to realize that ChatGPT is unreliable in the first place. I guess many will have to learn their lesson the hard way.
- pessimizer 4y ago
  All people know stuff about stuff, will talk (have already talked) to AI about that stuff that they know, and will figure out that AI is lying about stuff they know. Maybe this can reduce the number of people who believe everything they read to a less dangerous level.
  - BenjiWiebe 4y ago
    Gell-Mann amnesia...
- mercer 4y ago
  Everyone I know already knows this, because ChatGPT makes such obvious mistakes in between brilliant takes. Hell, my grandmother knows that this 'ai' thing does something called 'hallucinate' because it was on the news a few times.
  - Sharlin 4y ago
    Everyone you know may be a less representative sample of the general population than you think.
thrashh 4y ago
I agree. You only need to be done dirty once, then you'll remember.

random3 4y ago

Let's call them gpt-cyborgs. Some will be worse, some better off. Leveraging the "real" world would be in their advantage. Here's an example - can't prove it's correct, but it could be measured. Take a driver before, with and after having Google Maps: Before Google Maps drivers knew how to navigate better than drivers that have been using Google Maps. I.e. After removing Google Maps some drivers will be lost. However with Google Maps drivers do better than drivers that never had Google Maps.

RC_ITR 4y ago

>The cynic in me says that there will be Chat-GPT generation of kids that won't be able to think rationally on their own It’s interesting to me that your implication is “ChatGPT” will give reasonable but wrong answers to things, therefore people will accept those answers. It reminds me quite a bit of “no one will know math because calculators.” There’s plenty of feedback loops that exist for exactly this problem. Kids submit GPT homework and it’s either correct (rendering the skill the homework was testing worthless), or it’s not, and those kids will be punished with bad grades. If anything it will teach a whole new generation how to analyze semi-unreliable text, in the same way Google taught millennials how to search though disparate sources and synthesize answers. >the same way that kids and young people these days are completely unable to have a normal sleep schedule. Conclusions: A lack of empirical evidence for sleep recommendations was universally acknowledged. Inadequate sleep was seen as a consequence of "modern life," associated with technologies of the time. No matter how much sleep children are getting, it has always been assumed that they need more. https://pubmed.ncbi.nlm.nih.gov/22331340/ https://pubmed.ncbi.nlm.nih.gov/22331340/

asimpleusecase 4y ago
On the kids and calculators… in the past week I have had two times that it was clear the person at the till could not make change. In one case the total was £4.90 I gave them a £10 note. Quickly I added a 20p coin so they could give me £5.10 back and save them having to give me so many coins. The person was in the 20’s and I could see immediately they did not know what to do. The phone came out of the pocket and calculator function stated and some tapping proceeded but they did not look comforted and then there was tapping on the till and then I was handed £5 and they rushed to the back room. I would have chalked it up a one off but it happened again a few days later. Admittedly it dealt with an antiquated form of money. However, I would guess that level of innumeracy has to impact them in many way in normal life.
- snozolli 4y ago
  In one case the total was £4.90 I gave them a £10 note. Quickly I added a 20p coin so they could give me £5.10 10 - 4.90 = 5.10. Why was another 20 involved? Edit: I'm guessing the total was 5.10, so you gave 10.20 to get 5.10 back.

ryanar 4y ago

Can you elaborate more on “kids and young people these days are completely unable to have a normal sleep schedule”? I have seen cases of this in young people, but I didn’t know the issue was widespread and would be interested to learn more.

barrysteve 4y ago

The second most terrifying thing would be an expert university professor on the warpath because they have to correct every single passing remark ever coughed up. We are giving critical experts way too much responsibility to hold the untrained chatgpt-er masses to account. Open-standards fact checking, or a global heirarchy of reputation could be plausible solutions. I really don't want to listen to another set of public academics, so a global resource like wikipedia, with a secure (or conceptually secure-ish) blockchain-alike technology to ensure the untampered communication and authenticity of the original fact source, to relay "base factual" information on the web, would be preferable to my mind.

bondarchuk 4y ago

On the contrary, I think there will be a chatgpt generation of kids who understand exactly to take these outputs with a huge grain of salt, while there will be a cohort of currently-adult people who are so used to believing almost anything written in text that they will do the same with ai outputs.

sterlind 4y ago
100%. Most of the people I see posting fake news (as in, rumors from bot tweets/generated pages that I've determined is false from easily accessible, primary sources) are older. Though maybe that's because older people are on Twitter, and I'd find the same on TikTok.
wruza 4y ago
They’ll have to take something without salt to learn anyway. It seems pretty inevitable that AIs will replace generic search engines and change the way “blogs” work. Like it doesn’t matter to google today that you wanted ["foo" "bar"]. You may be smart, but a completely irrelevant “S”E“R”P is all you can have.