5 ms·

I don’t find this particular example very impressive. The U-238 answer is a bunch of disconnected sentences. U-238 is radioactive and releases something like 4

by amluto 4y ago

I don’t find this particular example very impressive. The U-238 answer is a bunch of disconnected sentences. U-238 is radioactive and releases something like 4 MeV per decay, not 200 MeV. It is not fissile, although one might calculate the net output of whacking it with enough neutrons that it eventually fissions. And if you ate it, unless your stomach was a breeder reactor, you would only have access to the decay energy, which would be large but would release so slowly as to be useless. ChatGPT is usually more impressive than this, IMO, at least when asked shallow questions.

sebzim4500 4y ago
Yeah I think it was thinking of U-235. At least in this case ChatGPT made it clear that the number did not come from Wolfram, so you don't need to be especially knowledgable to see that it shouldn't be trusted.
dsign 4y ago
I've been playing with the combination of Chat GPT and Wolfram, and while there are gold nuggets in the output sometimes, it is mostly terrifying non-sense for anybody who couldn't already verify those outputs on their own. The cynic in me says that there will be Chat-GPT generation of kids that won't be able to think rationally on their own, the same way that kids and young people these days are completely unable to have a normal sleep schedule.
- s3p 4y ago
  I'm not sure the last claim you make follows from your line of reasoning. If you are sure tech(?) is ruining kids' sleep schedules, but only kids, I would love to see evidence of it.
- duskwuff 4y ago
  > The cynic in me says that there will be Chat-GPT generation of kids that won't be able to think rationally on their own This is, unfortunately, already happening. I've seen a terrifying number of users on Reddit uncritically using ChatGPT as a "source", or expecting (largely unsuccessfully) to have it give them expert guidance in performing tasks like writing software. Making ChatGPT available to the general public, even as a "test", was a mistake. It's simply too good at sounding convincingly authoritative while being completely wrong.
  - philistine 4y ago
    They simply should have charged for it.
  - blibble 4y ago
    hopefully the lesson that enters our culture is "AI produces garbage", which then kills off the AI revolution permanently
    - tempodox 4y ago
      Nah, it will be just another “AI winter”, if even that much, until the next hype cycle begins. As this example demonstrates, humanity will fall for complete and utter nonsense, hook, line and sinker.
      - mercer 4y ago
        Yeah let's just chuck it all in the bin unless it's 100% useful!
  - zelphirkalt 4y ago
    > It's simply too good at sounding convincingly authoritative while being completely wrong. In a way that is what we humans do way too often. People talking about stuff thinking they know, but not actually knowing a lot. I guess that is where it learned that from.
    - ryanar 4y ago
      yep, my dad would always say I have an answer for everything, even topics I know nothing about, I was/am just really good at stringing together bits of knowledge in a way that sounds very plausible, combine that with cis/white/het male speaker and you have HumanGPT
      - peepeepoopoo3 4y ago
        Way to throw identity politics into it.
- mrcwinn 4y ago
  It's a fair point. I actually slept 8 hours last night (made sure not to eat after 7 or so, set my phone down early on). What's noteworthy is that... I noticed and was so excited I'd slept 8 hours. That used to not be noteworthy!
- ToValueFunfetti 4y ago
  I would expect an unreliable source to have the opposite effect. When I use ChatGPT, I find myself thinking critically and questioning everything in a way that I really should be applying to everything else I read.
  - Sharlin 4y ago
    First you have to realize that ChatGPT is unreliable in the first place. I guess many will have to learn their lesson the hard way.
    - pessimizer 4y ago
      All people know stuff about stuff, will talk (have already talked) to AI about that stuff that they know, and will figure out that AI is lying about stuff they know. Maybe this can reduce the number of people who believe everything they read to a less dangerous level.
      - BenjiWiebe 4y ago
        Gell-Mann amnesia...
    - mercer 4y ago
      Everyone I know already knows this, because ChatGPT makes such obvious mistakes in between brilliant takes. Hell, my grandmother knows that this 'ai' thing does something called 'hallucinate' because it was on the news a few times.
      - Sharlin 4y ago
        Everyone you know may be a less representative sample of the general population than you think.
  - thrashh 4y ago
    I agree. You only need to be done dirty once, then you'll remember.
- random3 4y ago
  Let's call them gpt-cyborgs. Some will be worse, some better off. Leveraging the "real" world would be in their advantage. Here's an example - can't prove it's correct, but it could be measured. Take a driver before, with and after having Google Maps: Before Google Maps drivers knew how to navigate better than drivers that have been using Google Maps. I.e. After removing Google Maps some drivers will be lost. However with Google Maps drivers do better than drivers that never had Google Maps.
- RC_ITR 4y ago
  >The cynic in me says that there will be Chat-GPT generation of kids that won't be able to think rationally on their own It’s interesting to me that your implication is “ChatGPT” will give reasonable but wrong answers to things, therefore people will accept those answers. It reminds me quite a bit of “no one will know math because calculators.” There’s plenty of feedback loops that exist for exactly this problem. Kids submit GPT homework and it’s either correct (rendering the skill the homework was testing worthless), or it’s not, and those kids will be punished with bad grades. If anything it will teach a whole new generation how to analyze semi-unreliable text, in the same way Google taught millennials how to search though disparate sources and synthesize answers. >the same way that kids and young people these days are completely unable to have a normal sleep schedule. Conclusions: A lack of empirical evidence for sleep recommendations was universally acknowledged. Inadequate sleep was seen as a consequence of "modern life," associated with technologies of the time. No matter how much sleep children are getting, it has always been assumed that they need more. https://pubmed.ncbi.nlm.nih.gov/22331340/ https://pubmed.ncbi.nlm.nih.gov/22331340/
  - asimpleusecase 4y ago
    On the kids and calculators… in the past week I have had two times that it was clear the person at the till could not make change. In one case the total was £4.90 I gave them a £10 note. Quickly I added a 20p coin so they could give me £5.10 back and save them having to give me so many coins. The person was in the 20’s and I could see immediately they did not know what to do. The phone came out of the pocket and calculator function stated and some tapping proceeded but they did not look comforted and then there was tapping on the till and then I was handed £5 and they rushed to the back room. I would have chalked it up a one off but it happened again a few days later. Admittedly it dealt with an antiquated form of money. However, I would guess that level of innumeracy has to impact them in many way in normal life.
    - snozolli 4y ago
      In one case the total was £4.90 I gave them a £10 note. Quickly I added a 20p coin so they could give me £5.10 10 - 4.90 = 5.10. Why was another 20 involved? Edit: I'm guessing the total was 5.10, so you gave 10.20 to get 5.10 back.
- ryanar 4y ago
  Can you elaborate more on “kids and young people these days are completely unable to have a normal sleep schedule”? I have seen cases of this in young people, but I didn’t know the issue was widespread and would be interested to learn more.
- barrysteve 4y ago
  The second most terrifying thing would be an expert university professor on the warpath because they have to correct every single passing remark ever coughed up. We are giving critical experts way too much responsibility to hold the untrained chatgpt-er masses to account. Open-standards fact checking, or a global heirarchy of reputation could be plausible solutions. I really don't want to listen to another set of public academics, so a global resource like wikipedia, with a secure (or conceptually secure-ish) blockchain-alike technology to ensure the untampered communication and authenticity of the original fact source, to relay "base factual" information on the web, would be preferable to my mind.
- bondarchuk 4y ago
  On the contrary, I think there will be a chatgpt generation of kids who understand exactly to take these outputs with a huge grain of salt, while there will be a cohort of currently-adult people who are so used to believing almost anything written in text that they will do the same with ai outputs.
  - sterlind 4y ago
    100%. Most of the people I see posting fake news (as in, rumors from bot tweets/generated pages that I've determined is false from easily accessible, primary sources) are older. Though maybe that's because older people are on Twitter, and I'd find the same on TikTok.
  - wruza 4y ago
    They’ll have to take something without salt to learn anyway. It seems pretty inevitable that AIs will replace generic search engines and change the way “blogs” work. Like it doesn’t matter to google today that you wanted ["foo" "bar"]. You may be smart, but a completely irrelevant “S”E“R”P is all you can have.
cheald 4y ago
This is going to be the big problem with the integration of LLMs into normal peoples' everyday lives; if you accept the answer as valid because it's eloquent and appears to follow some semblance of logical reasoning, you're going to be quickly misled. I've had numerous instances where ChatGPT has produced output which looks confidently authoritative, but which I as a domain expert can recognize as wrong or even nonsensical. One needs to _heavily_ tune their Gell-Mann amnesia detector when working with LLMs, and recall that if it's not getting the things right you are an expert on, it's likely not getting the things right that you're not an expert on, either.
- trzy 4y ago
  This reminds me of speaking with SV investors. They’re eloquent and say things that sound plausible in the moment but then I snap out of it and realize they’ve said nothing meaningful or correct at all.
  - atlasunshrugged 4y ago
    I think you'll find this in many places, I've noticed the same from SV also in DC, Berlin, etc. I think the upper echelons of society are just filled with people who often say very plausible eloquent things that aren't questioned for a variety of factors and it mostly just works for them.
- lamontcg 4y ago
  Given how humans are quickly misled by sociopathic humans who are eloquent and appear to follow logical reasoning, I don't really have any hope. People on the whole are incredibly bad at that particular skill (at least in our culture).
  - cheald 4y ago
    I'm inclined to agree with your assessment, given the existence of the concept of Gell-Mann amnesia at all. That said, I think we might have a glimmer of a chance to escape the executioner's axe here, as it were, because while a psychopath is intentionally deceptive, and will become evasive or aggressive when they sense that you're suspicious and probing of them, LLMs have no such quality, and will "happily" keep talking and exposing their shortcomings to those willing to listen. Ultimately, though, peoples' ability to understand how appropriately to trust LLMs will rest on their ability to understand their own capacity for being too-trusting and easily misled, a feature on which the human psyche does not have a great track record.
dist-epoch 4y ago
Thermal batteries, the kind used on spaceships, weight 15 kg and can generate 100W for decades. The human baseline metabolism is around 100W, so with two thermal batteries and another classic battery for energy usage spikes (physical activity) you would have enough if we could feed electricity directly to cells (instead of in a round-about way through food)
Last5Digits 4y ago
I don't really understand what point you're making here. Gasoline doesn't release energy on its own either, and the average person doesn't have an ignition coil or pistons in their stomach. This person obviously wanted to know how long a human being could be "powered" by a variety of energy sources. It's a silly, abstract question - details don't matter, only the energy that we can theoretically extract from the material does (in the way energy is usually extracted from the material; gasoline: ignition, U-238: fission). The number it put out, 200 MeV, is a very close approximation of the net energy you'd gain from a neutron induced fission reaction of U-238. Honestly, it feels like ChatGPT understood the point of the question better than you did. And if it answered in your nitpicky style, then we'd probably be criticizing that.
- peteradio 4y ago
  If someone asked you how much gasoline to drink for a week of energy you'd probably ask for some context. It's a shitty question, it shouldn't really get an answer.
  - edvinbesic 4y ago
    Except in this case an adequate answer was provided without any additional context required.
    - deleted 4y ago
      [deleted]
  - Last5Digits 4y ago
    If I was an LLM and one of my main tasks was to answer questions, then I would expect a lot of "what if" type abstract queries by bored people. Obviously, people like that don't want a literal answer - the context is crystal clear. It's not a bad question either - it can illustrate the efficiency of the human body, or help you get a better feel for different quantities of energy. It even hedged its bet by very explicitly explaining why consuming gasoline or uranium is a bad idea.

by amluto 4y ago

sebzim4500 4y ago

Yeah I think it was thinking of U-235. At least in this case ChatGPT made it clear that the number did not come from Wolfram, so you don't need to be especially knowledgable to see that it shouldn't be trusted.

dsign 4y ago

I've been playing with the combination of Chat GPT and Wolfram, and while there are gold nuggets in the output sometimes, it is mostly terrifying non-sense for anybody who couldn't already verify those outputs on their own. The cynic in me says that there will be Chat-GPT generation of kids that won't be able to think rationally on their own, the same way that kids and young people these days are completely unable to have a normal sleep schedule.

s3p 4y ago
I'm not sure the last claim you make follows from your line of reasoning. If you are sure tech(?) is ruining kids' sleep schedules, but only kids, I would love to see evidence of it.
duskwuff 4y ago
> The cynic in me says that there will be Chat-GPT generation of kids that won't be able to think rationally on their own This is, unfortunately, already happening. I've seen a terrifying number of users on Reddit uncritically using ChatGPT as a "source", or expecting (largely unsuccessfully) to have it give them expert guidance in performing tasks like writing software. Making ChatGPT available to the general public, even as a "test", was a mistake. It's simply too good at sounding convincingly authoritative while being completely wrong.
- philistine 4y ago
  They simply should have charged for it.
- blibble 4y ago
  hopefully the lesson that enters our culture is "AI produces garbage", which then kills off the AI revolution permanently
  - tempodox 4y ago
    Nah, it will be just another “AI winter”, if even that much, until the next hype cycle begins. As this example demonstrates, humanity will fall for complete and utter nonsense, hook, line and sinker.
    - mercer 4y ago
      Yeah let's just chuck it all in the bin unless it's 100% useful!
- zelphirkalt 4y ago
  > It's simply too good at sounding convincingly authoritative while being completely wrong. In a way that is what we humans do way too often. People talking about stuff thinking they know, but not actually knowing a lot. I guess that is where it learned that from.
  - ryanar 4y ago
    yep, my dad would always say I have an answer for everything, even topics I know nothing about, I was/am just really good at stringing together bits of knowledge in a way that sounds very plausible, combine that with cis/white/het male speaker and you have HumanGPT
    - peepeepoopoo3 4y ago
      Way to throw identity politics into it.
mrcwinn 4y ago
It's a fair point. I actually slept 8 hours last night (made sure not to eat after 7 or so, set my phone down early on). What's noteworthy is that... I noticed and was so excited I'd slept 8 hours. That used to not be noteworthy!
ToValueFunfetti 4y ago
I would expect an unreliable source to have the opposite effect. When I use ChatGPT, I find myself thinking critically and questioning everything in a way that I really should be applying to everything else I read.
- Sharlin 4y ago
  First you have to realize that ChatGPT is unreliable in the first place. I guess many will have to learn their lesson the hard way.
  - pessimizer 4y ago
    All people know stuff about stuff, will talk (have already talked) to AI about that stuff that they know, and will figure out that AI is lying about stuff they know. Maybe this can reduce the number of people who believe everything they read to a less dangerous level.
    - BenjiWiebe 4y ago
      Gell-Mann amnesia...
  - mercer 4y ago
    Everyone I know already knows this, because ChatGPT makes such obvious mistakes in between brilliant takes. Hell, my grandmother knows that this 'ai' thing does something called 'hallucinate' because it was on the news a few times.
    - Sharlin 4y ago
      Everyone you know may be a less representative sample of the general population than you think.
- thrashh 4y ago
  I agree. You only need to be done dirty once, then you'll remember.
random3 4y ago
Let's call them gpt-cyborgs. Some will be worse, some better off. Leveraging the "real" world would be in their advantage. Here's an example - can't prove it's correct, but it could be measured. Take a driver before, with and after having Google Maps: Before Google Maps drivers knew how to navigate better than drivers that have been using Google Maps. I.e. After removing Google Maps some drivers will be lost. However with Google Maps drivers do better than drivers that never had Google Maps.
RC_ITR 4y ago
>The cynic in me says that there will be Chat-GPT generation of kids that won't be able to think rationally on their own It’s interesting to me that your implication is “ChatGPT” will give reasonable but wrong answers to things, therefore people will accept those answers. It reminds me quite a bit of “no one will know math because calculators.” There’s plenty of feedback loops that exist for exactly this problem. Kids submit GPT homework and it’s either correct (rendering the skill the homework was testing worthless), or it’s not, and those kids will be punished with bad grades. If anything it will teach a whole new generation how to analyze semi-unreliable text, in the same way Google taught millennials how to search though disparate sources and synthesize answers. >the same way that kids and young people these days are completely unable to have a normal sleep schedule. Conclusions: A lack of empirical evidence for sleep recommendations was universally acknowledged. Inadequate sleep was seen as a consequence of "modern life," associated with technologies of the time. No matter how much sleep children are getting, it has always been assumed that they need more. https://pubmed.ncbi.nlm.nih.gov/22331340/ https://pubmed.ncbi.nlm.nih.gov/22331340/
- asimpleusecase 4y ago
  On the kids and calculators… in the past week I have had two times that it was clear the person at the till could not make change. In one case the total was £4.90 I gave them a £10 note. Quickly I added a 20p coin so they could give me £5.10 back and save them having to give me so many coins. The person was in the 20’s and I could see immediately they did not know what to do. The phone came out of the pocket and calculator function stated and some tapping proceeded but they did not look comforted and then there was tapping on the till and then I was handed £5 and they rushed to the back room. I would have chalked it up a one off but it happened again a few days later. Admittedly it dealt with an antiquated form of money. However, I would guess that level of innumeracy has to impact them in many way in normal life.
  - snozolli 4y ago
    In one case the total was £4.90 I gave them a £10 note. Quickly I added a 20p coin so they could give me £5.10 10 - 4.90 = 5.10. Why was another 20 involved? Edit: I'm guessing the total was 5.10, so you gave 10.20 to get 5.10 back.
ryanar 4y ago
Can you elaborate more on “kids and young people these days are completely unable to have a normal sleep schedule”? I have seen cases of this in young people, but I didn’t know the issue was widespread and would be interested to learn more.
barrysteve 4y ago
The second most terrifying thing would be an expert university professor on the warpath because they have to correct every single passing remark ever coughed up. We are giving critical experts way too much responsibility to hold the untrained chatgpt-er masses to account. Open-standards fact checking, or a global heirarchy of reputation could be plausible solutions. I really don't want to listen to another set of public academics, so a global resource like wikipedia, with a secure (or conceptually secure-ish) blockchain-alike technology to ensure the untampered communication and authenticity of the original fact source, to relay "base factual" information on the web, would be preferable to my mind.
bondarchuk 4y ago
On the contrary, I think there will be a chatgpt generation of kids who understand exactly to take these outputs with a huge grain of salt, while there will be a cohort of currently-adult people who are so used to believing almost anything written in text that they will do the same with ai outputs.
- sterlind 4y ago
  100%. Most of the people I see posting fake news (as in, rumors from bot tweets/generated pages that I've determined is false from easily accessible, primary sources) are older. Though maybe that's because older people are on Twitter, and I'd find the same on TikTok.
- wruza 4y ago
  They’ll have to take something without salt to learn anyway. It seems pretty inevitable that AIs will replace generic search engines and change the way “blogs” work. Like it doesn’t matter to google today that you wanted ["foo" "bar"]. You may be smart, but a completely irrelevant “S”E“R”P is all you can have.

cheald 4y ago

This is going to be the big problem with the integration of LLMs into normal peoples' everyday lives; if you accept the answer as valid because it's eloquent and appears to follow some semblance of logical reasoning, you're going to be quickly misled. I've had numerous instances where ChatGPT has produced output which looks confidently authoritative, but which I as a domain expert can recognize as wrong or even nonsensical. One needs to _heavily_ tune their Gell-Mann amnesia detector when working with LLMs, and recall that if it's not getting the things right you are an expert on, it's likely not getting the things right that you're not an expert on, either.

trzy 4y ago
This reminds me of speaking with SV investors. They’re eloquent and say things that sound plausible in the moment but then I snap out of it and realize they’ve said nothing meaningful or correct at all.
- atlasunshrugged 4y ago
  I think you'll find this in many places, I've noticed the same from SV also in DC, Berlin, etc. I think the upper echelons of society are just filled with people who often say very plausible eloquent things that aren't questioned for a variety of factors and it mostly just works for them.
lamontcg 4y ago
Given how humans are quickly misled by sociopathic humans who are eloquent and appear to follow logical reasoning, I don't really have any hope. People on the whole are incredibly bad at that particular skill (at least in our culture).
- cheald 4y ago
  I'm inclined to agree with your assessment, given the existence of the concept of Gell-Mann amnesia at all. That said, I think we might have a glimmer of a chance to escape the executioner's axe here, as it were, because while a psychopath is intentionally deceptive, and will become evasive or aggressive when they sense that you're suspicious and probing of them, LLMs have no such quality, and will "happily" keep talking and exposing their shortcomings to those willing to listen. Ultimately, though, peoples' ability to understand how appropriately to trust LLMs will rest on their ability to understand their own capacity for being too-trusting and easily misled, a feature on which the human psyche does not have a great track record.

dist-epoch 4y ago

Thermal batteries, the kind used on spaceships, weight 15 kg and can generate 100W for decades. The human baseline metabolism is around 100W, so with two thermal batteries and another classic battery for energy usage spikes (physical activity) you would have enough if we could feed electricity directly to cells (instead of in a round-about way through food)

Last5Digits 4y ago

I don't really understand what point you're making here. Gasoline doesn't release energy on its own either, and the average person doesn't have an ignition coil or pistons in their stomach. This person obviously wanted to know how long a human being could be "powered" by a variety of energy sources. It's a silly, abstract question - details don't matter, only the energy that we can theoretically extract from the material does (in the way energy is usually extracted from the material; gasoline: ignition, U-238: fission). The number it put out, 200 MeV, is a very close approximation of the net energy you'd gain from a neutron induced fission reaction of U-238. Honestly, it feels like ChatGPT understood the point of the question better than you did. And if it answered in your nitpicky style, then we'd probably be criticizing that.

peteradio 4y ago
If someone asked you how much gasoline to drink for a week of energy you'd probably ask for some context. It's a shitty question, it shouldn't really get an answer.
- edvinbesic 4y ago
  Except in this case an adequate answer was provided without any additional context required.
  - deleted 4y ago
    [deleted]
- Last5Digits 4y ago
  If I was an LLM and one of my main tasks was to answer questions, then I would expect a lot of "what if" type abstract queries by bored people. Obviously, people like that don't want a literal answer - the context is crystal clear. It's not a bad question either - it can illustrate the efficiency of the human body, or help you get a better feel for different quantities of energy. It even hedged its bet by very explicitly explaining why consuming gasoline or uranium is a bad idea.