5 ms·

If you don't know the subject, how can you be sure what it's telling you is true? Do you vet what ChatGPT tells you with other sources? I don't really know Typ

If you don't know the subject, how can you be sure what it's telling you is true? Do you vet what ChatGPT tells you with other sources? I don't really know Typescript, so I've been using it a lot to supplement my learning, but I find it really hard to accept any of its answers that aren't straight code examples I can test.

_boffin_ 4y ago
Can you go into more depth about >I don't really know Typescript, so I've been using it a lot to supplement my learning, but I find it really hard to accept any of its answers that aren't straight code examples I can test. - How are you using it? - What are the questions you're asking it? - What are your thoughts about the answers and how are you cross checking them? Edit: >If you don't know the subject, how can you be sure what it's telling you is true? Do you vet what ChatGPT tells you with other sources? I can't, but i can take a look at books i have or search google to find additional sources. To me, the biggest power of it is to help me understand and build mental models of something new.
- georgebcrawford 4y ago
  They asked you questions too, y’know…
  - _boffin_ 4y ago
    Guess my brain skipped over that part. Thanks for pointing that out -- updating my answer
- yoyohello13 4y ago
  At this point I generally stick to specific small problems like "How can I write a script to convert a Product from the Stripe API into my custom interface?" or "How do I do this thing in SQL". I trust these answers because I can verify by reading and running the actual code. For more open ended questions I tend to treat it more like a random comment in a forum. For example, I often notice that Typescript code examples don't use the `function` keyword often, they tend to use anonymous functions like `const func = () => blah`. I asked ChatGPT why this is and it gave a plausible answer, I have no idea if what it's saying is true, but it seemed true enough. I give the answer the same amount of trust as I would some random comment on Stack Overflow. The benefit of Stack Overflow though is at least you know the reputation of the person you're talking to.
publius_ 4y ago
How do you know what a PhD is telling you is truth? Unlike the PhD, the AI model has benchmark scores on truthfulness. Right now, they're looking pretty good.
- BaseballPhysics 4y ago
  How do we know anything is true??! Seriously, you're veering into sophistry. People have reputations. They cite sources. Unless they're compulsive liars, they don't tend to just make stuff up on the spot based on what will be probabilistically pleasing to you. There are countless examples of ChatGPT not just making mistakes but making up "facts" entirely from whole cloth, not based on misunderstanding or bias or anything else, but simply because the math says it's the best way to complete a sentence. Let's not use vacuous arguments to dismiss that very real concern. Edit: As an aside, it somehow only now just occurred to me that LLM bullshit generation may actually be more insidious than the human-generated variety as LLMs are specifically trained to create language that's pleasing, which means it's going to try to make sure it sounds right, and therefore the misinformation may turn out to be more subtle and convincing...
  - deleted 4y ago
    [deleted]
  - westoncb 4y ago
    The way in which this kind of error deviates from what a human would do is generally trivial: “confidently stating bs” is the same as how mistakes from human professionals often manifest—it will be this way anytime the person doesn’t realize they’re making a mistake. The only real difference is that you’re imputing a particular kind of intention to the ai whereas the human’s intention can be assumed good in the above scenario. The BS vs unknowing falsehood distinction is purely intention based, a category error to attribute to an llm.
    - BaseballPhysics 4y ago
      > The way in which this kind of error deviates from what a human would do is generally trivial That's not even remotely true and if you've worked with these technologies at all you'd know that. For example, as I previously mentioned, humans don't typically make up complete fiction out of whole cloth and present it as fact unless those humans possess some sort of mental illness. > The only real difference is that you’re imputing a particular kind of intention to the ai No, in fact I'm imputing the precise opposite. These AIs have no intention because they have no comprehension or intelligence. The result is that when they generate false information, it can be unexpected and unpredictable. If I'm talking to a human I can make some reasonable inferences about what they might get wrong, where their biases lie, etc. Machines fail in surprising, unexpected, and often subtle ways that make them difficult for humans to predict.
everfree 4y ago
> Do you vet what ChatGPT tells you with other sources? I find that ChatGPT is good at helping me with "unknown unknown" questions, where I don't know how to properly phrase my question for a search engine, so I explain to ChatGPT in vague terms how I am feeling about a certain thing. ChatGPT helps me understand what to search for, and then I take it from there by looking for a reputable answer on a search engine.
- yura 4y ago
  That's true. I've also used it for these "unknown unknowns" questions with very good results. Basically talking with ChatGPT to find out what should I put on Google, and how we go from there is business as usual. But other than that it makes me nervous when people say they're "learning with ChatGPT": any serious conversation with ChatGPT about a subject I know about quickly shows just how much nonsense and bullshit it conjures out of thin air. ChatGPT is extremely good at sounding convincing and authoritative, and you'll feel like you're learning a lot, when in fact you could be learning 100% made-up facts and the only way to tell is if you understand the subject already.
  - test6554 4y ago
    Perhaps you underestimate how much average people lack the most basic surface-level knowledge of various subjects, and how much value learning the basics can provide. Some of these people are just learning about the relationship between temperature and pressure or current and voltage, etc. something well within the bounds of LLMs and its enriching their lives dramatically. I asked it a question once to clarify a fact from a book I was reading that temporarily baffled my 2am barely awake mind. “Why is humid air less dense than dry air? Isn’t water heavier than air” It went on to explain the composition of air, the atomic weights of all the most common air molecules and how the atomic weight of water molecules is lighter than nitrogen (N2) and oxygen (O2) And my fallacy was in comparing air to liquid water that people are more familiar with rather than water vapor which is what would be found in humid air.
kulikalov 4y ago
> If you don't know the subject, how can you be sure what it's telling you is true? That applies to any article, book, or a verbal communication with any human being, not only to LLMs
- throwaway675309 4y ago
  This is a pointless whataboutism, but I'll humor you. I can pick up a college textbook on interval calculus and be reasonably assured of its veracity because it's been checked over by a proofreader, other mathematicians, the publisher, and finally has been previously used in a classroom environment by experts in the field.
  - pixl97 4y ago
    It's unfortunate but the vast amount of human literature is not up to those standards.
    - yura 4y ago
      The vast amount of human literature is not worth a read. As long as you pick reputable sources, read great books and so on, they will be up to those standards. Of course, it's not a trivial task to find the reputable sources and the great books about a subject you don't know about. But there are many ways to find that out, for example by checking out the curriculum of respected universities to see which textbooks they use.
  - altilunium 4y ago
    > I can pick up a college textbook on interval calculus and be reasonably assured of its veracity because it's been checked over by a proofreader, other mathematicians, the publisher, and finally has been previously used in a classroom environment by experts in the field. Well, even a very popular scientific theory, that supported by the whole consensus of academic society at its time, could be proved wrong decades later.
    - 4y ago
BeetleB 4y ago
> If you don't know the subject, how can you be sure what it's telling you is true? People are reading too much into the comment. You wouldn't use ChatGPT to become as knowledgeable as obtaining a PhD. The idea is "If I wanted to ask an expert something, I have easy access to one now." The real questions are: 1. For a given domain, how much more/less accurate is ChatGPT? 2. How available are the PhDs? It makes sense to accept a somewhat lower accuracy if they are 10 times more available than a real PhD - you'll still learn a lot more, even though you also learn more wrong things. I'll take a ChatGPT that is accurate 80% of the times and is available all day and night vs a PhD who is accurate 90% of the times but I get only 30 minutes with him per week.
altilunium 4y ago
> If you don't know the subject, how can you be sure what it's telling you is true? The same question could be asked when we're learning through books or an expert. There's no guarantee that books or experts are always spitting out the truth.

_boffin_ 4y ago

Can you go into more depth about >I don't really know Typescript, so I've been using it a lot to supplement my learning, but I find it really hard to accept any of its answers that aren't straight code examples I can test. - How are you using it? - What are the questions you're asking it? - What are your thoughts about the answers and how are you cross checking them? Edit: >If you don't know the subject, how can you be sure what it's telling you is true? Do you vet what ChatGPT tells you with other sources? I can't, but i can take a look at books i have or search google to find additional sources. To me, the biggest power of it is to help me understand and build mental models of something new.

georgebcrawford 4y ago
They asked you questions too, y’know…
- _boffin_ 4y ago
  Guess my brain skipped over that part. Thanks for pointing that out -- updating my answer
yoyohello13 4y ago
At this point I generally stick to specific small problems like "How can I write a script to convert a Product from the Stripe API into my custom interface?" or "How do I do this thing in SQL". I trust these answers because I can verify by reading and running the actual code. For more open ended questions I tend to treat it more like a random comment in a forum. For example, I often notice that Typescript code examples don't use the `function` keyword often, they tend to use anonymous functions like `const func = () => blah`. I asked ChatGPT why this is and it gave a plausible answer, I have no idea if what it's saying is true, but it seemed true enough. I give the answer the same amount of trust as I would some random comment on Stack Overflow. The benefit of Stack Overflow though is at least you know the reputation of the person you're talking to.

publius_ 4y ago

How do you know what a PhD is telling you is truth? Unlike the PhD, the AI model has benchmark scores on truthfulness. Right now, they're looking pretty good.

BaseballPhysics 4y ago
How do we know anything is true??! Seriously, you're veering into sophistry. People have reputations. They cite sources. Unless they're compulsive liars, they don't tend to just make stuff up on the spot based on what will be probabilistically pleasing to you. There are countless examples of ChatGPT not just making mistakes but making up "facts" entirely from whole cloth, not based on misunderstanding or bias or anything else, but simply because the math says it's the best way to complete a sentence. Let's not use vacuous arguments to dismiss that very real concern. Edit: As an aside, it somehow only now just occurred to me that LLM bullshit generation may actually be more insidious than the human-generated variety as LLMs are specifically trained to create language that's pleasing, which means it's going to try to make sure it sounds right, and therefore the misinformation may turn out to be more subtle and convincing...
- deleted 4y ago
  [deleted]
- westoncb 4y ago
  The way in which this kind of error deviates from what a human would do is generally trivial: “confidently stating bs” is the same as how mistakes from human professionals often manifest—it will be this way anytime the person doesn’t realize they’re making a mistake. The only real difference is that you’re imputing a particular kind of intention to the ai whereas the human’s intention can be assumed good in the above scenario. The BS vs unknowing falsehood distinction is purely intention based, a category error to attribute to an llm.
  - BaseballPhysics 4y ago
    > The way in which this kind of error deviates from what a human would do is generally trivial That's not even remotely true and if you've worked with these technologies at all you'd know that. For example, as I previously mentioned, humans don't typically make up complete fiction out of whole cloth and present it as fact unless those humans possess some sort of mental illness. > The only real difference is that you’re imputing a particular kind of intention to the ai No, in fact I'm imputing the precise opposite. These AIs have no intention because they have no comprehension or intelligence. The result is that when they generate false information, it can be unexpected and unpredictable. If I'm talking to a human I can make some reasonable inferences about what they might get wrong, where their biases lie, etc. Machines fail in surprising, unexpected, and often subtle ways that make them difficult for humans to predict.

everfree 4y ago

> Do you vet what ChatGPT tells you with other sources? I find that ChatGPT is good at helping me with "unknown unknown" questions, where I don't know how to properly phrase my question for a search engine, so I explain to ChatGPT in vague terms how I am feeling about a certain thing. ChatGPT helps me understand what to search for, and then I take it from there by looking for a reputable answer on a search engine.

yura 4y ago
That's true. I've also used it for these "unknown unknowns" questions with very good results. Basically talking with ChatGPT to find out what should I put on Google, and how we go from there is business as usual. But other than that it makes me nervous when people say they're "learning with ChatGPT": any serious conversation with ChatGPT about a subject I know about quickly shows just how much nonsense and bullshit it conjures out of thin air. ChatGPT is extremely good at sounding convincing and authoritative, and you'll feel like you're learning a lot, when in fact you could be learning 100% made-up facts and the only way to tell is if you understand the subject already.
- test6554 4y ago
  Perhaps you underestimate how much average people lack the most basic surface-level knowledge of various subjects, and how much value learning the basics can provide. Some of these people are just learning about the relationship between temperature and pressure or current and voltage, etc. something well within the bounds of LLMs and its enriching their lives dramatically. I asked it a question once to clarify a fact from a book I was reading that temporarily baffled my 2am barely awake mind. “Why is humid air less dense than dry air? Isn’t water heavier than air” It went on to explain the composition of air, the atomic weights of all the most common air molecules and how the atomic weight of water molecules is lighter than nitrogen (N2) and oxygen (O2) And my fallacy was in comparing air to liquid water that people are more familiar with rather than water vapor which is what would be found in humid air.

kulikalov 4y ago

> If you don't know the subject, how can you be sure what it's telling you is true? That applies to any article, book, or a verbal communication with any human being, not only to LLMs

throwaway675309 4y ago
This is a pointless whataboutism, but I'll humor you. I can pick up a college textbook on interval calculus and be reasonably assured of its veracity because it's been checked over by a proofreader, other mathematicians, the publisher, and finally has been previously used in a classroom environment by experts in the field.
- pixl97 4y ago
  It's unfortunate but the vast amount of human literature is not up to those standards.
  - yura 4y ago
    The vast amount of human literature is not worth a read. As long as you pick reputable sources, read great books and so on, they will be up to those standards. Of course, it's not a trivial task to find the reputable sources and the great books about a subject you don't know about. But there are many ways to find that out, for example by checking out the curriculum of respected universities to see which textbooks they use.
- altilunium 4y ago
  > I can pick up a college textbook on interval calculus and be reasonably assured of its veracity because it's been checked over by a proofreader, other mathematicians, the publisher, and finally has been previously used in a classroom environment by experts in the field. Well, even a very popular scientific theory, that supported by the whole consensus of academic society at its time, could be proved wrong decades later.
  - 4y ago

BeetleB 4y ago

> If you don't know the subject, how can you be sure what it's telling you is true? People are reading too much into the comment. You wouldn't use ChatGPT to become as knowledgeable as obtaining a PhD. The idea is "If I wanted to ask an expert something, I have easy access to one now." The real questions are: 1. For a given domain, how much more/less accurate is ChatGPT? 2. How available are the PhDs? It makes sense to accept a somewhat lower accuracy if they are 10 times more available than a real PhD - you'll still learn a lot more, even though you also learn more wrong things. I'll take a ChatGPT that is accurate 80% of the times and is available all day and night vs a PhD who is accurate 90% of the times but I get only 30 minutes with him per week.

altilunium 4y ago

> If you don't know the subject, how can you be sure what it's telling you is true? The same question could be asked when we're learning through books or an expert. There's no guarantee that books or experts are always spitting out the truth.