3 ms·

Am I the only one left with a deep distrust of ChatGPT? It still lies ("hallucinates"), doesn't it? And no, I will not accept a rebuttal that says, "well ever

by krupan 3y ago

Am I the only one left with a deep distrust of ChatGPT? It still lies ("hallucinates"), doesn't it? And no, I will not accept a rebuttal that says, "well everyone lies" because no, not everyone is as confidently wrong as ChatGPT. It's doesn't qualify statements with, "I was told..." or, "some believe..." or, "I'm not sure but it might be..." It just flat out lies. People are enthralled with this tech when they should still be very very skeptical.

MattGaiser 3y ago
It does make mistakes, but it gets stuff right enough to be worth it. And if you catch it in a lie, you can just tell it and usually it will correct itself. To me, it is like a junior developer. It does the bulk of the work and whenever there is an issue, you can tell it like in a code review to fix a particular piece. Look past the confidence level and focus on the work product. Remember, it doesn’t need to always be accurate. It doesn’t need to be perfect. It just needs to outperform the alternative (passable job here) for the price paid (amazing value here).
- _flux 3y ago
  A junior developer will figure out and hopefully confess when he's way over his head, but ChatGPT will happily respond "oh right you are, that makes no sense, here's a corrected version"—just to respond with a version that is also false in a way that might not be readily apparent. ChatGPT ultimately doesn't know what it doesn't know, unless it's been told so explicitly (this is my current understanding which may be flawed; I'd like to know the "not knowing" would otherwise work in this system!). But indeed it's still quite useful. It's best used in situations where it doesn't matter much if the answer is right, or when the answer can be easily checked if it's correct, yet more annoying to come up by oneself in the first place.
leosanchez 3y ago
You are definitely not alone :)
ryanjshaw 3y ago
It doesn't "lie" (there's no intent to deceive). It makes "mistakes" [1]. It's a tool that is increasingly a part of the modern world whether you like it or not. Students who learn how to use the tool, which includes learning its limitations, will benefit overall. [1] technically it doesn't even make "mistakes", it's just an LLM doing its thing, but we humans like to anthropomorphise
- em-bee 3y ago
  when a child makes up things that it knows not to be real there is no intent to deceive either. the child lacks the capacity to understand the difference. yet we will tell the child that it is lying. just like a child, chatgpt can not tell the difference between reality and fiction. chatgpt lies because it presents fiction as fact. there is no intent to deceive necessary.
- teapot7 3y ago
  We do like our anthropomorphisation, and I prefer to say that ChatGPT and its pal likes to "bullshit". They're neither "trying" to tell the truth nor trying to lie - they trying to sound plausible, with a cheerful disregard of the value of truth. I'm done anthopomorphising now.
Staple_Diet 3y ago
Hmm, all I'm hearing is that ChatGPT should run for President. But on a serious note I have only gotten good content out of ChatGPT when it's used as a smart Thesaurus, or a code checker (even that was 50:50). If I have a paragraph that I feel needs rewriting or condensing I can throw it in and resubmit until I get a good result - which I still then need to edit to make it less... ChatGPTish. If I open up old scripts (data extraction/analysis) that I forgot what they do I chuck it in and find out what will happen. Like any large technological advance, the hype is much larger than the eventual reality (see VR goggles).
calderknight 3y ago
The students are using it anyway, so Oslo decided that teaching students to use it would be the best path. That way they can teach the students about the hallucinations and about ethical ways to use the technology.
codingminds 3y ago
I came here to say this as well. Every time someone replies with "I've asked ChatGPT.." or "Why not ask ChatGPT?" I get seriously mad. It's like religious people. Totally convinced of a fake story and therefore tells it as truth. > People are enthralled with this tech when they should still be very very skeptical. Yes, that's the real issue. Everyone should be very skeptical about ChatGPT responses. But instead it's often trusted blindly, because "it's often enough correct". But I get the idea behind this particular move and at least it can be a chance to teach the students to distrust it.
- onethought 3y ago
  It’s pretty solid on well understood facts, physics, mathematics. It’s not so good at problem solving, it’s pretty good at code in some languages but it others. It’s quite good at translation in a number of languages. It’s great and fictional writing in any particular style. Isn’t denying these facts just as dogmatic as accepting them blindly?
  - cageface 3y ago
    I wouldn’t call it great at writing fiction. Its prose style is bland and generic. There’s no authorial voice.
    - onethought 3y ago
      There is if you give it one. Literally write your criticism in and it will improve
darkerside 3y ago
I rarely hear any of those phrases from a teacher, fwiw
- krupan 3y ago
  So two wrongs make a right?
calibas 3y ago
It's often confidently wrong, but it's still a useful tool. I spent 5 minutes searching Google for what an email error code meant, it kept showing me threads on Microsoft's forums that never gave me a clear answer. Took about 10 seconds to get the correct answer on ChatGPT. If you treat it like it's a bumbling assistant that has remarkably encyclopedic knowledge, it's a very helpful tool. If you treat it like it's truly intelligent and let it do your thinking for you, then you're going to have some problems.
- cageface 3y ago
  Took about 10 seconds to get the correct answer on ChatGPT. But, generally speaking, if you don’t have some way of verifying what the correct answer is, how can you be sure that the answers you get in 10 seconds from the LLM are actually correct?
  - lpapez 3y ago
    If you don't have a way of verifying what the correct answer is, then is there actually a correct answer at all? In many cases, programming included, you don't actually care if it's 100% correct, it is enough to just give you a nudge, show you the way forward, suggest some ideas etc. If whatever it is you are doing can be verified for correctness, then of course you will check the answer. But for creative pursuits there often is no correct answer, and generative AI (LLMs included) excel at that.
    - cageface 3y ago
      Right. The two areas where I've found current AI to be useful are code generation where I can fairly easily check the results and creative stuff where it doesn't matter. I don't understand how people feel they can use it as a search engine replacement. If you have to go separately check the results then it doesn't seem like it's saving any time. I suspect people are just trusting the output a lot more than they should.
      - lpapez 3y ago
interpol_p 3y ago
As long as you _know the right answer_, ChatGPT is a very useful tool 90% of the time I know the code I want to write. ChatGPT just writes it for me, when I read it I can confirm it is what I would have written, or fix it if not (or more likely: tell ChatGPT to fix it) This means I don’t have to google for syntax, canonical ways of doing things, etc. If you are asking ChatGPT for facts, triple check everything it tells you. Or better still, paste the Wikipedia entry to it first, then ask for the style or type of writing you are interested in about those facts
huytersd 3y ago
It doesn’t matter. The way it’s set up now, you can actually ask it to look up real information with sources linked.
tverbeure 3y ago
You need to distrust ChatGPT if you want to use if effectively. I'm paying for the 4.0 version, I know that it will hallucinate BS, yet I also think it's a fantastic tool that will get some things done faster than any other way. There's its ability to write small one-off scripts for me. I've given it an example of a structured log file and told it to "write a pyparser script to parse this string" and it will do that. What would take me a hour to learn pyparser and get somewhere now takes a few minutes. You can literally see it iterate on different solutions until it comes up with something acceptable. (ChatGPT 3.5 doesn't do that.) Another great use case is as an assistant when I'm learning some new topic. I was reading A Mathematical Theory of Communications, the breakthrough thesis by Claude Shannon. It's quite accessible, but there are always points where you don't quite understand a point that is being made and you want to get more examples. I can ask ChatGPT to create a concrete example, use actual numbers, plot a graph that illustrates the concept. And it will do that, along with the Python script that it used to create the graph so I can double check what it did. "Create an Eb/No graph for the following parameters." And then ask to modify the graph to illustrate a particular point. Google can deliver similar information, but it will push me a document that contains the answer, along with a truckload of info that I wasn't looking for. ChatGPT pulls just the info that I'm looking for. Or lately I was looking into Laplace transformations, low pass filters, pole/zero plots etc. ChatGPT 4.0 uses the SymPy symbolic math package to derive the math, I can once again examine the scripts, ask for concrete examples with graphs and so forth. It's way more efficient than doing the same thing with Google, which will usually send you StackExchange with an answer that is not quite what you're looking for. In these kind of cases, I know enough math and subject knowledge to know when it's bullshitting, but it doesn't have to be 100% accurate to a huge help in learning.
golergka 3y ago
You're judging a fish by its ability to climb a tree. Retrieving truthful information about the world is one of the possible ways to use of GPT models, but neither is the best working, nor the most helpful. Instead, think of it as a computer that can perform transformations on text, described in plain language and based on text's tone and meaning.
vunderba 3y ago
I don't wanna be the guy who pulls out the "you're holding the phone wrong" from Apple, but... you're holding the phone wrong. Here's an example of some thing I did off the cuff. I have a bunch of hand written notes that are mind maps and graphs written on old notebooks from 10 years ago. I snapped a picture of all of the different graphs, threw them into chatGPT and asked it to convert them to mermaid UML syntax. Every single one of them converted flawlessly when I brought them into my markdown editor. The amazing thing here is that that the ChatGPT large language model wasn't even explicitly trained for this at all, but with the massive quantity of data that it's trained on its able to just connect the dots. If you're using chatGPT as nothing more than a glorified fact checker and not taking advantage of the multimodal capabilities such as vision, OCR, Python VM, generative imagery, you're really missing the point.

by krupan 3y ago

MattGaiser 3y ago

It does make mistakes, but it gets stuff right enough to be worth it. And if you catch it in a lie, you can just tell it and usually it will correct itself. To me, it is like a junior developer. It does the bulk of the work and whenever there is an issue, you can tell it like in a code review to fix a particular piece. Look past the confidence level and focus on the work product. Remember, it doesn’t need to always be accurate. It doesn’t need to be perfect. It just needs to outperform the alternative (passable job here) for the price paid (amazing value here).

_flux 3y ago
A junior developer will figure out and hopefully confess when he's way over his head, but ChatGPT will happily respond "oh right you are, that makes no sense, here's a corrected version"—just to respond with a version that is also false in a way that might not be readily apparent. ChatGPT ultimately doesn't know what it doesn't know, unless it's been told so explicitly (this is my current understanding which may be flawed; I'd like to know the "not knowing" would otherwise work in this system!). But indeed it's still quite useful. It's best used in situations where it doesn't matter much if the answer is right, or when the answer can be easily checked if it's correct, yet more annoying to come up by oneself in the first place.

leosanchez 3y ago

You are definitely not alone :)

ryanjshaw 3y ago

It doesn't "lie" (there's no intent to deceive). It makes "mistakes" [1]. It's a tool that is increasingly a part of the modern world whether you like it or not. Students who learn how to use the tool, which includes learning its limitations, will benefit overall. [1] technically it doesn't even make "mistakes", it's just an LLM doing its thing, but we humans like to anthropomorphise

em-bee 3y ago
when a child makes up things that it knows not to be real there is no intent to deceive either. the child lacks the capacity to understand the difference. yet we will tell the child that it is lying. just like a child, chatgpt can not tell the difference between reality and fiction. chatgpt lies because it presents fiction as fact. there is no intent to deceive necessary.
teapot7 3y ago
We do like our anthropomorphisation, and I prefer to say that ChatGPT and its pal likes to "bullshit". They're neither "trying" to tell the truth nor trying to lie - they trying to sound plausible, with a cheerful disregard of the value of truth. I'm done anthopomorphising now.

Staple_Diet 3y ago

Hmm, all I'm hearing is that ChatGPT should run for President. But on a serious note I have only gotten good content out of ChatGPT when it's used as a smart Thesaurus, or a code checker (even that was 50:50). If I have a paragraph that I feel needs rewriting or condensing I can throw it in and resubmit until I get a good result - which I still then need to edit to make it less... ChatGPTish. If I open up old scripts (data extraction/analysis) that I forgot what they do I chuck it in and find out what will happen. Like any large technological advance, the hype is much larger than the eventual reality (see VR goggles).

calderknight 3y ago

The students are using it anyway, so Oslo decided that teaching students to use it would be the best path. That way they can teach the students about the hallucinations and about ethical ways to use the technology.

codingminds 3y ago

I came here to say this as well. Every time someone replies with "I've asked ChatGPT.." or "Why not ask ChatGPT?" I get seriously mad. It's like religious people. Totally convinced of a fake story and therefore tells it as truth. > People are enthralled with this tech when they should still be very very skeptical. Yes, that's the real issue. Everyone should be very skeptical about ChatGPT responses. But instead it's often trusted blindly, because "it's often enough correct". But I get the idea behind this particular move and at least it can be a chance to teach the students to distrust it.

onethought 3y ago
It’s pretty solid on well understood facts, physics, mathematics. It’s not so good at problem solving, it’s pretty good at code in some languages but it others. It’s quite good at translation in a number of languages. It’s great and fictional writing in any particular style. Isn’t denying these facts just as dogmatic as accepting them blindly?
- cageface 3y ago
  I wouldn’t call it great at writing fiction. Its prose style is bland and generic. There’s no authorial voice.
  - onethought 3y ago
    There is if you give it one. Literally write your criticism in and it will improve

darkerside 3y ago

I rarely hear any of those phrases from a teacher, fwiw

krupan 3y ago
So two wrongs make a right?

calibas 3y ago

It's often confidently wrong, but it's still a useful tool. I spent 5 minutes searching Google for what an email error code meant, it kept showing me threads on Microsoft's forums that never gave me a clear answer. Took about 10 seconds to get the correct answer on ChatGPT. If you treat it like it's a bumbling assistant that has remarkably encyclopedic knowledge, it's a very helpful tool. If you treat it like it's truly intelligent and let it do your thinking for you, then you're going to have some problems.

cageface 3y ago
Took about 10 seconds to get the correct answer on ChatGPT. But, generally speaking, if you don’t have some way of verifying what the correct answer is, how can you be sure that the answers you get in 10 seconds from the LLM are actually correct?
- lpapez 3y ago
  If you don't have a way of verifying what the correct answer is, then is there actually a correct answer at all? In many cases, programming included, you don't actually care if it's 100% correct, it is enough to just give you a nudge, show you the way forward, suggest some ideas etc. If whatever it is you are doing can be verified for correctness, then of course you will check the answer. But for creative pursuits there often is no correct answer, and generative AI (LLMs included) excel at that.
  - cageface 3y ago
    Right. The two areas where I've found current AI to be useful are code generation where I can fairly easily check the results and creative stuff where it doesn't matter. I don't understand how people feel they can use it as a search engine replacement. If you have to go separately check the results then it doesn't seem like it's saving any time. I suspect people are just trusting the output a lot more than they should.
    - lpapez 3y ago

interpol_p 3y ago

As long as you _know the right answer_, ChatGPT is a very useful tool 90% of the time I know the code I want to write. ChatGPT just writes it for me, when I read it I can confirm it is what I would have written, or fix it if not (or more likely: tell ChatGPT to fix it) This means I don’t have to google for syntax, canonical ways of doing things, etc. If you are asking ChatGPT for facts, triple check everything it tells you. Or better still, paste the Wikipedia entry to it first, then ask for the style or type of writing you are interested in about those facts

huytersd 3y ago

It doesn’t matter. The way it’s set up now, you can actually ask it to look up real information with sources linked.

tverbeure 3y ago

You need to distrust ChatGPT if you want to use if effectively. I'm paying for the 4.0 version, I know that it will hallucinate BS, yet I also think it's a fantastic tool that will get some things done faster than any other way. There's its ability to write small one-off scripts for me. I've given it an example of a structured log file and told it to "write a pyparser script to parse this string" and it will do that. What would take me a hour to learn pyparser and get somewhere now takes a few minutes. You can literally see it iterate on different solutions until it comes up with something acceptable. (ChatGPT 3.5 doesn't do that.) Another great use case is as an assistant when I'm learning some new topic. I was reading A Mathematical Theory of Communications, the breakthrough thesis by Claude Shannon. It's quite accessible, but there are always points where you don't quite understand a point that is being made and you want to get more examples. I can ask ChatGPT to create a concrete example, use actual numbers, plot a graph that illustrates the concept. And it will do that, along with the Python script that it used to create the graph so I can double check what it did. "Create an Eb/No graph for the following parameters." And then ask to modify the graph to illustrate a particular point. Google can deliver similar information, but it will push me a document that contains the answer, along with a truckload of info that I wasn't looking for. ChatGPT pulls just the info that I'm looking for. Or lately I was looking into Laplace transformations, low pass filters, pole/zero plots etc. ChatGPT 4.0 uses the SymPy symbolic math package to derive the math, I can once again examine the scripts, ask for concrete examples with graphs and so forth. It's way more efficient than doing the same thing with Google, which will usually send you StackExchange with an answer that is not quite what you're looking for. In these kind of cases, I know enough math and subject knowledge to know when it's bullshitting, but it doesn't have to be 100% accurate to a huge help in learning.

golergka 3y ago

You're judging a fish by its ability to climb a tree. Retrieving truthful information about the world is one of the possible ways to use of GPT models, but neither is the best working, nor the most helpful. Instead, think of it as a computer that can perform transformations on text, described in plain language and based on text's tone and meaning.

vunderba 3y ago

I don't wanna be the guy who pulls out the "you're holding the phone wrong" from Apple, but... you're holding the phone wrong. Here's an example of some thing I did off the cuff. I have a bunch of hand written notes that are mind maps and graphs written on old notebooks from 10 years ago. I snapped a picture of all of the different graphs, threw them into chatGPT and asked it to convert them to mermaid UML syntax. Every single one of them converted flawlessly when I brought them into my markdown editor. The amazing thing here is that that the ChatGPT large language model wasn't even explicitly trained for this at all, but with the massive quantity of data that it's trained on its able to just connect the dots. If you're using chatGPT as nothing more than a glorified fact checker and not taking advantage of the multimodal capabilities such as vision, OCR, Python VM, generative imagery, you're really missing the point.