5 ms·

As long as we keep having to ask, "ChatGPT answered this questions but was it right?" it is a nonstarter. It is currently a novelty, not a tool. To be a tool it

As long as we keep having to ask, "ChatGPT answered this questions but was it right?" it is a nonstarter. It is currently a novelty, not a tool. To be a tool it needs to be reliable.

FlyingAvatar 4y ago
In most subjects I have found ChatGPT to be fairly reliable. Are there specific scenarios that you have found concerning? I think perfect reliability is not absolutely necessary. None of the human teachers that I have learned from were perfectly reliable even in their area of expertise. Thankfully, skepticism seems to be an innate human trait.
- shadowfox 4y ago
  > In most subjects I have found ChatGPT to be fairly reliable. Not the OP, but I am very curious what subjects you have experimented with and whether you used any special prompt techniques. My experience has been quite on the contrary: on the few subjects I have some understanding of, ChatGPT appears to do fine for basic questions and attempts to summarize concepts at a high-level, but any attempt at drilling in to specifics seem to cause it to hallucinate quite a bit, particularly with regards to concrete facts or citations or anything of that sort.
  - FlyingAvatar 4y ago
    Most of my questioning have been practical questions relating to computer science / programming and language learning (English / Spanish). I have asked a lot of other general knowledge questions, but nothing exceptionally deep. Can you give more detail as to the hallucinations that you found?
- vharuck 4y ago
  Prompt: Cite three important academic journal articles about cybersecurity, including DOI numbers Response: 1. Anderson, R., & Moore, T. (2006). The Economics of Information Security. Science, 314(5799), 610-613. DOI: 10.1126/science.1130992 2. Schneier, B. (2015). Data and Goliath: The Hidden Battles to Collect Your Data and Control Your World. WW Norton & Company. DOI: 10.1080/13600834.2015.1093541 3. Fung, B., & Graham, M. (2019). AI and cybersecurity: Safety, privacy, and security in the age of AI. IEEE Security & Privacy, 17(3), 10-14. DOI: 10.1109/MSEC.2019.2904199 #1 is good. #2 is a book (the DOI points to nothing). I can't find any evidence #3 exists. I don't consider this a difficult question to ask of cybersecurity students, let alone teachers.
  - AussieWog93 4y ago
    Were you using GPT-3.5 or GPT-4? Apparently GPT-4 is much more accurate and natural in its responses.
    - FlyingAvatar 4y ago
      At the moment, this will happen in GPT-4 as well.
  - FlyingAvatar 4y ago
    I guess I see ChatGPT as a very broadly knowledgeable, yet fallible person, not as a database. It would be a very rare person who could come up with the data you asked for off the top of their heads. Granted, it would be much better if it could tell you "I don't know", rather than hallucinate facts. As far as it being able to explain base concepts in a conversational manner and helping bridge gaps in my own understanding, it has been very helpful to me. I'm sure there are certain subjects where this will work better than others, but I would guess especially for most K-12 classwork, ChatGPT even with its current warts would be a net gain for many students in helping to increase their understanding just by talking with it.
    - deleted 4y ago
      [deleted]
- curiousgibbon 4y ago
  ChatGPT does not get full credit on the homework I create, even when the topic is some standard-ish algorithm with significant presence in the training data. Or maybe I'm just a bad prompt engineer.
  - FlyingAvatar 4y ago
    What grade level is the homework you are creating?
    - curiousgibbon 3y ago
      MS in computer science. The specific question I was surprised about was a O(log (m+n)) algorithm for finding the median of the union of two sorted arrays. This is in-scope for elite leetcoders and similar. The code ChatGPT generated had some issues resulting in out of bounds array accesses. It also wasn't clear if the algorithm was the desired log(m+n) one or a simpler but inferior log(m) + log(n) one. Was the model was confused and blended parts of different approaches? Is that statement too anthropomorphic for a LLM?
- ChancyChance 4y ago
  If you read the news, it is libeling people. https://www.reuters.com/technology/australian-mayor-readies-worlds-first-defamation-lawsuit-over-chatgpt-content-2023-04-05/ https://www.reuters.com/technology/australian-mayor-readies-... EDIT: I commented that is often wrong when talking about RF analog circuits or motor control, but this is a rabbit hole. It's like a bad textbook.
FractalHQ 4y ago
I’m still on the waitlist but I hear hallucinations have been largely solved already by connecting it to the internet with ChatGPT Plugins and telling it to verify its sources.
- ChancyChance 4y ago
  Unfortunately there is no way to test if every answer is correct. There's no magic plugin that "solves" this, because this is beyond P=NP.
  - NoZebra120vClip 4y ago
    Just connect it to Facebook, and process any comment that begins with "Well, actually, ..."
  - KyeRussell 4y ago
    OK. Let’s keep the context of this conversation in mind. As is typical for these sorts of comments, you slowly move the goalposts from “non-starter”, to “my arbitrary idea of ‘good’, to “superhuman”. At my day job we get a pretty good idea of the shortcomings of a typical teacher. We are also getting a very good, measurable, idea of what LLMs are capable of in this area. There’s skeptical, and there’s being in denial.
Our_Benefactors 4y ago
This is so wrong it’s not even funny. It is absolutely a tool and can help with _many_ rote tasks.

As long as we keep having to ask, "ChatGPT answered this questions but was it right?" it is a nonstarter. It is currently a novelty, not a tool. To be a tool it needs to be reliable.

FlyingAvatar 4y ago

In most subjects I have found ChatGPT to be fairly reliable. Are there specific scenarios that you have found concerning? I think perfect reliability is not absolutely necessary. None of the human teachers that I have learned from were perfectly reliable even in their area of expertise. Thankfully, skepticism seems to be an innate human trait.

shadowfox 4y ago
> In most subjects I have found ChatGPT to be fairly reliable. Not the OP, but I am very curious what subjects you have experimented with and whether you used any special prompt techniques. My experience has been quite on the contrary: on the few subjects I have some understanding of, ChatGPT appears to do fine for basic questions and attempts to summarize concepts at a high-level, but any attempt at drilling in to specifics seem to cause it to hallucinate quite a bit, particularly with regards to concrete facts or citations or anything of that sort.
- FlyingAvatar 4y ago
  Most of my questioning have been practical questions relating to computer science / programming and language learning (English / Spanish). I have asked a lot of other general knowledge questions, but nothing exceptionally deep. Can you give more detail as to the hallucinations that you found?
vharuck 4y ago
Prompt: Cite three important academic journal articles about cybersecurity, including DOI numbers Response: 1. Anderson, R., & Moore, T. (2006). The Economics of Information Security. Science, 314(5799), 610-613. DOI: 10.1126/science.1130992 2. Schneier, B. (2015). Data and Goliath: The Hidden Battles to Collect Your Data and Control Your World. WW Norton & Company. DOI: 10.1080/13600834.2015.1093541 3. Fung, B., & Graham, M. (2019). AI and cybersecurity: Safety, privacy, and security in the age of AI. IEEE Security & Privacy, 17(3), 10-14. DOI: 10.1109/MSEC.2019.2904199 #1 is good. #2 is a book (the DOI points to nothing). I can't find any evidence #3 exists. I don't consider this a difficult question to ask of cybersecurity students, let alone teachers.
- AussieWog93 4y ago
  Were you using GPT-3.5 or GPT-4? Apparently GPT-4 is much more accurate and natural in its responses.
  - FlyingAvatar 4y ago
    At the moment, this will happen in GPT-4 as well.
- FlyingAvatar 4y ago
  I guess I see ChatGPT as a very broadly knowledgeable, yet fallible person, not as a database. It would be a very rare person who could come up with the data you asked for off the top of their heads. Granted, it would be much better if it could tell you "I don't know", rather than hallucinate facts. As far as it being able to explain base concepts in a conversational manner and helping bridge gaps in my own understanding, it has been very helpful to me. I'm sure there are certain subjects where this will work better than others, but I would guess especially for most K-12 classwork, ChatGPT even with its current warts would be a net gain for many students in helping to increase their understanding just by talking with it.
  - deleted 4y ago
    [deleted]
curiousgibbon 4y ago
ChatGPT does not get full credit on the homework I create, even when the topic is some standard-ish algorithm with significant presence in the training data. Or maybe I'm just a bad prompt engineer.
- FlyingAvatar 4y ago
  What grade level is the homework you are creating?
  - curiousgibbon 3y ago
    MS in computer science. The specific question I was surprised about was a O(log (m+n)) algorithm for finding the median of the union of two sorted arrays. This is in-scope for elite leetcoders and similar. The code ChatGPT generated had some issues resulting in out of bounds array accesses. It also wasn't clear if the algorithm was the desired log(m+n) one or a simpler but inferior log(m) + log(n) one. Was the model was confused and blended parts of different approaches? Is that statement too anthropomorphic for a LLM?
ChancyChance 4y ago
If you read the news, it is libeling people. https://www.reuters.com/technology/australian-mayor-readies-worlds-first-defamation-lawsuit-over-chatgpt-content-2023-04-05/ https://www.reuters.com/technology/australian-mayor-readies-... EDIT: I commented that is often wrong when talking about RF analog circuits or motor control, but this is a rabbit hole. It's like a bad textbook.

FractalHQ 4y ago

I’m still on the waitlist but I hear hallucinations have been largely solved already by connecting it to the internet with ChatGPT Plugins and telling it to verify its sources.

ChancyChance 4y ago
Unfortunately there is no way to test if every answer is correct. There's no magic plugin that "solves" this, because this is beyond P=NP.
- NoZebra120vClip 4y ago
  Just connect it to Facebook, and process any comment that begins with "Well, actually, ..."
- KyeRussell 4y ago
  OK. Let’s keep the context of this conversation in mind. As is typical for these sorts of comments, you slowly move the goalposts from “non-starter”, to “my arbitrary idea of ‘good’, to “superhuman”. At my day job we get a pretty good idea of the shortcomings of a typical teacher. We are also getting a very good, measurable, idea of what LLMs are capable of in this area. There’s skeptical, and there’s being in denial.

Our_Benefactors 4y ago

This is so wrong it’s not even funny. It is absolutely a tool and can help with _many_ rote tasks.