4 ms·

I finally went ahead and tried ChatGPT this morning (along with everyone else seemingly - it is now heavily rate-limited!) and I am truly blown away. I ask ques

I finally went ahead and tried ChatGPT this morning (along with everyone else seemingly - it is now heavily rate-limited!) and I am truly blown away. I ask questions about various things (in my case, embedded systems development on the PineNote) and it gives the same quality of answers I would expect from a median coworker. Sure they're maybe not 100% correct, but most coworker answers aren't 100% correct either. They're at least in the right ballpark, and very rarely do we achieve detailed of knowledge of something from a single source - it's all about taking in info from various places and integrating them (conflicts and all) to derive a more-and-more-detailed grasp of the things we're learning about. The ability to ask questions about a specific thing (example: What are waveform files for e-ink screens? Followup: Is it possible to damage e-ink screens by modifying wave form files?) very quickly without having to sift through mountains of blogs and random Q/A websites and documentation and textbooks for the answer is incredibly useful.

roperj 4y ago
Youre commenting on an article where the answer was not even just a little wrong - it was completely wrong. Sometimes it’s “in the ballpark” - which is apparently t good enough these days - but often times it is just confidently entirely correct. How are you able to use such a tool as you propose practically?
- ahelwer 4y ago
  I could easily imagine an ordinary person giving this exact same wrong answer (confusing Hobbes and Locke) - we're talking about value-over-replacement here! In the process of learning things we take in wrong information all the time. A lot of it is even intentionally wrong, in the sense of being simplified. These are rarely large obstacles to true understanding. Sometimes they're even beneficial, as correcting prior beliefs can drive home the more accurate belief with greater force.
  - jcranmer 4y ago
    If this were a test question, the response given would be marked wrong, likely with no partial credit awarded. It's that egregiously wrong, even if the attribution is perhaps understandable.
    - ahelwer 4y ago
      Well thankfully I haven't been in school for over a decade at this point, so rarely (never?) encounter these hard-cutoff no-second-chances trivia situations. I operate in the real world, where continually making something that's not-quite-correct then refining it and fixing the errors is how basically everything is accomplished.
      - roperj 4y ago
        > not-quite-correct then refining it and fixing the errors is how basically everything is accomplished. Not-quite-correct is a wordy misrepresentation when something is plainly incorrect. In many fields being confidently wrong is how you get fucked over hard. That ChatGPT will gleefully fabricate references like a schizophrenic is just more icing on this shit cake.
      - jcranmer 4y ago
        > I operate in the real world, where continually making something that's not-quite-correct then refining it and fixing the errors is how basically everything is accomplished. This isn't really not-quite-correct; it's egregiously wrong in the central aspect of the thesis. Refining and fixing it requires understanding that, and how, it's wrong--and if you have that level of knowledge, why are you using a tool to write this kind of thing? It's not going to save you much time from actually writing the whole thing yourself.
    - appletrotter 4y ago
      You can correct the bot. If you tell it that a fact it has wrong is wrong, you can get it to change its answer. Also, you can just make tweaks manually. Seems super useful to me as long as you understand the limitations. Trust it the right it amount - don't trust it too much, and you're golden.
  - adamsmith143 4y ago
    I think the problem is assuming GPTChat is a reliable source at all. You can probably assume your median coworker knows something correct about embedded systems but it's not clear why you would assume or if you should ever assume ChatGPT is correct about anything.
cromwellian 4y ago
If you ask it about things which require deduction like Math, even simple Math questions like multiply binomials or solve a quadratic it gets it totally wrong, confidently, and even if you correct it, it often still gets it wrong. It’s not even close to something like Wolfram Alpha. I think we’re blown away more by its command of language and prose than by its reasoning ability. It’s fantastic at generation, but like stable diffusion, things can fit together and look beautiful yet still be not what you asked.
- alain94040 4y ago
  Sure. But if you combine the understanding that this chatbot has with a Wolfram Alpha backend, you could build an even more amazing system. I'm sure someone is working on hooking up language models to math backends (anywhere from a simple calculator to Wolfram Alpha).
  - cromwellian 4y ago
    DeepMind published a system that does sort this with a backend theorem prover a year ago. My point is, I don’t think transformer based text prediction systems are the right model here. I could be wrong, but it think about how formal systems work, they seem a far cry from what decoder architectures are doing. https://www.nature.com/articles/s41586-021-04086-x https://www.nature.com/articles/s41586-021-04086-x
constantcrying 4y ago
>What are waveform files for e-ink screens? Followup: Is it possible to damage e-ink screens by modifying wave form files? What is the value of these answers if there is a 1% chance of them being wrong? From my experience it is a better way to google things, which certainly is an advancement and quite novel.

roperj 4y ago

Youre commenting on an article where the answer was not even just a little wrong - it was completely wrong. Sometimes it’s “in the ballpark” - which is apparently t good enough these days - but often times it is just confidently entirely correct. How are you able to use such a tool as you propose practically?

ahelwer 4y ago
I could easily imagine an ordinary person giving this exact same wrong answer (confusing Hobbes and Locke) - we're talking about value-over-replacement here! In the process of learning things we take in wrong information all the time. A lot of it is even intentionally wrong, in the sense of being simplified. These are rarely large obstacles to true understanding. Sometimes they're even beneficial, as correcting prior beliefs can drive home the more accurate belief with greater force.
- jcranmer 4y ago
  If this were a test question, the response given would be marked wrong, likely with no partial credit awarded. It's that egregiously wrong, even if the attribution is perhaps understandable.
  - ahelwer 4y ago
    Well thankfully I haven't been in school for over a decade at this point, so rarely (never?) encounter these hard-cutoff no-second-chances trivia situations. I operate in the real world, where continually making something that's not-quite-correct then refining it and fixing the errors is how basically everything is accomplished.
    - roperj 4y ago
      > not-quite-correct then refining it and fixing the errors is how basically everything is accomplished. Not-quite-correct is a wordy misrepresentation when something is plainly incorrect. In many fields being confidently wrong is how you get fucked over hard. That ChatGPT will gleefully fabricate references like a schizophrenic is just more icing on this shit cake.
    - jcranmer 4y ago
      > I operate in the real world, where continually making something that's not-quite-correct then refining it and fixing the errors is how basically everything is accomplished. This isn't really not-quite-correct; it's egregiously wrong in the central aspect of the thesis. Refining and fixing it requires understanding that, and how, it's wrong--and if you have that level of knowledge, why are you using a tool to write this kind of thing? It's not going to save you much time from actually writing the whole thing yourself.
  - appletrotter 4y ago
    You can correct the bot. If you tell it that a fact it has wrong is wrong, you can get it to change its answer. Also, you can just make tweaks manually. Seems super useful to me as long as you understand the limitations. Trust it the right it amount - don't trust it too much, and you're golden.
- adamsmith143 4y ago
  I think the problem is assuming GPTChat is a reliable source at all. You can probably assume your median coworker knows something correct about embedded systems but it's not clear why you would assume or if you should ever assume ChatGPT is correct about anything.

cromwellian 4y ago

If you ask it about things which require deduction like Math, even simple Math questions like multiply binomials or solve a quadratic it gets it totally wrong, confidently, and even if you correct it, it often still gets it wrong. It’s not even close to something like Wolfram Alpha. I think we’re blown away more by its command of language and prose than by its reasoning ability. It’s fantastic at generation, but like stable diffusion, things can fit together and look beautiful yet still be not what you asked.

alain94040 4y ago
Sure. But if you combine the understanding that this chatbot has with a Wolfram Alpha backend, you could build an even more amazing system. I'm sure someone is working on hooking up language models to math backends (anywhere from a simple calculator to Wolfram Alpha).
- cromwellian 4y ago
  DeepMind published a system that does sort this with a backend theorem prover a year ago. My point is, I don’t think transformer based text prediction systems are the right model here. I could be wrong, but it think about how formal systems work, they seem a far cry from what decoder architectures are doing. https://www.nature.com/articles/s41586-021-04086-x https://www.nature.com/articles/s41586-021-04086-x

constantcrying 4y ago

>What are waveform files for e-ink screens? Followup: Is it possible to damage e-ink screens by modifying wave form files? What is the value of these answers if there is a 1% chance of them being wrong? From my experience it is a better way to google things, which certainly is an advancement and quite novel.