5 ms·

Not too interesting to solve a "problem" that solves itself. Just going from GPT-3.5 to GPT-4 I'd say intent inference has improved by an order of magnitude. I

Not too interesting to solve a "problem" that solves itself. Just going from GPT-3.5 to GPT-4 I'd say intent inference has improved by an order of magnitude. It almost feels mind-reader-ish. I've been waffling because I'm trying to cope with the absurdity of this product by highlighting the flaws (it can fail badly on tough problems), but we've gone from CleverBot to a genie that can instantly analyze your written intent and produce a highly logically coherent, well-worded analysis in a very short period of time.

ChatGTP 4y ago
I’d double check the code. I have a similar experience but when I actually look at what’s been returned I find strange things in there. Like it often works, but it’s rarely as correct as it seems. At least for a given use case.
- boringuser2 4y ago
  I read all of the code it writes.
- rjbwork 4y ago
  I agree. One of the engineers on my team came to me with a script (maybe 20 lines of bash) like "hey isn't it cool that I got ChatGPT to generate this?" and after studying it for a few minutes I was like "it is cool, but now spot the bug". It was near the end of the day so I asked them to follow up with me in the morning after they'd found and fixed the bug. I ended up having to point it out directly the next day. It was a spurious -A NUM argument that would, in a production scenario, cause it to recognize valid data as invalid because the full context of the data would be missing. All this to say, it is a very useful tool, but we should be highly suspicious of its output, and really make sure we understand what it's giving us. The ability to read and mentally parse and execute code will become extremely important, and give people with that ability a leg up.
  - boringuser2 4y ago
    Give me the snippet and I guarantee I can get GPT-4 to identify the bug with no hints and minimal prompting.
    - 8n4vidtmkvmk 4y ago
      it often works if you yourself can spot the bug, you can coax it to fix it but if you don't know it's there...
      - int_19h 4y ago
        With GPT-4, quite often, if you just ask it to review its own result for correctness, it'll spot the issue without further prompting. So you can just always ask that, although then you risk dealing with hallucinated non-bugs.
        deleted 4y ago
        [deleted]
    - rjbwork 4y ago
      I won't do that...but I will try it myself. Give me a few. EDIT: Well I'll be damned. "However, there is a bug in the script as it only checks for the tags attribute in the first 10 lines of the resource block using the grep -A10 command. "
      - boringuser2 4y ago
        It takes GPT-4 perhaps 25-30 tries on the API to help me write an entire class with complex functionality, including testing, correcting and asking for new output. Sometimes, it can one-shot the problem. Gonna go with Carmack and say that "hand-coding" isn't going to last. Another thing it can do is reliably debug a file, or point me in the right direction, if I explain the bug and provide the code. It's scary as hell because I can reliably assume that it will always understand the nuance to the content I provide.
otabdeveloper4 4y ago
Reading minds is not a hard problem to solve. The Akinator can do that reliably and it's not even a neural network. (I won't even mention the venerable millenia-old fortune telling industry.)
- boringuser2 4y ago
  20 questions is easy. Extracting intent from one piece of text when a number of interpretations exist that I cannot count is not.

ChatGTP 4y ago

I’d double check the code. I have a similar experience but when I actually look at what’s been returned I find strange things in there. Like it often works, but it’s rarely as correct as it seems. At least for a given use case.

boringuser2 4y ago
I read all of the code it writes.
rjbwork 4y ago
I agree. One of the engineers on my team came to me with a script (maybe 20 lines of bash) like "hey isn't it cool that I got ChatGPT to generate this?" and after studying it for a few minutes I was like "it is cool, but now spot the bug". It was near the end of the day so I asked them to follow up with me in the morning after they'd found and fixed the bug. I ended up having to point it out directly the next day. It was a spurious -A NUM argument that would, in a production scenario, cause it to recognize valid data as invalid because the full context of the data would be missing. All this to say, it is a very useful tool, but we should be highly suspicious of its output, and really make sure we understand what it's giving us. The ability to read and mentally parse and execute code will become extremely important, and give people with that ability a leg up.
- boringuser2 4y ago
  Give me the snippet and I guarantee I can get GPT-4 to identify the bug with no hints and minimal prompting.
  - 8n4vidtmkvmk 4y ago
    it often works if you yourself can spot the bug, you can coax it to fix it but if you don't know it's there...
    - int_19h 4y ago
      With GPT-4, quite often, if you just ask it to review its own result for correctness, it'll spot the issue without further prompting. So you can just always ask that, although then you risk dealing with hallucinated non-bugs.
      - deleted 4y ago
        [deleted]
  - rjbwork 4y ago
    I won't do that...but I will try it myself. Give me a few. EDIT: Well I'll be damned. "However, there is a bug in the script as it only checks for the tags attribute in the first 10 lines of the resource block using the grep -A10 command. "
    - boringuser2 4y ago
      It takes GPT-4 perhaps 25-30 tries on the API to help me write an entire class with complex functionality, including testing, correcting and asking for new output. Sometimes, it can one-shot the problem. Gonna go with Carmack and say that "hand-coding" isn't going to last. Another thing it can do is reliably debug a file, or point me in the right direction, if I explain the bug and provide the code. It's scary as hell because I can reliably assume that it will always understand the nuance to the content I provide.

otabdeveloper4 4y ago

Reading minds is not a hard problem to solve. The Akinator can do that reliably and it's not even a neural network. (I won't even mention the venerable millenia-old fortune telling industry.)

boringuser2 4y ago
20 questions is easy. Extracting intent from one piece of text when a number of interpretations exist that I cannot count is not.