5 ms·

"when you ask for a snippet of code and it gets it right" Most examples I have seen are about basic code that is highly likely to be already present in the tra

by aredox 3y ago

"when you ask for a snippet of code and it gets it right" Most examples I have seen are about basic code that is highly likely to be already present in the training data - making the language model an expert system, but with a worse rate of failure. Meanwhile it has a hard time creating something it hasn't already seen, or a close variant.

simonw 3y ago
> Meanwhile it has a hard time creating something it hasn't already seen, or a close variant. How much time have you spent with the strongest models, such as GPT-4? I've been using it on a daily basis for 6+ months now and your statement there doesn't fit my experience at all.
- ryandvm 3y ago
  Indeed. I feel like people are not even talking about the same thing in here. I've been using GPT-4 daily for a couple months and it's not outputting code that "only looks correct". It is writing functional solutions that often work as is. Just yesterday I pasted in a hastily written function and told GPT-4 to "please evaluate this function and make sure it's bulletproof". It outputted a bulleted list of criticisms (mostly true) and then spit out a version of the function that was shorter, easier to understand, and was safer. That's not just the "next most likely token". Or if it is, then that's all any of us are doing.
  - rbranson 3y ago
    It’s probably more accurate to say that what your brain is doing when writing code is more like emulating an LLM than an LLM is emulating your brain. Converting human language to code is an ideal task for LLMs, which is part of why they are such an exciting development.
- zerbinxx 3y ago
  If you ask it to start explaining/extending/implementing classes/interfaces, start asking it to reason about leaky abstractions or when it’s appropriate to use Mixins, it frequently generates code that either doesn’t work or just misses the point. It’s especially hard because some of these problems actually are not tractable in the language you’re taking about due to their features, or due to limitations of your code/your knowledge it may suggest you do something slightly worse because you haven’t prompted it on factories or other design patterns that you might want to use.
  - simonw 3y ago
    There are an almost unlimited number of ways you can prompt it that will produce obviously bad results. The trick to using it effectively is to figure out the ways of prompting it that produce GOOD results.
    - zerbinxx 3y ago
      Right, but obviously you can see the unknown unknown problem or Dunning-Krueger popping up when software developers who don’t know everything (unlike me, of course) start using it extensively without knowing the ways in which their prompt engineering is flawed.
      - simonw 3y ago
        One of the big open questions in LLM-assisted learning is whether it helps or hinders less experienced developers. At this point I'm pretty confident it helps them. They need to understand that it's not infallible, but I think most people figure that out pretty quickly after using it for more than a few days. There's absolutely a skill to using these things well, and it's a difficult one to teach because it's based more on intuition that you develop over time. The key thing is helping people understand the strengths and limitations of these tools. That's something I do think we can teach.
    - lawlessone 3y ago
      >The trick to using it effectively is to figure out the ways of prompting it that produce GOOD results Might as well just learn to code then.
      - simonw 3y ago
        My argument is that AI-assistance makes it easier and faster to learn to code.
    - jazzyjackson 3y ago
      it's not just good prompts vs bad prompts, there's luck involved, since the result is determined by dice rolls. I used to be smug about getting good results from GPT while coworkers failed to get it to produce anything useful, but over time my luck changed, and GPT stopped nailing it. If you remember there was a bunch of conspiracy talk that GPT got lobotomized, was no longer writing full programs anymore etc, but what really happened is people who got used to rolling 7s started rolling 2s and 3s and shook their fists in anger at their change of luck.
j4yav 3y ago
You can invent a programming language on the spot and it will help you write programs in it.

by aredox 3y ago

simonw 3y ago

> Meanwhile it has a hard time creating something it hasn't already seen, or a close variant. How much time have you spent with the strongest models, such as GPT-4? I've been using it on a daily basis for 6+ months now and your statement there doesn't fit my experience at all.

ryandvm 3y ago
Indeed. I feel like people are not even talking about the same thing in here. I've been using GPT-4 daily for a couple months and it's not outputting code that "only looks correct". It is writing functional solutions that often work as is. Just yesterday I pasted in a hastily written function and told GPT-4 to "please evaluate this function and make sure it's bulletproof". It outputted a bulleted list of criticisms (mostly true) and then spit out a version of the function that was shorter, easier to understand, and was safer. That's not just the "next most likely token". Or if it is, then that's all any of us are doing.
- rbranson 3y ago
  It’s probably more accurate to say that what your brain is doing when writing code is more like emulating an LLM than an LLM is emulating your brain. Converting human language to code is an ideal task for LLMs, which is part of why they are such an exciting development.
zerbinxx 3y ago
If you ask it to start explaining/extending/implementing classes/interfaces, start asking it to reason about leaky abstractions or when it’s appropriate to use Mixins, it frequently generates code that either doesn’t work or just misses the point. It’s especially hard because some of these problems actually are not tractable in the language you’re taking about due to their features, or due to limitations of your code/your knowledge it may suggest you do something slightly worse because you haven’t prompted it on factories or other design patterns that you might want to use.
- simonw 3y ago
  There are an almost unlimited number of ways you can prompt it that will produce obviously bad results. The trick to using it effectively is to figure out the ways of prompting it that produce GOOD results.
  - zerbinxx 3y ago
    Right, but obviously you can see the unknown unknown problem or Dunning-Krueger popping up when software developers who don’t know everything (unlike me, of course) start using it extensively without knowing the ways in which their prompt engineering is flawed.
    - simonw 3y ago
      One of the big open questions in LLM-assisted learning is whether it helps or hinders less experienced developers. At this point I'm pretty confident it helps them. They need to understand that it's not infallible, but I think most people figure that out pretty quickly after using it for more than a few days. There's absolutely a skill to using these things well, and it's a difficult one to teach because it's based more on intuition that you develop over time. The key thing is helping people understand the strengths and limitations of these tools. That's something I do think we can teach.
  - lawlessone 3y ago
    >The trick to using it effectively is to figure out the ways of prompting it that produce GOOD results Might as well just learn to code then.
    - simonw 3y ago
      My argument is that AI-assistance makes it easier and faster to learn to code.
  - jazzyjackson 3y ago
    it's not just good prompts vs bad prompts, there's luck involved, since the result is determined by dice rolls. I used to be smug about getting good results from GPT while coworkers failed to get it to produce anything useful, but over time my luck changed, and GPT stopped nailing it. If you remember there was a bunch of conspiracy talk that GPT got lobotomized, was no longer writing full programs anymore etc, but what really happened is people who got used to rolling 7s started rolling 2s and 3s and shook their fists in anger at their change of luck.

j4yav 3y ago

You can invent a programming language on the spot and it will help you write programs in it.