13 ms·

1) verifying code is harder than writing it and 2) verifying code requires domain knowledge, which implies that the utility of these models is limited to thing

1) verifying code is harder than writing it and 2) verifying code requires domain knowledge, which implies that the utility of these models is limited to things could write myself if I weren't too lazy. That's hugely constricting.

dmix 4y ago
Yes but I don’t see it as generating entire blocks of code you’re supposed to copy and paste into your project. It’s like a template, a suggestion from which you can build your own version. Only rarely does it have the context or understanding of the wider codebase to do a programmers job for them. The times when it does generate a copy/pastable function it’s usually some isolated utility function like “format date as DD-YYYY” something really simple and easy to verify. The type of stuff you’d copy entirely from Stackoverflow rather than finding a specific solution you adapt. It’s mostly a glorified autocomplete and example suggestion service. It is not a full code writing service. Domain expertise will obviously still be a job requirement. It’s an assistant to the programmer, not an occasional replacement for the programmer (and if you have domain expertise you usually use a 3rd party library). Maybe future versions will try to do more but that’s not what we have today.
- AlotOfReading 4y ago
  I think that many people will treat it as something that can generate entire blocks of code. Unfortunately it can be quite broken even just writing basic functions. One of my tests was doing some quaternion rotations. It did them, but refused to stop multiplying by the conjugate afterwards. Another was converting dates to "years ago", which was broken for BCE because ChatGPT doesn't understand the underlying concept.
  - dmix 4y ago
    > I think that many people will treat it as something that can generate entire blocks of code. Unfortunately it can be quite broken even just Have you tried to use it for this purpose? It basically can’t unless you’re only building a toy app. Even after multiple levels of refinement it still requires tons of real programming work. Which is largely my point, it won’t because it’s fundamentally incapable of providing that in its current state. Even without the buggy part it’s mostly just outputs generic stuff that will always need to be integrated into the wider codebase and to specifically what you’re trying to build.
    - AlotOfReading 4y ago
      I have, actually. Today I had it write a basic JS map for my wedding site. Yesterday I had it produce a puzzle solver with CLI and DSL. Obviously I'm still doing manual interventions at key points, but it's changed my personal cost/benefit calculation on whether various random ideas are worth doing.

dmix 4y ago

Yes but I don’t see it as generating entire blocks of code you’re supposed to copy and paste into your project. It’s like a template, a suggestion from which you can build your own version. Only rarely does it have the context or understanding of the wider codebase to do a programmers job for them. The times when it does generate a copy/pastable function it’s usually some isolated utility function like “format date as DD-YYYY” something really simple and easy to verify. The type of stuff you’d copy entirely from Stackoverflow rather than finding a specific solution you adapt. It’s mostly a glorified autocomplete and example suggestion service. It is not a full code writing service. Domain expertise will obviously still be a job requirement. It’s an assistant to the programmer, not an occasional replacement for the programmer (and if you have domain expertise you usually use a 3rd party library). Maybe future versions will try to do more but that’s not what we have today.

AlotOfReading 4y ago
I think that many people will treat it as something that can generate entire blocks of code. Unfortunately it can be quite broken even just writing basic functions. One of my tests was doing some quaternion rotations. It did them, but refused to stop multiplying by the conjugate afterwards. Another was converting dates to "years ago", which was broken for BCE because ChatGPT doesn't understand the underlying concept.
- dmix 4y ago
  > I think that many people will treat it as something that can generate entire blocks of code. Unfortunately it can be quite broken even just Have you tried to use it for this purpose? It basically can’t unless you’re only building a toy app. Even after multiple levels of refinement it still requires tons of real programming work. Which is largely my point, it won’t because it’s fundamentally incapable of providing that in its current state. Even without the buggy part it’s mostly just outputs generic stuff that will always need to be integrated into the wider codebase and to specifically what you’re trying to build.
  - AlotOfReading 4y ago
    I have, actually. Today I had it write a basic JS map for my wedding site. Yesterday I had it produce a puzzle solver with CLI and DSL. Obviously I'm still doing manual interventions at key points, but it's changed my personal cost/benefit calculation on whether various random ideas are worth doing.