4 ms·

That's the worst part about generative AI IMO - it makes writing new code faster - it barely helps with editing existing code. So when someone eventually update

That's the worst part about generative AI IMO - it makes writing new code faster - it barely helps with editing existing code. So when someone eventually updates the code and forgets to update the comments I wouldn't be surprised if the misleading comments made AI hallucinate.

coliveira 3y ago
I believe that AI will get so good at creating new code that a lot of existing libraries will be let unused. What is the point of using lots of libraries if AI can generate the code we need directly? The AI will be the library itself, and the generated code will embed the knowledge about doing lots of things for which we used libraries.
- pydry 3y ago
  >What is the point of using lots of libraries if AI can generate the code we need directly? Theyve been debugged.
  - vasili111 3y ago
    And also have documentation.
  - throwuwu 3y ago
    At some point the models will produce code with a lower error rate than existing libraries.
    - funnymony 3y ago
      How do you ensure quality? Review and testing. Reviewing is easier when there is less code (i.e. libraries are in use)
      - throwuwu 3y ago
        It’s crazy how many people miss this. GPT models can review code too! They can also write and run tests. Once the context window is big enough to fit the whole code base into it they will be better at review than you are. Eventually we’ll have fine tuned models that are experts in any subject you can think of, the only barrier is data and a lot of recent research is showing that that can be machine generated too.
        moonchrome 3y ago
        GPT 4 pre nerf was terrible at reviewing non-trivial or non textbook code. I've decided to test it for a few weeks by checking stuff I caught in review or as bugs, to see if it would spot it. It was like 0% on first try (would always talk about something irrelevant) and after leading it with follow up questions it would figure out the problem half of the time and half of the time I'd just give up leading it. These were tricky problems that were small scope - I've picked them so I could easily provide it to GPT for review. So I doubt larger context window will do much.
        throwuwu 3y ago
        It’s hard to tell why you ran into such a problem without seeing how you prompted but I can offer a few pointers. Use the OpenAI playground instead of chat, it allows you to specify the system prompt and edit the conversation before each submission. System prompt is good for providing general context, tools and options but you absolutely must provide a few example interactions in the conversation. Even just two prompt and response pairs will strongly influence the rest of the conversation. You can use that to shape the responses however you like and it focuses the model on the task at hand. If you get a bad response, delete or edit it. Bad examples beget more bad responses.

coliveira 3y ago

I believe that AI will get so good at creating new code that a lot of existing libraries will be let unused. What is the point of using lots of libraries if AI can generate the code we need directly? The AI will be the library itself, and the generated code will embed the knowledge about doing lots of things for which we used libraries.

pydry 3y ago
>What is the point of using lots of libraries if AI can generate the code we need directly? Theyve been debugged.
- vasili111 3y ago
  And also have documentation.
- throwuwu 3y ago
  At some point the models will produce code with a lower error rate than existing libraries.
  - funnymony 3y ago
    How do you ensure quality? Review and testing. Reviewing is easier when there is less code (i.e. libraries are in use)
    - throwuwu 3y ago
      It’s crazy how many people miss this. GPT models can review code too! They can also write and run tests. Once the context window is big enough to fit the whole code base into it they will be better at review than you are. Eventually we’ll have fine tuned models that are experts in any subject you can think of, the only barrier is data and a lot of recent research is showing that that can be machine generated too.
      - moonchrome 3y ago
        GPT 4 pre nerf was terrible at reviewing non-trivial or non textbook code. I've decided to test it for a few weeks by checking stuff I caught in review or as bugs, to see if it would spot it. It was like 0% on first try (would always talk about something irrelevant) and after leading it with follow up questions it would figure out the problem half of the time and half of the time I'd just give up leading it. These were tricky problems that were small scope - I've picked them so I could easily provide it to GPT for review. So I doubt larger context window will do much.
        throwuwu 3y ago
        It’s hard to tell why you ran into such a problem without seeing how you prompted but I can offer a few pointers. Use the OpenAI playground instead of chat, it allows you to specify the system prompt and edit the conversation before each submission. System prompt is good for providing general context, tools and options but you absolutely must provide a few example interactions in the conversation. Even just two prompt and response pairs will strongly influence the rest of the conversation. You can use that to shape the responses however you like and it focuses the model on the task at hand. If you get a bad response, delete or edit it. Bad examples beget more bad responses.