4 ms·

What the articles like this miss is that we understand pretty well how ChatGPT writes code. But what about maintenance, and particularly debugging? How would it

What the articles like this miss is that we understand pretty well how ChatGPT writes code. But what about maintenance, and particularly debugging? How would it make these tasks. And we know that these things take much much more time than writing code.

realusername 4y ago
Especially that code generators are pretty old, ChatGPT is very late to the game. I see all this people spending time to create a landing page with ChatGPT whereas in reality you can just install Wordpress and it even looks nicer.
- manojlds 4y ago
  As though creating landing pages is where it's tricky. So many cookie cutter examples to just create one. One of the examples from GPT wasn't even responsive and didn't work on mobile. But...some folks are creating iOS games without any Swift experience, so that's cool.
krsdcbl 4y ago
Plain and simple: Feed the code it produced back into it and ask it to modify the sections needing maintenance or changes, or just tell it the issues you're having and ask it to explain/debug them.
- solomatov 4y ago
  IMO, the problem is that it doesn't work. It could introduce other bugs, make incorrect changes, or mess up the code. If a good developer does this, it converges, with GPT I am not so sure. We need to find a way for the system to iterate, debug, write unit tests, and repeat the loop. In theory it's doable, but we are yet to see the system which does this.
- globalise83 4y ago
  Be careful! I have been experimenting with just this approach and ChatGPT 4 often forgets important parts of the code when generating the new files. If you ask ChatGPT to break down every change into step-by-step changes with code examples, it does a really great job and you can check every change yourself before implementing it.
  - throwthrowuknow 4y ago
    A good prompt is key. If pasting code make sure to run it through the tokenizer first to make sure it’s not too long. Be very descriptive about what you want it to do. If it goes off the rails just hit the stop button and then edit your prompt with the little edit button at the top right and try again, this saves on context length and it saves the previous prompt and response so you can switch between them and compare.
manojlds 4y ago
Which is why I still believe proper developers will still have jobs :) GPT is just Spreadsheets in a way. Lot of companies had business users and SMEs create contraptions based on Excel and then when it's a hairy mess or a rube goldberg machine, there are "digital transformation" projects that are done to make them proper apps.
- lordnacho 4y ago
  That might be the exact right analogy. People who see Excel for the first time seem to think they have superpowers. In actual fact the superpower is knowing how to use it effectively and knowing when you are on the edge of its abilities.
blagie 4y ago
I find ChatGPT is pretty helpful for debugging. A big chunk of debugging is missing stupid things, and for debugging, it's like having a second set of eyes. It's almost useless for high-level design and architecture. Those tasks do take more time than writing code, but they're super-charged by rapid prototyping, which ChatGPT excels at. For example, right now, if I'm picking between three libraries, I can have working versions of my code implemented in all three in a few minutes, and built out a bigger mockup in a few hours. I can make a more educated decision from there. In the olden days, that'd be a few hours and many days, respectively. It also does really nicely for code reviews, which results in better code. That also speeds up maintenance and, later, debugging. That's especially true in domains I'm less familiar with. Some maintenance, it excels at. I'm the author of a major piece of open source infrastructure, which is widely used, well-architected, but every technology used is obsolete (JQuery, older Python web frameworks, legacy database, etc.). I feel like I could rewrite the whole thing in React+Node+etc. in a few weeks. There is close to zero architecture work -- it's mostly having GPT write -- or sometimes just translate -- code piece-by-piece. I don't feel like GPT-4 obsoletes software engineers as it's used yet, but it does really raise the bar on what's possible to build.
- nolawi89 4y ago
  Not yet; I spend 60% my day reviewing PRs. I don't have to if chat GPT gets integrated as a PR reviewer..
  - grumple 4y ago
    I review PRs primarily for correctness and to look for nonobvious problems. I rarely find buggy code. Sometimes I'll find performance issues. In order to do what I do when reviewing, ChatGPT would need clear specs (yeah right) and a deep understanding of our system, domain, and even our customers. Of course I'll try it and see how it does. But it struggled and failed to produce what I would consider trivial code when I tried to create a single usable page to spec in react native, so I'm not terribly optimistic.
  - Lewton 4y ago
    ChatGPT will turn your day into spending 90% of your time reviewing code...
elif 4y ago
GPT is a much better debugger than coder in my experience. It is a drop in replacement for googling random linux errors too. "I'm getting this error <error> with the following code. Please add log statements that will test for all the things that could be going wrong <code>" Saved me about 15 minutes yesterday. Not only does chatGPT generate the probably causes of the error, it knows how to test the code for those errors. The human doesn't even have to read the error message anymore.

realusername 4y ago

Especially that code generators are pretty old, ChatGPT is very late to the game. I see all this people spending time to create a landing page with ChatGPT whereas in reality you can just install Wordpress and it even looks nicer.

manojlds 4y ago
As though creating landing pages is where it's tricky. So many cookie cutter examples to just create one. One of the examples from GPT wasn't even responsive and didn't work on mobile. But...some folks are creating iOS games without any Swift experience, so that's cool.

krsdcbl 4y ago

Plain and simple: Feed the code it produced back into it and ask it to modify the sections needing maintenance or changes, or just tell it the issues you're having and ask it to explain/debug them.

solomatov 4y ago
IMO, the problem is that it doesn't work. It could introduce other bugs, make incorrect changes, or mess up the code. If a good developer does this, it converges, with GPT I am not so sure. We need to find a way for the system to iterate, debug, write unit tests, and repeat the loop. In theory it's doable, but we are yet to see the system which does this.
globalise83 4y ago
Be careful! I have been experimenting with just this approach and ChatGPT 4 often forgets important parts of the code when generating the new files. If you ask ChatGPT to break down every change into step-by-step changes with code examples, it does a really great job and you can check every change yourself before implementing it.
- throwthrowuknow 4y ago
  A good prompt is key. If pasting code make sure to run it through the tokenizer first to make sure it’s not too long. Be very descriptive about what you want it to do. If it goes off the rails just hit the stop button and then edit your prompt with the little edit button at the top right and try again, this saves on context length and it saves the previous prompt and response so you can switch between them and compare.

manojlds 4y ago

Which is why I still believe proper developers will still have jobs :) GPT is just Spreadsheets in a way. Lot of companies had business users and SMEs create contraptions based on Excel and then when it's a hairy mess or a rube goldberg machine, there are "digital transformation" projects that are done to make them proper apps.

lordnacho 4y ago
That might be the exact right analogy. People who see Excel for the first time seem to think they have superpowers. In actual fact the superpower is knowing how to use it effectively and knowing when you are on the edge of its abilities.

blagie 4y ago

I find ChatGPT is pretty helpful for debugging. A big chunk of debugging is missing stupid things, and for debugging, it's like having a second set of eyes. It's almost useless for high-level design and architecture. Those tasks do take more time than writing code, but they're super-charged by rapid prototyping, which ChatGPT excels at. For example, right now, if I'm picking between three libraries, I can have working versions of my code implemented in all three in a few minutes, and built out a bigger mockup in a few hours. I can make a more educated decision from there. In the olden days, that'd be a few hours and many days, respectively. It also does really nicely for code reviews, which results in better code. That also speeds up maintenance and, later, debugging. That's especially true in domains I'm less familiar with. Some maintenance, it excels at. I'm the author of a major piece of open source infrastructure, which is widely used, well-architected, but every technology used is obsolete (JQuery, older Python web frameworks, legacy database, etc.). I feel like I could rewrite the whole thing in React+Node+etc. in a few weeks. There is close to zero architecture work -- it's mostly having GPT write -- or sometimes just translate -- code piece-by-piece. I don't feel like GPT-4 obsoletes software engineers as it's used yet, but it does really raise the bar on what's possible to build.

nolawi89 4y ago
Not yet; I spend 60% my day reviewing PRs. I don't have to if chat GPT gets integrated as a PR reviewer..
- grumple 4y ago
  I review PRs primarily for correctness and to look for nonobvious problems. I rarely find buggy code. Sometimes I'll find performance issues. In order to do what I do when reviewing, ChatGPT would need clear specs (yeah right) and a deep understanding of our system, domain, and even our customers. Of course I'll try it and see how it does. But it struggled and failed to produce what I would consider trivial code when I tried to create a single usable page to spec in react native, so I'm not terribly optimistic.
- Lewton 4y ago
  ChatGPT will turn your day into spending 90% of your time reviewing code...

elif 4y ago

GPT is a much better debugger than coder in my experience. It is a drop in replacement for googling random linux errors too. "I'm getting this error <error> with the following code. Please add log statements that will test for all the things that could be going wrong <code>" Saved me about 15 minutes yesterday. Not only does chatGPT generate the probably causes of the error, it knows how to test the code for those errors. The human doesn't even have to read the error message anymore.