3 ms·

I have the same experience. Every time I try to have it code something that isn't completely trivial or all over the internet like quicksort, it always has bugs

I have the same experience. Every time I try to have it code something that isn't completely trivial or all over the internet like quicksort, it always has bugs and invents calls to functions that don't exist. And yes, I'm using GPT-4, the best model available. And I'm not even asking about an exotic language like F#, I'm asking it questions about C++ or Python. People are out there claiming that GPT is doing all their coding for them. I just don't see how, unless they simply did not know how to program at all. I feel like I'm either crazy, or all these people are lying.

throwawayanothe 3y ago
Same experience here
nkozyra 3y ago
> I feel like I'm either crazy, or all these people are lying. With some careful prompting I've been able to get some decent code that is 95% usable out of the box. If that saves me time and changes my role there into code review versus dev + code review, that's a win. If you just ask GPT4 to write a program and don't give it fairly specific guardrails I agree it spits out nearly junk.
- oblio 3y ago
  > If you just ask GPT4 to write a program and don't give it fairly specific guardrails I agree it spits out nearly junk. The thing is, if you do start drilling down and fixing all the issues, etc, is it a long term net time saver? I can't imagine we have research clarifying this question.
taco-hands 3y ago
> People are out there claiming that GPT is doing all their coding for them. I just don't see how, unless they simply did not know how to program at all. I doubt it and certainly not for anything beyond basic. I've seen (and tried GPT's for code input a lot) and often they come back with errors or weird implementations. I made one request yesterday for a linear regression function (yes, because I was being lazy). So was chatGPT... It spat out a trashy broken function that wasn't even remotely close to working - more along the lines of pseudo code. I complained saying "WTH, that doesn't even work" and it said "my apologies" and spits out a perfectly working accurate function! Go figure. Others have turned to testing tips or threats, which is an interesting avenue: https://minimaxir.com/2024/02/chatgpt-tips-analysis/ https://minimaxir.com/2024/02/chatgpt-tips-analysis/

throwawayanothe 3y ago

Same experience here

nkozyra 3y ago

> I feel like I'm either crazy, or all these people are lying. With some careful prompting I've been able to get some decent code that is 95% usable out of the box. If that saves me time and changes my role there into code review versus dev + code review, that's a win. If you just ask GPT4 to write a program and don't give it fairly specific guardrails I agree it spits out nearly junk.

oblio 3y ago
> If you just ask GPT4 to write a program and don't give it fairly specific guardrails I agree it spits out nearly junk. The thing is, if you do start drilling down and fixing all the issues, etc, is it a long term net time saver? I can't imagine we have research clarifying this question.

taco-hands 3y ago

> People are out there claiming that GPT is doing all their coding for them. I just don't see how, unless they simply did not know how to program at all. I doubt it and certainly not for anything beyond basic. I've seen (and tried GPT's for code input a lot) and often they come back with errors or weird implementations. I made one request yesterday for a linear regression function (yes, because I was being lazy). So was chatGPT... It spat out a trashy broken function that wasn't even remotely close to working - more along the lines of pseudo code. I complained saying "WTH, that doesn't even work" and it said "my apologies" and spits out a perfectly working accurate function! Go figure. Others have turned to testing tips or threats, which is an interesting avenue: https://minimaxir.com/2024/02/chatgpt-tips-analysis/ https://minimaxir.com/2024/02/chatgpt-tips-analysis/