4 ms·

I've been using it extensively and never once found ChatGPT to produce working code when the ask is just above basic. It might have to do with subject matter. I

I've been using it extensively and never once found ChatGPT to produce working code when the ask is just above basic. It might have to do with subject matter. It usually is close enough but my experience has been what you've found with Bard - missing arguments, phantom methods, etc. Haven't bothered with Bard at all.

UncleOxidant 4y ago
did you try with GPT4? I was able to run one GPT4 query/day for free on poe.com Yeah, given the random nature of these LLMs it's possible that I won the lottery in that particular case.
- busseio 4y ago
  "But did you try with GPT4?" is becoming a bit of a refrain for me when talking about this with people. I can get GPT-4 to produce working code from agile user stories surrounded by a well-crafted prompt.
- fhd2 4y ago
  In my experience, GPT-4 does get lots of textbook/common problems right in common languages on first try. But when I tried to solve a just mildly complex problem (finding a ring of cells in a Godot GridMap using GDScript), it produced nonsense: Solved the problem wrong, called functions that don't exist and so on. It did correct some of the mistakes after I pointed them out, but I couldn't get something working without manual editing. Then I asked it to optimise the solution and got nonsense again. For simple data wrangling in Python it worked a lot better. I have a suspicion there might be some... showmanship at work where all kinds of test problems and common tasks are made sure to work well. Also it generates code slower than I would type (if I know what to write), which makes me mad for some reason.
  - fnordpiglet 4y ago
    I think it has a lot to do with how common code examples in the language are. I bet if you tried with languages detailed here you will get a lot better results than more obscure languages (like GDscript) https://octoverse.github.com/2022/top-programming-languages https://octoverse.github.com/2022/top-programming-languages

UncleOxidant 4y ago

did you try with GPT4? I was able to run one GPT4 query/day for free on poe.com Yeah, given the random nature of these LLMs it's possible that I won the lottery in that particular case.

busseio 4y ago
"But did you try with GPT4?" is becoming a bit of a refrain for me when talking about this with people. I can get GPT-4 to produce working code from agile user stories surrounded by a well-crafted prompt.
fhd2 4y ago
In my experience, GPT-4 does get lots of textbook/common problems right in common languages on first try. But when I tried to solve a just mildly complex problem (finding a ring of cells in a Godot GridMap using GDScript), it produced nonsense: Solved the problem wrong, called functions that don't exist and so on. It did correct some of the mistakes after I pointed them out, but I couldn't get something working without manual editing. Then I asked it to optimise the solution and got nonsense again. For simple data wrangling in Python it worked a lot better. I have a suspicion there might be some... showmanship at work where all kinds of test problems and common tasks are made sure to work well. Also it generates code slower than I would type (if I know what to write), which makes me mad for some reason.
- fnordpiglet 4y ago
  I think it has a lot to do with how common code examples in the language are. I bet if you tried with languages detailed here you will get a lot better results than more obscure languages (like GDscript) https://octoverse.github.com/2022/top-programming-languages https://octoverse.github.com/2022/top-programming-languages