3 ms·
My experience with asking ChatGPT to write code is that is produces code that LOOKS like it will work and solve the question asked but it actually doesn't. For
by dpedu 4y ago
My experience with asking ChatGPT to write code is that is produces code that LOOKS like it will work and solve the question asked but it actually doesn't. For example, I've asked it to create code examples of how to use different features in some Python libraries. The samples it produces make me think "ok, that's exactly how I would expect X feature in this library to work", but upon more a detailed inspection, I find that it references library methods that don't even exist! You can complain to the bot that it is lying and it will apologize and spit out another sample with calls to nonexistent methods.
This experience makes me question if something else is going on here. It's easy to overlook a mistake in a whiteboard coding exercise.
- snovv_crash 4y agoI think of it as the model having lossy compression. The library it writes the feature against is the one you expect to exist, based on your overall understanding of the problem, not the one that actually exists. It has learned the low resolution version of the problem, not the specific library it is referring to.
- scarface74 4y agoI’ve asked ChatGPT to write plenty of Python scripts that uses the AWS SDK and it came back correct most of the time.
- gedy 4y agoI've asked it to make React components, with unit tests, etc and it works quite well. Also things like image to ASCII converter in a webpage, adding dithering, etc. Found only tiny bugs here and there.
- mindvirus 4y agoNot unlike a junior engineer! :)
- dboreham 4y agoOh wow. It's completely emulated a junior dev.
- lostmsu 4y agoThe thing is if it were a static language this state is just 1 step away from full-blown programs, as it could see the function does not exist, and ask it to generate the body.
- cheeselip420 4y agoLet's see you write a few pages of working code from memory without compiling or getting a feedback loop
- visarga 4y ago> produces code that LOOKS like it will work and solve the question asked but it actually doesn't You just described my first draft of any program.