3 ms·

I think AI can replace the "copy everything from Stack Overflow" type of developer but I doubt it is capable of replacing the competent developer. The currently

I think AI can replace the "copy everything from Stack Overflow" type of developer but I doubt it is capable of replacing the competent developer. The currently popular form of AI is likely to hit a local maximum just like machine translation has where it is reasonably close most of the time on easy or intermediate problems but sometimes breaks in bizarre ways and doesn't work at all for the hard problems. Since speeding up the work of incompetent developers by a factor of 100 is a good way to end up with an unmaintainable disaster that will break in bizarre ways that are very hard to fix, I think we probably are going to see complexity balloon way beyond human comprehension in every codebase foolish enough to use it. Just like those incompetent developers create more work for competent developers, AI will create more work for human developers.

BoorishBears 4y ago
Have you actually tried using an LM for programming? ChatGPT catches me off guard sometimes even when working on "boring" problems. For example, I was consuming structs from a UNIX domain socket in Java and had some tedious translation from C-types to ByteBuffer calls. It's brain dead easy thing to do, the structs were defined as packed so not alignment issues, ByteBuffers handle endianness, there's a little bit of trickiness with unsigned types but nothing terrible... but with 6 types and writing tests for all 6 definitions, I wondered if ChatGPT could do it and save me a couple of minutes. What impressed me wasn't that it managed to generate the near identical code to my example definition,it was the fact it recognized one of the fields as padding and pre-emptively discarded its value. The padding wasn't in any way shape or form covered by typing, yet the fact the comment in the C definition mentioned padding caused ChatGPT to generate code that skipped over the padding instead of trying to needlessly capture its value and commented that was doing so. And when I had it generate unit tests, it managed to maintained context on the padding and added a helpful comment pointing it out. - It's small corner cases like that which really make me see LMs as being the natural productivity multiplier for programmers. There's a ton of nuance in some problems that by the very nature of an LM, you'd never imagine it'd be able to capture. Yet when you press on it, it actually does better on those than it does on "copy everything from Stack Overflow" problems.

BoorishBears 4y ago

Have you actually tried using an LM for programming? ChatGPT catches me off guard sometimes even when working on "boring" problems. For example, I was consuming structs from a UNIX domain socket in Java and had some tedious translation from C-types to ByteBuffer calls. It's brain dead easy thing to do, the structs were defined as packed so not alignment issues, ByteBuffers handle endianness, there's a little bit of trickiness with unsigned types but nothing terrible... but with 6 types and writing tests for all 6 definitions, I wondered if ChatGPT could do it and save me a couple of minutes. What impressed me wasn't that it managed to generate the near identical code to my example definition,it was the fact it recognized one of the fields as padding and pre-emptively discarded its value. The padding wasn't in any way shape or form covered by typing, yet the fact the comment in the C definition mentioned padding caused ChatGPT to generate code that skipped over the padding instead of trying to needlessly capture its value and commented that was doing so. And when I had it generate unit tests, it managed to maintained context on the padding and added a helpful comment pointing it out. - It's small corner cases like that which really make me see LMs as being the natural productivity multiplier for programmers. There's a ton of nuance in some problems that by the very nature of an LM, you'd never imagine it'd be able to capture. Yet when you press on it, it actually does better on those than it does on "copy everything from Stack Overflow" problems.