3 ms·

One quibble I have with this: the items on the Joel Test are all language-independent, even if you're using a minor language with poor presence on GitHub. But t

One quibble I have with this: the items on the Joel Test are all language-independent, even if you're using a minor language with poor presence on GitHub. But that's not the case with AI codegen. GPT-3.5 was downright terrible at F#, and though Bing+GPT-4 does seem a lot better I'm not sure it's actually good enough for professional F#. I also doubt AI codegen is useful for Scheme/Racket developers who create DSLs (and of course it's totally useless for proprietary languages). So I am not sure AI codegen fits with the other items on the test. In general I am concerned that LLMs will discourage innovation in programming language design - why write a better Python if GPT can just automate the tedium away?

greggyb 2y ago
There is a code-gen feature in a product I know of. The product has a proprietary language and the user base clamors for AI generation. Part of the suggestion process in the tool is to feed generated suggestions through the parser and a basic semantic validation before putting the suggestion in front of users. There is still no guarantee of correctness, but the code will parse and run thanks to this. They have to implement some limitations, because it becomes expensive to re-generate repeatedly. I believe the knowledge I gained was NDA, so I'm keeping it generic.
aleph_minus_one 2y ago
> I am concerned that LLMs will discourage innovation in programming language design - why write a better Python if GPT can just automate the tedium away? I don't think so negatively. My bet: innovation in programming language design will emerge that will make it a lot "less necessary/helpful" to use AIs. Just one example: Quite some programmers claim that AI take a lot of "tedium" from the programming away. But what if we could create programming languages that mostly get rid of this "tedium" (e.g. by using higher-level abstractions to abstract away the tedious, repetitive tasks)?

greggyb 2y ago

There is a code-gen feature in a product I know of. The product has a proprietary language and the user base clamors for AI generation. Part of the suggestion process in the tool is to feed generated suggestions through the parser and a basic semantic validation before putting the suggestion in front of users. There is still no guarantee of correctness, but the code will parse and run thanks to this. They have to implement some limitations, because it becomes expensive to re-generate repeatedly. I believe the knowledge I gained was NDA, so I'm keeping it generic.

aleph_minus_one 2y ago

> I am concerned that LLMs will discourage innovation in programming language design - why write a better Python if GPT can just automate the tedium away? I don't think so negatively. My bet: innovation in programming language design will emerge that will make it a lot "less necessary/helpful" to use AIs. Just one example: Quite some programmers claim that AI take a lot of "tedium" from the programming away. But what if we could create programming languages that mostly get rid of this "tedium" (e.g. by using higher-level abstractions to abstract away the tedious, repetitive tasks)?