>> Do you remember how people on HN used to take comfort that AI couldn't write code? Now they're taking comfort in the fact that AI can't write code well. If

>> Do you remember how people on HN used to take comfort that AI couldn't write code? Now they're taking comfort in the fact that AI can't write code well. If you're referring to neural program synthesis with large language models (LLMs), the performance of current systems is pitifully low, with best results in the range of 5-7% correct programs when evaluating on a test set similar to the one used for training (and allowing a single "guess"; it goes up to 30% with 100 guesses; happy to point to references if required). So no, "AI" (meaning deep learning I presume) cannot write code very "well". Of course there exists an entire field of research in program synthesis that predates LLM code generators by a long time, and that can do much better than LLMs, but you haven't heard of it because DeepMind and OpenAI don't choose to champion it. But that's a story for another time. [Correction: dammit, I misquoted the metrics. OpenAI claims ~30% correct results for their Codex model when allowed one guess on their test set; ~70% when allowed 100 guesses. See link to Codex paper in child comment. The 5-7% is the rate of correct programs for SalesForce's CodeRL on the "Introductory" level problems of the third-party APPS dataset and when allowed one guess; see Table 1b in the CodeRL paper summarising results on APPS for various LLM code generators: https://arxiv.org/abs/2207.01780 https://arxiv.org/abs/2207.01780]