4 ms·
How would this anyone be able to evaluate whether the program they requested is correct or not? Automatic program generation from human language really feels l
by kccqzy 3y ago
How would this anyone be able to evaluate whether the program they requested is correct or not?
Automatic program generation from human language really feels like the same problem with machine translation between human languages. I have an elementary understanding of French and so when I see a passage machine translated into French (regardless of software, Google Translate or DeepL) I cannot find any mistakes; I may even learn a few new words. But to the professional translator, the passage is full of mistakes, non-idiomatic expressions and other weirdness. You aren't going to see publishers publishing entirely machine translated books.
I suspect the same thing happens for LLM-written programs. The average person finds them useful; the expert finds them riddled with bugs. When the stakes are low, like tourists not speaking the native language, machine translation is fine. So will many run-once programs destined for a specific purpose. When the stakes are high, human craft is still needed.
- qup 3y agoI was imagining a step past what you're talking about, when the outputs are just always correct, and the bots code better than we do.
- WJW 3y ago"Always" correct is a very high bar and likely unattainable. It seems much more likely that the amount of errors will trend downwards but never quite reach zero. How could it be otherwise? AIs are not magic god-machines, they have a limited set of information to work with just like the rest of us (though it might be larger than humans could handle) and sometimes the piece of information is just not known yet. Let's say that in a few years the amount of correct code becomes 99% instead of ~80%. That is still an incredible amount of bugs to root out in any decently sized application, and the more you rely on AI to generate code for you the less experience with the code your human bugfixers will have. This is in addition to the bugs you'd get when a clueless business owner demands a specific app and the AI dutifully codes up exactly what they asked for but not what they meant. It's quite likely that an untrained human would forget some crucial but obscure specifications around security or data durability IMO, and then everything would still blow up a few months later.
- physicsguy 3y agoWe’re already using ChatGPT at work to do machine translation because it takes weeks to get back translations for the 10 languages our application supports. It’s not a work of literature, it’s quite technical language and feedback we’ve had from customers is that it’s quite good. Before this, we wouldn’t have ever supported a language like Czech because the market isn’t big enough to justify the cost of translation, and Google Translate couldn’t handle large passages of text in the docs well enough.
- Xelynega 3y agoI chatgpt translated this: "Our business model can't afford to pay enough translators so we have been replacing them with chatGPT, and enough of our users haven't complained that we consider it a success"
- physicsguy 3y agoMost users in this market segment get the software in English, German or Chinese and nothing else because the cost doesn't justify doing it elsewhere.
- Tijdreiziger 3y agoI've encountered enough janky translations to prefer getting software in English.