3 ms·

Somewhere between the hype and the doom I think there’s a much simpler answer. We have always needed to use language to interface with computers. In early days,

Somewhere between the hype and the doom I think there’s a much simpler answer. We have always needed to use language to interface with computers. In early days, we spent more time learning to talk the language of a computer. As computers became more powerful, we made their language more like ours. This is just the next “higher level”. In the near future, I think programming language will be natural language and LLMs will be a translator to lower level code. Why should we have an LLM program Python, when it could probably just write low level instructions only meant to be tested and verified but not read? Translation is what LLMs are good at, and summarization is fundamentally a translation task of verbose text into key information. Code is a translation of our ideas into machine language. The reasoning aspects are not the strength of an LLM. Without detailed instructions to translate, they are not good at writing code.

logicprog 3y ago
There are two problems with this IMO. First of all, natural language is actually not good at precisely and concisely specifying logic, computations, behavior, etc. That's why things like math and propositional logic were invented. You can get around that and attempt to specify things extremely precisely in natural language, but it ends up producing gigantic specifications and legalese and so on, which are often harder to read and certainly harder to write accurately then at least the ideal conception of a language designed for expressing computability and logic and behavior. It would end up being more work trying to hedge in the edge cases and ambiguities of natural language then it would be to just communicate in a language designed without them. So while, yes, perhaps more advanced computer programs will allow us to write in much more high level programming languages that resemble pseudocode more, there will probably never be a situation in which someone can just communicate using average natural language and get a program that accurately reflects what they wanted. Whether we go the legalese route or the high level pseudocode route, there will always have to be some sort of special language for doing this that will require a change in how we conceptualize things and in how we communicate them to the computer that has to be learned. It's the classic "detailed enough specification" problem: whenever people talk about how in the future product managers will be able to just directly tell the computer what they want and the computer will be able to produce it, the problem with that is that you need to be able to produce a specification precise enough to accurately communicate all of the computations and behaviors you want in a fair amount of detail, which is normally what programmers do when they take the specifications given them by managers and turn them into code — there's an interpretive and additive process there and if you move it up into the specification, someone's still going to need to do that. Secondly, llms are not actually simple expanders like a compiler or a decompression algorithm, they are much more complicated and more difficult to predict and less deterministic, and they also have no actual conceptual understanding of anything or reasoning processes governed by self conscious principles, and therefore far less accurate and trustworthy in what they do, so asking them to produce vast amounts of code is a bad idea. You will still have to manually carefully check the logic and behavior of the code yourself, instead of just writing the logic and behavior yourself, and we all know that reading someone else's unfamiliar code is much more difficult than writing code to do something yourself, so it will probably be even more error prone. Having llms produce lower level code to skip out on the compiler just magnifies this checking problem, and also introduces other problems like the fact that you specified things at a certain level of abstraction comment and now it has to go down far more levels of abstraction, which means there are many more chances for its non-deterministic and unpredictable algorithm to mess things up. Now, I get the sense that you specified that we would test the behavior of the output that llms would produce via essentially tested driven development in order to stave off this problem, but unfortunately I don't think that's the right way to go about things. If you go that route, then you have to remember to test all code paths and all possible scenarios and edge cases, and all the behaviors you want, which is difficult to do, whereas if you write or check the code itself yourself, then you can more easily tell, at least in general, a priori whether the logic is correct. You can figure out if something has been implemented correctly by just looking at the kernel from which behavior is generated inside of having to look at the branching wave front of generated behavior, which has a much larger surface area. This isn't always true, of course, but I don't think relying on test driven development in this way is a good idea.

logicprog 3y ago

There are two problems with this IMO. First of all, natural language is actually not good at precisely and concisely specifying logic, computations, behavior, etc. That's why things like math and propositional logic were invented. You can get around that and attempt to specify things extremely precisely in natural language, but it ends up producing gigantic specifications and legalese and so on, which are often harder to read and certainly harder to write accurately then at least the ideal conception of a language designed for expressing computability and logic and behavior. It would end up being more work trying to hedge in the edge cases and ambiguities of natural language then it would be to just communicate in a language designed without them. So while, yes, perhaps more advanced computer programs will allow us to write in much more high level programming languages that resemble pseudocode more, there will probably never be a situation in which someone can just communicate using average natural language and get a program that accurately reflects what they wanted. Whether we go the legalese route or the high level pseudocode route, there will always have to be some sort of special language for doing this that will require a change in how we conceptualize things and in how we communicate them to the computer that has to be learned. It's the classic "detailed enough specification" problem: whenever people talk about how in the future product managers will be able to just directly tell the computer what they want and the computer will be able to produce it, the problem with that is that you need to be able to produce a specification precise enough to accurately communicate all of the computations and behaviors you want in a fair amount of detail, which is normally what programmers do when they take the specifications given them by managers and turn them into code — there's an interpretive and additive process there and if you move it up into the specification, someone's still going to need to do that. Secondly, llms are not actually simple expanders like a compiler or a decompression algorithm, they are much more complicated and more difficult to predict and less deterministic, and they also have no actual conceptual understanding of anything or reasoning processes governed by self conscious principles, and therefore far less accurate and trustworthy in what they do, so asking them to produce vast amounts of code is a bad idea. You will still have to manually carefully check the logic and behavior of the code yourself, instead of just writing the logic and behavior yourself, and we all know that reading someone else's unfamiliar code is much more difficult than writing code to do something yourself, so it will probably be even more error prone. Having llms produce lower level code to skip out on the compiler just magnifies this checking problem, and also introduces other problems like the fact that you specified things at a certain level of abstraction comment and now it has to go down far more levels of abstraction, which means there are many more chances for its non-deterministic and unpredictable algorithm to mess things up. Now, I get the sense that you specified that we would test the behavior of the output that llms would produce via essentially tested driven development in order to stave off this problem, but unfortunately I don't think that's the right way to go about things. If you go that route, then you have to remember to test all code paths and all possible scenarios and edge cases, and all the behaviors you want, which is difficult to do, whereas if you write or check the code itself yourself, then you can more easily tell, at least in general, a priori whether the logic is correct. You can figure out if something has been implemented correctly by just looking at the kernel from which behavior is generated inside of having to look at the branching wave front of generated behavior, which has a much larger surface area. This isn't always true, of course, but I don't think relying on test driven development in this way is a good idea.