5 ms·

On the foolishness of “natural language programming” (1978)

126 points by behnamoh 3y ago

kwhitefoot 3y ago
Anyone proposing that we use ChatGPT to write programs or query databases should read and ponder this and pretty much any other Djikstra papers they come across.
- jiggawatts 3y ago
  What is the SQL query for: “What was the name of the guy, you know from that consulting company like Deloitte but the other one, who worked on the design for some authentication system last year with Dick?” I’ve seen LLMs find matches for queries like this, even if “Dick” is a nickname for “Richard”. An awful lot of business questions are in this category and neither relational theory nor full text search is useful to solve such problems. An LLM coupled with a vector database could solve this, and provide entire new categories of products instead of a sub-standard substitute for an existing product.
  - photochemsyn 3y ago
    An LLM might also create a meta-analysis questioning the value of the basic assumptions involved in the desirability of creating 'entirely new categories of products' - which, if you look at the history of financial products invented in the United States over the past 40 years, has some validity. When Elon Musk et al. talk about 'slowing down AI' that's the kind of thing that seems to concern them. What if AI could make a convincing argument for unionization at Tesla and the institution of a VW-style system of corporate management? Very frightening, I guess...
    - dragonwriter 3y ago
      > When Elon Musk et al. talk about 'slowing down AI' that's the kind of thing that seems to concern them. No, Musk’s concern is just that his AI venture is starting out behind, so slowing down AI lets him catch up.
      - cubefox 3y ago
        How come you make such a disingenuous statement? He has warned about AI risk for 10 years. https://www.cnet.com/science/elon-musk-we-are-summoning-the-demon-with-artificial-intelligence/ https://www.cnet.com/science/elon-musk-we-are-summoning-the-... https://www.lesswrong.com/posts/kzHJ5BRhgkj9CSQ3N/musk-on-agi-timeframes https://www.lesswrong.com/posts/kzHJ5BRhgkj9CSQ3N/musk-on-ag... https://www.forbes.com/sites/ericmack/2015/01/15/elon-musk-puts-down-10-million-to-fight-skynet/ https://www.forbes.com/sites/ericmack/2015/01/15/elon-musk-p... https://www.youtube.com/watch?v=9jkRcrM6XKA https://www.youtube.com/watch?v=9jkRcrM6XKA https://twitter.com/elonmusk/status/1647601906087329794 https://twitter.com/elonmusk/status/1647601906087329794
        cyphereal 3y ago
        Sure he warned of dangers, but also invested heavily in AI for autonomous driving by hiring DL researchers ("a million robot taxis by 2020"). He also invested in OpenAI until he had a falling out about who would be in charge of the company. He recently founded X.AI, which is supposedly also building generative AI. Everyone sees danger, but many, including Musk, are forging ahead.
        kaba0 3y ago
        He is a scam artist with zero knowledge on anything. Just ignore him entirely.
        cubefox 3y ago
        Sure, zero knowledge, and Tesla and SpaceX were just accidents. Got it.
        kaba0 3y ago
        Both were existing companies bought up by him, with the "founder" title included. Also, paying salaries for talent is very different from having said talent oneself. He has no expert knowledge on anything, one might have thought that at least he is good at programming (paypal), but look at twitter, he ain't good even at that.
  - moffkalast 3y ago
    > LLM coupled with a vector database Speaking of that, has anyone shown a convincing demo of that working at all yet, or is it just hype? I've seen one short clip of allegedly working in long term memory with it, but the reliability could've been anything from zero to one.
  - drpixie 3y ago
    > I’ve seen LLMs find matches for queries like this, even if “Dick” is a nickname for “Richard”. An awful lot of business questions are in this category and neither relational theory nor full text search is useful to solve such problems. Question - have you seen an LLM that can answer queries like this correctly and consistently, for a range of natural inputs? The ones I've seen often produce correct answers, but likely often produce apparently correct but wrong answers, and often produce garbage. For those systems, we've just changed the human's work from "write a query" to "filter the correct answers from the garbage" :(
    - idopmstuff 3y ago
      Yes, provided the answer for the question is in context, not training. If you feed GPT4 a document and ask questions, it will virtually always answer those questions right. If you ask it for info in its training set, it will hallucinate sometimes. With Anthropic's 100k token context window, you can do a whole lot with this.
  - codebje 3y ago
    Sure - once the hallucination problem is solved. Otherwise it'll just tell you the guy's name is Harry whether that's supported by the data available or not.
  - kaba0 3y ago
    > provide entire new categories of products instead of a sub-standard substitute for an existing product. The question is, do people really want to give up deterministic execution or not. Sure, it might find that “Dick”, but will it reliably find the plain simple “Richard, order by date” query? Or will it sometimes omit/add a few additional records?
  - deleted 3y ago
    [deleted]
- rqtwteye 3y ago
  I had some conversations with ChatGPT about python coding. I am not so sure that an AI that’s specialized on code couldn’t write and modify code with the user having a dialog in regular language. I think that’s where it will be going.
  - galaxyLogic 3y ago
    When I write code it's usually not working the way I want it on the first try. There are errors and it also doesn't do exactly what I and users would want it to do. So I iterate. I compile and test and debug and modify. But how could an LLM do that kind of thing if it can not actually run the code and inspect the results, and compare them to some criteria it has in its "head"? Does any LLM do such a thing? Of yes you say surely next year but let's focus on what we know to be possible now.
    - CharlesW 3y ago
      > But how could an LLM do that kind of thing if it can not actually run the code and inspect the results… https://openai.com/blog/chatgpt-plugins#code-interpreter https://openai.com/blog/chatgpt-plugins#code-interpreter
    - williamcotton 3y ago
      I’ve been working on a tool called autorails. It is a GitHub application that reads issues and create PRs. The ReAct tool in LangChain with a custom DockerGitGitHub tool results in this PR: https://github.com/williamcotton/autorails-alpha/pull/38 https://github.com/williamcotton/autorails-alpha/pull/38 The log of the work done by the LLM is in the PR description.
- deleted 3y ago
  [deleted]
- userbinator 3y ago
  Or just try it --- and quickly realise that LLMs can tell lies confidently, and mislead you in ways that no human would do (except perhaps the best --- or worst --- of politicians.) To paraphrase a memorable comment I saw here recently: AI combines the speed of machines with the imprecision of humans.
killthebuddha 3y ago
IMO the argument is sound in one direction: the precision, structure, and unforgiving nature of programming languages is a feature rather than a bug. That said, I think this does not mean that there is no benefit to natural language interfaces to machines. I think he goes wrong here: > When all is said and told, the "naturalness" with which we use our native tongues boils down to the ease with which we can use them for making statements the nonsense of which is not obvious. I would say that he leaves out a critical detail: the nonsense of which is not obvious _and also not important_.
- jameshart 3y ago
  Exactly. When you instruct a human in natural language you can rely on them to do disambiguating and nonsense filtering, which means you don’t need to take the time to phrase things precisely. If you’re really lucky, when you give them an ambiguous instruction, they will even ask for clarification. And these are precisely the sorts of things LLMs can bring to the table. With the added benefit that they are infinitely patient, and don’t suffer from embarrassment if they get it wrong. As long as anything they produce is reversible and malleable they can be induced to try again or refine their output indefinitely, which helps solve ambiguities and confusions - I don’t have to prematurely jump to precise language if the LLM can figure out what I want from a vague description.
  - JohnBooty 3y ago
    And these are precisely the sorts of things LLMs can bring to the table. LLMs have shown some promise here, obviously, but I think at the end of the day you are still going to have to enumerate all of the edge cases and error states w.r.t. a particular segment of code and it's not clear to me that LLMs are going to make this faster in many real world cases. LLMs shine with boilerplate stuff that zillions have done before ("create a linked list implementation for me in language XYZ", "create a dropdown list with these five options", etc) but it is less clear to me that they are (or can be) useful for more custom domain- or application- logic. There are certainly many classes of things for which specialized notation/representations are far clearer than natural language. Imagine describing how to play Beethoven's 5th with [insert human language here] rather than standard musical notation. It's probably a continuum of suitability, and not a binary yes/no thing, but I think even after LLMs really hit their full potential most software development will probably still be best achieved with the precision of specialized programming languages and not natural language.
lifeisstillgood 3y ago
I agree - we should not try to make computers that understand us better - we should learn to understand computers better. I argue that software is a form of literacy (Hell I'm rewriting a book ok it for the third time). But we don't say "instead of novels let's just stick to rearranging cut out panels from Marvel back issues." We spend years training children to handle words and speech and presenting arguments - and we need to spend more on empirical testing and skepticism. Literacy is a technology - and we bend our minds to encompass it - and we should expect no less of software
- dclowd9901 3y ago
  Is, though, the literacy of software strictly necessary, or an artifact of lower technology? Isn’t it the case that you can use plain English to explain _exactly_ what a piece of software does, and as such, you can, _exactly_ supplant software with plain English? I think everyone in this thread is patting their backs right now about the necessity of precise syntax and symbology to express programs, but that’s still missing the point. Literacy and communication is built around the ways our minds already work (indeed, it’s often an artifact of a culture distilled to its essence). We _don’t_ need computer language symbols and grammar and syntax — it’s just another form of expression, and definitely not one that fits the human mind nicely.
  - kaba0 3y ago
    > Isn’t it the case that you can use plain English to explain _exactly_ what a piece of software does, and as such, you can, _exactly_ supplant software with plain English? Is it any easier to do that than to “program”? In my mind that’s the exact same thing, just a different, less syntactically strict programming language. Writing good specification is crazy hard, and it usually incomplete, so you end up going back and forth with the human compiler (the programmer) to get a common understanding. One might even argue that a cleaned up, domain-specific language actually helps mutual understanding. It’s not always the case that written text is easier to comprehend — see for example math notation.
    - xp84 3y ago
deleted 3y ago
[deleted]
dang 3y ago
Related threads below. Does anybody know how to decide the year? On the foolishness of “natural language programming” (1979) - https://news.ycombinator.com/item?id=24529900 https://news.ycombinator.com/item?id=24529900 - Sept 2020 (121 comments) On the foolishness of “natural language programming” - https://news.ycombinator.com/item?id=8222017 https://news.ycombinator.com/item?id=8222017 - Aug 2014 (152 comments) E.W.Dijkstra: On the foolishness of "natural language programming" - https://news.ycombinator.com/item?id=1913308 https://news.ycombinator.com/item?id=1913308 - Nov 2010 (69 comments) On the foolishness of "natural language programming" - https://news.ycombinator.com/item?id=233780 https://news.ycombinator.com/item?id=233780 - July 2008 (1 comment)
- philipkglass 3y ago
  This is from the Edsger W. Dijkstra Archive. This entry is 667, included in the numerical index here that covers the years 1976 to 1979: https://www.cs.utexas.edu/~EWD/index06xx.html https://www.cs.utexas.edu/~EWD/index06xx.html The entries ascend in date as they ascend in number. I believe that the date for this is 1978 since it sits between entry 665 "Trip report E.W.Dijkstra, U.K. – Bahamas – U.S.A., 11–30 April 1978" and entry 674 "Trip report E.W.Dijkstra, Zürich, 2–7 July 1978."
  - dang 3y ago
    Thanks! I've put 1978 in the title above.
auxfil 3y ago
[dead]
idopmstuff 3y ago
This is definitely the perspective of an academic - focused on the ideal at the expense of what's actually useful. Programming in natural language will definitely be less precise than programming in an actual programming language, but with the very beneficial tradeoff that a broader swathe of people can use it. I'm not a programmer, but I've been using GPT to write scripts to automate pulling data from my Shopify store for an ecommerce business for reporting purposes. I'm sure that professor would be disgusted by the code, but the thing is... it works. In my case, using natural language via GPT to create the script is obviously superior to any programming language. To write it myself in Python, I'd have to learn Python. Alternatively, I could hire a developer to do it, but then I'd have to write a product spec for them to implement, which takes just as much time as writing that same level of detailed spec and giving it to GPT (less, because I can half-ass the spec initially and iterate as I go with no communication overhead). Then, of course, I'd have to pay the developer. And while the results might be better in some very theoretical sense of code quality, in practice they would be the same, because the script would do the thing I need it to do without breaking anything. There's a reason there's more than one programming language - each has upsides and downsides. As natural language becomes a programming language, it will have the upside of broad accessibility with the downside of relatively low precision. The good news for the author is that you don't have to use it.
- YeGoblynQueenne 3y ago
  >> To write it myself in Python, I'd have to learn Python. Yes, you need to learn a programming language in order to be able to program in that language, it takes a long time, and a lot of hard work to do it, and there are no shortcuts: https://www.norvig.com/21-days.html https://www.norvig.com/21-days.html And yes, with ChatGPT et al you can get the job done without paying the cost of learning the language - albeit you can only do that in a very limited way because the performance of systems like ChatGPT in programming, when evaluated systematically (rather than listening to peoples' subjective opinions about it) is always very poor (and not in a "very theoretical sense of code quality", but in the sense of code that just doesn't do the job). But my question is: ChatGPT is one product, by one company, that isn't even particularly tailored to the programming task. Are you really comfortable handing off your ability to write programs and create applications to one product, owned by one company? Doesn't that just completely take away your independence and even your choice of what you can program?
intalentive 3y ago
Even today it's probably impossible to train an LLM that always correctly translates natural language to machine code.
stevebmark 3y ago
> some people found error messages they couldn't ignore more annoying than wrong results, and, when judging the relative merits of programming languages, some still seem to equate "the ease of programming" with the ease of making undetected mistakes. And thus, Ruby flourished
agalunar 3y ago
Perhaps I'm hopelessly out of touch with pragmatism, but I think Dijkstra's comments would still be applicable even if we could trust GPT to write code correctly. Getting some task done is only half the reason I write programs, even professionally; just as important (if not more important) to me is that I understand something precisely and completely. Writing a program is how I do that. That program is usually in the form of code. Programs aren't just a means to an end, they're an end unto themselves. Or more correctly, a precise and complete understanding of a problem and its solution is a desirable even if it's unnecessary for the task immediately at hand. Although I do mean that understanding is valuable in the way that art or music are valuable, or that education is valuable [1], I also mean it's more valuable than any particular program that's a product of it (and I think there are practical reasons why [2]). [1] https://college.uchicago.edu/student-life/aims-education-address-2002-andrew-abbott https://college.uchicago.edu/student-life/aims-education-add... or https://home.uchicago.edu/~aabbott/Papers/aims2.pdf https://home.uchicago.edu/~aabbott/Papers/aims2.pdf [2] https://pablo.rauzy.name/dev/naur1985programming.pdf https://pablo.rauzy.name/dev/naur1985programming.pdf
- namaria 3y ago
  I see the same issue with the interminable line of todo list apps, mental map apps. They confound the utility of the process - surveying my immediate responsibilities and ranking them, connecting concepts being learned in a new domain - with the product. In my view it has to do with a 'productivity fetish'. We feel like we need to be always outputting something, when the real goal often is to internalize concepts.
- tinus_hn 3y ago
  In the meantime we’re using programming languages that cause us all kinds of problems because there’s a ton of constructs that cause undefined behavior.
eternalban 3y ago
> the fetters of medieval scholasticism — a vain attempt at verbal precision & > computer science would consist mainly of the indeed black art how to bootstrap from [using only language as an interface to machines] Dijkstra was making a prophecy even though he looked (at times hypothetically) to the past. The first quote cuts through to the essential paradigm shift that led to enlightenment. The second is an apt description of what "programming" in the next decade may look like. I'll add my own little ironic prophecy: if machines do become intelligent, they will most likely use formal symbolic systems to interact with each other.
bionhoward 3y ago
Bion: What’s your modern update on this article from 1978? Today is 2023-05-17 and this is on the first page of hacker news. Can you find actionable advice here for prompt engineers? $ARTICLE GPT-4: This article from 1978 discusses the idea of natural language programming, which is the concept of programming computers using languages that are more like human languages, as opposed to the structured and formal programming languages that we use today. The author argues that such an approach is not feasible due to the imprecision and ambiguity inherent in natural languages, and the historical precedent of mathematics flourishing only after adopting formal symbolisms. Looking at this concept from a modern perspective (2023), natural language programming is still not the norm, but there have been significant advancements in natural language processing (NLP) technologies. The development of sophisticated AI models such as GPT-3 and GPT-4 by OpenAI have demonstrated the ability of machines to understand and generate human-like text based on prompts. However, these models do not imply the feasibility of natural language programming. Instead, they are used in applications such as chatbots, translation services, and tools for generating human-like text. In terms of applying the article's insights to today's software engineers, there are a few key points to consider: 1. *Understanding of Formal Languages:* Despite advancements in NLP, the importance of formal languages for programming remains. Engineers should have a deep understanding of the programming languages they use. The ability to think logically and algorithmically is crucial, and this is best facilitated by formal languages. 2. *Designing Interfaces:* The argument for "narrow interfaces" is more relevant than ever. As software systems become more complex, the need for well-designed and specific interfaces increases. Engineers should strive to create interfaces that are clear, concise, and precise in their function and communication. 3. *Natural Language Processing Skills:* While natural language programming might not be feasible, the importance of NLP is growing. Understanding how to use and implement NLP technologies could be beneficial for creating more intuitive user interfaces and interactions. 4. *Communication Skills:* The article criticizes the ambiguity of natural language, but in the human realm, effective communication is crucial. Engineers should cultivate their ability to communicate clearly and precisely, both in their code and their interactions with others. In conclusion, while natural language programming might not be viable, the lessons from this 1978 article remain relevant. The need for precision, formalism, and clear interfaces in programming continues to be important, and the growing role of natural language processing technologies presents new opportunities and challenges for today's engineers.
stevebmark 3y ago
The grand irony of this is that Dijkstra is pretty bad writer. Verbose, obtuse, and condescending.

126 points by behnamoh 3y ago

Anyone proposing that we use ChatGPT to write programs or query databases should read and ponder this and pretty much any other Djikstra papers they come across.

jiggawatts 3y ago
What is the SQL query for: “What was the name of the guy, you know from that consulting company like Deloitte but the other one, who worked on the design for some authentication system last year with Dick?” I’ve seen LLMs find matches for queries like this, even if “Dick” is a nickname for “Richard”. An awful lot of business questions are in this category and neither relational theory nor full text search is useful to solve such problems. An LLM coupled with a vector database could solve this, and provide entire new categories of products instead of a sub-standard substitute for an existing product.
- photochemsyn 3y ago
  An LLM might also create a meta-analysis questioning the value of the basic assumptions involved in the desirability of creating 'entirely new categories of products' - which, if you look at the history of financial products invented in the United States over the past 40 years, has some validity. When Elon Musk et al. talk about 'slowing down AI' that's the kind of thing that seems to concern them. What if AI could make a convincing argument for unionization at Tesla and the institution of a VW-style system of corporate management? Very frightening, I guess...
  - dragonwriter 3y ago
    > When Elon Musk et al. talk about 'slowing down AI' that's the kind of thing that seems to concern them. No, Musk’s concern is just that his AI venture is starting out behind, so slowing down AI lets him catch up.
    - cubefox 3y ago
      How come you make such a disingenuous statement? He has warned about AI risk for 10 years. https://www.cnet.com/science/elon-musk-we-are-summoning-the-demon-with-artificial-intelligence/ https://www.cnet.com/science/elon-musk-we-are-summoning-the-... https://www.lesswrong.com/posts/kzHJ5BRhgkj9CSQ3N/musk-on-agi-timeframes https://www.lesswrong.com/posts/kzHJ5BRhgkj9CSQ3N/musk-on-ag... https://www.forbes.com/sites/ericmack/2015/01/15/elon-musk-puts-down-10-million-to-fight-skynet/ https://www.forbes.com/sites/ericmack/2015/01/15/elon-musk-p... https://www.youtube.com/watch?v=9jkRcrM6XKA https://www.youtube.com/watch?v=9jkRcrM6XKA https://twitter.com/elonmusk/status/1647601906087329794 https://twitter.com/elonmusk/status/1647601906087329794
      - cyphereal 3y ago
        Sure he warned of dangers, but also invested heavily in AI for autonomous driving by hiring DL researchers ("a million robot taxis by 2020"). He also invested in OpenAI until he had a falling out about who would be in charge of the company. He recently founded X.AI, which is supposedly also building generative AI. Everyone sees danger, but many, including Musk, are forging ahead.
      - kaba0 3y ago
        He is a scam artist with zero knowledge on anything. Just ignore him entirely.
        cubefox 3y ago
        Sure, zero knowledge, and Tesla and SpaceX were just accidents. Got it.
        kaba0 3y ago
        Both were existing companies bought up by him, with the "founder" title included. Also, paying salaries for talent is very different from having said talent oneself. He has no expert knowledge on anything, one might have thought that at least he is good at programming (paypal), but look at twitter, he ain't good even at that.
- moffkalast 3y ago
  > LLM coupled with a vector database Speaking of that, has anyone shown a convincing demo of that working at all yet, or is it just hype? I've seen one short clip of allegedly working in long term memory with it, but the reliability could've been anything from zero to one.
- drpixie 3y ago
  > I’ve seen LLMs find matches for queries like this, even if “Dick” is a nickname for “Richard”. An awful lot of business questions are in this category and neither relational theory nor full text search is useful to solve such problems. Question - have you seen an LLM that can answer queries like this correctly and consistently, for a range of natural inputs? The ones I've seen often produce correct answers, but likely often produce apparently correct but wrong answers, and often produce garbage. For those systems, we've just changed the human's work from "write a query" to "filter the correct answers from the garbage" :(
  - idopmstuff 3y ago
    Yes, provided the answer for the question is in context, not training. If you feed GPT4 a document and ask questions, it will virtually always answer those questions right. If you ask it for info in its training set, it will hallucinate sometimes. With Anthropic's 100k token context window, you can do a whole lot with this.
- codebje 3y ago
  Sure - once the hallucination problem is solved. Otherwise it'll just tell you the guy's name is Harry whether that's supported by the data available or not.
- kaba0 3y ago
  > provide entire new categories of products instead of a sub-standard substitute for an existing product. The question is, do people really want to give up deterministic execution or not. Sure, it might find that “Dick”, but will it reliably find the plain simple “Richard, order by date” query? Or will it sometimes omit/add a few additional records?
- deleted 3y ago
  [deleted]
rqtwteye 3y ago
I had some conversations with ChatGPT about python coding. I am not so sure that an AI that’s specialized on code couldn’t write and modify code with the user having a dialog in regular language. I think that’s where it will be going.
- galaxyLogic 3y ago
  When I write code it's usually not working the way I want it on the first try. There are errors and it also doesn't do exactly what I and users would want it to do. So I iterate. I compile and test and debug and modify. But how could an LLM do that kind of thing if it can not actually run the code and inspect the results, and compare them to some criteria it has in its "head"? Does any LLM do such a thing? Of yes you say surely next year but let's focus on what we know to be possible now.
  - CharlesW 3y ago
    > But how could an LLM do that kind of thing if it can not actually run the code and inspect the results… https://openai.com/blog/chatgpt-plugins#code-interpreter https://openai.com/blog/chatgpt-plugins#code-interpreter
  - williamcotton 3y ago
    I’ve been working on a tool called autorails. It is a GitHub application that reads issues and create PRs. The ReAct tool in LangChain with a custom DockerGitGitHub tool results in this PR: https://github.com/williamcotton/autorails-alpha/pull/38 https://github.com/williamcotton/autorails-alpha/pull/38 The log of the work done by the LLM is in the PR description.
deleted 3y ago
[deleted]
userbinator 3y ago
Or just try it --- and quickly realise that LLMs can tell lies confidently, and mislead you in ways that no human would do (except perhaps the best --- or worst --- of politicians.) To paraphrase a memorable comment I saw here recently: AI combines the speed of machines with the imprecision of humans.

killthebuddha 3y ago

IMO the argument is sound in one direction: the precision, structure, and unforgiving nature of programming languages is a feature rather than a bug. That said, I think this does not mean that there is no benefit to natural language interfaces to machines. I think he goes wrong here: > When all is said and told, the "naturalness" with which we use our native tongues boils down to the ease with which we can use them for making statements the nonsense of which is not obvious. I would say that he leaves out a critical detail: the nonsense of which is not obvious _and also not important_.

jameshart 3y ago
Exactly. When you instruct a human in natural language you can rely on them to do disambiguating and nonsense filtering, which means you don’t need to take the time to phrase things precisely. If you’re really lucky, when you give them an ambiguous instruction, they will even ask for clarification. And these are precisely the sorts of things LLMs can bring to the table. With the added benefit that they are infinitely patient, and don’t suffer from embarrassment if they get it wrong. As long as anything they produce is reversible and malleable they can be induced to try again or refine their output indefinitely, which helps solve ambiguities and confusions - I don’t have to prematurely jump to precise language if the LLM can figure out what I want from a vague description.
- JohnBooty 3y ago
  And these are precisely the sorts of things LLMs can bring to the table. LLMs have shown some promise here, obviously, but I think at the end of the day you are still going to have to enumerate all of the edge cases and error states w.r.t. a particular segment of code and it's not clear to me that LLMs are going to make this faster in many real world cases. LLMs shine with boilerplate stuff that zillions have done before ("create a linked list implementation for me in language XYZ", "create a dropdown list with these five options", etc) but it is less clear to me that they are (or can be) useful for more custom domain- or application- logic. There are certainly many classes of things for which specialized notation/representations are far clearer than natural language. Imagine describing how to play Beethoven's 5th with [insert human language here] rather than standard musical notation. It's probably a continuum of suitability, and not a binary yes/no thing, but I think even after LLMs really hit their full potential most software development will probably still be best achieved with the precision of specialized programming languages and not natural language.

lifeisstillgood 3y ago

I agree - we should not try to make computers that understand us better - we should learn to understand computers better. I argue that software is a form of literacy (Hell I'm rewriting a book ok it for the third time). But we don't say "instead of novels let's just stick to rearranging cut out panels from Marvel back issues." We spend years training children to handle words and speech and presenting arguments - and we need to spend more on empirical testing and skepticism. Literacy is a technology - and we bend our minds to encompass it - and we should expect no less of software

dclowd9901 3y ago
Is, though, the literacy of software strictly necessary, or an artifact of lower technology? Isn’t it the case that you can use plain English to explain _exactly_ what a piece of software does, and as such, you can, _exactly_ supplant software with plain English? I think everyone in this thread is patting their backs right now about the necessity of precise syntax and symbology to express programs, but that’s still missing the point. Literacy and communication is built around the ways our minds already work (indeed, it’s often an artifact of a culture distilled to its essence). We _don’t_ need computer language symbols and grammar and syntax — it’s just another form of expression, and definitely not one that fits the human mind nicely.
- kaba0 3y ago
  > Isn’t it the case that you can use plain English to explain _exactly_ what a piece of software does, and as such, you can, _exactly_ supplant software with plain English? Is it any easier to do that than to “program”? In my mind that’s the exact same thing, just a different, less syntactically strict programming language. Writing good specification is crazy hard, and it usually incomplete, so you end up going back and forth with the human compiler (the programmer) to get a common understanding. One might even argue that a cleaned up, domain-specific language actually helps mutual understanding. It’s not always the case that written text is easier to comprehend — see for example math notation.
  - xp84 3y ago

deleted 3y ago

[deleted]

dang 3y ago

Related threads below. Does anybody know how to decide the year? On the foolishness of “natural language programming” (1979) - https://news.ycombinator.com/item?id=24529900 https://news.ycombinator.com/item?id=24529900 - Sept 2020 (121 comments) On the foolishness of “natural language programming” - https://news.ycombinator.com/item?id=8222017 https://news.ycombinator.com/item?id=8222017 - Aug 2014 (152 comments) E.W.Dijkstra: On the foolishness of "natural language programming" - https://news.ycombinator.com/item?id=1913308 https://news.ycombinator.com/item?id=1913308 - Nov 2010 (69 comments) On the foolishness of "natural language programming" - https://news.ycombinator.com/item?id=233780 https://news.ycombinator.com/item?id=233780 - July 2008 (1 comment)

philipkglass 3y ago
This is from the Edsger W. Dijkstra Archive. This entry is 667, included in the numerical index here that covers the years 1976 to 1979: https://www.cs.utexas.edu/~EWD/index06xx.html https://www.cs.utexas.edu/~EWD/index06xx.html The entries ascend in date as they ascend in number. I believe that the date for this is 1978 since it sits between entry 665 "Trip report E.W.Dijkstra, U.K. – Bahamas – U.S.A., 11–30 April 1978" and entry 674 "Trip report E.W.Dijkstra, Zürich, 2–7 July 1978."
- dang 3y ago
  Thanks! I've put 1978 in the title above.

auxfil 3y ago

[dead]

idopmstuff 3y ago

This is definitely the perspective of an academic - focused on the ideal at the expense of what's actually useful. Programming in natural language will definitely be less precise than programming in an actual programming language, but with the very beneficial tradeoff that a broader swathe of people can use it. I'm not a programmer, but I've been using GPT to write scripts to automate pulling data from my Shopify store for an ecommerce business for reporting purposes. I'm sure that professor would be disgusted by the code, but the thing is... it works. In my case, using natural language via GPT to create the script is obviously superior to any programming language. To write it myself in Python, I'd have to learn Python. Alternatively, I could hire a developer to do it, but then I'd have to write a product spec for them to implement, which takes just as much time as writing that same level of detailed spec and giving it to GPT (less, because I can half-ass the spec initially and iterate as I go with no communication overhead). Then, of course, I'd have to pay the developer. And while the results might be better in some very theoretical sense of code quality, in practice they would be the same, because the script would do the thing I need it to do without breaking anything. There's a reason there's more than one programming language - each has upsides and downsides. As natural language becomes a programming language, it will have the upside of broad accessibility with the downside of relatively low precision. The good news for the author is that you don't have to use it.

YeGoblynQueenne 3y ago
>> To write it myself in Python, I'd have to learn Python. Yes, you need to learn a programming language in order to be able to program in that language, it takes a long time, and a lot of hard work to do it, and there are no shortcuts: https://www.norvig.com/21-days.html https://www.norvig.com/21-days.html And yes, with ChatGPT et al you can get the job done without paying the cost of learning the language - albeit you can only do that in a very limited way because the performance of systems like ChatGPT in programming, when evaluated systematically (rather than listening to peoples' subjective opinions about it) is always very poor (and not in a "very theoretical sense of code quality", but in the sense of code that just doesn't do the job). But my question is: ChatGPT is one product, by one company, that isn't even particularly tailored to the programming task. Are you really comfortable handing off your ability to write programs and create applications to one product, owned by one company? Doesn't that just completely take away your independence and even your choice of what you can program?

intalentive 3y ago

Even today it's probably impossible to train an LLM that always correctly translates natural language to machine code.

stevebmark 3y ago

> some people found error messages they couldn't ignore more annoying than wrong results, and, when judging the relative merits of programming languages, some still seem to equate "the ease of programming" with the ease of making undetected mistakes. And thus, Ruby flourished

agalunar 3y ago

Perhaps I'm hopelessly out of touch with pragmatism, but I think Dijkstra's comments would still be applicable even if we could trust GPT to write code correctly. Getting some task done is only half the reason I write programs, even professionally; just as important (if not more important) to me is that I understand something precisely and completely. Writing a program is how I do that. That program is usually in the form of code. Programs aren't just a means to an end, they're an end unto themselves. Or more correctly, a precise and complete understanding of a problem and its solution is a desirable even if it's unnecessary for the task immediately at hand. Although I do mean that understanding is valuable in the way that art or music are valuable, or that education is valuable [1], I also mean it's more valuable than any particular program that's a product of it (and I think there are practical reasons why [2]). [1] https://college.uchicago.edu/student-life/aims-education-address-2002-andrew-abbott https://college.uchicago.edu/student-life/aims-education-add... or https://home.uchicago.edu/~aabbott/Papers/aims2.pdf https://home.uchicago.edu/~aabbott/Papers/aims2.pdf [2] https://pablo.rauzy.name/dev/naur1985programming.pdf https://pablo.rauzy.name/dev/naur1985programming.pdf

namaria 3y ago
I see the same issue with the interminable line of todo list apps, mental map apps. They confound the utility of the process - surveying my immediate responsibilities and ranking them, connecting concepts being learned in a new domain - with the product. In my view it has to do with a 'productivity fetish'. We feel like we need to be always outputting something, when the real goal often is to internalize concepts.
tinus_hn 3y ago
In the meantime we’re using programming languages that cause us all kinds of problems because there’s a ton of constructs that cause undefined behavior.

eternalban 3y ago

> the fetters of medieval scholasticism — a vain attempt at verbal precision & > computer science would consist mainly of the indeed black art how to bootstrap from [using only language as an interface to machines] Dijkstra was making a prophecy even though he looked (at times hypothetically) to the past. The first quote cuts through to the essential paradigm shift that led to enlightenment. The second is an apt description of what "programming" in the next decade may look like. I'll add my own little ironic prophecy: if machines do become intelligent, they will most likely use formal symbolic systems to interact with each other.

bionhoward 3y ago

Bion: What’s your modern update on this article from 1978? Today is 2023-05-17 and this is on the first page of hacker news. Can you find actionable advice here for prompt engineers? $ARTICLE GPT-4: This article from 1978 discusses the idea of natural language programming, which is the concept of programming computers using languages that are more like human languages, as opposed to the structured and formal programming languages that we use today. The author argues that such an approach is not feasible due to the imprecision and ambiguity inherent in natural languages, and the historical precedent of mathematics flourishing only after adopting formal symbolisms. Looking at this concept from a modern perspective (2023), natural language programming is still not the norm, but there have been significant advancements in natural language processing (NLP) technologies. The development of sophisticated AI models such as GPT-3 and GPT-4 by OpenAI have demonstrated the ability of machines to understand and generate human-like text based on prompts. However, these models do not imply the feasibility of natural language programming. Instead, they are used in applications such as chatbots, translation services, and tools for generating human-like text. In terms of applying the article's insights to today's software engineers, there are a few key points to consider: 1. *Understanding of Formal Languages:* Despite advancements in NLP, the importance of formal languages for programming remains. Engineers should have a deep understanding of the programming languages they use. The ability to think logically and algorithmically is crucial, and this is best facilitated by formal languages. 2. *Designing Interfaces:* The argument for "narrow interfaces" is more relevant than ever. As software systems become more complex, the need for well-designed and specific interfaces increases. Engineers should strive to create interfaces that are clear, concise, and precise in their function and communication. 3. *Natural Language Processing Skills:* While natural language programming might not be feasible, the importance of NLP is growing. Understanding how to use and implement NLP technologies could be beneficial for creating more intuitive user interfaces and interactions. 4. *Communication Skills:* The article criticizes the ambiguity of natural language, but in the human realm, effective communication is crucial. Engineers should cultivate their ability to communicate clearly and precisely, both in their code and their interactions with others. In conclusion, while natural language programming might not be viable, the lessons from this 1978 article remain relevant. The need for precision, formalism, and clear interfaces in programming continues to be important, and the growing role of natural language processing technologies presents new opportunities and challenges for today's engineers.

stevebmark 3y ago

The grand irony of this is that Dijkstra is pretty bad writer. Verbose, obtuse, and condescending.