9 ms·

Linguistics and Programming Languages

77 points by rumcajz 9y ago

JoeDaDude 9y ago
"We have again the popularity of "Wouldn't it be nice if our machines were smart enough to allow programming in natural language?". Well, natural languages are most suitable for their original purposes, viz. to be ambiguous in, to tell jokes in and to make love in, but most unsuitable for any form of even mildly sophisticated precision. And if you don't believe that, either try to read a modern legal document and you will immediately see how the need for precision has created a most unnatural language, called "legalese", or try to read one of Euclid's original verbal proofs (preferably in Greek). That should cure you, and should make you realize that formalisms have not been introduced to make things difficult, but to make things possible. And if, after that, you still believe that we express ourselves most easily in our native tongues, you will be sentenced to the reading of five student essays." E. W. Djikstra, EWD952 [1] [1] https://www.cs.utexas.edu/users/EWD/ewd09xx/EWD952.PDF https://www.cs.utexas.edu/users/EWD/ewd09xx/EWD952.PDF
- didibus 9y ago
  Love this, and agree. Though I would love to see a computer system which can resolve ambiguity in natural languages and formalize it for us. It would get it wrong often, but if it is good at it, no more so then a programmer gets the requirements wrong.
  - Jtsummers 9y ago
    We may not be able to do this automatically, but you can use a kind of refinement calculus to accomplish this. Start off with vague requirements from your client: We need a machine which can wash clothes at various temperatures and speeds. The user should have options to control the temperature and speed since they'll have different types of materials. Maybe we can give them some preset options? Conversation: What are the temperature options? Hot, warm and cold; but it's in two cycles and each can have a different temperature. How long should it wash the clothes for? That depends on the material and how dirty it is. What if it's really dirty? They should be able to do an extra rinse cycle. How fast should it spin? That depends on the material and how dirty it is. Refined (control panel only): Dial presenting options for: Sanitary Normal Sensitive Bulky Buttons allowing further refinement: Water: Hot/Cold, Warm/Warm, Warm/Cold, Cold/Cold Extra Rinse: (yes or no) "Dirtiness": light, normal, heavy (impacts wash duration) If you use a proper requirements management tool, you can trace every one of those final system requirements back to some (slightly editorialized) conversation or initial (ambiguous, plain language) requirement description. Of course, this will, in reality, be an iterative process as you discover ambiguities in the specification and code and go back to the client for feedback and clarification.
  - another-dave 9y ago
    You could potentially have a compiler that would infer meaning & only fail compilation if the fuzzy natural language terms were ambiguous in context — just like languages like Scala currently infer types when unambiguous. Also, feels like Inform7 should be mentioned in any discussion on NLP and programming: http://inform7.com http://inform7.com First thing to mind based on the article's premise and hits a lot of the article's desired points already.
    - didibus 9y ago
      Interesting. This feels like a great way to prototype functional requirements. Express the full process as an interactive fiction novel, and let people play it. Get feedback, iterate on the novel, until they are satisfied, then implement.
    - slaymaker1907 9y ago
      One thing that would be interesting is if you could write a compiler front end that was interactive so that it could give you better error messages.
- neilparikh 9y ago
  Another related EWD is 667[1] The paragraph that stuck out to me was: "Instead of regarding the obligation to use formal symbols as a burden, we should regard the convenience of using them as a privilege: thanks to them, school children can learn to do what in earlier days only genius could achieve." [1] https://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/EWD667.html https://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/E...
- rumcajz 9y ago
  The investigation here goes in the opposite direction. It's not "wouldn't it be nice if computers understood natural language" but rather "wouldn't it be nice if people understood programming languages". The former is just silly. The latter is a call to create, in programming languages, constructs that can be parsed by the language machinery in our brains. If the goal of programming languages was to instruct the computer, after all, we'd be all writing machine code.
  - sriku 9y ago
    Humble submission to the cause - using "the" and "it" in a scripting language to avoid having to come up with variable names all the time. Way back, I'd designed a scheme-based scripting language called "muSE" [1] in which we created specifications for video editing styles. I also toyed with the language. One such experiment was having a way to refer to values already computed using "the" and "it". [2] [1] https://github.com/srikumarks/muSE https://github.com/srikumarks/muSE [2] https://github.com/srikumarks/muSE/wiki/TheAndIt https://github.com/srikumarks/muSE/wiki/TheAndIt edit: This project isn't maintained, but the latest version 711 still works on macosx [3] [3] https://code.google.com/archive/p/muvee-symbolic-expressions/downloads https://code.google.com/archive/p/muvee-symbolic-expressions...
    - slaymaker1907 9y ago
      There are already constructs like this. In all programming languages, you see variable names like data or result all over the place. Additionally, when using the python interpreter, there is a special variable _ for the last expression evaluated.
      - euyyn 9y ago
        Either Kotlin or eXtend already have "it" for the default variable name of single-parameter lambdas.
      - sriku 9y ago
        "it" is common in that sense, but muse additionally models "the" as a function that turns verbs into nouns - like we do. For example, (open-file "blah" 'for-writing) (write (the open-file) "hello world") (close it) Or (length (list "milk" "pudding")) (print "number of items is " (the length))
- deleted 9y ago
  [deleted]
- AnimalMuppet 9y ago
  I think Perl is interesting here. When we're talking about computer code, we humans say things like "Read in a line of text. If it ends in a newline, remove it." But we can't program computers that way. The compiler says, in effect, "Read in a line of text from where? And put it where? If it ends in a newline? If what ends in a newline?" And so on. But Perl actually lets you program that way. Perl says, "Read in a line of text? You didn't say from where, so I'll assume the default place, which is the files that were named as arguments in the program invocation. You didn't say where to put it, so I'll put it in the default variable. If it ends in a newline? You didn't say if what ends in a newline, so I'll assume that you're talking about the default variable, which happens to contain the line we just read in." And so on. Effectively, you can use "it" in your conversation with Perl, and it will do reasonable things. This is one of the places that it shows that Perl was designed by a linguist.
  - throwaway7645 9y ago
    Yep, default variables and context are important to Perl. I like a lot of the design, but there is a lot of cruft that has piled up in 30 years. Perl6 addresses that while adding support for FP, OO, array programming, grammar based programming...etc. I find it quite pleasant to read and write. The problem is the implementation is immature and I'm not sure it will ever be as fast as we need for a production language.
    - falsedan 9y ago
      The topic variable came from awk; most of the whipitupitude variables are direct ports of awks.
      - throwaway7645 9y ago
        I completely forgot about that, good point!
  - schoen 9y ago
    By the way, an important problem in natural language processing, which is related to this at a high level, is anaphora resolution. When we use pronouns to refer to people and things we've mentioned before, there are often ambiguities about what the referent of a particular pronoun is, but native speakers almost never have to consciously think about this question. But in practice, resolving these references correctly may require sophisticated reasoning about our knowledge of the world in order to determine which interpretation is plausible. An example adapted from the Winograd Schema Challenge is: The file couldn't fit on the hard drive because it was too big. The file couldn't fit on the hard drive because it was too small. In the first sentence, "it" refers to the file; in the second sentence, "it" refers to the hard drive. Native speakers who know what files and hard drives or (or even who don't) should have no trouble understanding the references and might not even have noticed that there was any ambiguity (!), even though resolving the ambiguity requires bringing to bear specific knowledge about the world. There is a whole family of AI language understanding problems based around this, such as https://en.wikipedia.org/wiki/Winograd_Schema_Challenge https://en.wikipedia.org/wiki/Winograd_Schema_Challenge Looking over some examples shows just how challenging this can be, because of the way the sentences can require people to know arbitrary things about the world. ("The atom emitted a photon because it was entering a lower energy state.")
- jdonaldson 9y ago
  Specificity is always on a scale. In most cases, you don't want to tell the computer exactly how to lay out the memory (common in imperative programming). In other cases, you don't want to tell the computer exactly what sequence to perform the operations in (common in functional programming). In other cases, you allow for contradictory information to be provided to the computer (common in constraint based programming). There's plenty of room for ambiguity or contradiction in programming, and good things happen on either end of the spectrum.
DonbunEf7 9y ago
Learn Lojban today: https://mw.lojban.org/papri/la_karda https://mw.lojban.org/papri/la_karda Lojban is based on predicate and relational logic, and parses unambiguously for both humans and computers, meaning that we can get straight to semantics instead of faffing about with syntax.
- throwaway7645 9y ago
  Anytime Lojban comes up I recommend taking a look at linguist Arika Okrent's book "Into the Land of Invented Languages". She learns several conlangs (constructed languages) during the book and has several chapters devoted to the history and her learning of Lojban and some of its pitfalls (some things take a lot of words to say). There's a reason why natural languages have so much ambiguity, so you don't have to write a novel to have a basic conversation. If you go to wikitongues on YouTube you'll see a prominent community member who can only read it and can't speak it...it isn't easy. Edit: Lojban is ambiguous with how something is said, so when a speaker says something you should never misinterpret the words, but the meaning could be misinterpreted as I understand it. I'm no expert, but would love to see something like this take off.
  - rumcajz 9y ago
    When it comes to book recommendations I would recommend Umberto Eco's The Search for the Perfect Language.
    - throwaway7645 9y ago
      Speaking of Larry Wall (creator of Perl) above, "the search for the perfect language" is his favorite book I believe.
  - zephyrfalcon 9y ago
    """I recommend taking a look at linguist Arika Okrent's book "Into the Land of Invented Languages""" I used to have that book... I didn't think it was very good. I wanted to learn more about the various conlangs, but instead the author mostly focused on their creators, and then especially their idiosyncrasies and quirks. For those who want to know more about conlangs, I would recommend "The Language Construction Kit" by Mark Rosenfelder.
Ngunyan 9y ago
English ≠ Computerish but: English = C || English = Pascal || English = SQL .......
falsedan 9y ago
Note that Martin Sústrik is a programmer, not a linguist. Larry Wall (creator of perl) is a linguist and a programmer, and has written on the topic[0] [0]: http://world.std.com/~swmcd/steven/perl/linguistics.html http://world.std.com/~swmcd/steven/perl/linguistics.html
- forgotpwtomain 9y ago
  Curiously I find myself disagreeing quite a bit with Larry. > If a language is designed so that you can "learn as you go", then the expectation is that everyone is learning, and that's okay. It's okay if we never reach understanding or agreement on what Faulkner intended by a particular sentence (we can still grasp most of the whole). For a programming language this is explicitly not okay! This goes for ambiguity as well. > Multiple ways to say the same thing > This one is more of an anthropological feature. People not only learn as they go, but come from different backgrounds, and will learn a different subset of the language first. This increases cognative load with no particular benefit.
  - falsedan 9y ago
    For the former, I think Larry does not imply a post-modernist “death of the author” position: there is still an objectively correct interpretation of the code as the author intended, but it may be understood differently by people of different experience levels. For example, a map with reference to a sub can be though of as a loop that calls the sub. With perl, the objective truth is opcodes, which are well-understood by a small group. Everyone else bases their understanding on heuristics and analogies, and the goal is to write your code to trigger the same heuristics/etc. in the reader. For the latter, you are declaring your opinion as fact. Every language allows redundancy and variation of expression; if it truly provided no benefit, why have we not seen a popular language that only allowed a single expressive style?
    - forgotpwtomain 9y ago
      > For the latter, you are declaring your opinion as fact. Every language allows redundancy and variation of expression; if it truly provided no benefit, why have we not seen a popular language that only allowed a single expressive style? It is indeed my opinion (as prefaced by 'I find myself disagreeing..') but the premise that a programming language benefits from resembling or mimicking features of natural languages is also an opinion. Also I should note that your phrasing of the question isn't quite correct ("if it truly provided no benefit"), something that provides no benefit is unlikely to be excluded from a language (e.g. double negatives "there ain't nothing here to see!") unless there is a clear benefit to doing so, and in fact there are times that there is. Particularly I would draw your attention to for example more limited lexicon sets (such as those used by dispatchers, rescue workers, climbers, EMT professionals etc.). Those explicitly exclude variation of expression, since that leads to an increased risk of being misinterpretation in often critical situations. It is my inclination that while interpretation of code isn't time critical, reducing variance (e.g. a common coding style does this as well) reduces cognitive load (makes comprehension more efficient).

77 points by rumcajz 9y ago

"We have again the popularity of "Wouldn't it be nice if our machines were smart enough to allow programming in natural language?". Well, natural languages are most suitable for their original purposes, viz. to be ambiguous in, to tell jokes in and to make love in, but most unsuitable for any form of even mildly sophisticated precision. And if you don't believe that, either try to read a modern legal document and you will immediately see how the need for precision has created a most unnatural language, called "legalese", or try to read one of Euclid's original verbal proofs (preferably in Greek). That should cure you, and should make you realize that formalisms have not been introduced to make things difficult, but to make things possible. And if, after that, you still believe that we express ourselves most easily in our native tongues, you will be sentenced to the reading of five student essays." E. W. Djikstra, EWD952 [1] [1] https://www.cs.utexas.edu/users/EWD/ewd09xx/EWD952.PDF https://www.cs.utexas.edu/users/EWD/ewd09xx/EWD952.PDF

didibus 9y ago
Love this, and agree. Though I would love to see a computer system which can resolve ambiguity in natural languages and formalize it for us. It would get it wrong often, but if it is good at it, no more so then a programmer gets the requirements wrong.
- Jtsummers 9y ago
  We may not be able to do this automatically, but you can use a kind of refinement calculus to accomplish this. Start off with vague requirements from your client: We need a machine which can wash clothes at various temperatures and speeds. The user should have options to control the temperature and speed since they'll have different types of materials. Maybe we can give them some preset options? Conversation: What are the temperature options? Hot, warm and cold; but it's in two cycles and each can have a different temperature. How long should it wash the clothes for? That depends on the material and how dirty it is. What if it's really dirty? They should be able to do an extra rinse cycle. How fast should it spin? That depends on the material and how dirty it is. Refined (control panel only): Dial presenting options for: Sanitary Normal Sensitive Bulky Buttons allowing further refinement: Water: Hot/Cold, Warm/Warm, Warm/Cold, Cold/Cold Extra Rinse: (yes or no) "Dirtiness": light, normal, heavy (impacts wash duration) If you use a proper requirements management tool, you can trace every one of those final system requirements back to some (slightly editorialized) conversation or initial (ambiguous, plain language) requirement description. Of course, this will, in reality, be an iterative process as you discover ambiguities in the specification and code and go back to the client for feedback and clarification.
- another-dave 9y ago
  You could potentially have a compiler that would infer meaning & only fail compilation if the fuzzy natural language terms were ambiguous in context — just like languages like Scala currently infer types when unambiguous. Also, feels like Inform7 should be mentioned in any discussion on NLP and programming: http://inform7.com http://inform7.com First thing to mind based on the article's premise and hits a lot of the article's desired points already.
  - didibus 9y ago
    Interesting. This feels like a great way to prototype functional requirements. Express the full process as an interactive fiction novel, and let people play it. Get feedback, iterate on the novel, until they are satisfied, then implement.
  - slaymaker1907 9y ago
    One thing that would be interesting is if you could write a compiler front end that was interactive so that it could give you better error messages.
neilparikh 9y ago
Another related EWD is 667[1] The paragraph that stuck out to me was: "Instead of regarding the obligation to use formal symbols as a burden, we should regard the convenience of using them as a privilege: thanks to them, school children can learn to do what in earlier days only genius could achieve." [1] https://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/EWD667.html https://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/E...
rumcajz 9y ago
The investigation here goes in the opposite direction. It's not "wouldn't it be nice if computers understood natural language" but rather "wouldn't it be nice if people understood programming languages". The former is just silly. The latter is a call to create, in programming languages, constructs that can be parsed by the language machinery in our brains. If the goal of programming languages was to instruct the computer, after all, we'd be all writing machine code.
- sriku 9y ago
  Humble submission to the cause - using "the" and "it" in a scripting language to avoid having to come up with variable names all the time. Way back, I'd designed a scheme-based scripting language called "muSE" [1] in which we created specifications for video editing styles. I also toyed with the language. One such experiment was having a way to refer to values already computed using "the" and "it". [2] [1] https://github.com/srikumarks/muSE https://github.com/srikumarks/muSE [2] https://github.com/srikumarks/muSE/wiki/TheAndIt https://github.com/srikumarks/muSE/wiki/TheAndIt edit: This project isn't maintained, but the latest version 711 still works on macosx [3] [3] https://code.google.com/archive/p/muvee-symbolic-expressions/downloads https://code.google.com/archive/p/muvee-symbolic-expressions...
  - slaymaker1907 9y ago
    There are already constructs like this. In all programming languages, you see variable names like data or result all over the place. Additionally, when using the python interpreter, there is a special variable _ for the last expression evaluated.
    - euyyn 9y ago
      Either Kotlin or eXtend already have "it" for the default variable name of single-parameter lambdas.
    - sriku 9y ago
      "it" is common in that sense, but muse additionally models "the" as a function that turns verbs into nouns - like we do. For example, (open-file "blah" 'for-writing) (write (the open-file) "hello world") (close it) Or (length (list "milk" "pudding")) (print "number of items is " (the length))
deleted 9y ago
[deleted]
AnimalMuppet 9y ago
I think Perl is interesting here. When we're talking about computer code, we humans say things like "Read in a line of text. If it ends in a newline, remove it." But we can't program computers that way. The compiler says, in effect, "Read in a line of text from where? And put it where? If it ends in a newline? If what ends in a newline?" And so on. But Perl actually lets you program that way. Perl says, "Read in a line of text? You didn't say from where, so I'll assume the default place, which is the files that were named as arguments in the program invocation. You didn't say where to put it, so I'll put it in the default variable. If it ends in a newline? You didn't say if what ends in a newline, so I'll assume that you're talking about the default variable, which happens to contain the line we just read in." And so on. Effectively, you can use "it" in your conversation with Perl, and it will do reasonable things. This is one of the places that it shows that Perl was designed by a linguist.
- throwaway7645 9y ago
  Yep, default variables and context are important to Perl. I like a lot of the design, but there is a lot of cruft that has piled up in 30 years. Perl6 addresses that while adding support for FP, OO, array programming, grammar based programming...etc. I find it quite pleasant to read and write. The problem is the implementation is immature and I'm not sure it will ever be as fast as we need for a production language.
  - falsedan 9y ago
    The topic variable came from awk; most of the whipitupitude variables are direct ports of awks.
    - throwaway7645 9y ago
      I completely forgot about that, good point!
- schoen 9y ago
  By the way, an important problem in natural language processing, which is related to this at a high level, is anaphora resolution. When we use pronouns to refer to people and things we've mentioned before, there are often ambiguities about what the referent of a particular pronoun is, but native speakers almost never have to consciously think about this question. But in practice, resolving these references correctly may require sophisticated reasoning about our knowledge of the world in order to determine which interpretation is plausible. An example adapted from the Winograd Schema Challenge is: The file couldn't fit on the hard drive because it was too big. The file couldn't fit on the hard drive because it was too small. In the first sentence, "it" refers to the file; in the second sentence, "it" refers to the hard drive. Native speakers who know what files and hard drives or (or even who don't) should have no trouble understanding the references and might not even have noticed that there was any ambiguity (!), even though resolving the ambiguity requires bringing to bear specific knowledge about the world. There is a whole family of AI language understanding problems based around this, such as https://en.wikipedia.org/wiki/Winograd_Schema_Challenge https://en.wikipedia.org/wiki/Winograd_Schema_Challenge Looking over some examples shows just how challenging this can be, because of the way the sentences can require people to know arbitrary things about the world. ("The atom emitted a photon because it was entering a lower energy state.")
jdonaldson 9y ago
Specificity is always on a scale. In most cases, you don't want to tell the computer exactly how to lay out the memory (common in imperative programming). In other cases, you don't want to tell the computer exactly what sequence to perform the operations in (common in functional programming). In other cases, you allow for contradictory information to be provided to the computer (common in constraint based programming). There's plenty of room for ambiguity or contradiction in programming, and good things happen on either end of the spectrum.

DonbunEf7 9y ago

Learn Lojban today: https://mw.lojban.org/papri/la_karda https://mw.lojban.org/papri/la_karda Lojban is based on predicate and relational logic, and parses unambiguously for both humans and computers, meaning that we can get straight to semantics instead of faffing about with syntax.

throwaway7645 9y ago
Anytime Lojban comes up I recommend taking a look at linguist Arika Okrent's book "Into the Land of Invented Languages". She learns several conlangs (constructed languages) during the book and has several chapters devoted to the history and her learning of Lojban and some of its pitfalls (some things take a lot of words to say). There's a reason why natural languages have so much ambiguity, so you don't have to write a novel to have a basic conversation. If you go to wikitongues on YouTube you'll see a prominent community member who can only read it and can't speak it...it isn't easy. Edit: Lojban is ambiguous with how something is said, so when a speaker says something you should never misinterpret the words, but the meaning could be misinterpreted as I understand it. I'm no expert, but would love to see something like this take off.
- rumcajz 9y ago
  When it comes to book recommendations I would recommend Umberto Eco's The Search for the Perfect Language.
  - throwaway7645 9y ago
    Speaking of Larry Wall (creator of Perl) above, "the search for the perfect language" is his favorite book I believe.
- zephyrfalcon 9y ago
  """I recommend taking a look at linguist Arika Okrent's book "Into the Land of Invented Languages""" I used to have that book... I didn't think it was very good. I wanted to learn more about the various conlangs, but instead the author mostly focused on their creators, and then especially their idiosyncrasies and quirks. For those who want to know more about conlangs, I would recommend "The Language Construction Kit" by Mark Rosenfelder.

Ngunyan 9y ago

English ≠ Computerish but: English = C || English = Pascal || English = SQL .......

falsedan 9y ago

Note that Martin Sústrik is a programmer, not a linguist. Larry Wall (creator of perl) is a linguist and a programmer, and has written on the topic[0] [0]: http://world.std.com/~swmcd/steven/perl/linguistics.html http://world.std.com/~swmcd/steven/perl/linguistics.html

forgotpwtomain 9y ago
Curiously I find myself disagreeing quite a bit with Larry. > If a language is designed so that you can "learn as you go", then the expectation is that everyone is learning, and that's okay. It's okay if we never reach understanding or agreement on what Faulkner intended by a particular sentence (we can still grasp most of the whole). For a programming language this is explicitly not okay! This goes for ambiguity as well. > Multiple ways to say the same thing > This one is more of an anthropological feature. People not only learn as they go, but come from different backgrounds, and will learn a different subset of the language first. This increases cognative load with no particular benefit.
- falsedan 9y ago
  For the former, I think Larry does not imply a post-modernist “death of the author” position: there is still an objectively correct interpretation of the code as the author intended, but it may be understood differently by people of different experience levels. For example, a map with reference to a sub can be though of as a loop that calls the sub. With perl, the objective truth is opcodes, which are well-understood by a small group. Everyone else bases their understanding on heuristics and analogies, and the goal is to write your code to trigger the same heuristics/etc. in the reader. For the latter, you are declaring your opinion as fact. Every language allows redundancy and variation of expression; if it truly provided no benefit, why have we not seen a popular language that only allowed a single expressive style?
  - forgotpwtomain 9y ago
    > For the latter, you are declaring your opinion as fact. Every language allows redundancy and variation of expression; if it truly provided no benefit, why have we not seen a popular language that only allowed a single expressive style? It is indeed my opinion (as prefaced by 'I find myself disagreeing..') but the premise that a programming language benefits from resembling or mimicking features of natural languages is also an opinion. Also I should note that your phrasing of the question isn't quite correct ("if it truly provided no benefit"), something that provides no benefit is unlikely to be excluded from a language (e.g. double negatives "there ain't nothing here to see!") unless there is a clear benefit to doing so, and in fact there are times that there is. Particularly I would draw your attention to for example more limited lexicon sets (such as those used by dispatchers, rescue workers, climbers, EMT professionals etc.). Those explicitly exclude variation of expression, since that leads to an increased risk of being misinterpretation in often critical situations. It is my inclination that while interpretation of code isn't time critical, reducing variance (e.g. a common coding style does this as well) reduces cognitive load (makes comprehension more efficient).