18 ms·

Lisp Isn’t Really a Programming Language

21 points by gnosis 16y ago

tom_b 16y ago
I'm trying to wrap my head around "thinking like a language designer" when approaching software dev. It's the viewpoint in this article that I'm intrigued by these days - that building DSLs in Lisp is a source of great expressiveness in coding.
- jerrya 16y ago
  A trillion years ago, I had to get 50 Lisp machines speaking in an SNA network to American Express' IBM mainframes. The programmers of two companies got our network engineers together, and we designed a protocol to minimize network overhead and still be all robust, fast, reliable, recoverable.... American Express mainframes were available for our use six months down the road, at 2am and for 2 hours. Once a month. They didn't have (or offer us) any sort of testing environment. We went live at 2am for two hours and could debug our code after that offline. At least that was the common Amex procedure. So I wanted to build our half of the protocol, plus a simulator of their half the protocol. What I ended up doing was creating a DSL for describing the finite state machine at the heart of our protocol. Allowing the user (me) to easily specify the parameters of each state and associate them with behavior. Timeouts, transitions, actions to take, error conditions, etc. (def-state ...) Upon doing that my simulator basically fell out trivially, and we could test the network, test our responses, inject errors, all sorts of fun things. It's 2am at Amex and we're going live and many people at Amex dislike our crazy group with our non-IBM machines and they'd love to see us fail. And guess what? It doesn't work. American Express' mainframe code implementation of or protocol is just wrong and not what we agreed to. Eh whatever. It took 30 minutes to confirm there was a problem, another 30 minutes to figure out what the erroneous state implementations were doing, and then instead of going back to cry at our hotel, I just changed our protocol to match theirs, and ... it worked. Not as exciting as saving the Apollo moon landing, but it's my story. So I was able to "think like a language designer", and I created a language very specific to our needs in Lisp. Describe a communications protocol and decorate the states with the constraints and behavior you need. And at 2am, it was very nice to be able to "think like a finite state machine" developer, and just focus directly on re-arranging the states and their transitions and be confident the underlying machinery would deal with the rest. fwiw.
DanielBMarkham 16y ago
Any procedure you write becomes a part of the programming language– your procedures are indistinguishable from the built-in ones. You don’t write Lisp code so much as you extend the Lisp language until it becomes a custom language for whatever it is you’re trying to do. Its a subtle difference, but a significant one. Yes, and you can do the same thing when you extend the type system in C++, or C#. The ability to extend the type system to allow the eventual DSL created to integrate is a key consideration, and LISP is a leader in this area, but the important thing to realize is that you should work at the level of DSL, you should become a Language-Oriented Programmer.
- rbanffy 16y ago
  You can, but you can't change the meaning of if or #define. In Lisp (like Forth, where I learned it) it's not like "first class procedures" but "all names are created equal". At least until you take the red pill and grok macros, but that's a story for another day. When you extend types you don't create a DSL - you create a domain-specific object hierarchy that fits within a given language.
  - DanielBMarkham 16y ago
    You're arguing a technical detail. I agree with you, but in my opinion it's not important, so I'm not going to chase down every little language option here out there. It would turn our discussion quickly into a weird kind of trivia contest. The point is that understanding DSLs is a key concept for a programmer. Wrapping that up into your toolbox gives you a huge advantage. From there you can appreciate (or not) these other things.
    - rbanffy 16y ago
      I see what you mean, but creating a DSL on top of a grammar like C's is more like creating a jargon than a language. Like when you catch two astronauts talking, you can't make out any of the nouns, some of the verbs, but the structure is still familiar - it's English (or Chinese, or Russian) with a new set of nouns and verbs. You don't have a domain-specific language - you just have domain-specific words. Languages such as Lisp are uniquely well suited for that because you can redefine almost anything, including language parts that other languages consider reserved words. With it, you can create other language that's free to create not only its own verbs and nouns, but its own grammar rules.
deleted 16y ago
[deleted]
edw 16y ago
Someone please explain to me how this article, which adds nothing to human knowledge, was posted with some intention other than to cull karma points from those of us with a weakness for anything Lisp-related. That article about the perverse incentives that emerge from the gamification of discussion forums is starting to weigh on me.
- kaiwetzel 16y ago
  I agree the article isn't very strong, and it's from 2007. Nevertheless, I think some of the points could result in an insightful discussion with posts from people who can say something about Lisps (or DSLs) and (ideally) how they applied to their startup. I actually upvoted the submission in that hope. For example, the author points out the importance of "first class procedures" to Lisps' usefulness to create DSLs whereas uniform syntax, code-as-data, macros, and the REPL seem to be at least as relevant. I would have loved to read reputes or points in favour by people with actual experience using Lisps writing DSLs for their business. Another point fromthe article: Lisps having trouble with programming at the large. The article doen't get very specific here, but hearing how companies like ITA (presumably) solved those problems (if they are even real) would be awesome. And highly relevant to a software startup toying with the idea of using a Lisp as part of the technology stack. It would be a lot better to read contemporary, amazingly well written articles about Lisps which are (ideally) relevant to startups (where are those posted ?) but accusing the OP of culling karma points sounds a tad harsh to me :(
deleted 16y ago
[deleted]
metageek 16y ago
I groan every time I see someone advocating building data structures out of cons cells. I use Lisp at my day job, and I can tell you that some of the hardest code to maintain is the code that uses lists instead of defstruct. Once you need to add a field to that structure, you have to manually trace down every spot that uses it, and make sure it can cope with your longer list. Last time it turned out to be easier to just replace the list with a defstruct.
- RodgerTheGreat 16y ago
  Building data structures out of pairs is one of the features of Lisp that introductions to the language tend to focus on, (particularly in academia) as it feels very simple and elegant. This kind of disconnect between theory and practice is part of why I often find it hard to take writing about Lisp programming at face value.
- pavelludiq 16y ago
  I agree in general, it is better to inform your readers, your type system and yourself in a few weeks of your intentions. But in simple cases, if your testing out something, it might be better to use lists and cons cells, at least initially(for example plists or alists instead of hash tables). But like a lot of things in common lisp, this requires good judgment.

21 points by gnosis 16y ago

tom_b 16y ago

I'm trying to wrap my head around "thinking like a language designer" when approaching software dev. It's the viewpoint in this article that I'm intrigued by these days - that building DSLs in Lisp is a source of great expressiveness in coding.

jerrya 16y ago
A trillion years ago, I had to get 50 Lisp machines speaking in an SNA network to American Express' IBM mainframes. The programmers of two companies got our network engineers together, and we designed a protocol to minimize network overhead and still be all robust, fast, reliable, recoverable.... American Express mainframes were available for our use six months down the road, at 2am and for 2 hours. Once a month. They didn't have (or offer us) any sort of testing environment. We went live at 2am for two hours and could debug our code after that offline. At least that was the common Amex procedure. So I wanted to build our half of the protocol, plus a simulator of their half the protocol. What I ended up doing was creating a DSL for describing the finite state machine at the heart of our protocol. Allowing the user (me) to easily specify the parameters of each state and associate them with behavior. Timeouts, transitions, actions to take, error conditions, etc. (def-state ...) Upon doing that my simulator basically fell out trivially, and we could test the network, test our responses, inject errors, all sorts of fun things. It's 2am at Amex and we're going live and many people at Amex dislike our crazy group with our non-IBM machines and they'd love to see us fail. And guess what? It doesn't work. American Express' mainframe code implementation of or protocol is just wrong and not what we agreed to. Eh whatever. It took 30 minutes to confirm there was a problem, another 30 minutes to figure out what the erroneous state implementations were doing, and then instead of going back to cry at our hotel, I just changed our protocol to match theirs, and ... it worked. Not as exciting as saving the Apollo moon landing, but it's my story. So I was able to "think like a language designer", and I created a language very specific to our needs in Lisp. Describe a communications protocol and decorate the states with the constraints and behavior you need. And at 2am, it was very nice to be able to "think like a finite state machine" developer, and just focus directly on re-arranging the states and their transitions and be confident the underlying machinery would deal with the rest. fwiw.

DanielBMarkham 16y ago

Any procedure you write becomes a part of the programming language– your procedures are indistinguishable from the built-in ones. You don’t write Lisp code so much as you extend the Lisp language until it becomes a custom language for whatever it is you’re trying to do. Its a subtle difference, but a significant one. Yes, and you can do the same thing when you extend the type system in C++, or C#. The ability to extend the type system to allow the eventual DSL created to integrate is a key consideration, and LISP is a leader in this area, but the important thing to realize is that you should work at the level of DSL, you should become a Language-Oriented Programmer.

rbanffy 16y ago
You can, but you can't change the meaning of if or #define. In Lisp (like Forth, where I learned it) it's not like "first class procedures" but "all names are created equal". At least until you take the red pill and grok macros, but that's a story for another day. When you extend types you don't create a DSL - you create a domain-specific object hierarchy that fits within a given language.
- DanielBMarkham 16y ago
  You're arguing a technical detail. I agree with you, but in my opinion it's not important, so I'm not going to chase down every little language option here out there. It would turn our discussion quickly into a weird kind of trivia contest. The point is that understanding DSLs is a key concept for a programmer. Wrapping that up into your toolbox gives you a huge advantage. From there you can appreciate (or not) these other things.
  - rbanffy 16y ago
    I see what you mean, but creating a DSL on top of a grammar like C's is more like creating a jargon than a language. Like when you catch two astronauts talking, you can't make out any of the nouns, some of the verbs, but the structure is still familiar - it's English (or Chinese, or Russian) with a new set of nouns and verbs. You don't have a domain-specific language - you just have domain-specific words. Languages such as Lisp are uniquely well suited for that because you can redefine almost anything, including language parts that other languages consider reserved words. With it, you can create other language that's free to create not only its own verbs and nouns, but its own grammar rules.

deleted 16y ago

[deleted]

edw 16y ago

Someone please explain to me how this article, which adds nothing to human knowledge, was posted with some intention other than to cull karma points from those of us with a weakness for anything Lisp-related. That article about the perverse incentives that emerge from the gamification of discussion forums is starting to weigh on me.

kaiwetzel 16y ago
I agree the article isn't very strong, and it's from 2007. Nevertheless, I think some of the points could result in an insightful discussion with posts from people who can say something about Lisps (or DSLs) and (ideally) how they applied to their startup. I actually upvoted the submission in that hope. For example, the author points out the importance of "first class procedures" to Lisps' usefulness to create DSLs whereas uniform syntax, code-as-data, macros, and the REPL seem to be at least as relevant. I would have loved to read reputes or points in favour by people with actual experience using Lisps writing DSLs for their business. Another point fromthe article: Lisps having trouble with programming at the large. The article doen't get very specific here, but hearing how companies like ITA (presumably) solved those problems (if they are even real) would be awesome. And highly relevant to a software startup toying with the idea of using a Lisp as part of the technology stack. It would be a lot better to read contemporary, amazingly well written articles about Lisps which are (ideally) relevant to startups (where are those posted ?) but accusing the OP of culling karma points sounds a tad harsh to me :(

deleted 16y ago

[deleted]

metageek 16y ago

I groan every time I see someone advocating building data structures out of cons cells. I use Lisp at my day job, and I can tell you that some of the hardest code to maintain is the code that uses lists instead of defstruct. Once you need to add a field to that structure, you have to manually trace down every spot that uses it, and make sure it can cope with your longer list. Last time it turned out to be easier to just replace the list with a defstruct.

RodgerTheGreat 16y ago
Building data structures out of pairs is one of the features of Lisp that introductions to the language tend to focus on, (particularly in academia) as it feels very simple and elegant. This kind of disconnect between theory and practice is part of why I often find it hard to take writing about Lisp programming at face value.
pavelludiq 16y ago
I agree in general, it is better to inform your readers, your type system and yourself in a few weeks of your intentions. But in simple cases, if your testing out something, it might be better to use lists and cons cells, at least initially(for example plists or alists instead of hash tables). But like a lot of things in common lisp, this requires good judgment.