7 ms·

I hear sentiments like this often with lisp, but if this were the case, why isn't everyone using lisp?

mrcode007 5y ago
Largely because of myths surrounding it, for example parenthesis syntax and lack of editor support. With paredit, you get meta level direct AST manipulation support in emacs that is still light years ahead of any programming IDE out there. I think the other fear factor is you have to think a lot more about how to approach the problem when writing Common Lisp and it requires a more complete engineer due to a smaller library ecosystem. Usually there is one and sometimes two libraries for doing X and if not you’re up for making a library yourself. I’m attaching an example of how quickly you can write TLS 1.3 by a single person in Common Lisp. At the time I wrote this library most websites still struggled with TLS 1.3 support. The point is that most software creators don’t have the breath and depth of knowledge required to be proficient in LISP if they only posses siloed domain knowledge. Links: https://github.com/mateuszb/tls1.3 https://github.com/mateuszb/tls1.3 If you want an example of expressivity you can take a look at EC crypto with a full NIST test vector set in about 300 lines of code https://github.com/mateuszb/tls1.3/blob/master/elliptic-curves.lisp https://github.com/mateuszb/tls1.3/blob/master/elliptic-curv...
- bmitc 5y ago
  > With paredit, you get meta level direct AST manipulation support in emacs that is still light years ahead of any programming IDE out there. Do you happen to know of a video or article with visual examples that demonstrates this? I always hear about these things with Common Lisp, but I have personally never seen it. I’d love to understand how this works or enables people.
  - rags2riches 5y ago
    There's the Animated Guide to Paredit: http://danmidwood.com/content/2014/11/21/animated-paredit.html http://danmidwood.com/content/2014/11/21/animated-paredit.ht... It really feels more like you're working on a tree of code than just editing text.
    - bmitc 5y ago
      That’s helpful. I’ll read it more in depth when I’m off mobile as that site is hard to read on mobile. Although, I think there’s still something missing between those demonstrations and the quoted claims. I have never used Paredit when doing Scheme or Racket, so maybe I should just hunker down with it at some point (I have looked into it before). I don’t use Emacs (tried it a few times), but there are several Paredit-based VSCode extensions.
      - mrcode007 5y ago
        Paredit is the editor part, but then there is also SLIME REPL. In lisp, unlike in other programming languages, you don't start writing your code with a blank slate and an empty source file. A common approach is to start with an initial core. The idea is that all programs start the same, with an initial core, that contains runtime and standard library. You can test things out in a REPL, that you run side by side with a text editor. As you work things out in REPL, you can start moving them to the text file and the program starts taking form and shape. It's a much more exploration driven design and with fast structure editing of paredit you can really transplant or reorganize pieces of code really quickly. The idea is to grow the program from the initial core into something else inside-out style. It's akin to being dropped into a running program, and modifying it as it is running but that is not possible with most languages easily and as interactively as in Common Lisp for example. With SLIME the REPL and text editor integration, you can for example write a buggy function, test it in REPL and find out about the error. Modify it in the text editor, press Ctrl-c twice, and the function is hot reloaded on the fly. No reloads, no re-imports, no-caching, no program restarts. Next time the function is called, it is replaced with a new stuff you just wrote. It really is pretty magical. It just works, and works for classes, objects, meta classes, functions, variables, etc. The language is unlike other languages, because it offers object introspection (often mistakenly referred to as reflection) and object intercession; introspection and intercession combined give you a true reflection. It is the latter technique that supports all of the above, for an imho superior development experience. It really just works, and it works marvelously well. No yak shaving for hours on end trying to get some dependency chain to work, compiler errors, package version mismatches, etc. No need for build team, no need for testing/qa team, no need for most of your usual dev overhead we are so used to these days :)
  - diatone 5y ago
    Next Advent of Code, grab yourself a download of Portacle and see how far you can get with Lisp. By day 7 you should have a feel for what people mean when they say you are directly editing the AST. Basically: there is the same amount of friction to moving constructs around at a high level as there is at a low level, and this is due to Lisp’s uniform syntax, and paredit leverages this a lot. Contrast this to a language like Java, where the friction between modifying a single expression and modifying class-level interactions are totally different, syntactically. Now you find yourself leaning on highly specific IDE-level support or Lombok for things like setting up Builders for every POJO, to get around these constraints. From a Lisper’s perspective, these syntactic constraints are unnecessary, and should at most require a library that you can trivially port to any editor. Which is exactly what paredit achieves.
    - bmitc 5y ago
      I have used Scheme and Racket a fair amount and know bits of Common Lisp. What I don’t understand is what I stated and what I quoted. Without seeing it demonstrated, I just don’t understand the quoted claims. When I’ve done Scheme and Racket, I never really needed to be copying and moving ASTs (i.e., parenthetic groups) all over the place. Maybe that’s just me and my style, for better or worse.
      - reikonomusha 5y ago
        I think "structural editing" is a better way to talk about it. You manipulate code as units of S-expressions. You don't type parentheses so much as you create and modify (trees of) S-expressions. So things like unbalanced parentheses or improperly indented forms are not possible since there's no way to create them in the first place. In a somewhat vague sense, the "units of editing" are not characters. There are some videos on YouTube demonstrating ParEdit and SLIME.
        bmitc 5y ago
        That’s fine, but I don’t see where one gets "light years ahead of any programming IDE out there" from that. I like Lisps/Schemes as much as the next person, but other languages don’t have parentheses to deal with, so by that same token, you’re getting that for free, such as an ML like F#.
  - Jach 5y ago
    I think the paredit stuff is a bit overblown but apart from managing parens for you, another simple example is editing single expressions. e.g. in Java you might have a line: "int a = blah.bar(something, thing, whatever);" If you realize you need to actually pass "whatever" first, not last, unless you know an IDE shortcut that can make the edit for you, you're going to have to type stuff. I would probably just move my cursor to the start, type "whatever, ", move my cursor to the comma after "thing" and highlight to the end then delete. If "whatever" was a longer variable, or even more interestingly an entire sub-function call like "whatever(x, y, z)", I might instead highlight it all, cut, backspace the comma, move cursor to the start, paste, type a comma. Moving the cursor to the start might be character by character, or using the operating system wide ctrl+left/right to move in 'word' chunks, or possibly it's faster to use the 'home' key first or god forbid the mouse. Oh no, I might miss a comma or somehow mess up a paren/semicolon or typo a name?! Whatever, it's rare for me, and for most mistakes I'd get a red squiggly alerting me to it immediately. I like typing, and prefer most 'helpful' plugins get out of my way for most things, so such a process isn't that annoying to me. But I do at least see there's a nicer process if you have something like paredit: you just move you cursor to the "whatever" (even if it's instead "whatever(a,b,c)") and a command will move it to the left/right/etc. and fix up anything that needs fixing up. In Lisp though the base syntax is so simple and uniform that there's not usually much needing "fixing up" -- there's no pesky commas to deal with for instance, and having the opening paren come in front of the function name instead of after simplifies a lot of things. The worst is adding/removing/moving a form that's at the end of a let binding, or sometimes introducing a let binding, or sometimes adding something to the end of a function that previously ended with ))). I like to use vim (which does have paredit though I have it disabled) and just having the ability to jump between open/close parens by pressing "%" and to cut jumps as a whole, or the insides, without having to move my cursor character by character, is good enough for me. I still use some paredit-like commands in some instances like moving forms around or in those "worst case issues" I mentioned but I use them with these mappings: https://github.com/tpope/vim-sexp-mappings-for-regular-people https://github.com/tpope/vim-sexp-mappings-for-regular-peopl... There are more advanced things but how much I care about them varies; I don't tend to need them for Lisp, though every so often I'll miss something from Eclipse that I suspect not even emacs does (or does well). e.g. I know emacs can do a "templateized" completion just like a Java IDE where you type a function name and it completes it and inserts its required arguments as placeholder variables to later define/type over, I don't know though whether emacs can then let you place the cursor over each one in turn and with something as easy as 'ctrl+1' hoist that var to an assignment form just above (I did this all the time in Eclipse to avoid having to choose a name, type it, and type its correct type). (In Lisp it's complicated by needing to introduce a let binding if it doesn't exist or append to one if it does. It wouldn't surprise me if paredit can do this, it's just that I'm aware of some refactoring tools in Slime but they don't tend to approach what Eclipse or IntelliJ users expect even if in theory they could.)
  - mrcode007 5y ago
    you could take a look at this in action here: https://www.youtube.com/watch?v=D6h5dFyyUX0 https://www.youtube.com/watch?v=D6h5dFyyUX0
speed_spread 5y ago
Because you need to find those 10 lisp programmers. The ratio of mediocre Java to good lisp programmers is probably more than 100:1. With only 10 specialists, attrition can become a problem quickly. It's a different optimization.
- User23 5y ago
  On the other hand there are virtually no mediocre Lisp programmers.
  - exdsq 5y ago
    Why not?
    - quickthrower2 5y ago
      Probably the same in APL, Coq, Prolog and Haskell etc. Weird but mathematically oriented languages that don’t easily get you jobs are persued by passionate programmers. It is like there are no mediocre hikers on the north pole!
      - exdsq 5y ago
        You should see some Haskell I've written in production and you'll change your mind quick enough ;)
  - klyrs 5y ago
    I've met quite a few lisp programmers over the years, about half of them mediocre -- particularly those who only knew lisp. Reason being that they learned lisp for a job, and that was just the language the job was in. I might believe that only good programmers pick up lisp on their own, but mediocre programmers follow the money. The excellent lisp programmers I've known have been polyglots -- folks who can dig deep, know how memory works, know how cpus work, know how their lisp works under the hood. The mediocre ones just bash stuff together, oblivious of the details, and the result is no better than javascript.
User23 5y ago
Because MPAI.
Jach 5y ago
Because such simple one-factor sentiments, even if they are true, are not how people evaluate using one thing over another. At best they might consider it in a multi-factor analysis. But with this particular sentiment it still probably won't be considered much because it's too hard to evaluate the truthiness of such a general claim -- I suspect it's more true in some cases but not so true in others. You'd be more interested in considering the specifics of language+ecosystem (not language alone) and how they relate to your problem domain.
bitwize 5y ago
Lots of reasons. 1) Lisp acquired a reputation of being associated with AI, back when that was a bad thing, lots of government-funded "expert system" boondoggles and the like. It is not considered a general purpose language, despite being one. 2) Lisp is thought to have poor library support. Unlike Java or JavaScript, which have standard libraries for every task, Lisp is thought to lack this. Quicklisp helps, but it's not as comprehensive as Maven or npm. 3) Lisp is associated with a certain programmer type: highly intelligent, persnickety, only interested in solving interesting problems. Companies prefer programmers who have only a moderate amount of technical skill but who are diligent, who will take orders and get the job done without complaint. 4) Not many programmers actually know Lisp, because not many bother to learn it. If CS students are exposed to Lisp in college, they will complain about it and won't touch it again after passing required classes that use it. This means Lisp has a much smaller qualified candidate pool and it's much harder to hire Lisp programmers. 5) Software projects, especially enterprise software projects, are optimized to require lots of programmers each of whom develop one component without stepping on each other's toes. In the enterprise, not breaking existing things is more important than building new things rapidly. This is why OO languages like Java are so well loved... they help encapsulate programmers from each other as well as encapsulating units of functionality. Each programmer poses more risk in a small team where every programmer's responsibility is larger (and every programmer could theoretically rebuild the whole system themself given enough time) than in a large team where none of the programmers have a complete understanding of the system and everybody stays in their lane. 7) Larger programming teams are just more impressive. As a manager if you have 100 direct reports instead of 10, you are a bigger deal. You can command a bigger budget, which also makes you seem more important. So the odds are pretty stacked against Lisp. It doesn't scale well to large teams, but it lets a small team get more done. The incentives in most software development reward large teams of programmers just barely smart enough to get the job done, not small surgical teams of wizardly Lisp hackers. So Lisp remains niche, perhaps rightly so.
- daniel-cussen 5y ago
  You might be able to hire 100 Lisp engineers. It's just super expensive, they're going for really high prices right now, and it might be way overkill.

Largely because of myths surrounding it, for example parenthesis syntax and lack of editor support. With paredit, you get meta level direct AST manipulation support in emacs that is still light years ahead of any programming IDE out there. I think the other fear factor is you have to think a lot more about how to approach the problem when writing Common Lisp and it requires a more complete engineer due to a smaller library ecosystem. Usually there is one and sometimes two libraries for doing X and if not you’re up for making a library yourself. I’m attaching an example of how quickly you can write TLS 1.3 by a single person in Common Lisp. At the time I wrote this library most websites still struggled with TLS 1.3 support. The point is that most software creators don’t have the breath and depth of knowledge required to be proficient in LISP if they only posses siloed domain knowledge. Links: https://github.com/mateuszb/tls1.3 https://github.com/mateuszb/tls1.3 If you want an example of expressivity you can take a look at EC crypto with a full NIST test vector set in about 300 lines of code https://github.com/mateuszb/tls1.3/blob/master/elliptic-curves.lisp https://github.com/mateuszb/tls1.3/blob/master/elliptic-curv...

bmitc 5y ago
> With paredit, you get meta level direct AST manipulation support in emacs that is still light years ahead of any programming IDE out there. Do you happen to know of a video or article with visual examples that demonstrates this? I always hear about these things with Common Lisp, but I have personally never seen it. I’d love to understand how this works or enables people.
- rags2riches 5y ago
  There's the Animated Guide to Paredit: http://danmidwood.com/content/2014/11/21/animated-paredit.html http://danmidwood.com/content/2014/11/21/animated-paredit.ht... It really feels more like you're working on a tree of code than just editing text.
  - bmitc 5y ago
    That’s helpful. I’ll read it more in depth when I’m off mobile as that site is hard to read on mobile. Although, I think there’s still something missing between those demonstrations and the quoted claims. I have never used Paredit when doing Scheme or Racket, so maybe I should just hunker down with it at some point (I have looked into it before). I don’t use Emacs (tried it a few times), but there are several Paredit-based VSCode extensions.
    - mrcode007 5y ago
      Paredit is the editor part, but then there is also SLIME REPL. In lisp, unlike in other programming languages, you don't start writing your code with a blank slate and an empty source file. A common approach is to start with an initial core. The idea is that all programs start the same, with an initial core, that contains runtime and standard library. You can test things out in a REPL, that you run side by side with a text editor. As you work things out in REPL, you can start moving them to the text file and the program starts taking form and shape. It's a much more exploration driven design and with fast structure editing of paredit you can really transplant or reorganize pieces of code really quickly. The idea is to grow the program from the initial core into something else inside-out style. It's akin to being dropped into a running program, and modifying it as it is running but that is not possible with most languages easily and as interactively as in Common Lisp for example. With SLIME the REPL and text editor integration, you can for example write a buggy function, test it in REPL and find out about the error. Modify it in the text editor, press Ctrl-c twice, and the function is hot reloaded on the fly. No reloads, no re-imports, no-caching, no program restarts. Next time the function is called, it is replaced with a new stuff you just wrote. It really is pretty magical. It just works, and works for classes, objects, meta classes, functions, variables, etc. The language is unlike other languages, because it offers object introspection (often mistakenly referred to as reflection) and object intercession; introspection and intercession combined give you a true reflection. It is the latter technique that supports all of the above, for an imho superior development experience. It really just works, and it works marvelously well. No yak shaving for hours on end trying to get some dependency chain to work, compiler errors, package version mismatches, etc. No need for build team, no need for testing/qa team, no need for most of your usual dev overhead we are so used to these days :)
- diatone 5y ago
  Next Advent of Code, grab yourself a download of Portacle and see how far you can get with Lisp. By day 7 you should have a feel for what people mean when they say you are directly editing the AST. Basically: there is the same amount of friction to moving constructs around at a high level as there is at a low level, and this is due to Lisp’s uniform syntax, and paredit leverages this a lot. Contrast this to a language like Java, where the friction between modifying a single expression and modifying class-level interactions are totally different, syntactically. Now you find yourself leaning on highly specific IDE-level support or Lombok for things like setting up Builders for every POJO, to get around these constraints. From a Lisper’s perspective, these syntactic constraints are unnecessary, and should at most require a library that you can trivially port to any editor. Which is exactly what paredit achieves.
  - bmitc 5y ago
    I have used Scheme and Racket a fair amount and know bits of Common Lisp. What I don’t understand is what I stated and what I quoted. Without seeing it demonstrated, I just don’t understand the quoted claims. When I’ve done Scheme and Racket, I never really needed to be copying and moving ASTs (i.e., parenthetic groups) all over the place. Maybe that’s just me and my style, for better or worse.
    - reikonomusha 5y ago
      I think "structural editing" is a better way to talk about it. You manipulate code as units of S-expressions. You don't type parentheses so much as you create and modify (trees of) S-expressions. So things like unbalanced parentheses or improperly indented forms are not possible since there's no way to create them in the first place. In a somewhat vague sense, the "units of editing" are not characters. There are some videos on YouTube demonstrating ParEdit and SLIME.
      - bmitc 5y ago
        That’s fine, but I don’t see where one gets "light years ahead of any programming IDE out there" from that. I like Lisps/Schemes as much as the next person, but other languages don’t have parentheses to deal with, so by that same token, you’re getting that for free, such as an ML like F#.
- Jach 5y ago
  I think the paredit stuff is a bit overblown but apart from managing parens for you, another simple example is editing single expressions. e.g. in Java you might have a line: "int a = blah.bar(something, thing, whatever);" If you realize you need to actually pass "whatever" first, not last, unless you know an IDE shortcut that can make the edit for you, you're going to have to type stuff. I would probably just move my cursor to the start, type "whatever, ", move my cursor to the comma after "thing" and highlight to the end then delete. If "whatever" was a longer variable, or even more interestingly an entire sub-function call like "whatever(x, y, z)", I might instead highlight it all, cut, backspace the comma, move cursor to the start, paste, type a comma. Moving the cursor to the start might be character by character, or using the operating system wide ctrl+left/right to move in 'word' chunks, or possibly it's faster to use the 'home' key first or god forbid the mouse. Oh no, I might miss a comma or somehow mess up a paren/semicolon or typo a name?! Whatever, it's rare for me, and for most mistakes I'd get a red squiggly alerting me to it immediately. I like typing, and prefer most 'helpful' plugins get out of my way for most things, so such a process isn't that annoying to me. But I do at least see there's a nicer process if you have something like paredit: you just move you cursor to the "whatever" (even if it's instead "whatever(a,b,c)") and a command will move it to the left/right/etc. and fix up anything that needs fixing up. In Lisp though the base syntax is so simple and uniform that there's not usually much needing "fixing up" -- there's no pesky commas to deal with for instance, and having the opening paren come in front of the function name instead of after simplifies a lot of things. The worst is adding/removing/moving a form that's at the end of a let binding, or sometimes introducing a let binding, or sometimes adding something to the end of a function that previously ended with ))). I like to use vim (which does have paredit though I have it disabled) and just having the ability to jump between open/close parens by pressing "%" and to cut jumps as a whole, or the insides, without having to move my cursor character by character, is good enough for me. I still use some paredit-like commands in some instances like moving forms around or in those "worst case issues" I mentioned but I use them with these mappings: https://github.com/tpope/vim-sexp-mappings-for-regular-people https://github.com/tpope/vim-sexp-mappings-for-regular-peopl... There are more advanced things but how much I care about them varies; I don't tend to need them for Lisp, though every so often I'll miss something from Eclipse that I suspect not even emacs does (or does well). e.g. I know emacs can do a "templateized" completion just like a Java IDE where you type a function name and it completes it and inserts its required arguments as placeholder variables to later define/type over, I don't know though whether emacs can then let you place the cursor over each one in turn and with something as easy as 'ctrl+1' hoist that var to an assignment form just above (I did this all the time in Eclipse to avoid having to choose a name, type it, and type its correct type). (In Lisp it's complicated by needing to introduce a let binding if it doesn't exist or append to one if it does. It wouldn't surprise me if paredit can do this, it's just that I'm aware of some refactoring tools in Slime but they don't tend to approach what Eclipse or IntelliJ users expect even if in theory they could.)
- mrcode007 5y ago
  you could take a look at this in action here: https://www.youtube.com/watch?v=D6h5dFyyUX0 https://www.youtube.com/watch?v=D6h5dFyyUX0

speed_spread 5y ago

Because you need to find those 10 lisp programmers. The ratio of mediocre Java to good lisp programmers is probably more than 100:1. With only 10 specialists, attrition can become a problem quickly. It's a different optimization.

User23 5y ago
On the other hand there are virtually no mediocre Lisp programmers.
- exdsq 5y ago
  Why not?
  - quickthrower2 5y ago
    Probably the same in APL, Coq, Prolog and Haskell etc. Weird but mathematically oriented languages that don’t easily get you jobs are persued by passionate programmers. It is like there are no mediocre hikers on the north pole!
    - exdsq 5y ago
      You should see some Haskell I've written in production and you'll change your mind quick enough ;)
- klyrs 5y ago
  I've met quite a few lisp programmers over the years, about half of them mediocre -- particularly those who only knew lisp. Reason being that they learned lisp for a job, and that was just the language the job was in. I might believe that only good programmers pick up lisp on their own, but mediocre programmers follow the money. The excellent lisp programmers I've known have been polyglots -- folks who can dig deep, know how memory works, know how cpus work, know how their lisp works under the hood. The mediocre ones just bash stuff together, oblivious of the details, and the result is no better than javascript.

User23 5y ago

Because MPAI.

Jach 5y ago

Because such simple one-factor sentiments, even if they are true, are not how people evaluate using one thing over another. At best they might consider it in a multi-factor analysis. But with this particular sentiment it still probably won't be considered much because it's too hard to evaluate the truthiness of such a general claim -- I suspect it's more true in some cases but not so true in others. You'd be more interested in considering the specifics of language+ecosystem (not language alone) and how they relate to your problem domain.

bitwize 5y ago

Lots of reasons. 1) Lisp acquired a reputation of being associated with AI, back when that was a bad thing, lots of government-funded "expert system" boondoggles and the like. It is not considered a general purpose language, despite being one. 2) Lisp is thought to have poor library support. Unlike Java or JavaScript, which have standard libraries for every task, Lisp is thought to lack this. Quicklisp helps, but it's not as comprehensive as Maven or npm. 3) Lisp is associated with a certain programmer type: highly intelligent, persnickety, only interested in solving interesting problems. Companies prefer programmers who have only a moderate amount of technical skill but who are diligent, who will take orders and get the job done without complaint. 4) Not many programmers actually know Lisp, because not many bother to learn it. If CS students are exposed to Lisp in college, they will complain about it and won't touch it again after passing required classes that use it. This means Lisp has a much smaller qualified candidate pool and it's much harder to hire Lisp programmers. 5) Software projects, especially enterprise software projects, are optimized to require lots of programmers each of whom develop one component without stepping on each other's toes. In the enterprise, not breaking existing things is more important than building new things rapidly. This is why OO languages like Java are so well loved... they help encapsulate programmers from each other as well as encapsulating units of functionality. Each programmer poses more risk in a small team where every programmer's responsibility is larger (and every programmer could theoretically rebuild the whole system themself given enough time) than in a large team where none of the programmers have a complete understanding of the system and everybody stays in their lane. 7) Larger programming teams are just more impressive. As a manager if you have 100 direct reports instead of 10, you are a bigger deal. You can command a bigger budget, which also makes you seem more important. So the odds are pretty stacked against Lisp. It doesn't scale well to large teams, but it lets a small team get more done. The incentives in most software development reward large teams of programmers just barely smart enough to get the job done, not small surgical teams of wizardly Lisp hackers. So Lisp remains niche, perhaps rightly so.

daniel-cussen 5y ago
You might be able to hire 100 Lisp engineers. It's just super expensive, they're going for really high prices right now, and it might be way overkill.