9 ms·

Why I chose Common Lisp over Python, Ruby, and Clojure

134 points by smanek 17y ago

ricree 17y ago
The fact that he proposed removing map, reduce, filter, and lambda from Python 3. In the case of map and filter, is there any compelling reason to use either of those instead of list comprehensions or generator expressions?
- smanek 17y ago
  Nope - just a style thing. You don't need lambdas either - named functions are just as powerful. But I often think 'functionally.' Since I have options, I'd rather go with a language that allows me to program how I think - instead of being forced to translate my thought into its semantics.
  - azanar 17y ago
    Nope - just a style thing. You don't need lambdas either - named functions are just as powerful. It depends on what you define as power, and how wide of a continuum you are willing to presume that it runs. Named functions add two levels of clutter. The first is to the actual code, because you have to add a name to something that never wanted one, and it has to be defined apart from where it is used. You could give it a fluff name, but that is worse than no name at all. The reader is left wondering what the function is for, which requires carrying unnecessary mental baggage. It would seem more powerful to me that you just define a function where it is used, and not worry about having to carry around that extra mental information for later on; the less state you need to keep the better. But I may only say that because I, like you, tend to think functionally. The second is to the namespace, because now there is a symbol in the environment that doesn't need to be there. It's not a huge deal, but it is just another bit of extraneous fluff. Anyway, not meaning to turn this into a language war. Just a couple of thoughts that popped into my head when I read your first paragraph.
    - ellyagg 17y ago
      I agree with this big time. I hate increasing the "vocabulary" of my program unnecessarily.
    - d0mine 17y ago
      The "correct" alternative to multiline lambda might be not a named function but an explicit code block: no clutter at the prize of vertical space. result = [] for item in iterable: # here goes multiline lambda as an explicit suite # that computes value result.append(value)
- maxtilford 17y ago
  Obviously, map and filter are better because ... ah, hmm, I don't know. I think it is a matter of language design. Higher order functions are not as prominent in Python as say, Haskell, where map & friends are generally preferred for composability. Lispers prefer map & friends just because list comprehensions add all that messy syntax. If I were a Python programmer, I would probably use list comprehensions, since they seem to be the preferred idiom.
- riobard 17y ago
  A single, non-nested list comprehension or generator exp is basically map(filter). You need nesting to get filter(map). e.g. map(expensive_call, filter(cond, seq)) equals [expensive_call(each) for each in seq if cond(each)] but filter(cond, map(expensive_call, seq)) equals [each for each in [expensive_call(x) for x in seq] if cond(each)] note because of "expensive_call", it's inefficient (and silly) to do [expensive_call(each) for each in seq if cond(expensive_call(each))] So map/filter combination gives more flexibility than list comprehension, and for functional-thinking minds, it's just so natural to think in abstract terms of passing functions around. List comprehension is pretty syntactical sugar to do similar things, but it forces you think about the "how to do" instead of "what to do". That said, it's not really "compelling" though -- there is no real "compelling" reason to switch from one Turing-complete language to another given that you can do the same thing eventually. But hey, it's the itches that drive us nuts, isn't it? :) Just my 2 cents
  - amix 17y ago
    Most times you are interested in doing the simple, e.g.: filtered = [x for x in seq if x>10] Python's list comprehension is much more readable than using map/filter/reduce - at least for Python programmers :) Anyhow, I really like Guido's decision on dropping these - it creates a cleaner language and forces people to think Pythonic when programming in Python.
    - riobard 17y ago
      > Python's list comprehension is much more readable than using map/filter/reduce - at least for Python programmers For simple cases, yes. But I wouldn't say [each for each in [expensive_call(x) for x in seq] if cond(each)] is more readable than filter(cond, map(expensive_call, seq)) at least for functional-thinking minds. The level of thinking in abstract is different here. Now the problem is, some people see Python as a very functional language (with first-class functions etc) and want to use it that way (like Lisp), but BDFL and some core Python devs believe it is better to keep it Pythonic, thus those functional people are kinda pissed off by this and switch away from Python. Personally I don't think it will make Python a lot cleaner to remove two auxiliary functions and force people to use list comprehension when it is completely trivial to add these missing pair back (two lines of code). (disclosure: I prefer FP, but I also think keeping things Pythonic is fine most of the time. It's just that in this case, I think map/filter is pretty "Pythonic" according to me. :)
- pwnstigator 17y ago
  I like lambda because it allows anonymous functions-- functions that can be defined on the fly (dynamically) and then thrown away. IMO, it's generally bad form for a program to be dynamically creating named functions, filling the namespace. Also, I like map and filter because they aren't just functions in the procedural sense but combinators: you can pass them around, using them as arguments and returning them. It's much harder to pass around a syntactic entity like a list comprehension (although I'm glad list comprehensions exist; as shorthand they are great in source code.) All this said, I haven't used Python in 3 years, in favor of purely functional languages such as Clojure and ML, so I might be way out of date.
  - megaman821 17y ago
    You bring up a good point about passing map and filter around. I hadn't really thought of that. I still prefer to user list comprehensions, and don't think it would be so bad to have map, filter, reduce live in the itertools module. This would stylistically match what is done with the comparison operator functions living in their own module, which you can import if you need to pass the functions around.
    - pwnstigator 17y ago
      List comprehensions are great, but they're sort of a DSL-- something the syntax recognizes as special and converts to something else. It's unusual that you can pass around higher-order syntactic elements, while every modern language worth its salt allows you to pass around functions. Common Lisp has something similar, called LOOP. Implemented as a macro, it's a within-Lisp DSL for expressing looping constructs, e.g. (loop for i from 1 to 10 sum i) => 55 (loop for c across "bar" collect c) => (#\b #\a #\r) It's controversial within the CL community, because the loop language looks much more like traditional languages than Lisp.
      - gruseom 17y ago
        Indeed, we figured out recently that it's basically embedded Algol 68... which seems superbly fitting somehow.
- algorias 17y ago
  map, filter and reduce were not removed from python3.x, however. They'll be there for the next decade at least.
tjr 17y ago
After that, I was planning to dig into some specifics about the best way to manage/deploy a production Common Lisp webapp, which I hope would help a new Lisper get off the ground. Looking forward to that.
icey 17y ago
You could probably get rid of the "(fixed)" in your title; I bet most people didn't see your original submission, and the ones that did will know what happened.
- smanek 17y ago
  Thanks. Sorry for the trouble before. I had an old personal wordpress blog that I'd originally posted this on - but there were some technical issues with upgrading WordPress to a modern version. Just reposted the same content on Posterous.
pcof 17y ago
I predict you will get a great deal of pythonic fire. Nevertheless, I find it very nice that you plan to deal with the specific libraries you're using in the next posts. The greatest problem in adopting CL seems to be the extremely decentralized (not to say disorganized) library spacs.
- benhoyt 17y ago
  "Python 2 is fine now – but in the coming months and years new libraries, features, and performance improvements are only going to be introduced in Python 3, and I didn’t want to get left behind or forced to take on an expensive and time consuming port in the future." Does this argument hold some water, or is Python 2-to-3 conversion often pretty trivial in practice?
  - etal 17y ago
    Word on the street is that the 2to3 tool is magical. Just run the Py2.6 interpreter with the "-3" flag for warnings, fix the warnings, and run the 2to3 tool to get your port.
WilliamLP 17y ago
Basically: I started with my conclusion, and then went through all the other options and rationalized them away until it seemed like my choice was based on objective reasoning.
- smanek 17y ago
  I think you drawing that conclusion may be a result of how I structured my essay (I introduced my conclusion, then my reasoning). I freely admit my personal biases and experiences eliminated a lot of potentially good options without ever giving them due consideration (e.g., Haskell, Scheme, Lua). But I really did try to give the 3 options listed in the article a fair shake at the time. For what it's worth - I was about 50-50 on using Python till the very end ...
  - wheels 17y ago
    I too thought the logic was weak. Here was my basic objection: you chose not to use two popular web programming languages because of their potential volatility, but then went with one that has been so infrequently used in web programming that it almost certainly has opaque deffeciencies. It's not that your reasons were bogus, they just didn't carry the decision.
    - smanek 17y ago
      That's a fair point. For a programmer without Lisp experience, that would be true. But I've written enough Lisp-based webapps that there aren't really that many 'unforseen' problems for me anymore (rule of thumb: use software by Edi Weitz - everything he writes is golden :-D)). That being said, I still ran into trouble with Elephant, so I probably should have taken these sort of 'unknown unknowns' (which are more prevalent in unusual languages like Lisp) into consideration upfront. To not do so was certainly an oversight.
      - ube 17y ago
        If you are going to be the only programmer on the project then your reasons make sense. However, if you are going to have other people on the projects - then your reasons do not make sense from a longer term point of view. How many lisp programmers do you have out there as compared to python or ruby programmers? If you and the founders decide to go your separate ways how likely is the project going to be continued in lisp? It seems like a somewhat precarious decision to let you use Lisp for this project. There's nothing supremely special about the problem you are attacking that warrants Lisp (outside of your comfort level which you clarified very clearly in your post). Perhaps I'm just old and cynical but I've been on too many projects where the first (and therefore "lead") developer chose a particular language and architecture that in the long term was not sustainable.
scott_s 17y ago
Python's multiprocessing library (http://docs.python.org/library/multiprocessing.html http://docs.python.org/library/multiprocessing.html) provides a similar interface that one would expect from a multithreaded library. For example, from multiprocessing import Pool p = Pool(5) def f(x): return x*x p.map(f, [1,2,3]) I do not have direct experience with the library, but from what I've seen, the code you would write is not significantly different from the code you'd write using a decent multithreading interface.
- fauigerzigerk 17y ago
  Multiple processes cannot share regular in memory data structures. That makes the multiprocessing library unsuitable for a growing class of applications. However, unladen swallow aims to remove the GIL, so there's still hope for Python.
  - scott_s 17y ago
    There is an API for both communicating data explicitly, and for allocating shared memory data structures which can be shared implicitly.
    - fauigerzigerk 17y ago
      I know. That's why I said "regular in-memory data structures".
mahmud 17y ago
Did you really choose Lisp over alternatives? Before learning CL I was a fairly decent, C, C++ and Perl programmer. Did assembly, Pascal, TCL and Awk. Up to that point, I always had to pause a for a minute when starting a new project/script, think about its scope, and choose a language based on the necessary performance, development speed, expressiveness, available libraries, etc. (and whether whoever was going to read the code afterward knew the language; C was often a natural choice for code shared with others on Unix, C++ for MFC/COM, Perl for sysadmin stuff, and TCL and Awk for my own tools.) I learned Lisp in over a month, to spite someone (I dared a notorious troll I would write an AI bot of his choice if he stopped spamming us, youthful bravado for sure, and I lost the bet) While researching "AI" I came across Winston and Horn's "Common Lisp", then the hyperspec, then a few more books over the course of a month. I sat down with SICP and did the exercises on my break, while I was in school and waiting tables. After I learned it however, specially with CLOS, there was no contest. Three months after buying Sonya Keene's CLOS book it was fair to say I forgot all other programming languages. There were no more "projects"; I no longer had to sketch out designs on paper or do "requirement analysis" (something I was told in school was necessary for all software.) For once, the great ideas in my head were a new emacs buffer away. I could write code faster than I would in Perl, Awk or TCL, it ran as fast as C++, and it was more expressive than the English in my head. I could type "commands" into a shell get a dialog embedded in my window, a few more commands and it would move to the upper right corner, I could change its name property and add text to it, then I could fold that dialog box into a menu-item named "Help" in the menubar and call that dialogbox "About". Amazing. I went on hacking like this for about year when I realized I was doing the "wrong thing". You see, I have been using CMUCL with its builtin editor and writing GUI applications in Motif (it was 2001 and Motif wasn't open source yet, so I got the hang on Lesstif and learned its quirks.) Right around this time, Linux GUIs were maturing and people were being snobs about their Enlightenment themes and dissing each other over their choices of Windows Manager. So I was peer-pressured into learning DHTML and Web Design. I read comp.lang.lisp and those too were snobbish condescending idiots who flamed everyone, specially competent programmers whose work I admired (including Scott McKay and Robert Fahlman (the very people who gave me my CMUCL.)) It was really hard to be a Lisper for a while, specially a young impressionable one who read cll uncritically; news of corporate giants coming with new tools and programming languages to enslave humanity were abound. First C++, then Java, then XML, and finally .NET. You literally had to pick your battles and choose a corporate sponsor or you would have no future in computing! (you think I am kidding?) cll is all doom and gloom, and of course, there is the obligatory stabs at Lisp vendors by Open Source proponents, and stabs at Open Source for people alleging it's killing our beloved vendors. Every once in a while there was news of a Lisp dialect that's going to kill Common Lisp (Smalltalk, Dylan, and the ancient religions of Mesopotamia.) Fuck, that was painful. All the while I was following this 4-year long intellectual funeral, becoming ever more "hardcore" and learning mathematics, there was a small group of "Yobos" silently kicking ass and churning out great software. CMUCL got forked to SBCL, added unicode support and threads, not to mention easy building, SLIME was a new Emacs mode better than anything before and since, Cliki was launched, C-L.net, and the #lisp IRC channel was born and hit puberty overnight. Perfect ecosystem. Today, Lisp is nothing like what it was 8,7,6, even 2 years ago. It's not just "good" in the well-explored text book fashion; no, it's _good shit_. Get work done good. Think, hack, ship, bill for it good. 2-3 products per month good. You still have to know where things are, who is working on what, what's maintained and what's obsoleted by what. Sure. But there is absolutely no lack of libraries.
babo 17y ago
I don't buy the author's argument against Python and clojure but anyway, that's they system.
wgj 17y ago
Some of the conclusions are inconsistent. For example, Ruby got canned mainly for having a heavyweight framework, since he wants to write everything from scratch. What keeps one from doing that in Ruby? Ruby does not depend on Rails. His complaint about different versions and implementations is a red herring. Lisp is not immune from this.
- smanek 17y ago
  There are different implementations, of course. But the the multiple version problem was worked out over a decade ago.
  - deleted 17y ago
    [deleted]
- gruseom 17y ago
  It's the other way around. For things you've decided to write from scratch, especially if they're hard, CL is a dream. There's nothing comparable in my experience [~]: you can build unbelievably powerful and concise programs easily, then just as easily transform them in major ways. It's the ultimate combination of malleability and expressive power. (Oh, and performance, when you need it.) I can think of only three reasons to prefer Ruby over it: (1) you want Rails etc.; (2) you don't like Lisp; (3) you really like Ruby. The latter two are matters of taste, which leaves #1. [~] I haven't tried Clojure yet. Then again, I don't need to: the JVM is a drawback for me, and I don't need all those Java libraries right now.
akkartik 17y ago
No mention of mzscheme?!
herdrick 17y ago
I like the choice. I love to see people using lisps but some of the path to that choice makes me cringe. Like this: I probably could have written a ‘bare-bones’ implementation of the site’s back-end in Rails in a week instead of two weeks, but I would rather ‘waste’ that one week up front to have more flexibility later. YAGNI. Speed kills (the competition).
- gruseom 17y ago
  It was the previous sentence that resonated with me: I’m a bit wary of using ‘heavy weight’ frameworks like Rails (or Django) on large custom projects. In my experience they make the first 90% of what I’m trying to do be really easy – but then make the last 10% a living hell since I need to modify something the framework never intended me to control. That is a very true statement, in my experience. It's also an elusive insight, because once you commit to a framework you begin to interpret your problem, the universe, and everything in terms of the framework's conceptual space.
helium 17y ago
Ok, so Rails is too heavy weight for you? What about Merb? Or Sinatra, which was designed for just that reason? Dismissing Ruby as language because only one of a dozen available web frameworks is too "heavy-weight"sounds a bit short-sighted to me.
- seiji 17y ago
  Not everybody has time to keep up with the continuous and rapidly increasing micro-speciation of all ruby projects from the VMs down to template formats.
  - jamesbritt 17y ago
    " ...continuous and rapidly increasing micro-speciation of all ruby projects from the VMs down to template formats." This is simply bullshit. There are merely a fair number of solid choices, as one would hope any decent language would offer. Being able to select appropriate tools for basic Web framework, ORM,and template language is a Good Thing. There are only a handful of viable VMs, and for Web development the practical choices are MRI, REE, and JRuby. "micro-speciation"? That's hysterical. Literally. If Ruby doesn't interest you, fine. Bogus characterizations don't help anyone though.
- Tichy 17y ago
  I am not the OP, but I have a problem with Rails and alternatives: there does not seem to be a viable alternative to ActiveRecord around? I looked into DataMapper (I think that was it's name) which initially looked great. But then I could not find any information on how to use transactions, and also received no answers on the newsgroup. I guess I should have read the source, but I gave up at that point. Since I hate ActiveRecord, I have now subconsciously decided that Rails probably isn't for me. It's great that Sinatra is spawning lots of imitators for other languages, though.
  - steveklabnik 17y ago
    Just so you know, one of the stated goals of Rails 3 is to make it super-easy to swap in your own ORM layer, DataMapper included.
IgorPartola 17y ago
I forget where this came from but this applies 100%: You ask a person to rate 5 paintings 1-5. You then tell them that you have spare paintings that they happened to have rated as 3 and 4 and ask them if they would like one. They will take a "4". Come back in a week and ask them to rate the paintings again. The "4" becomes a "5" and the "3" becomes a "2". Our brains will rationalize decisions to make us happy (whatever that means).
- revorad 17y ago
  You may be referring to this TED talk by Dan Gilbert - http://www.ted.com/talks/lang/eng/dan_gilbert_asks_why_are_we_happy.html http://www.ted.com/talks/lang/eng/dan_gilbert_asks_why_are_w...
vseloved 17y ago
It's funny, that the decision to choose Common Lisp for some project often has to defend itself (because of so much FUD around the language). CL has one killer feature, that will appeal to any mature developer: it's the only production-ready language around, in which you are virtually not constrained by any third-party design decision. That's it. Discussing other details just boils down to the question of tastes (that differ).
- pwnstigator 17y ago
  What about Clojure, circa now? Do you not consider it "production ready"? I've used both CL and Clojure. I'd prefer either over Blub, but Clojure wins in my opinion.
  - vseloved 17y ago
    Clojure is nice, although I didn't use it in production. But it is Java deep underneath (so not completely homoiconic) and comparing to CL there are restrictions, albeit mild. For some specific tasks I might have preferred it (until CL catches up with concurrency support). But in general CL gives you more freedom.
natmaster 17y ago
"The completely broken implementation of closures." That's what the nonlocal keyword is for.
- silentbicycle 17y ago
  When the fundamental design of the language is stacked against your style of working, the cognitive dissonance of using so many workarounds is a serious problem. I just switched to Lua during the Python 2 to 3 transition, and I haven't looked back. Real closures and tail calls, no GIL, a vastly more tasteful design, a dead-simple C API, and the whole shebang is a tenth the size of Python.
  - silentbicycle 17y ago
    Meant to add: Lua's not perfect, but its whole design favors embedding it in another language, so it's a hell of a lot better about gracefully getting out of the way when it's better to write the parts that matter in other languages.
pge 17y ago
I appreciated the article, but I didn't find the reasoning compelling. What I took away from the article was the following line of reasoning: 1. You strongly prefer functional programming to OO programming (no reason given other than personal preference) 2. You don't have enough experience with any languages that are well-suited to FP other than CL that you would be comfortable undertaking a large project 3. You confirmed that many languages you are familiar with (Java, Python, Ruby) are better suited to OO than FP style (not a surprise at all except perhaps Python as I touch on below), so as a direct consequence of (1) and (2), you were left with CL. CL is my first-choice language for certain kinds of projects as well, but I didn't come away any new insights from your article. Your choice seems to hinge around two key conclusions for which you give very little explanation: FP is better than OO for your project, and Python is not a good language for FP. I single out Python because I don't think anyone would argue that Java or Ruby is well-suited to FP. Guido insists that Python is not an FP language, but many users (myself included) would argue that it is reasonably well-suited for FP. That FP was better for your project does not require any further explanation. As others have noted, for a project where scalability of the development team is not an issue (i.e. you are the primary or only developer), it makes sense to choose the style and language you are most comfortable with. If you were starting a project that was going to require several years and a large number of developers, I would have found it more surprising if you committed to a functional programming model. But, given the nature of the project, it didn't seem like there was any need to defend that decision further. All in all, I'm very excited to see CL being used for more commercial web apps, but I didn't find any insight in the thought process you laid out. I am, however, much more interested in Pt. II - your description of rolling out a web app in CL. I learned CL ten years before I learned Python and would prefer to write in CL when possible, but for web apps, I have turned to Python for a number of reasons, most notably the obvious availability of libraries. I would love to hear what you used and hear a cogent argument that rapid commercial web app deployment can be done in CL as easily as in Python. Thanks for sharing your story.

134 points by smanek 17y ago

The fact that he proposed removing map, reduce, filter, and lambda from Python 3. In the case of map and filter, is there any compelling reason to use either of those instead of list comprehensions or generator expressions?

smanek 17y ago
Nope - just a style thing. You don't need lambdas either - named functions are just as powerful. But I often think 'functionally.' Since I have options, I'd rather go with a language that allows me to program how I think - instead of being forced to translate my thought into its semantics.
- azanar 17y ago
  Nope - just a style thing. You don't need lambdas either - named functions are just as powerful. It depends on what you define as power, and how wide of a continuum you are willing to presume that it runs. Named functions add two levels of clutter. The first is to the actual code, because you have to add a name to something that never wanted one, and it has to be defined apart from where it is used. You could give it a fluff name, but that is worse than no name at all. The reader is left wondering what the function is for, which requires carrying unnecessary mental baggage. It would seem more powerful to me that you just define a function where it is used, and not worry about having to carry around that extra mental information for later on; the less state you need to keep the better. But I may only say that because I, like you, tend to think functionally. The second is to the namespace, because now there is a symbol in the environment that doesn't need to be there. It's not a huge deal, but it is just another bit of extraneous fluff. Anyway, not meaning to turn this into a language war. Just a couple of thoughts that popped into my head when I read your first paragraph.
  - ellyagg 17y ago
    I agree with this big time. I hate increasing the "vocabulary" of my program unnecessarily.
  - d0mine 17y ago
    The "correct" alternative to multiline lambda might be not a named function but an explicit code block: no clutter at the prize of vertical space. result = [] for item in iterable: # here goes multiline lambda as an explicit suite # that computes value result.append(value)
maxtilford 17y ago
Obviously, map and filter are better because ... ah, hmm, I don't know. I think it is a matter of language design. Higher order functions are not as prominent in Python as say, Haskell, where map & friends are generally preferred for composability. Lispers prefer map & friends just because list comprehensions add all that messy syntax. If I were a Python programmer, I would probably use list comprehensions, since they seem to be the preferred idiom.
riobard 17y ago
A single, non-nested list comprehension or generator exp is basically map(filter). You need nesting to get filter(map). e.g. map(expensive_call, filter(cond, seq)) equals [expensive_call(each) for each in seq if cond(each)] but filter(cond, map(expensive_call, seq)) equals [each for each in [expensive_call(x) for x in seq] if cond(each)] note because of "expensive_call", it's inefficient (and silly) to do [expensive_call(each) for each in seq if cond(expensive_call(each))] So map/filter combination gives more flexibility than list comprehension, and for functional-thinking minds, it's just so natural to think in abstract terms of passing functions around. List comprehension is pretty syntactical sugar to do similar things, but it forces you think about the "how to do" instead of "what to do". That said, it's not really "compelling" though -- there is no real "compelling" reason to switch from one Turing-complete language to another given that you can do the same thing eventually. But hey, it's the itches that drive us nuts, isn't it? :) Just my 2 cents
- amix 17y ago
  Most times you are interested in doing the simple, e.g.: filtered = [x for x in seq if x>10] Python's list comprehension is much more readable than using map/filter/reduce - at least for Python programmers :) Anyhow, I really like Guido's decision on dropping these - it creates a cleaner language and forces people to think Pythonic when programming in Python.
  - riobard 17y ago
    > Python's list comprehension is much more readable than using map/filter/reduce - at least for Python programmers For simple cases, yes. But I wouldn't say [each for each in [expensive_call(x) for x in seq] if cond(each)] is more readable than filter(cond, map(expensive_call, seq)) at least for functional-thinking minds. The level of thinking in abstract is different here. Now the problem is, some people see Python as a very functional language (with first-class functions etc) and want to use it that way (like Lisp), but BDFL and some core Python devs believe it is better to keep it Pythonic, thus those functional people are kinda pissed off by this and switch away from Python. Personally I don't think it will make Python a lot cleaner to remove two auxiliary functions and force people to use list comprehension when it is completely trivial to add these missing pair back (two lines of code). (disclosure: I prefer FP, but I also think keeping things Pythonic is fine most of the time. It's just that in this case, I think map/filter is pretty "Pythonic" according to me. :)
pwnstigator 17y ago
I like lambda because it allows anonymous functions-- functions that can be defined on the fly (dynamically) and then thrown away. IMO, it's generally bad form for a program to be dynamically creating named functions, filling the namespace. Also, I like map and filter because they aren't just functions in the procedural sense but combinators: you can pass them around, using them as arguments and returning them. It's much harder to pass around a syntactic entity like a list comprehension (although I'm glad list comprehensions exist; as shorthand they are great in source code.) All this said, I haven't used Python in 3 years, in favor of purely functional languages such as Clojure and ML, so I might be way out of date.
- megaman821 17y ago
  You bring up a good point about passing map and filter around. I hadn't really thought of that. I still prefer to user list comprehensions, and don't think it would be so bad to have map, filter, reduce live in the itertools module. This would stylistically match what is done with the comparison operator functions living in their own module, which you can import if you need to pass the functions around.
  - pwnstigator 17y ago
    List comprehensions are great, but they're sort of a DSL-- something the syntax recognizes as special and converts to something else. It's unusual that you can pass around higher-order syntactic elements, while every modern language worth its salt allows you to pass around functions. Common Lisp has something similar, called LOOP. Implemented as a macro, it's a within-Lisp DSL for expressing looping constructs, e.g. (loop for i from 1 to 10 sum i) => 55 (loop for c across "bar" collect c) => (#\b #\a #\r) It's controversial within the CL community, because the loop language looks much more like traditional languages than Lisp.
    - gruseom 17y ago
      Indeed, we figured out recently that it's basically embedded Algol 68... which seems superbly fitting somehow.
algorias 17y ago
map, filter and reduce were not removed from python3.x, however. They'll be there for the next decade at least.

tjr 17y ago

After that, I was planning to dig into some specifics about the best way to manage/deploy a production Common Lisp webapp, which I hope would help a new Lisper get off the ground. Looking forward to that.

icey 17y ago

You could probably get rid of the "(fixed)" in your title; I bet most people didn't see your original submission, and the ones that did will know what happened.

smanek 17y ago
Thanks. Sorry for the trouble before. I had an old personal wordpress blog that I'd originally posted this on - but there were some technical issues with upgrading WordPress to a modern version. Just reposted the same content on Posterous.

pcof 17y ago

I predict you will get a great deal of pythonic fire. Nevertheless, I find it very nice that you plan to deal with the specific libraries you're using in the next posts. The greatest problem in adopting CL seems to be the extremely decentralized (not to say disorganized) library spacs.

benhoyt 17y ago
"Python 2 is fine now – but in the coming months and years new libraries, features, and performance improvements are only going to be introduced in Python 3, and I didn’t want to get left behind or forced to take on an expensive and time consuming port in the future." Does this argument hold some water, or is Python 2-to-3 conversion often pretty trivial in practice?
- etal 17y ago
  Word on the street is that the 2to3 tool is magical. Just run the Py2.6 interpreter with the "-3" flag for warnings, fix the warnings, and run the 2to3 tool to get your port.

WilliamLP 17y ago

Basically: I started with my conclusion, and then went through all the other options and rationalized them away until it seemed like my choice was based on objective reasoning.

smanek 17y ago
I think you drawing that conclusion may be a result of how I structured my essay (I introduced my conclusion, then my reasoning). I freely admit my personal biases and experiences eliminated a lot of potentially good options without ever giving them due consideration (e.g., Haskell, Scheme, Lua). But I really did try to give the 3 options listed in the article a fair shake at the time. For what it's worth - I was about 50-50 on using Python till the very end ...
- wheels 17y ago
  I too thought the logic was weak. Here was my basic objection: you chose not to use two popular web programming languages because of their potential volatility, but then went with one that has been so infrequently used in web programming that it almost certainly has opaque deffeciencies. It's not that your reasons were bogus, they just didn't carry the decision.
  - smanek 17y ago
    That's a fair point. For a programmer without Lisp experience, that would be true. But I've written enough Lisp-based webapps that there aren't really that many 'unforseen' problems for me anymore (rule of thumb: use software by Edi Weitz - everything he writes is golden :-D)). That being said, I still ran into trouble with Elephant, so I probably should have taken these sort of 'unknown unknowns' (which are more prevalent in unusual languages like Lisp) into consideration upfront. To not do so was certainly an oversight.
    - ube 17y ago
      If you are going to be the only programmer on the project then your reasons make sense. However, if you are going to have other people on the projects - then your reasons do not make sense from a longer term point of view. How many lisp programmers do you have out there as compared to python or ruby programmers? If you and the founders decide to go your separate ways how likely is the project going to be continued in lisp? It seems like a somewhat precarious decision to let you use Lisp for this project. There's nothing supremely special about the problem you are attacking that warrants Lisp (outside of your comfort level which you clarified very clearly in your post). Perhaps I'm just old and cynical but I've been on too many projects where the first (and therefore "lead") developer chose a particular language and architecture that in the long term was not sustainable.

scott_s 17y ago

Python's multiprocessing library (http://docs.python.org/library/multiprocessing.html http://docs.python.org/library/multiprocessing.html) provides a similar interface that one would expect from a multithreaded library. For example, from multiprocessing import Pool p = Pool(5) def f(x): return x*x p.map(f, [1,2,3]) I do not have direct experience with the library, but from what I've seen, the code you would write is not significantly different from the code you'd write using a decent multithreading interface.

fauigerzigerk 17y ago
Multiple processes cannot share regular in memory data structures. That makes the multiprocessing library unsuitable for a growing class of applications. However, unladen swallow aims to remove the GIL, so there's still hope for Python.
- scott_s 17y ago
  There is an API for both communicating data explicitly, and for allocating shared memory data structures which can be shared implicitly.
  - fauigerzigerk 17y ago
    I know. That's why I said "regular in-memory data structures".

mahmud 17y ago

Did you really choose Lisp over alternatives? Before learning CL I was a fairly decent, C, C++ and Perl programmer. Did assembly, Pascal, TCL and Awk. Up to that point, I always had to pause a for a minute when starting a new project/script, think about its scope, and choose a language based on the necessary performance, development speed, expressiveness, available libraries, etc. (and whether whoever was going to read the code afterward knew the language; C was often a natural choice for code shared with others on Unix, C++ for MFC/COM, Perl for sysadmin stuff, and TCL and Awk for my own tools.) I learned Lisp in over a month, to spite someone (I dared a notorious troll I would write an AI bot of his choice if he stopped spamming us, youthful bravado for sure, and I lost the bet) While researching "AI" I came across Winston and Horn's "Common Lisp", then the hyperspec, then a few more books over the course of a month. I sat down with SICP and did the exercises on my break, while I was in school and waiting tables. After I learned it however, specially with CLOS, there was no contest. Three months after buying Sonya Keene's CLOS book it was fair to say I forgot all other programming languages. There were no more "projects"; I no longer had to sketch out designs on paper or do "requirement analysis" (something I was told in school was necessary for all software.) For once, the great ideas in my head were a new emacs buffer away. I could write code faster than I would in Perl, Awk or TCL, it ran as fast as C++, and it was more expressive than the English in my head. I could type "commands" into a shell get a dialog embedded in my window, a few more commands and it would move to the upper right corner, I could change its name property and add text to it, then I could fold that dialog box into a menu-item named "Help" in the menubar and call that dialogbox "About". Amazing. I went on hacking like this for about year when I realized I was doing the "wrong thing". You see, I have been using CMUCL with its builtin editor and writing GUI applications in Motif (it was 2001 and Motif wasn't open source yet, so I got the hang on Lesstif and learned its quirks.) Right around this time, Linux GUIs were maturing and people were being snobs about their Enlightenment themes and dissing each other over their choices of Windows Manager. So I was peer-pressured into learning DHTML and Web Design. I read comp.lang.lisp and those too were snobbish condescending idiots who flamed everyone, specially competent programmers whose work I admired (including Scott McKay and Robert Fahlman (the very people who gave me my CMUCL.)) It was really hard to be a Lisper for a while, specially a young impressionable one who read cll uncritically; news of corporate giants coming with new tools and programming languages to enslave humanity were abound. First C++, then Java, then XML, and finally .NET. You literally had to pick your battles and choose a corporate sponsor or you would have no future in computing! (you think I am kidding?) cll is all doom and gloom, and of course, there is the obligatory stabs at Lisp vendors by Open Source proponents, and stabs at Open Source for people alleging it's killing our beloved vendors. Every once in a while there was news of a Lisp dialect that's going to kill Common Lisp (Smalltalk, Dylan, and the ancient religions of Mesopotamia.) Fuck, that was painful. All the while I was following this 4-year long intellectual funeral, becoming ever more "hardcore" and learning mathematics, there was a small group of "Yobos" silently kicking ass and churning out great software. CMUCL got forked to SBCL, added unicode support and threads, not to mention easy building, SLIME was a new Emacs mode better than anything before and since, Cliki was launched, C-L.net, and the #lisp IRC channel was born and hit puberty overnight. Perfect ecosystem. Today, Lisp is nothing like what it was 8,7,6, even 2 years ago. It's not just "good" in the well-explored text book fashion; no, it's _good shit_. Get work done good. Think, hack, ship, bill for it good. 2-3 products per month good. You still have to know where things are, who is working on what, what's maintained and what's obsoleted by what. Sure. But there is absolutely no lack of libraries.

babo 17y ago

I don't buy the author's argument against Python and clojure but anyway, that's they system.

wgj 17y ago

Some of the conclusions are inconsistent. For example, Ruby got canned mainly for having a heavyweight framework, since he wants to write everything from scratch. What keeps one from doing that in Ruby? Ruby does not depend on Rails. His complaint about different versions and implementations is a red herring. Lisp is not immune from this.

smanek 17y ago
There are different implementations, of course. But the the multiple version problem was worked out over a decade ago.
- deleted 17y ago
  [deleted]
gruseom 17y ago
It's the other way around. For things you've decided to write from scratch, especially if they're hard, CL is a dream. There's nothing comparable in my experience [~]: you can build unbelievably powerful and concise programs easily, then just as easily transform them in major ways. It's the ultimate combination of malleability and expressive power. (Oh, and performance, when you need it.) I can think of only three reasons to prefer Ruby over it: (1) you want Rails etc.; (2) you don't like Lisp; (3) you really like Ruby. The latter two are matters of taste, which leaves #1. [~] I haven't tried Clojure yet. Then again, I don't need to: the JVM is a drawback for me, and I don't need all those Java libraries right now.

akkartik 17y ago

No mention of mzscheme?!

herdrick 17y ago

I like the choice. I love to see people using lisps but some of the path to that choice makes me cringe. Like this: I probably could have written a ‘bare-bones’ implementation of the site’s back-end in Rails in a week instead of two weeks, but I would rather ‘waste’ that one week up front to have more flexibility later. YAGNI. Speed kills (the competition).

gruseom 17y ago
It was the previous sentence that resonated with me: I’m a bit wary of using ‘heavy weight’ frameworks like Rails (or Django) on large custom projects. In my experience they make the first 90% of what I’m trying to do be really easy – but then make the last 10% a living hell since I need to modify something the framework never intended me to control. That is a very true statement, in my experience. It's also an elusive insight, because once you commit to a framework you begin to interpret your problem, the universe, and everything in terms of the framework's conceptual space.

helium 17y ago

Ok, so Rails is too heavy weight for you? What about Merb? Or Sinatra, which was designed for just that reason? Dismissing Ruby as language because only one of a dozen available web frameworks is too "heavy-weight"sounds a bit short-sighted to me.

seiji 17y ago
Not everybody has time to keep up with the continuous and rapidly increasing micro-speciation of all ruby projects from the VMs down to template formats.
- jamesbritt 17y ago
  " ...continuous and rapidly increasing micro-speciation of all ruby projects from the VMs down to template formats." This is simply bullshit. There are merely a fair number of solid choices, as one would hope any decent language would offer. Being able to select appropriate tools for basic Web framework, ORM,and template language is a Good Thing. There are only a handful of viable VMs, and for Web development the practical choices are MRI, REE, and JRuby. "micro-speciation"? That's hysterical. Literally. If Ruby doesn't interest you, fine. Bogus characterizations don't help anyone though.
Tichy 17y ago
I am not the OP, but I have a problem with Rails and alternatives: there does not seem to be a viable alternative to ActiveRecord around? I looked into DataMapper (I think that was it's name) which initially looked great. But then I could not find any information on how to use transactions, and also received no answers on the newsgroup. I guess I should have read the source, but I gave up at that point. Since I hate ActiveRecord, I have now subconsciously decided that Rails probably isn't for me. It's great that Sinatra is spawning lots of imitators for other languages, though.
- steveklabnik 17y ago
  Just so you know, one of the stated goals of Rails 3 is to make it super-easy to swap in your own ORM layer, DataMapper included.

IgorPartola 17y ago

I forget where this came from but this applies 100%: You ask a person to rate 5 paintings 1-5. You then tell them that you have spare paintings that they happened to have rated as 3 and 4 and ask them if they would like one. They will take a "4". Come back in a week and ask them to rate the paintings again. The "4" becomes a "5" and the "3" becomes a "2". Our brains will rationalize decisions to make us happy (whatever that means).

revorad 17y ago
You may be referring to this TED talk by Dan Gilbert - http://www.ted.com/talks/lang/eng/dan_gilbert_asks_why_are_we_happy.html http://www.ted.com/talks/lang/eng/dan_gilbert_asks_why_are_w...

vseloved 17y ago

It's funny, that the decision to choose Common Lisp for some project often has to defend itself (because of so much FUD around the language). CL has one killer feature, that will appeal to any mature developer: it's the only production-ready language around, in which you are virtually not constrained by any third-party design decision. That's it. Discussing other details just boils down to the question of tastes (that differ).

pwnstigator 17y ago
What about Clojure, circa now? Do you not consider it "production ready"? I've used both CL and Clojure. I'd prefer either over Blub, but Clojure wins in my opinion.
- vseloved 17y ago
  Clojure is nice, although I didn't use it in production. But it is Java deep underneath (so not completely homoiconic) and comparing to CL there are restrictions, albeit mild. For some specific tasks I might have preferred it (until CL catches up with concurrency support). But in general CL gives you more freedom.

natmaster 17y ago

"The completely broken implementation of closures." That's what the nonlocal keyword is for.

silentbicycle 17y ago
When the fundamental design of the language is stacked against your style of working, the cognitive dissonance of using so many workarounds is a serious problem. I just switched to Lua during the Python 2 to 3 transition, and I haven't looked back. Real closures and tail calls, no GIL, a vastly more tasteful design, a dead-simple C API, and the whole shebang is a tenth the size of Python.
- silentbicycle 17y ago
  Meant to add: Lua's not perfect, but its whole design favors embedding it in another language, so it's a hell of a lot better about gracefully getting out of the way when it's better to write the parts that matter in other languages.

pge 17y ago

I appreciated the article, but I didn't find the reasoning compelling. What I took away from the article was the following line of reasoning: 1. You strongly prefer functional programming to OO programming (no reason given other than personal preference) 2. You don't have enough experience with any languages that are well-suited to FP other than CL that you would be comfortable undertaking a large project 3. You confirmed that many languages you are familiar with (Java, Python, Ruby) are better suited to OO than FP style (not a surprise at all except perhaps Python as I touch on below), so as a direct consequence of (1) and (2), you were left with CL. CL is my first-choice language for certain kinds of projects as well, but I didn't come away any new insights from your article. Your choice seems to hinge around two key conclusions for which you give very little explanation: FP is better than OO for your project, and Python is not a good language for FP. I single out Python because I don't think anyone would argue that Java or Ruby is well-suited to FP. Guido insists that Python is not an FP language, but many users (myself included) would argue that it is reasonably well-suited for FP. That FP was better for your project does not require any further explanation. As others have noted, for a project where scalability of the development team is not an issue (i.e. you are the primary or only developer), it makes sense to choose the style and language you are most comfortable with. If you were starting a project that was going to require several years and a large number of developers, I would have found it more surprising if you committed to a functional programming model. But, given the nature of the project, it didn't seem like there was any need to defend that decision further. All in all, I'm very excited to see CL being used for more commercial web apps, but I didn't find any insight in the thought process you laid out. I am, however, much more interested in Pt. II - your description of rolling out a web app in CL. I learned CL ten years before I learned Python and would prefer to write in CL when possible, but for web apps, I have turned to Python for a number of reasons, most notably the obvious availability of libraries. I would love to hear what you used and hear a cogent argument that rapid commercial web app deployment can be done in CL as easily as in Python. Thanks for sharing your story.