5 ms·

Surely if lisp makes a programmer N times more efficient then it would be easy to study scientifically. Have there been any studies comparing productivity? Why

by trapper 17y ago

Surely if lisp makes a programmer N times more efficient then it would be easy to study scientifically. Have there been any studies comparing productivity? Why is it that in RAD application competitions, "mainstream" languages often win? Also, if it were truly n times better the library disparity wouldn't exist; libraries should be n times faster to make or duplicate. I'm asking for evidence here: how can something be so much better and have such a paucity of evidence for it's efficacy? If there really is such scientific evidence for lisps effectiveness I would love to read it & and distribute it to my colleagues in charge of large development teams.

randallsquared 17y ago
libraries should be n times faster to make or duplicate Oh, they often are... and that's why nearly everyone just builds their own. Ease of building it yourself compared to learning someone else's system is a major reason Lisp is languishing, in my opinion.
- xpaulbettsx 17y ago
  I agree - I really think that when it comes down to it, Lisp isn't popular because it's harder to read. Even if it's more terse and far more elegant, the mental tax of trying to understand another person's code is far higher than C, where there's only "the dumb way" to do something. If people can't understand the code, they can't contribute, so you don't end up with the network effect of F/OSS contributions that make a language popular (note that I say popular, not better)
  - nocman 17y ago
    As a person who started learning Lisp after years and years of using mostly mainstream languages, I have to at least partially disagree. I will say this, however: Lisp is, at first, difficult to read for many people. But I think it is mainly difficult to read for people who are used to reading programs written in languages with Algol-like syntax (C, C++, Java, Perl, etc). If you've been writing programs for 5, 10, or 15 or more years mostly in languages that all have that common ancestry, it is understandable that your brain will (at first) work against you when reading something like Lisp. However, after using it for a reasonable period of time, I found that it became much easier to read. And now I don't find it any more difficult to read than any of the mainstream languages that I have used in the past. I think that, in fairness to Lisp, if you take two people who have never programmed in their life, and if each of the two has a reasonable and approximately equal amount of skill and potential for being a decent developer, and taking those two individuals, you teach one to develop in Lisp, and one in, say Java. Given that situation, I don't think the Lisp developer will find his code more difficult to read really at any stage of his learning (more difficult that is, than the Java developer finds his code to read at a similar stage of learning). It seems to me, from every testimony I've read from developers who have come to Lisp after extensive careers using other more popular languages, that there is a "breaking in" period where it takes some time to get used to things. However, anyone I've read about who put in any reasonable effort to learn a Lisp (or a language related to Lisp, like Scheme) seems to repeat what others like him or her have said -- you get used to the syntax after a while, and then at the very least it no longer bothers you or gets in your way.
    - joe_the_user 17y ago
      I program in C++/etc but also in SQL. SQL is terse logical language. Everyone may has SQL but I think given constraints of being a logical language SQL is far more readable than lisp - but I've only read bits of lisp online.
      - jules 17y ago
        I find CSS to be far more readable than SQL.
        silentbicycle 17y ago
        I'm not clear what point you're making. CSS isn't a logical language, and they don't really have enough in common to compare them. (And FWIW, I find SQL clearer than Nahuatl.)
        jules 17y ago
        > CSS isn't a logical language, and they don't really have enough in common to compare them. s/CSS/Lisp/
    - Jach 17y ago
      http://img264.imageshack.us/img264/1397/lispnd7.png http://img264.imageshack.us/img264/1397/lispnd7.png
  - 10ren 17y ago
    Maybe that generalizes, to say that it's the code that lisp doesn't turn into "Fungible Cogs", and so it can't be used by others in a standardized way. Standardization seems to be anathema to the individuals who are attracted to lisp's power and flexibility. If lisp ever achieved code fungibility, lispers would hate it.
    - 10ren 17y ago
      6. And the LORD said, Behold, the people is one, and they have all one language; and this they begin to do: and now nothing will be restrained from them, which they have imagined to do. 7. Go to, let us go down, and there confound their language, that they may not understand one another's speech. Genesis 11:6-7 http://www.omniglot.com/babel/english.htm#kj http://www.omniglot.com/babel/english.htm#kj
      - camccann 17y ago
        And here I thought that was talking about perl...
  - gaius 17y ago
    Lisp isn't popular because it's harder to read. Even if it's more terse and far more elegant, the mental tax of trying to understand another person's code is far higher than C I am deeply suspicious of explanations that boil down to, Lisp isn't popular because the average programmer isn't smart enough. Let me tell you, anyone smart enough to use C++ in anger is smart enough to use any language. The real world is highly competitive, and organizations that think they can win by using niche languages, do. There's APL at Morgan Stanley, Ocaml at Jane Street Capital, K at Lehman's (OK, maybe that's not so good an example), F#, Haskell, blah blah. Lisp isn't even on the radar, anywhere. What does that tell you?
    - yummyfajitas 17y ago
      Mathematica is widely used in the financial sector. Don't let the square brackets, prefix notation and fugly symbolic shortcuts fool you: Mathematica is lisp.
      - gaius 17y ago
        Well, yes, you could argue that Python is Lisp too, but my point stands: if Lisp, as in, CL, really was a huge productivity boost, organizations that have demonstrated their willingness to adopt niche languages where well-suited to their problem domain would have adopted it, but that isn't actually happening. If anything Haskell and APL are nichier (if that's even a word). The big Lisp success stories all start from the premise of a bunch of programmers who are already hugely experienced in Lisp happening to do this project in it (e.g. Viaweb, Orbitz). But when people with 20 years experience in C complete a project successfully, it isn't blogworthy news...
        yummyfajitas 17y ago
        Python is a primarily imperative/OO language. Good python code consists of writing loops and building objects. Syntactic abstraction consists primarily of looking up strings in obj.__dict__. Whenever more complicated metaprogramming is needed, strings are parsed. "obj__parent__name_eq"=foo is django for obj.parent.name == foo. Mathematica is a primarily functional language. The primary abstraction method is patterns (macros), which allow you to modify/build up the parse tree before evaluating it. http://reference.wolfram.com/mathematica/guide/Patterns.html http://reference.wolfram.com/mathematica/guide/Patterns.html I've programmed Python and Lisp, and have begun writing some projects of reasonable size in Mathematica. There are only a couple of major differences between Mathematica and lisp. One is syntax: f[a,b,c] vs (f a b c), as well as some infix syntactic sugar. The other is that arrays are used far more widely in Mathematica than in lisp (but that may be due to the problem domain of mathematica).
  - jacquesm 17y ago
    I think a bigger factor is the amount of prior knowledge required before you can be productive. In C (and c-like languages) that barrier is relatively low, a handful of concepts and a very limited subset of library modules and you can go a long long way.
    - gaius 17y ago
      Indeed. Lisp programmers with 20 years experience are more productive than Java programmers with 2 years experience. That doesn't say anything about Lisp vs Java (or, not nearly as much as some would claim it does).
- joe_the_user 17y ago
  Ease of building it yourself compared to learning someone else's system is a major reason Lisp is languishing, in my opinion. Oh, that why Perl is languishing too. Well, the problem is how hard it is to learn someone else's system in Perl. If Lisp has that problem, well, there you are.
  - randallsquared 17y ago
    I don't think Lisp has that problem. Rather, it has the "problem" that it's easier to build the 20% you need of that library than to find the library, understand enough to do the bit you need in the face of that inevitable impedance mismatch between your abstractions and the library author's, figure out how including it impacts the rest of your system including the dependencies... In other languages, this is just something everyone has to live with, and because of that, libraries face strong selection pressure to be one or more of simple, fast, standard, etc. There is no particular pressure like that in Lisp, since there's a much smaller domain where it's easy to understand why you need a thing, and troublesome to build it. To a much greater degree than mainstream languages, then, there are almost as many "language + standard library" sets as there are Lisp developers. You can see this same effect at work in Python web frameworks, I think.
  - gnosis 17y ago
    Perl is definitely not languishing. It just gets a bad rap from fans of other languages that are competing with it (particularly from the fans of Python and Ruby, the two upstarts trying to take its place). Let's look at the number of jobs advertised for Perl, Python, Ruby, and Lisp over the last 5 years: http://www.indeed.com/jobtrends?q=perl%2Cpython%2Cruby%2Clisp&l= http://www.indeed.com/jobtrends?q=perl%2Cpython%2Cruby%2Clis... The number of Perl jobs is more than those of Python and Ruby put together. Lisp has so few jobs it doesn't even really show up on the chart. Then there's Perl's CPAN archive, with over 17,000 modules, with about 30 modules updated or added per day. That much development is a sign of an active, alive community. So the rumors of Perl's demise are greatly exaggerated.
- silentbicycle 17y ago
  Oddly, it can also be a downside. A language that makes reinventing the wheel too easy means that people can spend more time saying, "Psh, I could do that in a couple hours" than sharing. Then, when people want to know what the library is to use for parsing HTTP headers (or whatever), everybody and their nephew shows up to suggest their own half-assed framework. It doesn't inspire confidence. Take that to its logical extent, and ... wait, which Lisp implementation do you use again? Mine only works with SBCL, crap, nevermind. When somebody has a really thoroughly debugged C library, it can be like huddling everyone together by a fire for warmth in the tundra. If you don't share, man, that's just cruel.
- trapper 17y ago
  This is a testable hypothesis due to the number of popular open source projects around we can draw from.
- rbanffy 17y ago
  Ease of building new stuff brings fragmentation: http://news.ycombinator.com/item?id=1013716 http://news.ycombinator.com/item?id=1013716
YuriNiyazov 17y ago
There has never been a rigorous scientific study done, no, but there's plenty of anecdotal evidence, and, to quote Paul Krugman's tongue-in-cheek title, "the plural of anecdote is data"
- trapper 17y ago
  That's the same explanation psychics give. If the effect size is large (as is claimed), it should be an extremely simple study. So why has no one done it? Why don't the lispers organize a RAD competition between lisp and all-comers, surely if lisp is that much more productive you would see the top ten places being lisp dialects. Would you agree?
  - camccann 17y ago
    A competition to solve a toy problem wouldn't necessarily say much, though. So the question is, how big of a problem would you need, and where are you going to find enough programmers (of reasonable and roughly similar competence) who don't have anything better to do than spend multiple hours participating in a glorified programming language pissing match? In other words, the reason why not is probably "because organizing things is hard work".
    - trapper 17y ago
      If the effect size is large (i.e. the magnitude of difference between the means of lisp vs non-lisp programmers) the size of problem should be irrelevant. Many thousands of programmers spend hours each week in pissing matches online. Why is it none of them feel like truly putting their money where their mouth is?
      - camccann 17y ago
        If the effect size is large (i.e. the magnitude of difference between the means of lisp vs non-lisp programmers) the size of problem should be irrelevant. Writing a "Hello, world" is going to take about the same amount of time in any language that isn't a turing tarpit. At issue is generally some sort of asymptotic advantage, not a guarantee of "language X will always be superior every time always". In fact, for any pair of one easy and one hard problem, I'm pretty confident I could find (or invent, if necessary) two languages that each beat the other for solving one of the problems. The claimed benefits of a more expressive language generally involve things like better management of application complexity, or greater ability to modify and restructure late in the project. Hammering out a program of a scale such that the programmer can keep an accurate model of the entire thing in mind at one time, and that is only written and never modified, would be more a measure of the programmer's typing speed and the sophistication of the IDE used than anything else.
jcapote 17y ago
Paul Graham writes about this extensively in http://www.paulgraham.com/avg.html http://www.paulgraham.com/avg.html
- trapper 17y ago
  It's computer science not computer woo. I'm asking for evidence not anecdote :) Surely, there has to be at least some evidence in it's 50 years? Especially given the claimed huge effect size.
  - sjs 17y ago
    It's simply impossible to isolate and control all of the important variables involved in developing any software of any significant size. There's a reason for the lack of productivity data for programming in general.
    - Herring 17y ago
      That sounds like a lisp proponent's problem, quoting stats he can't back up.
      - sjs 17y ago
        There are a dozen reasons the data would be bogus but I only need to state the most obvious: you can't use the same person in 2 comparable tests because the tasks would be either too trivial or too similar; likewise you cannot use different people in groups for the tests because people are too different. Smart and dumb people have succeeded with all kinds of languages on all kinds of platforms that are widely viewed as amazing and terrible and everywhere in between, on all kinds of projects.
    - trapper 17y ago
      One impossible study: http://news.ycombinator.com/item?id=1016762 http://news.ycombinator.com/item?id=1016762
10ren 17y ago
Perhaps one source of data is YC startups. What's the success rate of lisp vs. non-lisp? Yes, of course, many things contribute to success other than language choice. That's my point.
ams6110 17y ago
Maybe it's not that lisp makes any programmer more productive, but that it offers the anecdotal "order of magnitude more productive" programmer features that Java, being designed for the "interchangeable cog" programmer, does not.
- deleted 17y ago
  [deleted]
naveensundar 17y ago
trapper, this study might be in the flavor of what you are seeking. http://www.flownet.com/gat/papers/lisp-java.pdf http://www.flownet.com/gat/papers/lisp-java.pdf (2000)
asciilifeform 17y ago
> Surely if lisp makes a programmer N times more efficient then it would be easy to study scientifically. Have there been any studies comparing productivity? I began to write a reply to your question, but decided that it deserved its own post: http://www.loper-os.org/?p=85 http://www.loper-os.org/?p=85
- trapper 17y ago
  I refer you to: http://news.ycombinator.com/item?id=1016762 http://news.ycombinator.com/item?id=1016762 What's wrong with that as evidence for lisps superiority?
  - jules 17y ago
    What's wrong is that they measured the performance of Lisp programmers vs Java/C++ programmers. This is fine but you're not comparing the merits of the programming language. The Lisp programmers would win even if they were forced to program in Java.
- plinkplonk 17y ago
  "I began to write a reply to your question, but decided that it deserved its own post" That isn't really a reply to the question posed. Just more ranting and raving. Sample: "Yes, my dear bean counters, you can measure productivity. I would no more ask you to stop in your attempts at its measurement than I would ask mosquitoes to stop sucking blood. You can measure productivity – even of Lisp; even of political philosophies. You will simply need to secure a very large bag of beans – one deep enough to hold a bean for every twist and turn of a century of tinkering, politicking, and everything else associated with the messy business of successfully thinking new thoughts." The best interpretation of this drivel is that you want people to accept your claims of superior productivity just on your say so but not ask for any evidence because asking you back up your unsubstantiated claims of superiority would be "bean counting". Forget the scientific studies. There is another kind of "evidence" in programming - write code that is evidently superior. The best way to show that (say) Linux is an "inferior" operating system is to build one better. Not talk about making one sometime in the misty future. Build. Then talk. Endless rants aren't as good. As silentbicycle says elsewhere "I just take issue with how some particularly vocal Lispers have been sitting on the sidelines going, "Our language is THE BEST LANGUAGE" for decades. ("Sick of being a blub programmer and working in Blub? Try our Language For Smart People.") They're so used to being smug because their language was thirty years ahead of its time, fifty years ago."
pg 17y ago
Curiously, it's inherently impossible to compare what I think matters most for languages: solving new problems. To compare two languages you have to have people solve the same problem, and that means it has to be a predefined one.
- trapper 17y ago
  This is what statistics are for, and is exactly the same situation that many other disciplines find themselves in where treatment a influences treatment b.
  - bOR_ 17y ago
    Not the easiests of problems though. Do you know what statistics you would apply? IQ measurements seem to have a similar problem: doing test a, influences your score on test b, because you get more used to the problems presented.. but doing test a and b just marginally increases your general intelligence, which is what you are trying to measure, while it might have a big influence on someone's score in these tests. With IQ tests, you can expect that doing test c till z will eventually saturate someone.. but that might only work because you generally do not get the answers. In coding, you do get the answer back (whether your code compiles/works or not), so eventually both programmers will just be limited by the speed with which they type.
    - trapper 17y ago
      >Not the easiests of problems though. Do you know what statistics you would apply? There are many ways. Off the top of my head: - to focus on speed of development you could use a practice or warm-up > provide each with a practice in another language > provide a sample solution in pseudocode in both functional and imperative style - To focus on speed of problem solving you could use topcoder data + research of participants programming language backgrounds. The important thing is that if the effect is large it's easy to spot. According to some, the difference is an order of magnitude. If that's true, it will be trivial to spot.
- gruseom 17y ago

by trapper 17y ago

randallsquared 17y ago

libraries should be n times faster to make or duplicate Oh, they often are... and that's why nearly everyone just builds their own. Ease of building it yourself compared to learning someone else's system is a major reason Lisp is languishing, in my opinion.

xpaulbettsx 17y ago
I agree - I really think that when it comes down to it, Lisp isn't popular because it's harder to read. Even if it's more terse and far more elegant, the mental tax of trying to understand another person's code is far higher than C, where there's only "the dumb way" to do something. If people can't understand the code, they can't contribute, so you don't end up with the network effect of F/OSS contributions that make a language popular (note that I say popular, not better)
- nocman 17y ago
  As a person who started learning Lisp after years and years of using mostly mainstream languages, I have to at least partially disagree. I will say this, however: Lisp is, at first, difficult to read for many people. But I think it is mainly difficult to read for people who are used to reading programs written in languages with Algol-like syntax (C, C++, Java, Perl, etc). If you've been writing programs for 5, 10, or 15 or more years mostly in languages that all have that common ancestry, it is understandable that your brain will (at first) work against you when reading something like Lisp. However, after using it for a reasonable period of time, I found that it became much easier to read. And now I don't find it any more difficult to read than any of the mainstream languages that I have used in the past. I think that, in fairness to Lisp, if you take two people who have never programmed in their life, and if each of the two has a reasonable and approximately equal amount of skill and potential for being a decent developer, and taking those two individuals, you teach one to develop in Lisp, and one in, say Java. Given that situation, I don't think the Lisp developer will find his code more difficult to read really at any stage of his learning (more difficult that is, than the Java developer finds his code to read at a similar stage of learning). It seems to me, from every testimony I've read from developers who have come to Lisp after extensive careers using other more popular languages, that there is a "breaking in" period where it takes some time to get used to things. However, anyone I've read about who put in any reasonable effort to learn a Lisp (or a language related to Lisp, like Scheme) seems to repeat what others like him or her have said -- you get used to the syntax after a while, and then at the very least it no longer bothers you or gets in your way.
  - joe_the_user 17y ago
    I program in C++/etc but also in SQL. SQL is terse logical language. Everyone may has SQL but I think given constraints of being a logical language SQL is far more readable than lisp - but I've only read bits of lisp online.
    - jules 17y ago
      I find CSS to be far more readable than SQL.
      - silentbicycle 17y ago
        I'm not clear what point you're making. CSS isn't a logical language, and they don't really have enough in common to compare them. (And FWIW, I find SQL clearer than Nahuatl.)
        jules 17y ago
        > CSS isn't a logical language, and they don't really have enough in common to compare them. s/CSS/Lisp/
  - Jach 17y ago
    http://img264.imageshack.us/img264/1397/lispnd7.png http://img264.imageshack.us/img264/1397/lispnd7.png
- 10ren 17y ago
  Maybe that generalizes, to say that it's the code that lisp doesn't turn into "Fungible Cogs", and so it can't be used by others in a standardized way. Standardization seems to be anathema to the individuals who are attracted to lisp's power and flexibility. If lisp ever achieved code fungibility, lispers would hate it.
  - 10ren 17y ago
    6. And the LORD said, Behold, the people is one, and they have all one language; and this they begin to do: and now nothing will be restrained from them, which they have imagined to do. 7. Go to, let us go down, and there confound their language, that they may not understand one another's speech. Genesis 11:6-7 http://www.omniglot.com/babel/english.htm#kj http://www.omniglot.com/babel/english.htm#kj
    - camccann 17y ago
      And here I thought that was talking about perl...
- gaius 17y ago
  Lisp isn't popular because it's harder to read. Even if it's more terse and far more elegant, the mental tax of trying to understand another person's code is far higher than C I am deeply suspicious of explanations that boil down to, Lisp isn't popular because the average programmer isn't smart enough. Let me tell you, anyone smart enough to use C++ in anger is smart enough to use any language. The real world is highly competitive, and organizations that think they can win by using niche languages, do. There's APL at Morgan Stanley, Ocaml at Jane Street Capital, K at Lehman's (OK, maybe that's not so good an example), F#, Haskell, blah blah. Lisp isn't even on the radar, anywhere. What does that tell you?
  - yummyfajitas 17y ago
    Mathematica is widely used in the financial sector. Don't let the square brackets, prefix notation and fugly symbolic shortcuts fool you: Mathematica is lisp.
    - gaius 17y ago
      Well, yes, you could argue that Python is Lisp too, but my point stands: if Lisp, as in, CL, really was a huge productivity boost, organizations that have demonstrated their willingness to adopt niche languages where well-suited to their problem domain would have adopted it, but that isn't actually happening. If anything Haskell and APL are nichier (if that's even a word). The big Lisp success stories all start from the premise of a bunch of programmers who are already hugely experienced in Lisp happening to do this project in it (e.g. Viaweb, Orbitz). But when people with 20 years experience in C complete a project successfully, it isn't blogworthy news...
      - yummyfajitas 17y ago
        Python is a primarily imperative/OO language. Good python code consists of writing loops and building objects. Syntactic abstraction consists primarily of looking up strings in obj.__dict__. Whenever more complicated metaprogramming is needed, strings are parsed. "obj__parent__name_eq"=foo is django for obj.parent.name == foo. Mathematica is a primarily functional language. The primary abstraction method is patterns (macros), which allow you to modify/build up the parse tree before evaluating it. http://reference.wolfram.com/mathematica/guide/Patterns.html http://reference.wolfram.com/mathematica/guide/Patterns.html I've programmed Python and Lisp, and have begun writing some projects of reasonable size in Mathematica. There are only a couple of major differences between Mathematica and lisp. One is syntax: f[a,b,c] vs (f a b c), as well as some infix syntactic sugar. The other is that arrays are used far more widely in Mathematica than in lisp (but that may be due to the problem domain of mathematica).
- jacquesm 17y ago
  I think a bigger factor is the amount of prior knowledge required before you can be productive. In C (and c-like languages) that barrier is relatively low, a handful of concepts and a very limited subset of library modules and you can go a long long way.
  - gaius 17y ago
    Indeed. Lisp programmers with 20 years experience are more productive than Java programmers with 2 years experience. That doesn't say anything about Lisp vs Java (or, not nearly as much as some would claim it does).
joe_the_user 17y ago
Ease of building it yourself compared to learning someone else's system is a major reason Lisp is languishing, in my opinion. Oh, that why Perl is languishing too. Well, the problem is how hard it is to learn someone else's system in Perl. If Lisp has that problem, well, there you are.
- randallsquared 17y ago
  I don't think Lisp has that problem. Rather, it has the "problem" that it's easier to build the 20% you need of that library than to find the library, understand enough to do the bit you need in the face of that inevitable impedance mismatch between your abstractions and the library author's, figure out how including it impacts the rest of your system including the dependencies... In other languages, this is just something everyone has to live with, and because of that, libraries face strong selection pressure to be one or more of simple, fast, standard, etc. There is no particular pressure like that in Lisp, since there's a much smaller domain where it's easy to understand why you need a thing, and troublesome to build it. To a much greater degree than mainstream languages, then, there are almost as many "language + standard library" sets as there are Lisp developers. You can see this same effect at work in Python web frameworks, I think.
- gnosis 17y ago
  Perl is definitely not languishing. It just gets a bad rap from fans of other languages that are competing with it (particularly from the fans of Python and Ruby, the two upstarts trying to take its place). Let's look at the number of jobs advertised for Perl, Python, Ruby, and Lisp over the last 5 years: http://www.indeed.com/jobtrends?q=perl%2Cpython%2Cruby%2Clisp&l= http://www.indeed.com/jobtrends?q=perl%2Cpython%2Cruby%2Clis... The number of Perl jobs is more than those of Python and Ruby put together. Lisp has so few jobs it doesn't even really show up on the chart. Then there's Perl's CPAN archive, with over 17,000 modules, with about 30 modules updated or added per day. That much development is a sign of an active, alive community. So the rumors of Perl's demise are greatly exaggerated.
silentbicycle 17y ago
Oddly, it can also be a downside. A language that makes reinventing the wheel too easy means that people can spend more time saying, "Psh, I could do that in a couple hours" than sharing. Then, when people want to know what the library is to use for parsing HTTP headers (or whatever), everybody and their nephew shows up to suggest their own half-assed framework. It doesn't inspire confidence. Take that to its logical extent, and ... wait, which Lisp implementation do you use again? Mine only works with SBCL, crap, nevermind. When somebody has a really thoroughly debugged C library, it can be like huddling everyone together by a fire for warmth in the tundra. If you don't share, man, that's just cruel.
trapper 17y ago
This is a testable hypothesis due to the number of popular open source projects around we can draw from.
rbanffy 17y ago
Ease of building new stuff brings fragmentation: http://news.ycombinator.com/item?id=1013716 http://news.ycombinator.com/item?id=1013716

YuriNiyazov 17y ago

There has never been a rigorous scientific study done, no, but there's plenty of anecdotal evidence, and, to quote Paul Krugman's tongue-in-cheek title, "the plural of anecdote is data"

trapper 17y ago
That's the same explanation psychics give. If the effect size is large (as is claimed), it should be an extremely simple study. So why has no one done it? Why don't the lispers organize a RAD competition between lisp and all-comers, surely if lisp is that much more productive you would see the top ten places being lisp dialects. Would you agree?
- camccann 17y ago
  A competition to solve a toy problem wouldn't necessarily say much, though. So the question is, how big of a problem would you need, and where are you going to find enough programmers (of reasonable and roughly similar competence) who don't have anything better to do than spend multiple hours participating in a glorified programming language pissing match? In other words, the reason why not is probably "because organizing things is hard work".
  - trapper 17y ago
    If the effect size is large (i.e. the magnitude of difference between the means of lisp vs non-lisp programmers) the size of problem should be irrelevant. Many thousands of programmers spend hours each week in pissing matches online. Why is it none of them feel like truly putting their money where their mouth is?
    - camccann 17y ago
      If the effect size is large (i.e. the magnitude of difference between the means of lisp vs non-lisp programmers) the size of problem should be irrelevant. Writing a "Hello, world" is going to take about the same amount of time in any language that isn't a turing tarpit. At issue is generally some sort of asymptotic advantage, not a guarantee of "language X will always be superior every time always". In fact, for any pair of one easy and one hard problem, I'm pretty confident I could find (or invent, if necessary) two languages that each beat the other for solving one of the problems. The claimed benefits of a more expressive language generally involve things like better management of application complexity, or greater ability to modify and restructure late in the project. Hammering out a program of a scale such that the programmer can keep an accurate model of the entire thing in mind at one time, and that is only written and never modified, would be more a measure of the programmer's typing speed and the sophistication of the IDE used than anything else.

jcapote 17y ago

Paul Graham writes about this extensively in http://www.paulgraham.com/avg.html http://www.paulgraham.com/avg.html

trapper 17y ago
It's computer science not computer woo. I'm asking for evidence not anecdote :) Surely, there has to be at least some evidence in it's 50 years? Especially given the claimed huge effect size.
- sjs 17y ago
  It's simply impossible to isolate and control all of the important variables involved in developing any software of any significant size. There's a reason for the lack of productivity data for programming in general.
  - Herring 17y ago
    That sounds like a lisp proponent's problem, quoting stats he can't back up.
    - sjs 17y ago
      There are a dozen reasons the data would be bogus but I only need to state the most obvious: you can't use the same person in 2 comparable tests because the tasks would be either too trivial or too similar; likewise you cannot use different people in groups for the tests because people are too different. Smart and dumb people have succeeded with all kinds of languages on all kinds of platforms that are widely viewed as amazing and terrible and everywhere in between, on all kinds of projects.
  - trapper 17y ago
    One impossible study: http://news.ycombinator.com/item?id=1016762 http://news.ycombinator.com/item?id=1016762

10ren 17y ago

Perhaps one source of data is YC startups. What's the success rate of lisp vs. non-lisp? Yes, of course, many things contribute to success other than language choice. That's my point.

ams6110 17y ago

Maybe it's not that lisp makes any programmer more productive, but that it offers the anecdotal "order of magnitude more productive" programmer features that Java, being designed for the "interchangeable cog" programmer, does not.

deleted 17y ago
[deleted]

naveensundar 17y ago

trapper, this study might be in the flavor of what you are seeking. http://www.flownet.com/gat/papers/lisp-java.pdf http://www.flownet.com/gat/papers/lisp-java.pdf (2000)

asciilifeform 17y ago

> Surely if lisp makes a programmer N times more efficient then it would be easy to study scientifically. Have there been any studies comparing productivity? I began to write a reply to your question, but decided that it deserved its own post: http://www.loper-os.org/?p=85 http://www.loper-os.org/?p=85

trapper 17y ago
I refer you to: http://news.ycombinator.com/item?id=1016762 http://news.ycombinator.com/item?id=1016762 What's wrong with that as evidence for lisps superiority?
- jules 17y ago
  What's wrong is that they measured the performance of Lisp programmers vs Java/C++ programmers. This is fine but you're not comparing the merits of the programming language. The Lisp programmers would win even if they were forced to program in Java.
plinkplonk 17y ago
"I began to write a reply to your question, but decided that it deserved its own post" That isn't really a reply to the question posed. Just more ranting and raving. Sample: "Yes, my dear bean counters, you can measure productivity. I would no more ask you to stop in your attempts at its measurement than I would ask mosquitoes to stop sucking blood. You can measure productivity – even of Lisp; even of political philosophies. You will simply need to secure a very large bag of beans – one deep enough to hold a bean for every twist and turn of a century of tinkering, politicking, and everything else associated with the messy business of successfully thinking new thoughts." The best interpretation of this drivel is that you want people to accept your claims of superior productivity just on your say so but not ask for any evidence because asking you back up your unsubstantiated claims of superiority would be "bean counting". Forget the scientific studies. There is another kind of "evidence" in programming - write code that is evidently superior. The best way to show that (say) Linux is an "inferior" operating system is to build one better. Not talk about making one sometime in the misty future. Build. Then talk. Endless rants aren't as good. As silentbicycle says elsewhere "I just take issue with how some particularly vocal Lispers have been sitting on the sidelines going, "Our language is THE BEST LANGUAGE" for decades. ("Sick of being a blub programmer and working in Blub? Try our Language For Smart People.") They're so used to being smug because their language was thirty years ahead of its time, fifty years ago."

pg 17y ago

Curiously, it's inherently impossible to compare what I think matters most for languages: solving new problems. To compare two languages you have to have people solve the same problem, and that means it has to be a predefined one.

trapper 17y ago
This is what statistics are for, and is exactly the same situation that many other disciplines find themselves in where treatment a influences treatment b.
- bOR_ 17y ago
  Not the easiests of problems though. Do you know what statistics you would apply? IQ measurements seem to have a similar problem: doing test a, influences your score on test b, because you get more used to the problems presented.. but doing test a and b just marginally increases your general intelligence, which is what you are trying to measure, while it might have a big influence on someone's score in these tests. With IQ tests, you can expect that doing test c till z will eventually saturate someone.. but that might only work because you generally do not get the answers. In coding, you do get the answer back (whether your code compiles/works or not), so eventually both programmers will just be limited by the speed with which they type.
  - trapper 17y ago
    >Not the easiests of problems though. Do you know what statistics you would apply? There are many ways. Off the top of my head: - to focus on speed of development you could use a practice or warm-up > provide each with a practice in another language > provide a sample solution in pseudocode in both functional and imperative style - To focus on speed of problem solving you could use topcoder data + research of participants programming language backgrounds. The important thing is that if the effect is large it's easy to spot. According to some, the difference is an order of magnitude. If that's true, it will be trivial to spot.
gruseom 17y ago