8 ms·
Ask HN: Is there evidence regarding PG's theory about powerful languages?
Paul Graham claims that startups should use the most powerful language for a job, like Lisp. I'm looking for statistical evidence for or against this theory.
For example, amongst the thousands of YC startups, have ones that used Lisp or Smalltalk or other esoteric but powerful language succeeded more than startups that used a mainstream language like Python, Java or C++? Do we have statistics of success rate by language?
One should not look at a single startup (ITA software) that used Lisp and succeeded and conclude that it pays for the average startup to use Lisp (or other powerful language like Smalltalk).
- wasdfan 11y agoI can only speak from anecdotal memory, but I'm pretty sure Wit.AI primarily used clojure (arguably one of the most expressive lisps of all time) to go from 0 to acquisition in 18(!) months for what was essentially a pure software product.
- striking 11y agoI can't give you statistical evidence, but it's well known (or often said, at least) that using the right tool for the job makes the odds of success a lot higher. That being said, "right" is very different from pg's "powerful", especially as he says: > Lisp is so great not because of some magic quality visible only to devotees, but because it is simply the most powerful language available. In my opinion (and this person's[1]), Lisp is "powerful" in that it lets you express anything quickly and easily in a way that makes sense to you. That doesn't mean it makes sense to anyone, nor does that mean it's compatible with anything else. And where he says: > What's less often understood is that there is a more general principle here: that if you have a choice of several languages, it is, all other things being equal, a mistake to program in anything but the most powerful one. I feel he is sorely mistaken. The best programming language for the job is the one that most easily and quickly fits the requirements, given your experience level and the intrinsic attributes of the languages in question. That may be less true when you're working on a personal project rather than one for work... but from an engineering standpoint, development speed is very important. PHP may be near-universally reviled as a language, but I won't contest the fact that it runs anywhere, quickly. If you needed to quickly put up a blog at least some time ago, the answer was almost always "use Wordpress and PHP". Python is slow. Painfully slow. Slow as in "it stores the entire AST at runtime and lets you modify it while the program runs" slow. But it's also great at numerical computation with numpy. And it's enormously flexible (which is what makes it slow). And it's clean. It writes really well. For a quick mathematical processing program or website whose code needs to be legible, it's great. This goes on for just about any language. There are reasons to, and not to, use a programming language. It's not just "power" that's important, it's a number of factors. I hate many languages with a passion, but I'll agree that those languages still have their uses. So I disagree with pg here. 1: http://www.lambdassociates.org/blog/bipolar.htm http://www.lambdassociates.org/blog/bipolar.htm
- EuAndreh 11y agoI believe that is what he meant by "all other things being equal". They are equal in library availability, speed, etc.
- tyre 11y agoWhich is literally never the case, so kind of a moot point.
- EuAndreh 11y agoWasn't he just trying to isolate the variables?
- tyre 11y agoYes, but if you control for the variables that make the most difference then you do the hypothetical a disservice. It's like saying that, all response times being equal, it is a fast application server. What does that really tell you?
- DasIch 11y agoPython doesn't store the entire AST at runtime. You can hook into the import process and modify code before it's compiled to bytecode. You can modify the bytecode at runtime but not any that's on the stack. Python is also not all that slow, CPython is slow-ish but it's has effectively no optimizations at all. PyPy which has decent optimizations is actually fairly fast and getting faster with every release.
- Alphasite_ 11y agoZipPy is damn fast from what I've seen.
- stray 11y agoIsn't startup time painful though?
- nl 11y agoRead the date on those essays. You'll note that they were all written prior to YC building momentum. I think it's fair to say they are based on PG's early experiences, but it didn't appear to generalize. You'll note he stopped writing about it, and YC funded plenty of companies that used many, many different languages.
- wasdfan 11y agoThis is base rate neglect. The percentage of people who are competent enough (or crazy enough) to know a LISP well enough to build a product out of it AND be interested in a startup AND applying to YC is quite small in comparison to the number of people slinging code in the more popular languages (who are also interested in applying to YC). Of course YC is funding companies that don't use i.e. LISP/Haskell/whatever is cool. If it only limited itself to the elite/hipster programmer startups (heh), it wouldn't have that many companies to invest in per year.
- kartickv 11y agoI know this comment wasn't addressed to me, but that's why my original question was phrased as: Are Lisp startups more successful, not: Are there more Lisp startups?
- brudgers 11y agoThis was more or less my take. When Viaweb was built in the mid 1990's, Python was not a mainstream language, Java was new, and building web sites in C++ was approximately as efficient as it is today. The scale of the web was different in that Viaweb had a server in their office. It was also the server. By the first batch of YC, Graham had written about Python as an example of a leading edge language that was a good way to filter for "better programmers". Not long after the first batch started, Reddit had ditched Common Lisp for it. Anecdote is not the singular of data, but Whatsapp choosing Erlang is a more recent example of using the right tool for the right job.
- d0m 11y agoI think you should use whatever you're best with. There are so many risks in a startup that you might as well not add another technical risk. PG was a lisp expert so it made sense for him to use it.
- bcjordan 11y agoAaron Swartz wrote in 2005 about Reddit's switch from Lisp to Python, noting some upset reactions in the Lisp community. http://www.aaronsw.com/weblog/rewritingreddit http://www.aaronsw.com/weblog/rewritingreddit > Another figured something else must be going on: “Could this be…a lie? To throw off competition? It’s not as though Paul Graham hasn’t hinted at this tactic in his essays…”
- tyre 11y agoNo, but you can frame it in a way where he was correct. Look at some of the largest tech companies and what languages they used to get big: - Facebook: PHP - Twitter: Ruby - Dropbox: Python - Box: PHP - Github: Ruby - AirBnB: Ruby Probably not what PG would have had in mind. If you move away from his objective claim about Lisp's "power" to a subjective metric based on "the right tool for the job", then sure. (Though reading e.g. Hackers and Painters, his argument around Lisp was specific to the power of that language.) PHP perfectly fits the "move fast and break things" mantra of early Facebook. Ruby allowed for quick iteration at early Twitter. Note that Facebook built a PHP-to-C++ compiler called HipHop, while Twitter switched largely to Scala. Box (to the best of my knowledge) has supplemented PHP with Java and Scala. Github maintained a custom fork of Rails for a long time to achieve the performance it needed. So no, it doesn't appear that Lisp-flavoured companies have an advantage.
- incepted 11y agoAll good points. Even though it's hard to find any flaw in "Use the right tool for the job", I still think that the success of a start up in Silicon Valley (and probably the rest of the world) has a lot less to do with the technical choices of the company than the people who work there. And luck. A lot of luck. It's still baffling to me that pg doesn't realize how lucky he was. He was in the right place at the right time and he was probably surrounded by competent technical and non technical people. If you have such a team, the technical preferences of the engineers will probably have very little bearing on the success of the company as long as they use the tools they are proficient with and which allow them to execute quickly. I mean, the fact that Rails and PHP were at the origin of Twitter and Facebook alone shows that the quality of programming languages is irrelevant for the early year(s) of a start up.
- fleitz 11y agoNote the number of projects being witten in assembler, then try to calculate a rocket trajectory without calculus. More powerful language is the driver behind many advances and is generally considered to be the primary difference between humans and animals.
- ankurdhama 11y agoThere are enormous number of factors that play a role in the success of startups and most of them are more important than the programming language they use. By the way I have never understood how "powerful" can be the property of a programming language? If you have people with good experience in X language then no matter how "powerful" language Y is, it cannot compete with that experience.
- wasdfan 11y agoConsider Assembly as a counter-example.
- AnimalMuppet 11y agoIt's not. (All right, it usually is, but...) What if my definition of "power" is "the ability to write interrupt service routines and OS trap handlers"? That is, there are definitions of "power" where assembly is in fact the most powerful language out there. It all depends on what problem you're trying to solve.
- known 11y agoWhy wasn't the Linux kernel written in C++? https://news.ycombinator.com/item?id=2405387 https://news.ycombinator.com/item?id=2405387
- throwaway_exer 11y agopg wrote that he could out-compete Java shops at the time by using lisp or perl. (If you look at Web 1.0, most of the marketing/event space winners were perl shops.) However, focusing on lisp is pointless because of its low adoption commercially. (If you want me to provide stats for your class homework, sorry.)
- dang 11y ago> If you want me to provide stats for your class homework, sorry That's unduly personal. Please edit stuff like that out of comments here.
- deleted 11y ago[deleted]
- paulsutter 11y agoPaul Graham says that Viaweb was able to run circles around their competitors because Lisp gave them a big advantage[1]. Of course, Facebook ran circles around their competitors. Myspace, friendster, left in the dust. So is PHP more powerful than Lisp? (/s) His logic suggests so (given the relative performance of Viaweb vs Facebook). Having great programmers matters more than using any specific tool. [1] from http://www.paulgraham.com/avg.html http://www.paulgraham.com/avg.html "It must have seemed to our competitors that we had some kind of secret weapon-- that we were decoding their Enigma traffic or something. In fact we did have a secret weapon, but it was simpler than they realized. No one was leaking news of their features to us. We were just able to develop software faster than anyone thought possible." EDIT: Added (/s) since sarcasm was apparently nonobvious
- wasdfan 11y agoYou're engaging in base rate neglect: the pool of PHP programmers was vastly larger than the pool of LISP programmers when Facebook was founded.
- paulsutter 11y agoThere's a funny story about when cofounder Dustin Moskovitz first wanted to help on the site. Choice of language was obviously not such a big deal to them. From http://www.businessinsider.com/how-moskovitz-became-facebook-cofounder-2016-3 http://www.businessinsider.com/how-moskovitz-became-facebook... >Dustin was like, 'Hey, I want to help out. I want to help you do the expansion.' >Zuckerberg told Moskovitz, "That's pretty cool. But you don't know any PHP." >"That weekend he went home, bought the book 'PERL for Dummies,' came back, and was like, 'OK, I'm ready to go,'" Zuck said, and told him, "The site's written in PHP, not PERL, dude."
- crdb 11y agoI'll paraphrase Facebook themselves [1] back at the statement of yours that I disagree with ("His logic suggests so"): (A => B) !=> (B => A). That Viaweb was competitive because of blub, does not imply that blub is a necessary condition for any company to be competitive, or that not-blub would impede Facebook from becoming successful. For example, if fast new features are only possible with blub, and new features are not as essential to Facebook's success as they were to Viaweb's, Facebook's use of not-blub (or perhaps, non-use of blub) would not materially impact Facebook's success. One way in which this might happen is: Viaweb is B2B, and its customers value new features; whilst Facebook is B2C and a social network i.e. the value is in the number of people on the platform, not its new features. Another explanation might actually just be that there was no high enough level language in Viaweb's days to compete with Lispers, and plenty in the PHP era. To an extent we witness this today with the myriad frameworks available which in theory eliminate any edge for fast prototyping except in safety-critical or performance-critical applications. [1] https://facebook.github.io/react/blog/2016/01/08/A-implies-B-does-not-imply-B-implies-A.html https://facebook.github.io/react/blog/2016/01/08/A-implies-B...
- stray 11y agoYou will find no meaningful statistical evidence because it can't be done -- programming languages don't write software. Programmers do -- and programmers are made of meat. A bad programmer using Lisp will be less successful than a good programmer using PHP. And startups almost universally (regardless of the back-justifications they come up with) choose programming languages based on fashion: Résumé driven development. The successful programming language for a startup is one that the product can be built in -- and that investors will not be alarmed by. PHP would not be alarming at all to many investors -- so PHP is a decent choice if you intend to sell your company off anyway.
- dvirsky 11y agoUsually it's just one of the languages the technical founder is most familiar with. Mark Zuckerberg's weapon of choice before Facebook was PHP - which was one of the best options for web dev circa 2004 - so he just used the most productive tool for him.
- pmarreck 11y agoYou're better off looking for evidence that functional languages allow more productivity, of which there is some evidence
- CullingTheHerd 11y agohave fun regressing out everything else...
- jandrewrogers 11y agoNote that he says the most powerful language for the job. Which language is the "most powerful" varies greatly by application, and would include mainstream languages in a variety of contexts. For example, C++11 is the power language of choice for databases engines, and a reasonable argument could be made that languages like Python and PHP are as well in certain contexts. On the other hand, some application domains are arguably better served with esoteric languages. In other words, use the best tool for the job. LLVM IR also deserves mention for its role as a kind of ur-language that can express all others in many modern applications. The ability to easily design and dynamically compile purpose-built languages is proving to be useful in many contexts.
- BBL 11y agoI've used ruby to make a responsive web app and feedback on the product signifies that users like and trust the seemingly clunky design elements. I believe the founder of Facebook previously said that the company's early over-reliance on HTML5 was his biggest regret. The tradeoff paid off in terms of speed and agility. The use of the color blue was also quite smart. Note: My hidden agenda with this post is to increase my karma points.
- hiphipjorge 11y agoThis is totally subjective, but I think we're at a point where the language ecosystem is much more important than the actual language. This was probably not the case when pg was writing software for Viaweb and there wasn't nearly as much FOSS that you could just use. At that point in time the LANGUAGE was probably much more important because there was not a lot of differentiation beyond that. Now, we're at a point where the language isn't as important and the ECOSYSTEM around it is much more powerful that then language itself.
- a_bonobo 11y agoI've observed this in bioinformatics - Python, Perl, R are much more common than (let's say) lisp or Ruby, partially because more people know these first languages (possible exception in R - is that even taught in university?), but IMHO more because libraries such as BioPerl, Biopython, Matplotlib (Seaborn), numpy/scipy, ggplot2, the entire CRAN ecosystem are so much more mature than what you get in lisp/Ruby, you can get amazing stuff done very quickly.
- adenadel 11y agoI agree with you completely and just wanted to comment that R is fairly widely taught in universities. The difference with R is that it is often taught in statistics courses rather than CS.
- hga 11y agoThat's certainly true. While Clojure isn't exactly my favorite Lisp (it isn't even a LISP, as in a LISt Processing language any more, and is rough around the edges in various places), running on the JVM ecosystem makes it a great choice for many things (well, aside from logging, standard JVM logging is "sacrifice a chicken" territory, but there are native solutions now as well). Want to us a RDBMS? There are many libraries for doing that, my favorite is Korma, and they all use the JDBC interface and battle tested JDBC implementations, that part of the plumbing is trivial. Etc.
- collyw 11y agoI was thinking something similar. I used to use Perl until around 5 years ago, and its a pretty expressive language, great for building all the blocks for your application quickly. Then I discovered Django, and all those blocks I would have been building have been built for me.
- alayne 11y agoAlan Kay has a more generalized idea about building up abstractions that I like better. I think he mentioned it in this video https://www.youtube.com/watch?v=NdSD07U5uBs https://www.youtube.com/watch?v=NdSD07U5uBs I, like many in this thread, think that good people are more important than a particular programming language.
- a-saleh 11y agoI found a blog-post that tries to do this sort of comparison: http://codingvc.com/which-technologies-do-startups-use-an-exploration-of-angellist-data http://codingvc.com/which-technologies-do-startups-use-an-ex... I think there are two valid reasons for choosing non-mainstream language. First, you know the language and want to work in it. A.f.a.i.k that was the main reason why PG chose it to build his company. Second, if you know the technology is a good fit for your product, for example WhatsApp and Erlang seem to be good example for this.
- mbrodersen 11y agoThe ecosystem (libraries, documentation, tutorials etc.) are much more important than the language. AND the biz ideas/execution are MUCH more important than the programming language ecosystem. Programming languages doesn't build successful businesses (unless you are a programming language business).
- lpolovets 11y agoThis is somewhere between anecdotal and statistical evidence, but I wrote a post about technology vs. startup quality a few years ago: http://codingvc.com/which-technologies-do-startups-use-an-exploration-of-angellist-data http://codingvc.com/which-technologies-do-startups-use-an-ex... The gist of the post was to correlate technologies listed on companies' AngelList profiles with those companies' Signal scores. Signal scores are a bit of a black box, but I was told by an AngelList employee that they're roughly like PageRank over the graph of startups, founders, and investors. That is, a startup's Signal score will be higher if the people affiliated with that startup have high Signal scores. There were some correlations between higher Signal scores and lower usage of PHP, higher usage of languages like Go and Scala, etc. That supports PG's point, but probably lacks the statistical rigor that you'd want.
- kartickv 11y agoThis is brilliant. What I was looking for. Thanks very much. It seems that great startups use JS, Ruby, Python and Java, in that order.
- agumonkey 11y agoThere may be a little more structure behind this. Lisp, at his time, was incredibly different and prone to creativity and abstractions[1]. Amazing vehicle to explore. Most of the socially successful languages aren't about exploration but about economy of scale, lever. Most people are more happy building application by standing on shoulders of libs rather than finding new solutions to odd problems. I feel it's a bit similar to Haskell these days. They reinvent a world with their own abstractions, extremely powerful ones (concise yet general yet fast) but lots of people don't want to abstract this way. [1] at a cost, you'd need high end machines to enjoy the benefits without the memory/gc issues.
- stray 11y agoLisp (Common Lisp, anyway) has not changed in this century -- and even 20 years ago, Common Lisp didn't require a very high end machine (at work I had a modest Sun workstation and at home a Macintosh -- and both did just fine with Common Lisp). It's still an amazing vehicle for exploration -- and it's still very good for building software. Some of Lisp's benefits lost their impressiveness because other languages have included those things as well. But we live in BlubWorld. And most people don't see the value in languages that look weird to them. They can't.
- agumonkey 11y agoLisp came before the 60s. I try to put myself in the shoes of the industry where cons cells and GC would mean 'no' to your business. It wasn't until the 80s that lisp was usable in the mainstream (wild guess). And yes CL is perfectly adequate to write solid mundane software (quicklisp was a gift here). But in this context, lisp strength, strong metalevel facilities and culture, has no value. Plus they have costs too, macro-fu require a certain view point on code.
- stray 11y agoIt's going too far to say "has no value". It may be true that its value is not immediately obvious, though. In my opinion, Lisp is more attractive than any other language because of its Condition System -- though macros rightfully get more attention -- what it lacks is libraries. But hopefully, that condition won't last too long. If nobody beats me to the punch I'm working on a way to steal Python's libraries by transpiling them into Lisp.
- tim333 11y agoIt's hard to figure statistics for the stuff as it's all a bit vaguely defined which is kind of inherent to the nature of the problem. In the essay http://www.paulgraham.com/power.html http://www.paulgraham.com/power.html PG suggests power can be equated with susinctness. I'd kind of guess the most succinct language for startups to write in would tend to be Ruby as there is so much existing code to do things - Rails, gems and all that. It also seems associated with successful startups like Twitter, Github etc so maybe there's a correlation there? (Disclaimer - I'm not a Rails programmer - kinda guessing there).
- AnimalMuppet 11y agoI'll give you an argument that PG's theory (at least as stated) cannot be correct. PG says that Lisp is at the top of the power curve, looking down at all other languages. And Lisp practitioners are sure they're looking down, and they'll tell you why. "How can you get anything done in Haskell? It doesn't even have macros." But Haskell programmers are sure that they're at the top of the power curve, and that when they look at other languages (including Lisp!) that they're looking down. And they know why they're looking down. "How can you get anything done in Lisp? It doesn't even have a decent type system!" I have in fact heard almost the exact argument of PG's essay before, from proponents of a heavyweight CASE tool. This fact - that proponents of two languages are both sure that they're looking down when they look at the other language - tells us that something is very wrong with PG's idea. The problem is the idea that all languages can be ranked on a one-dimensional axis labeled "power". To see what's wrong with this idea, look at hardware. We know what "power" means there - it's MIPS. Until you think a bit more. Then you realize "Well, we've got that floating-point code, so we have think about FLOPS, too. And there's that one data set that won't fit into cache, so we need to worry about memory bandwidth... unless we can trade off some clock speed for cache size..." Suddenly "power" has at least four dimensions: MIPS, FLOPS, cache size, and memory bandwidth. Then some joker comes along and says, "What I mean by "power" is the ability to run for eight hours without having to charge the battery"; that is, "power" is something like the reciprocal of watts consumed. Back to software. It isn't just power of the language, it's power to do something - to write the program you're trying to write. What language is going to make that easiest, all things considered - the language itself, libraries, ecosystem, familiarity of team-members, ease of learning? Pick that language, because that language is "the most powerful" - for your situation.
- Kinnard 11y agoArc seems to be languishing: http://arclanguage.org/forum http://arclanguage.org/forum
- kristianp 11y agoInteresting that no-one has pointed out here the dynamic vs static or scripted vs compiled dimensions. Facebook's use of php arguably allows them to build new features quickly, just as Viaweb's use of Lisp. But is it the dynamic nature of those languages that allows the faster development, not the 'power'? It seems accepted that dynamic languages allow faster development, but static languages are more able to prevent bugs. It varies from language to language of course. Apparently the research studies on static vs dynamic have many problems though, so you can argue that there is no evidence that one is "better" than the other: http://danluu.com/empirical-pl/ http://danluu.com/empirical-pl/ Maybe we should have a coding competition that has an equal number of (lisp, perl, php, ruby, python) experts vs (haskell, ocaml) experts and see how fast they can add features, vs how buggy those features are. Haskell and Ocaml are powerful, but 'very static', so it would be an interesting comparison.