9 ms·

Why does anyone write web apps in Python? PHP? Ruby? Modern Java and Go are so much faster than the alternatives that it's stupid to consider anything else if

by slackingoff2017 9y ago

Why does anyone write web apps in Python? PHP? Ruby? Modern Java and Go are so much faster than the alternatives that it's stupid to consider anything else if performance is important. Golang and Java can manage over a half million HTTP responses a second. Node is pretty fast but why bother when Java is many times more mature in features, tooling, and and supports concurrency... And uses less ram and is usually faster. People moan about the "huge" Java runtime when JavaScript uses 3-5x more memory and has a huge runtime of its own. All the big companies are using Java and Go almost exclusively for high volume endpoints and it blows my mind the amount of mental gymnastics some companies go through to avoid following suit. Java has come a long way since J2EE. These days it's asynchronous, non-blocking, serverless, etc.. pretty much all the acronyms thrown around about node except its not JavaScript, which IMO is a huge win.

sidlls 9y ago
High quality Java and Go developers are more expensive than Ruby and JS webdevs, partly because there are just so many of the latter.
- slackingoff2017 9y ago
  Java and Go are much harder to learn. Much more complex from a language perspective, more complicated build process, bigger standard libraries. This leads to a lot of "first languages" being the latter and a perpetual glut of recent grads that know nothing else so will do those jobs for less
adambard 9y ago
The answer is that for a huge variety of software, performance is not important, or perhaps is only important for a subset of the application. My personal experience is that the dynamic languages you've laid out generally have frameworks that are extremely conducive to rapid prototyping (Django is my favorite). I've seen and done the dance many times -- start with a Django/Rails/Laravel app, get a free admin and build up some CRUD pages in no time flat, and then once you've got enough traffic to care, move parts of the application to more performant platforms (Go/JVM usually) as necessary.
- Jach 9y ago
  Yeah, plus even if performance is important, the app layer isn't necessarily the best place to optimize. It doesn't really matter how fast you sprint between database calls if the database and its IO dominate your site's performance profile, which they often do...
  - cgag 9y ago
    Everyone seems to say this and also to write really slow websites.
    - ajmurmann 9y ago
      The vast majority of slow websites I've seen written with RoR were slow because the DB later want optimised. 1+n query problems, pulling way more data than needed and then processing it in Ruby, missing indices etc.
      - yxhuvud 9y ago
        Either that or by slow views. URL generation is often a noticable culprit.
    - JamesBarney 9y ago
      Most websites are written to minimize developer time not processing time.
    - sidlls 9y ago
      Are these sites slow because the software on the server-side is slow or because they're JavaScript bloated garbage? That's a serious question: I find it's often hard to tell what the bottleneck might be in these applications.
    - hasenj 9y ago
      They say this and write slow websites because they don't care. They only care about their code being "beautiful" in some weird sense.
frozenport 9y ago
JS lets you share code between front-end and back-end. For example, if you're writing a networked game, you're going to need to transport a lot of state between the server and client.
luord 9y ago
> Modern Java and Go are so much faster than the alternatives that it's stupid to consider anything else if performance is important. You answered your own question. Why bother with those languages when the language isn't the bottleneck? In those cases, what language one uses becomes an entirely subjective matter. > All the big companies are [...] For every large company using Java, Go, Rust, C, etc. There's another one (hell, probably the same one) also using Python, Ruby, PHP or JS.
- slackingoff2017 9y ago
  And most of those other companies like Facebook are writing endless hacks, VM's, or entire languages to fix the slow runtime :)
  - luord 9y ago
    Sorry, I have no interest in getting into a religious flamewar. I will only say that I find it weird that you think that there aren't endless hacks and VMs in [insert here the languages you like].
    - slackingoff2017 9y ago
      Eh, vanilla Java or Go is at least an order of magnitude faster than those other languages without messing with anything. You can mess around with the JVM and interfacing to native code but it's rarely needed because it's already within a low multiple of C performance writing things normally. Arguments about what language is better are pretty flamey but it's hard to argue that performance is not an advantage of Go/Java
      - luord 9y ago
        > it's hard to argue that performance is not an advantage of Go/Java And I never even remotely argued such a thing so I'm confused as to why you are saying this in reply to me.
smegel 9y ago
Productivity.
stuartaxelowen 9y ago
For context, Instagram's primary web servers are Django.
- mi100hael 9y ago
  Pinterest, too, I believe.
- seabrookmx 9y ago
  And reddit. And a little thing called youtube (though to be fair last I read they're offloading a lot of the hot paths to golang and c via native modules).
  - kbaker 9y ago
    Curious, where do you see that youtube is running Django? Python, maybe, but I don't think they run Django?
    - techdragon 9y ago
      It's definitely python, but definitely not Django. I recall someone explaining to me that it actually predates WSGI standardisation so its not even WSGI its just all custom python from start to finish.
    - seabrookmx 9y ago
      My apologies, I thought OP said 'Python' not 'DJango'. You're correct. Youtube has their own framework and IIRC reddit runs a few different Pyramid modules rather than a full-fat framework. Reddit has some scattered Engineering blogs and my insight into youtube comes from a previous HN article by the author of GrumPy (who works on youtube at Google) in case you're curious.
deleted 9y ago
[deleted]
jaequery 9y ago
well then why go with any of them when erlang and elixir has them all beat in terms of performance AND stability? let alone the power of concurrent processes it can handle that none of those languages can hold a candle to. why? :)
- riku_iki 9y ago
  Because your statements are very challengeable. Erlang is much slower in raw performance than both go and jvm: https://benchmarksgame.alioth.debian.org/u64q/erlang.html https://benchmarksgame.alioth.debian.org/u64q/erlang.html Async approache similar to Erlang was reproduced for Java and Go already: https://akka.io https://akka.io
  - hack4supper 9y ago
    hi riku_iki, I just checked out the link akka.io and it says that it caters to java/scala only ... from your last statement I understood that it was meant for java and go.
    - riku_iki 9y ago
      Go also has tons of actor frameworks: https://github.com/AsynkronIT/protoactor-go https://github.com/AsynkronIT/protoactor-go But it also has native coroutines embedded into language, which provide excellent asynchronous performance.
      - hack4supper 9y ago
        Hey thanks for that link ... I was aware of go having coroutines available as part of the language but not that something similar to what erlang provided was available to the go system as well...
  - elcritch 9y ago
    You're right that Go/Java have async style similar to Erlang. But the majority of production systems today that these languages run is some sort of web application. In this area, the Erlang VM holds its own pretty well, especially for websockets [1]. In that study Elixir's memory usage is higher, but total connections were almost identical to Go. It'd be great if there were better benchmarks for common use cases of various languages. Spring on the Java side tends to be heavy on reflection usage, which is orders of magnitude slower than JIT'ed JVM methods. Benchmarks like the benchmark game don't capture this. Still despite that the Erlang VM performs very well on the benchmarks game compared to other dynamic/scripting languages. Often it's easily 5-10 times faster than Python or Ruby. Given the parents comment, I'd argue many programmers who enjoy developing with dynamic languages can do so with Elixir with comparable performance to Go/Java for high concurrency web applications. 1: https://hashrocket.com/blog/posts/websocket-shootout https://hashrocket.com/blog/posts/websocket-shootout
ubernostrum 9y ago
Why does anyone write web apps in Python? PHP? Ruby? First: Because squeezing every last nanosecond's worth of performance out of your web app is actually an extremely rare problem to have. And if you truly cared about performance over programmer convenience, you'd practice what you preach and build your web apps in hand-rolled assembly, but I'd bet a lot of money that you don't do that. The typical web application -- I'd be willing to bet over 99.999% of all deployed production web applications serving requests today -- has a bottleneck at the database and the network that dwarfs any overhead from language performance. I remember a bit over ten years ago when there was debate in the Python web world about which templating engine to use to generate HTML. And people argued endlessly over microbenchmarks of them, to figure out which was fastest, but I remember one blog post which showed a pie chart of time spent in the average request/response cycle. Nearly all of it was accessing the database, and template rendering was a tiny, tiny sliver, so the author humorously labeled it "obviously this is the part we need to focus all our optimization work on". Language choice is similar. Second: Because what else your company does matters. Where I work, web applications are how we expose data and interfaces to that data. But there's a gigantic stack behind that, of data intake, data parsing, data processing, analytics, the whole nine yards. It's all in Python, because Python has hands-down the strongest ecosystem of any popular programming language for that stuff. So the web applications which serve as the interfaces to the data are also written in Python; it means we have one language to worry about, one language to work in, one language every software engineer knows. I've been pulled onto projects doing things that didn't involve web applications at all, and I've been able to be productive because those projects were still in Python, and I could read code and get up to speed on what was happening, and take care of mundane things for a more domain-experienced person whose domain expertise was then free to apply to things I couldn't do. Third: Because programmer convenience really and truly does matter. When I first started doing this nearly twenty years ago, people posted comments like yours, incredulous at the idea that someone would use PHP or Perl given their performance characteristics compared to Java (or C -- plenty of web apps used to be written in C!). But even then we knew: servers are cheaper than people. The average salary of a quality software engineer (or "web developer" as we were known then) would buy you a lot of compute time, either on your own (in-house or colo'd) metal, or nowadays on someone else's cloud. So you choose based on convenience to humans. PHP, for all its faults, was an incredibly convenient language to write web apps in, and compared to the usual CGI model that preceded it, was a breath of fresh air when it took off. Today, frameworks written in Python, Ruby, PHP, etc. are similarly in a good position compared to more heavyweight things like the Java world (which for better or worse is still suffering the lingering effects of its mid-2000s "enterprise" reputation), or even Go (which is still young and still seems to come up short, both language- and ecosystem-wise, on some of the things actual working web programmers want. In particular, programmer-friendly ORMs in statically-typed language really really really want generics or a good equivalent, and Go's historic attitude toward that has not been great.
tyingq 9y ago
Efficiency can be language performance or developer performance or whatever. Higher level languages might offer lower raw performance and more bugs. But they offer more features per dollar. That isn't really specific to programming. You might also wonder why McDonalds is popular, even though their food is...meh.
yeukhon 9y ago
> All the big companies are using Java and Go almost exclusively for high volume endpoints Please cite “all the big companies.” Also exclusively? No seriously, that is a bold claim. Google isn’t one of them because most of those hotepots are still written in C++. I don’t know how you can claim this. Based on some occaion company blog posts, changing just one or two endpoints out of say 100? What about Rust? I also know companies rewrote some hotspots in Rust too.
- pjmlp 9y ago
  Pick the Fortune 500 list, take out the SV darlings, the majority of their backend stacks will be a mix of Java and .NET deployments. Easy to find out just by looking at their open job positions.
  - dx034 9y ago
    I'd say that you find a fair share of C++ as well. Especially for high volume endpoints.
    - pjmlp 9y ago
      Quite true, although it tends to be used in native libraries called from one of those managed languages.
  - slackingoff2017 9y ago
    This is true even with most SV darlings. Netflix, Google, Uber, and Amazon use tons of Java. Probably the majority of their systems. Microsoft is an exception because they built C# which is basically a more modern but less popular variant of Java. There was even a project for a long time that let you use Java code in C# projects by converting the bytecode, they're that similar. The notable exception is Facebook. They were stuck in PHP hell for so long that they redesigned the language to make it work.
- slackingoff2017 9y ago
  Google has a huge amount of code written in Java, I would say the majority of their systems. Just look at their open sourced projects and job listings. Over 50% Java easily.
pbreit 9y ago
I wonder the opposite. Hardly any system needs to process 500k response per second. But nearly all of them benefit from the developer productivity that Python/Rails offer. My current company has 15 Java & React engineers where 2 or 3 Rails would suffice. Load tops out at maybe a dozen requests/second. Feature development is super-slow. System complexity is off the charts.
- YawningAngel 9y ago
  I'm sure you'd need 10 or 15 bad Ruby developers as well. I've managed to kick things out the door very happily with one Java and one React developer before.
- matwood 9y ago
  > Feature development is super-slow. System complexity is off the charts. That probably has little to do with language/stack and everything to do with constantly changing requirements/system growth. To your required engineers comment, Spring Boot + jOOQ is easily one of the most productive backend stacks I've ever used. A single engineer could easily build a large API leveraging the stack.
  - slackingoff2017 9y ago
    Take a look a vert.x and Dropwizard. Spring Boot can handle maybe 10k requests per second, but Dropwizard is around 200k and vert.x maybe a million. If you're only using them for a REST API they offer similar features. Spring Boot supports loads of other stuff you probably don't need for something API driven
noselasd 9y ago
Most of the time, runtime speed is far, far less important than developer speed.
- mmargerum 9y ago
  Speed and memory usage don’t matter until you need to rent servers on AWS
  - ageofwant 9y ago
    At which time you beat 95% of the pack and you should be grateful that your current 'slow' language brought you where you are.
- matwood 9y ago
  I'd be curious what if any developer speed difference there is comparing a modern java stack (like Spring boot + jooq) with RAILs.
dragonwriter 9y ago
> All the big companies are using Java and Go almost exclusively for high volume endpoints Because mature companies in competitive markets live or die by operational cost effectiveness. Growth-phase companies or one with moats live or die by other means; if they are around long enough and are targeting a valuable enough market, they'll eventually probably be a mature firm in a competitive market, but a good way not to get there is to focus on the needs that such a firm would have rather than the needs the firm they actually are now actually has.
hasenj 9y ago
I was wondering the same thing, and it frustrates me. I think people obviously enjoy using Python/Ruby or whatever for small scripts, and by inductive reasoning they think they can enjoy programming even larger projects in these languages. They also rationalize the slowness by pretending that performance doesn't matter or that the bottle neck is the database. They abuse the adage about "premature optimization" being a bad thing. Not realizing that while it's stupid to prematurely optimize everything, it's equally unwise to write everything in a slow language. Using a fast language is not premature optimization; it's just the basic thing you need to do in order to not have a crappy optimization, which is a good thing , and everyone should do that.
- viraptor 9y ago
  > by pretending that performance doesn't matter or that the bottle neck is the database. Or, you know... You can actually measure those things. And unless you're growing like crazy, or have a massive initial audience, you're likely to find that 90%+ of the app time is spent in the database and your CPU time isn't even close to maxed out. Why would you assume people would pretend any of that is true?
  - hasenj 9y ago
    Well, we've seen for example, Twitter do that. I saw some slides[0] that other day from sometime in 2007[1] where they said they were using Ruby and spawning 180 rails instances to handle 600 requests per second. Like, that's insane. A compiled language can handle that load with just one instance. It's nothing. [0]: https://www.slideshare.net/Blaine/scaling-twitter/3-First_Some_Facts_600_requests https://www.slideshare.net/Blaine/scaling-twitter/3-First_So... Also, if your audience is small enough that a slow python can suffice, why even bother using a database server such as postgresql or mysql when you can just use sqlite and simplify the architecture? EDIT: [1] It's been 10 years since 2007 but people still do this more or less all the time. Write a webapp in js/ruby/python and spawn 20 instances of the application server
    - tychver 9y ago
anjc 9y ago
I like how for the past 10 years, everybody has been espousing language/framework xyz as being superior for the reasons you've just said (asynchronous, responses per second, memory footprint etc), but as soon as you say Java does it all better, suddenly developer efficiency is most important. FWIW, it always seems like a mostly zero sum game to me. Whatever efficiency you gain in using (e.g.) Python+Latest Frontend Framework+Backend Framework, you lose through having to wade through yet another set of new concepts and documents for those frameworks.
skeletal88 9y ago
Can you point a clueless (about Java web dev) person to a nice, lightweight framework that's easy to learn and to set up? Please no XML configuration files and other such nonsense. For me as a Python/Rails/C++ dev Java has a reputation of being too large, too complex and otherwise.. unwieldy. Hearing things like "To test a bug I had to start 6 services on my computer and then I ran out of memory (computer had 16GB)" doesn't encourage me to try to do web dev in Java.
- lmm 9y ago
  Not the most lightweight, but I'd highly recommend wicket running on embedded jetty, a la the second code block on https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=27848170 https://cwiki.apache.org/confluence/pages/viewpage.action?pa... . You do still have to use XML for Maven I'm afraid (there are alternatives but I wouldn't recommend them) but it's a relatively good use of XML and you can use eclipse's GUI to add dependencies rather than adding them directly if you like. Wicket is a true OO approach to GUI which is quite different from the page-template style of Rails/Django/..., but I find it makes for much more compositional style, with lots of small reusable components that are just compositions of a few smaller components. And while not being able to monkeypatch everything can chafe initially, when you come to upgrade to a newer version of the framework you'll really appreciate the safety a compiled/typechecked language can offer.
- mschulze 9y ago
  The Spark Framework [0] is pretty easy to pick up IMO. Spring (with Spring Boot) has approachable tutorials [1] on how to get started but will get intimidating pretty fast. I highly recommend using Kotlin instead of Java when trying out the JVM, though. [0] http://sparkjava.com/ http://sparkjava.com/ [1] https://spring.io/guides/gs/rest-service/ https://spring.io/guides/gs/rest-service/
  - jventura 9y ago
    I also recommend Java Spark for a minimalist framework. I would say that is quite similar to Flask in Python. But unlike Flask, Spark does not have a templating language. FOr something simple, I would recommend freemarker..
dna_polymerase 9y ago
Ehem: https://github.com/squeaky-pl/japronto https://github.com/squeaky-pl/japronto https://github.com/channelcat/sanic https://github.com/channelcat/sanic
Rotareti 9y ago
> Why does anyone write web apps in Python? The new generation of Python web frameworks is pretty fast (check out Sanic[0][1] for example), Python has a huge growing ecosystem, you get rapid time to marked and if you find some part of your application to be the bottleneck, you can replace it with C/Rust, which you probably don't need, because your company won't ever scale as far. [0] https://github.com/channelcat/sanic https://github.com/channelcat/sanic [1] https://magic.io/blog/uvloop-blazing-fast-python-networking/ https://magic.io/blog/uvloop-blazing-fast-python-networking/
- bpicolo 9y ago
  The problem with asyncio is that you can't take advantage of both it and the bazillion synchronous-io libraries already in the ecosystem. For large web apps it's still pretty much a nonstarter unless you want to not be able to take full advantage of libraries like sqlalchemy. Nodejs had a great advantage by starting with asyncio from the beginning. M:N threading would have been a much better fit, though that's out of question given the GIL.
apple4ever 9y ago
Because PHP is awesome, and so is Python. Java and Go are terrible. Simple as that.

sidlls 9y ago

High quality Java and Go developers are more expensive than Ruby and JS webdevs, partly because there are just so many of the latter.

slackingoff2017 9y ago
Java and Go are much harder to learn. Much more complex from a language perspective, more complicated build process, bigger standard libraries. This leads to a lot of "first languages" being the latter and a perpetual glut of recent grads that know nothing else so will do those jobs for less

adambard 9y ago

The answer is that for a huge variety of software, performance is not important, or perhaps is only important for a subset of the application. My personal experience is that the dynamic languages you've laid out generally have frameworks that are extremely conducive to rapid prototyping (Django is my favorite). I've seen and done the dance many times -- start with a Django/Rails/Laravel app, get a free admin and build up some CRUD pages in no time flat, and then once you've got enough traffic to care, move parts of the application to more performant platforms (Go/JVM usually) as necessary.

Jach 9y ago
Yeah, plus even if performance is important, the app layer isn't necessarily the best place to optimize. It doesn't really matter how fast you sprint between database calls if the database and its IO dominate your site's performance profile, which they often do...
- cgag 9y ago
  Everyone seems to say this and also to write really slow websites.
  - ajmurmann 9y ago
    The vast majority of slow websites I've seen written with RoR were slow because the DB later want optimised. 1+n query problems, pulling way more data than needed and then processing it in Ruby, missing indices etc.
    - yxhuvud 9y ago
      Either that or by slow views. URL generation is often a noticable culprit.
  - JamesBarney 9y ago
    Most websites are written to minimize developer time not processing time.
  - sidlls 9y ago
    Are these sites slow because the software on the server-side is slow or because they're JavaScript bloated garbage? That's a serious question: I find it's often hard to tell what the bottleneck might be in these applications.
  - hasenj 9y ago
    They say this and write slow websites because they don't care. They only care about their code being "beautiful" in some weird sense.

frozenport 9y ago

JS lets you share code between front-end and back-end. For example, if you're writing a networked game, you're going to need to transport a lot of state between the server and client.

luord 9y ago

> Modern Java and Go are so much faster than the alternatives that it's stupid to consider anything else if performance is important. You answered your own question. Why bother with those languages when the language isn't the bottleneck? In those cases, what language one uses becomes an entirely subjective matter. > All the big companies are [...] For every large company using Java, Go, Rust, C, etc. There's another one (hell, probably the same one) also using Python, Ruby, PHP or JS.

slackingoff2017 9y ago
And most of those other companies like Facebook are writing endless hacks, VM's, or entire languages to fix the slow runtime :)
- luord 9y ago
  Sorry, I have no interest in getting into a religious flamewar. I will only say that I find it weird that you think that there aren't endless hacks and VMs in [insert here the languages you like].
  - slackingoff2017 9y ago
    Eh, vanilla Java or Go is at least an order of magnitude faster than those other languages without messing with anything. You can mess around with the JVM and interfacing to native code but it's rarely needed because it's already within a low multiple of C performance writing things normally. Arguments about what language is better are pretty flamey but it's hard to argue that performance is not an advantage of Go/Java
    - luord 9y ago
      > it's hard to argue that performance is not an advantage of Go/Java And I never even remotely argued such a thing so I'm confused as to why you are saying this in reply to me.

smegel 9y ago

Productivity.

stuartaxelowen 9y ago

For context, Instagram's primary web servers are Django.

mi100hael 9y ago
Pinterest, too, I believe.
seabrookmx 9y ago
And reddit. And a little thing called youtube (though to be fair last I read they're offloading a lot of the hot paths to golang and c via native modules).
- kbaker 9y ago
  Curious, where do you see that youtube is running Django? Python, maybe, but I don't think they run Django?
  - techdragon 9y ago
    It's definitely python, but definitely not Django. I recall someone explaining to me that it actually predates WSGI standardisation so its not even WSGI its just all custom python from start to finish.
  - seabrookmx 9y ago
    My apologies, I thought OP said 'Python' not 'DJango'. You're correct. Youtube has their own framework and IIRC reddit runs a few different Pyramid modules rather than a full-fat framework. Reddit has some scattered Engineering blogs and my insight into youtube comes from a previous HN article by the author of GrumPy (who works on youtube at Google) in case you're curious.

deleted 9y ago

[deleted]

jaequery 9y ago

well then why go with any of them when erlang and elixir has them all beat in terms of performance AND stability? let alone the power of concurrent processes it can handle that none of those languages can hold a candle to. why? :)

riku_iki 9y ago
Because your statements are very challengeable. Erlang is much slower in raw performance than both go and jvm: https://benchmarksgame.alioth.debian.org/u64q/erlang.html https://benchmarksgame.alioth.debian.org/u64q/erlang.html Async approache similar to Erlang was reproduced for Java and Go already: https://akka.io https://akka.io
- hack4supper 9y ago
  hi riku_iki, I just checked out the link akka.io and it says that it caters to java/scala only ... from your last statement I understood that it was meant for java and go.
  - riku_iki 9y ago
    Go also has tons of actor frameworks: https://github.com/AsynkronIT/protoactor-go https://github.com/AsynkronIT/protoactor-go But it also has native coroutines embedded into language, which provide excellent asynchronous performance.
    - hack4supper 9y ago
      Hey thanks for that link ... I was aware of go having coroutines available as part of the language but not that something similar to what erlang provided was available to the go system as well...
- elcritch 9y ago
  You're right that Go/Java have async style similar to Erlang. But the majority of production systems today that these languages run is some sort of web application. In this area, the Erlang VM holds its own pretty well, especially for websockets [1]. In that study Elixir's memory usage is higher, but total connections were almost identical to Go. It'd be great if there were better benchmarks for common use cases of various languages. Spring on the Java side tends to be heavy on reflection usage, which is orders of magnitude slower than JIT'ed JVM methods. Benchmarks like the benchmark game don't capture this. Still despite that the Erlang VM performs very well on the benchmarks game compared to other dynamic/scripting languages. Often it's easily 5-10 times faster than Python or Ruby. Given the parents comment, I'd argue many programmers who enjoy developing with dynamic languages can do so with Elixir with comparable performance to Go/Java for high concurrency web applications. 1: https://hashrocket.com/blog/posts/websocket-shootout https://hashrocket.com/blog/posts/websocket-shootout

ubernostrum 9y ago

Why does anyone write web apps in Python? PHP? Ruby? First: Because squeezing every last nanosecond's worth of performance out of your web app is actually an extremely rare problem to have. And if you truly cared about performance over programmer convenience, you'd practice what you preach and build your web apps in hand-rolled assembly, but I'd bet a lot of money that you don't do that. The typical web application -- I'd be willing to bet over 99.999% of all deployed production web applications serving requests today -- has a bottleneck at the database and the network that dwarfs any overhead from language performance. I remember a bit over ten years ago when there was debate in the Python web world about which templating engine to use to generate HTML. And people argued endlessly over microbenchmarks of them, to figure out which was fastest, but I remember one blog post which showed a pie chart of time spent in the average request/response cycle. Nearly all of it was accessing the database, and template rendering was a tiny, tiny sliver, so the author humorously labeled it "obviously this is the part we need to focus all our optimization work on". Language choice is similar. Second: Because what else your company does matters. Where I work, web applications are how we expose data and interfaces to that data. But there's a gigantic stack behind that, of data intake, data parsing, data processing, analytics, the whole nine yards. It's all in Python, because Python has hands-down the strongest ecosystem of any popular programming language for that stuff. So the web applications which serve as the interfaces to the data are also written in Python; it means we have one language to worry about, one language to work in, one language every software engineer knows. I've been pulled onto projects doing things that didn't involve web applications at all, and I've been able to be productive because those projects were still in Python, and I could read code and get up to speed on what was happening, and take care of mundane things for a more domain-experienced person whose domain expertise was then free to apply to things I couldn't do. Third: Because programmer convenience really and truly does matter. When I first started doing this nearly twenty years ago, people posted comments like yours, incredulous at the idea that someone would use PHP or Perl given their performance characteristics compared to Java (or C -- plenty of web apps used to be written in C!). But even then we knew: servers are cheaper than people. The average salary of a quality software engineer (or "web developer" as we were known then) would buy you a lot of compute time, either on your own (in-house or colo'd) metal, or nowadays on someone else's cloud. So you choose based on convenience to humans. PHP, for all its faults, was an incredibly convenient language to write web apps in, and compared to the usual CGI model that preceded it, was a breath of fresh air when it took off. Today, frameworks written in Python, Ruby, PHP, etc. are similarly in a good position compared to more heavyweight things like the Java world (which for better or worse is still suffering the lingering effects of its mid-2000s "enterprise" reputation), or even Go (which is still young and still seems to come up short, both language- and ecosystem-wise, on some of the things actual working web programmers want. In particular, programmer-friendly ORMs in statically-typed language really really really want generics or a good equivalent, and Go's historic attitude toward that has not been great.

tyingq 9y ago

Efficiency can be language performance or developer performance or whatever. Higher level languages might offer lower raw performance and more bugs. But they offer more features per dollar. That isn't really specific to programming. You might also wonder why McDonalds is popular, even though their food is...meh.

yeukhon 9y ago

> All the big companies are using Java and Go almost exclusively for high volume endpoints Please cite “all the big companies.” Also exclusively? No seriously, that is a bold claim. Google isn’t one of them because most of those hotepots are still written in C++. I don’t know how you can claim this. Based on some occaion company blog posts, changing just one or two endpoints out of say 100? What about Rust? I also know companies rewrote some hotspots in Rust too.

pjmlp 9y ago
Pick the Fortune 500 list, take out the SV darlings, the majority of their backend stacks will be a mix of Java and .NET deployments. Easy to find out just by looking at their open job positions.
- dx034 9y ago
  I'd say that you find a fair share of C++ as well. Especially for high volume endpoints.
  - pjmlp 9y ago
    Quite true, although it tends to be used in native libraries called from one of those managed languages.
- slackingoff2017 9y ago
  This is true even with most SV darlings. Netflix, Google, Uber, and Amazon use tons of Java. Probably the majority of their systems. Microsoft is an exception because they built C# which is basically a more modern but less popular variant of Java. There was even a project for a long time that let you use Java code in C# projects by converting the bytecode, they're that similar. The notable exception is Facebook. They were stuck in PHP hell for so long that they redesigned the language to make it work.
slackingoff2017 9y ago
Google has a huge amount of code written in Java, I would say the majority of their systems. Just look at their open sourced projects and job listings. Over 50% Java easily.

pbreit 9y ago

I wonder the opposite. Hardly any system needs to process 500k response per second. But nearly all of them benefit from the developer productivity that Python/Rails offer. My current company has 15 Java & React engineers where 2 or 3 Rails would suffice. Load tops out at maybe a dozen requests/second. Feature development is super-slow. System complexity is off the charts.

YawningAngel 9y ago
I'm sure you'd need 10 or 15 bad Ruby developers as well. I've managed to kick things out the door very happily with one Java and one React developer before.
matwood 9y ago
> Feature development is super-slow. System complexity is off the charts. That probably has little to do with language/stack and everything to do with constantly changing requirements/system growth. To your required engineers comment, Spring Boot + jOOQ is easily one of the most productive backend stacks I've ever used. A single engineer could easily build a large API leveraging the stack.
- slackingoff2017 9y ago
  Take a look a vert.x and Dropwizard. Spring Boot can handle maybe 10k requests per second, but Dropwizard is around 200k and vert.x maybe a million. If you're only using them for a REST API they offer similar features. Spring Boot supports loads of other stuff you probably don't need for something API driven

noselasd 9y ago

Most of the time, runtime speed is far, far less important than developer speed.

mmargerum 9y ago
Speed and memory usage don’t matter until you need to rent servers on AWS
- ageofwant 9y ago
  At which time you beat 95% of the pack and you should be grateful that your current 'slow' language brought you where you are.
matwood 9y ago
I'd be curious what if any developer speed difference there is comparing a modern java stack (like Spring boot + jooq) with RAILs.

dragonwriter 9y ago

> All the big companies are using Java and Go almost exclusively for high volume endpoints Because mature companies in competitive markets live or die by operational cost effectiveness. Growth-phase companies or one with moats live or die by other means; if they are around long enough and are targeting a valuable enough market, they'll eventually probably be a mature firm in a competitive market, but a good way not to get there is to focus on the needs that such a firm would have rather than the needs the firm they actually are now actually has.

hasenj 9y ago

I was wondering the same thing, and it frustrates me. I think people obviously enjoy using Python/Ruby or whatever for small scripts, and by inductive reasoning they think they can enjoy programming even larger projects in these languages. They also rationalize the slowness by pretending that performance doesn't matter or that the bottle neck is the database. They abuse the adage about "premature optimization" being a bad thing. Not realizing that while it's stupid to prematurely optimize everything, it's equally unwise to write everything in a slow language. Using a fast language is not premature optimization; it's just the basic thing you need to do in order to not have a crappy optimization, which is a good thing , and everyone should do that.

viraptor 9y ago
> by pretending that performance doesn't matter or that the bottle neck is the database. Or, you know... You can actually measure those things. And unless you're growing like crazy, or have a massive initial audience, you're likely to find that 90%+ of the app time is spent in the database and your CPU time isn't even close to maxed out. Why would you assume people would pretend any of that is true?
- hasenj 9y ago
  Well, we've seen for example, Twitter do that. I saw some slides[0] that other day from sometime in 2007[1] where they said they were using Ruby and spawning 180 rails instances to handle 600 requests per second. Like, that's insane. A compiled language can handle that load with just one instance. It's nothing. [0]: https://www.slideshare.net/Blaine/scaling-twitter/3-First_Some_Facts_600_requests https://www.slideshare.net/Blaine/scaling-twitter/3-First_So... Also, if your audience is small enough that a slow python can suffice, why even bother using a database server such as postgresql or mysql when you can just use sqlite and simplify the architecture? EDIT: [1] It's been 10 years since 2007 but people still do this more or less all the time. Write a webapp in js/ruby/python and spawn 20 instances of the application server
  - tychver 9y ago

anjc 9y ago

I like how for the past 10 years, everybody has been espousing language/framework xyz as being superior for the reasons you've just said (asynchronous, responses per second, memory footprint etc), but as soon as you say Java does it all better, suddenly developer efficiency is most important. FWIW, it always seems like a mostly zero sum game to me. Whatever efficiency you gain in using (e.g.) Python+Latest Frontend Framework+Backend Framework, you lose through having to wade through yet another set of new concepts and documents for those frameworks.

skeletal88 9y ago

Can you point a clueless (about Java web dev) person to a nice, lightweight framework that's easy to learn and to set up? Please no XML configuration files and other such nonsense. For me as a Python/Rails/C++ dev Java has a reputation of being too large, too complex and otherwise.. unwieldy. Hearing things like "To test a bug I had to start 6 services on my computer and then I ran out of memory (computer had 16GB)" doesn't encourage me to try to do web dev in Java.

lmm 9y ago
Not the most lightweight, but I'd highly recommend wicket running on embedded jetty, a la the second code block on https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=27848170 https://cwiki.apache.org/confluence/pages/viewpage.action?pa... . You do still have to use XML for Maven I'm afraid (there are alternatives but I wouldn't recommend them) but it's a relatively good use of XML and you can use eclipse's GUI to add dependencies rather than adding them directly if you like. Wicket is a true OO approach to GUI which is quite different from the page-template style of Rails/Django/..., but I find it makes for much more compositional style, with lots of small reusable components that are just compositions of a few smaller components. And while not being able to monkeypatch everything can chafe initially, when you come to upgrade to a newer version of the framework you'll really appreciate the safety a compiled/typechecked language can offer.
mschulze 9y ago
The Spark Framework [0] is pretty easy to pick up IMO. Spring (with Spring Boot) has approachable tutorials [1] on how to get started but will get intimidating pretty fast. I highly recommend using Kotlin instead of Java when trying out the JVM, though. [0] http://sparkjava.com/ http://sparkjava.com/ [1] https://spring.io/guides/gs/rest-service/ https://spring.io/guides/gs/rest-service/
- jventura 9y ago
  I also recommend Java Spark for a minimalist framework. I would say that is quite similar to Flask in Python. But unlike Flask, Spark does not have a templating language. FOr something simple, I would recommend freemarker..

dna_polymerase 9y ago

Ehem: https://github.com/squeaky-pl/japronto https://github.com/squeaky-pl/japronto https://github.com/channelcat/sanic https://github.com/channelcat/sanic

Rotareti 9y ago

> Why does anyone write web apps in Python? The new generation of Python web frameworks is pretty fast (check out Sanic[0][1] for example), Python has a huge growing ecosystem, you get rapid time to marked and if you find some part of your application to be the bottleneck, you can replace it with C/Rust, which you probably don't need, because your company won't ever scale as far. [0] https://github.com/channelcat/sanic https://github.com/channelcat/sanic [1] https://magic.io/blog/uvloop-blazing-fast-python-networking/ https://magic.io/blog/uvloop-blazing-fast-python-networking/

bpicolo 9y ago
The problem with asyncio is that you can't take advantage of both it and the bazillion synchronous-io libraries already in the ecosystem. For large web apps it's still pretty much a nonstarter unless you want to not be able to take full advantage of libraries like sqlalchemy. Nodejs had a great advantage by starting with asyncio from the beginning. M:N threading would have been a much better fit, though that's out of question given the GIL.

apple4ever 9y ago

Because PHP is awesome, and so is Python. Java and Go are terrible. Simple as that.