5 ms·

Before everyone gets excited about these big numbers, I would like to remind you that even higher concurrency can be achieved with even lower CPU and memory usa

by ericz 14y ago

Before everyone gets excited about these big numbers, I would like to remind you that even higher concurrency can be achieved with even lower CPU and memory usage using Erlang. These numbers are good for Node, but don't use this as evidence that Node is magical and much better at handling large numbers of connections than other systems.

est 14y ago
what about other languages?
- boyter 14y ago
  I would be curious to know how something like Play! with all Async requests, or MVC with all Async would do. Probably similarly impressive, but I really have no idea.
  - fleitz 14y ago
    Probably not, the key to making 250,000 connections work is not doing a lot for each of those connections, or doing it very infrequently. Once you're rendering views, etc it's hard to maintain 50,000 req/sec. (250K connections @ 5 req/sec)
- ww520 14y ago
  The Netty library for Java routinely can do 500K connections without any sweat. Some people have tried 1M connections with beefy machines.
  - ZoFreX 14y ago
    I wonder how far you could push Java with a naive Thread implementation. If you reduce the per-thread stack size you can quite easily get 20K on very modest hardware.
jimparkins 14y ago
People are excited by Node doing numbers like this because there is a massive active Javascript community - with hundreds of thousands of people using Javascript all day every day at work. 0.01% of these people would ever consider learning Erlang, and even if they did they would not be able to use it at work - ever. As with everything having better features means nothing if nobody adopts. I am not saying nobody uses Erlang and I am not saying people are not adopting it - but the number are just not comparable to the Javascript community . Lastly just because you know a but of Javascript I realise that this does not mean you can architect massive real time systems. But it is like WOW even people that play casually aspire to having the best kit or playing for a top guild.
- mediocregopher 14y ago
  >As with everything having better features means nothing if nobody adopts Well, it means my product is going to be superior since I went the better, if less well known, architecture. It's not like erlang is a little unsupported side-project of a language, it's actually older then javascript if you count the time period before it was open-sourced, and only a few years younger if you don't, and is used extensively by many industries. Also, just because javascript the language is more well-known doesn't mean javascript the server architecture is more well-known. I would argue it isn't; when people want a highly concurrent, solid server, erlang is always mentioned. Lastly, erlang is a pretty easy language to learn. I had the basics down in a day, I had a prototype pubsub server that could handle 50k connections in two. The syntax is a bit strange, and honestly it does get in the way sometimes, but it's not hard.
  - maigret 14y ago
    You're missing the point of Node. You can construct a web rendering application and its AJAX parts in one codebase. You can move fastly code between server and client rendering. etc etc etc. Probably Erlang is "better". BTW, Java & C are quite fast also. Java applications, well written, do scale. If they don't, they are folks out there specialized to make them scale. Also, Erlang is probably easy to learn as language. But when you develop a web app, you have enough other skills to keep up with. Let's name CSS for one ;) The human brain is limited in its capacity to remember API and language specifics. Also, more popular means more libraries, which makes the product in turn better. This is why so many folks turn to PHP. It's not elegant, but everything you need is already here. Now I won't argue that you may have good reasons to use Erlang for yourself, may it be because you like the language structure, like to write libraries by yourself, or so on. But it doesn't make it "superior", foremost not as a platform.
scarmig 14y ago
Interesting example from 2008: http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-1 http://www.metabrew.com/article/a-million-user-comet-applica... I'd also add that this shouldn't be taken as something to say that Erlang is totally superior to node or that Erlang makes scaling to 1M concurrent connections a piece of cake. If you're working at that level, there's no magical out of the box solution.
ricardobeat 14y ago
The magic is in javascript. How many people can write Erlang?
- luriel 14y ago
  How many people can write clean, efficient and maintainable JavaScript? And Erlang is not the only sane option for this kinds of problems, there is also Go. And to a lesser degree you can do the same in many other languages given the right libraries and careful thinking, it takes more effort than with Erlang or Go, but almost anything beats JavaScript in both performance and code clarity (both at the 'low' code-readability level, and at the high 'project organization and design' level).
  - davidw 14y ago
    > almost anything beats JavaScript in both performance http://shootout.alioth.debian.org/u32/which-programming-languages-are-fastest.php http://shootout.alioth.debian.org/u32/which-programming-lang... Javascript with V8 stacks up pretty well.
- jeltz 14y ago
  Depends on where you are. In Stockholm I would not be surprised if more people can write decent Erlang compared to the people who can write decent JavaScript.
  - igouy 14y ago
    Would you be surprised if in Stockholm more people can write decent JavaScript compared to the people who can write decent Erlang?
pron 14y ago
... or Java.
ebiester 14y ago
How fast can I get a median programmer to learn Erlang, learn the libraries, and be productive enough to be able to make these high concurrency apps? Let's say they are a full stack programmer who knows some html, some css, some javascript, some java, and some sql. I have a pretty good idea how fast I can bring someone up to speed on node.js -- I have to teach them some advanced JS concepts, some node.js conventions, and the APIs of my library. Async takes a little bit to wrap your head around, but it's not terrible. Node.js seems like it is on the way to "worse is better."
- rubyrescue 14y ago
  about a week
- ericmoritz 14y ago
  A week with the Erlang language, which will enable to write projects that would be on par or better than anything written in node.js as far as resilient and scalability. Probably another week to get up to speed with OTP for all the promises of resilient Erlang applications. What is often missed about Erlang is that it's not really about highly concurrent applications. That property is actually a means to accomplish its primary goal: fault-tolerant applications. See: http://www.erlang.org/download/armstrong_thesis_2003.pdf http://www.erlang.org/download/armstrong_thesis_2003.pdf to understand the motives of the Erlang designers.
  - Isofarro 14y ago
    Firstly, I apologise for down-voting this response. A week to learn Erlang. Where do I start?
    - MetaCosm 14y ago
      http://learnyousomeerlang.com/ http://learnyousomeerlang.com/ You can pick up the basic syntax in one full day easily (if you are an experienced developer and already understand functional programming) ... 3 or 4 days if you are new to functional coding or just very inexperienced. It is a very brief / minimalist language from a syntax point of view. Then, it will take a week or two to get your head around OTP, which is the primary framework and has years (decades?) of mission critical work under its belt. Then, at the end of your journey will be the really hard problems... dealing with massive netsplits at a cluster level, elections for new masters, and all the other hard problems that happen at the upper-tier of massive clusters. If you are building an HTTP(S) app -- you can blessedly avoid a lot of these by avoiding a true massive cluster all together and using lots of individual "micro clusters"(note) balanced / routed by HTTP middle-ware. Also, check out Cowboy [webserver] (https://github.com/extend/cowboy/ https://github.com/extend/cowboy/), Agner [package manager] (http://erlagner.org/ http://erlagner.org/) and of course, the always awesome rebar [build tool] (https://github.com/basho/rebar/ https://github.com/basho/rebar/) .. and I love lager [log tool, make those erlang logs less alien looking] (http://basho.com/blog/technical/2011/07/20/Introducing-Lager-A-New-Logging-Framework-for-ErlangOTP/ http://basho.com/blog/technical/2011/07/20/Introducing-Lager...) (note) This is basically a strategy of using small clusters based on locations -- so if you are across lets say 3 locations, you would build 3 node clusters, 1 node per location and have them work as a unit localizing workloads and responding to requests, and then you allow your higher level middle-ware to deal with your many groups of "micro clusters". High reliability rather cheaply, but means you need your own system for pushing out updates.
dysinger 14y ago
Yawn. How about 2 million connections from a single server with Erlang & FreeBSD? http://blog.whatsapp.com/index.php/2012/01/1-million-is-so-2011/ http://blog.whatsapp.com/index.php/2012/01/1-million-is-so-2...
- testing12341234 14y ago
  250k concurrent connections on "... a $0.12/hr rackspace 2GB cloud server." vs 2M concurrent connections on a 24 core server with 96GB of memory.

by ericz 14y ago

est 14y ago

what about other languages?

boyter 14y ago
I would be curious to know how something like Play! with all Async requests, or MVC with all Async would do. Probably similarly impressive, but I really have no idea.
- fleitz 14y ago
  Probably not, the key to making 250,000 connections work is not doing a lot for each of those connections, or doing it very infrequently. Once you're rendering views, etc it's hard to maintain 50,000 req/sec. (250K connections @ 5 req/sec)
ww520 14y ago
The Netty library for Java routinely can do 500K connections without any sweat. Some people have tried 1M connections with beefy machines.
- ZoFreX 14y ago
  I wonder how far you could push Java with a naive Thread implementation. If you reduce the per-thread stack size you can quite easily get 20K on very modest hardware.

jimparkins 14y ago

People are excited by Node doing numbers like this because there is a massive active Javascript community - with hundreds of thousands of people using Javascript all day every day at work. 0.01% of these people would ever consider learning Erlang, and even if they did they would not be able to use it at work - ever. As with everything having better features means nothing if nobody adopts. I am not saying nobody uses Erlang and I am not saying people are not adopting it - but the number are just not comparable to the Javascript community . Lastly just because you know a but of Javascript I realise that this does not mean you can architect massive real time systems. But it is like WOW even people that play casually aspire to having the best kit or playing for a top guild.

mediocregopher 14y ago
>As with everything having better features means nothing if nobody adopts Well, it means my product is going to be superior since I went the better, if less well known, architecture. It's not like erlang is a little unsupported side-project of a language, it's actually older then javascript if you count the time period before it was open-sourced, and only a few years younger if you don't, and is used extensively by many industries. Also, just because javascript the language is more well-known doesn't mean javascript the server architecture is more well-known. I would argue it isn't; when people want a highly concurrent, solid server, erlang is always mentioned. Lastly, erlang is a pretty easy language to learn. I had the basics down in a day, I had a prototype pubsub server that could handle 50k connections in two. The syntax is a bit strange, and honestly it does get in the way sometimes, but it's not hard.
- maigret 14y ago
  You're missing the point of Node. You can construct a web rendering application and its AJAX parts in one codebase. You can move fastly code between server and client rendering. etc etc etc. Probably Erlang is "better". BTW, Java & C are quite fast also. Java applications, well written, do scale. If they don't, they are folks out there specialized to make them scale. Also, Erlang is probably easy to learn as language. But when you develop a web app, you have enough other skills to keep up with. Let's name CSS for one ;) The human brain is limited in its capacity to remember API and language specifics. Also, more popular means more libraries, which makes the product in turn better. This is why so many folks turn to PHP. It's not elegant, but everything you need is already here. Now I won't argue that you may have good reasons to use Erlang for yourself, may it be because you like the language structure, like to write libraries by yourself, or so on. But it doesn't make it "superior", foremost not as a platform.

scarmig 14y ago

Interesting example from 2008: http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-1 http://www.metabrew.com/article/a-million-user-comet-applica... I'd also add that this shouldn't be taken as something to say that Erlang is totally superior to node or that Erlang makes scaling to 1M concurrent connections a piece of cake. If you're working at that level, there's no magical out of the box solution.

ricardobeat 14y ago

The magic is in javascript. How many people can write Erlang?

luriel 14y ago
How many people can write clean, efficient and maintainable JavaScript? And Erlang is not the only sane option for this kinds of problems, there is also Go. And to a lesser degree you can do the same in many other languages given the right libraries and careful thinking, it takes more effort than with Erlang or Go, but almost anything beats JavaScript in both performance and code clarity (both at the 'low' code-readability level, and at the high 'project organization and design' level).
- davidw 14y ago
  > almost anything beats JavaScript in both performance http://shootout.alioth.debian.org/u32/which-programming-languages-are-fastest.php http://shootout.alioth.debian.org/u32/which-programming-lang... Javascript with V8 stacks up pretty well.
jeltz 14y ago
Depends on where you are. In Stockholm I would not be surprised if more people can write decent Erlang compared to the people who can write decent JavaScript.
- igouy 14y ago
  Would you be surprised if in Stockholm more people can write decent JavaScript compared to the people who can write decent Erlang?

pron 14y ago

... or Java.

ebiester 14y ago

How fast can I get a median programmer to learn Erlang, learn the libraries, and be productive enough to be able to make these high concurrency apps? Let's say they are a full stack programmer who knows some html, some css, some javascript, some java, and some sql. I have a pretty good idea how fast I can bring someone up to speed on node.js -- I have to teach them some advanced JS concepts, some node.js conventions, and the APIs of my library. Async takes a little bit to wrap your head around, but it's not terrible. Node.js seems like it is on the way to "worse is better."

rubyrescue 14y ago
about a week
ericmoritz 14y ago
A week with the Erlang language, which will enable to write projects that would be on par or better than anything written in node.js as far as resilient and scalability. Probably another week to get up to speed with OTP for all the promises of resilient Erlang applications. What is often missed about Erlang is that it's not really about highly concurrent applications. That property is actually a means to accomplish its primary goal: fault-tolerant applications. See: http://www.erlang.org/download/armstrong_thesis_2003.pdf http://www.erlang.org/download/armstrong_thesis_2003.pdf to understand the motives of the Erlang designers.
- Isofarro 14y ago
  Firstly, I apologise for down-voting this response. A week to learn Erlang. Where do I start?
  - MetaCosm 14y ago
    http://learnyousomeerlang.com/ http://learnyousomeerlang.com/ You can pick up the basic syntax in one full day easily (if you are an experienced developer and already understand functional programming) ... 3 or 4 days if you are new to functional coding or just very inexperienced. It is a very brief / minimalist language from a syntax point of view. Then, it will take a week or two to get your head around OTP, which is the primary framework and has years (decades?) of mission critical work under its belt. Then, at the end of your journey will be the really hard problems... dealing with massive netsplits at a cluster level, elections for new masters, and all the other hard problems that happen at the upper-tier of massive clusters. If you are building an HTTP(S) app -- you can blessedly avoid a lot of these by avoiding a true massive cluster all together and using lots of individual "micro clusters"(note) balanced / routed by HTTP middle-ware. Also, check out Cowboy [webserver] (https://github.com/extend/cowboy/ https://github.com/extend/cowboy/), Agner [package manager] (http://erlagner.org/ http://erlagner.org/) and of course, the always awesome rebar [build tool] (https://github.com/basho/rebar/ https://github.com/basho/rebar/) .. and I love lager [log tool, make those erlang logs less alien looking] (http://basho.com/blog/technical/2011/07/20/Introducing-Lager-A-New-Logging-Framework-for-ErlangOTP/ http://basho.com/blog/technical/2011/07/20/Introducing-Lager...) (note) This is basically a strategy of using small clusters based on locations -- so if you are across lets say 3 locations, you would build 3 node clusters, 1 node per location and have them work as a unit localizing workloads and responding to requests, and then you allow your higher level middle-ware to deal with your many groups of "micro clusters". High reliability rather cheaply, but means you need your own system for pushing out updates.

dysinger 14y ago

Yawn. How about 2 million connections from a single server with Erlang & FreeBSD? http://blog.whatsapp.com/index.php/2012/01/1-million-is-so-2011/ http://blog.whatsapp.com/index.php/2012/01/1-million-is-so-2...

testing12341234 14y ago
250k concurrent connections on "... a $0.12/hr rackspace 2GB cloud server." vs 2M concurrent connections on a 24 core server with 96GB of memory.