3 ms·

I'm not sure what your point here is. Facebook chat doesn't use CPU. Why would it need 16 cores? It's more than likely IO bound. It's just moving boring data a

by axod 16y ago

I'm not sure what your point here is. Facebook chat doesn't use CPU. Why would it need 16 cores? It's more than likely IO bound. It's just moving boring data around. If you're using serious CPU power to write a chat backend, you're doing something really badly or using a crappy tool. Your "But I need more CPU cores!!!" argument is moot.

nivertech 16y ago
I don't need more CPUs. But can you find a single core machine nova days? Most of the servers are 16 cores, which in case of evented framework will be heavily underutilized.
- axod 16y ago
  > "which in case of evented framework will be heavily underutilized." I'm not following your logic here. If something doesn't need 16 CPUs, why is splitting it over 16 cores a good idea? You're going to be adding a lot of inefficiency, complexity etc for no gain. Most of mine are dual or possibly 4 core. One core for OS stuff, other processes etc, one core to do networking IO for me. As I say, networking IO shouldn't be using CPU. If you're doing long or blocking jobs, pass it over to some other thread. (That's a far better way to utilize multiple cores than splitting the whole networking IO over multiple cores).
  - WALoeIII 16y ago
    "Pass it over to some other thread" What you are suggesting here is exactly what Erlang provides for you. Each connection is a light-weight process inside the VM. The VM then schedules these Erlang processes against native threads. The difference is Erlang has perfected and proven that it can do this reliably over 20 years of development. You would be implementing this by hand, and guaranteed to screw up because its a very hard problem and you are not the smartest person in the world. Hybrid solutions with Evented I/O and Multi-threading end up being the solution, Erlang is a hybrid out of the box.
    - axod 16y ago
      I have implemented it. It works well. You don't have to be the smartest person in the world to not screw up elementary programming problems. Mibbit handles a few million users without issue on a few VPSes. Erlang "doing it for me" isn't going to be anywhere near as efficient as me doing it myself. I'm probably not the smartest person in the world, but I'm certainly smart enough to write a comet/web server that's fast enough. I guess I'm not going to convince anyone here shrug. If you believe Erlang is a magic bullet great for you.
      - lzimm 16y ago
        Totally agree. Since when did a work queue/thread pool thing become anything beyond intro to systems work? Furthermore, I don't know where everyone gets off saying erlang is so fast either: I'm not sure where things are now, but last time I checked, their scheduling algorithms were leaving quite a lot on the table, standing to benefit quite substantially from even the most basic operations and logistics knowledge.
        revoltingx 16y ago
        Ugh, because you'd have to be pretty stupid to try to implement what Erlang/OTP has yourself. Yeah it's easy, but it'll get pretty hairy once you integrate hot-code updates, complex/simple CPU usage, load distribution, etc (i.e. you can write your app single threaded, then run 4 erlang VMS and distribute the load between all four of them.) The ability to easily swap between parallel and sequential functions easily, cross-process/cross-network seamless communication, etc. All of this without ever having to restart the VM or having 'downtime.' The point of Erlang is that it makes all of this so easy that you don't even think about it. It's the scripting language of parallel computing. You can rapidly prototype the app using the built-in tools and fine-tune whatever needs to be refined with C or better parallel algorithms or both.
cgbystrom 16y ago
An ordinary Comet server with long-poll can actually end up eating quite some CPU. I run beaconpush.com which is a cloud-based service for browser push messages. We're based on Netty and Java, which works out well for us. We've looked at Erlang and wrote some initial prototypes in it but Java became our language of choice. Mainly because we knew it and had some experiences with Netty, a NIO library for Java. Netty did also outperform Erlang and mochiweb (both vanilla configured). Anyway, the problem tend to be with long-poll that users re-establish their connection back to the servers each time they receive a message or navigate to another page (or reload the page). This end up taking a big toll on the system. We're to some extent limited to how fast we can perform accept() on our sockets. If you use long-polling with multi-part you can get away with sending more messages on a single established connection (it becomes request/response/response/response or similar). That can reduce the system load and the use of WebSockets can eliminate the use of reconnections altogether (disregarding any page reloads). Facebook's use of AJAX navigation (i.e not reloading the entire page when user clicks on links etc) also reduces this load. This due to not having to re-establish a connection each time a user reloads a page. So yes, we're actually CPU bound by the accept() behavior (at least to how the JVM does accept()). But would our connection be more permanent in nature then no, we would of course be more I/O bound if not completely.
- axod 16y ago
  I'd do some more in depth checks. I find it very hard to believe you're cpu bound within NIO itself. I do around 1k req/s per cpu and never see load avg above 0.5 >> "re-establish their connection back to the servers each time they receive a message or navigate to another page (or reload the page). This end up taking a big toll on the system. We're to some extent limited to how fast we can perform accept() on our sockets." I'd expect what you're seeing is the overhead of your system creating new objects initializing requests etc rather than anything low level. (Mibbit uses keep alive long polling XHR / websocket, using custom app server in Java +NIO)
  - cgbystrom 16y ago
    It turns out that Java synchronizes on accept() capping the number of socket accepts that can made each second on a single port. But even if that would be solved or if you used something more low-level like, say nginx, you'd still be limited of how fast you can accept(). Try using ApacheBench and hit one server with keep-alive on and off respectively. Even nginx, which I would say is a performant web server, show quite some discrepancies between having to reconnect each time and having a permanent connection. I file this under the penalty of having to set up and tear down a socket each time. Something that only can be avoided by doing less reconnecting. I could of course be wrong but these are the findings from doing performance testing and discussing it on the Netty mailing list. But I'd gladly accept suggestions on how to improve this situation for our service.

by axod 16y ago

nivertech 16y ago

I don't need more CPUs. But can you find a single core machine nova days? Most of the servers are 16 cores, which in case of evented framework will be heavily underutilized.

axod 16y ago
> "which in case of evented framework will be heavily underutilized." I'm not following your logic here. If something doesn't need 16 CPUs, why is splitting it over 16 cores a good idea? You're going to be adding a lot of inefficiency, complexity etc for no gain. Most of mine are dual or possibly 4 core. One core for OS stuff, other processes etc, one core to do networking IO for me. As I say, networking IO shouldn't be using CPU. If you're doing long or blocking jobs, pass it over to some other thread. (That's a far better way to utilize multiple cores than splitting the whole networking IO over multiple cores).
- WALoeIII 16y ago
  "Pass it over to some other thread" What you are suggesting here is exactly what Erlang provides for you. Each connection is a light-weight process inside the VM. The VM then schedules these Erlang processes against native threads. The difference is Erlang has perfected and proven that it can do this reliably over 20 years of development. You would be implementing this by hand, and guaranteed to screw up because its a very hard problem and you are not the smartest person in the world. Hybrid solutions with Evented I/O and Multi-threading end up being the solution, Erlang is a hybrid out of the box.
  - axod 16y ago
    I have implemented it. It works well. You don't have to be the smartest person in the world to not screw up elementary programming problems. Mibbit handles a few million users without issue on a few VPSes. Erlang "doing it for me" isn't going to be anywhere near as efficient as me doing it myself. I'm probably not the smartest person in the world, but I'm certainly smart enough to write a comet/web server that's fast enough. I guess I'm not going to convince anyone here shrug. If you believe Erlang is a magic bullet great for you.
    - lzimm 16y ago
      Totally agree. Since when did a work queue/thread pool thing become anything beyond intro to systems work? Furthermore, I don't know where everyone gets off saying erlang is so fast either: I'm not sure where things are now, but last time I checked, their scheduling algorithms were leaving quite a lot on the table, standing to benefit quite substantially from even the most basic operations and logistics knowledge.
      - revoltingx 16y ago
        Ugh, because you'd have to be pretty stupid to try to implement what Erlang/OTP has yourself. Yeah it's easy, but it'll get pretty hairy once you integrate hot-code updates, complex/simple CPU usage, load distribution, etc (i.e. you can write your app single threaded, then run 4 erlang VMS and distribute the load between all four of them.) The ability to easily swap between parallel and sequential functions easily, cross-process/cross-network seamless communication, etc. All of this without ever having to restart the VM or having 'downtime.' The point of Erlang is that it makes all of this so easy that you don't even think about it. It's the scripting language of parallel computing. You can rapidly prototype the app using the built-in tools and fine-tune whatever needs to be refined with C or better parallel algorithms or both.

cgbystrom 16y ago

An ordinary Comet server with long-poll can actually end up eating quite some CPU. I run beaconpush.com which is a cloud-based service for browser push messages. We're based on Netty and Java, which works out well for us. We've looked at Erlang and wrote some initial prototypes in it but Java became our language of choice. Mainly because we knew it and had some experiences with Netty, a NIO library for Java. Netty did also outperform Erlang and mochiweb (both vanilla configured). Anyway, the problem tend to be with long-poll that users re-establish their connection back to the servers each time they receive a message or navigate to another page (or reload the page). This end up taking a big toll on the system. We're to some extent limited to how fast we can perform accept() on our sockets. If you use long-polling with multi-part you can get away with sending more messages on a single established connection (it becomes request/response/response/response or similar). That can reduce the system load and the use of WebSockets can eliminate the use of reconnections altogether (disregarding any page reloads). Facebook's use of AJAX navigation (i.e not reloading the entire page when user clicks on links etc) also reduces this load. This due to not having to re-establish a connection each time a user reloads a page. So yes, we're actually CPU bound by the accept() behavior (at least to how the JVM does accept()). But would our connection be more permanent in nature then no, we would of course be more I/O bound if not completely.

axod 16y ago
I'd do some more in depth checks. I find it very hard to believe you're cpu bound within NIO itself. I do around 1k req/s per cpu and never see load avg above 0.5 >> "re-establish their connection back to the servers each time they receive a message or navigate to another page (or reload the page). This end up taking a big toll on the system. We're to some extent limited to how fast we can perform accept() on our sockets." I'd expect what you're seeing is the overhead of your system creating new objects initializing requests etc rather than anything low level. (Mibbit uses keep alive long polling XHR / websocket, using custom app server in Java +NIO)
- cgbystrom 16y ago
  It turns out that Java synchronizes on accept() capping the number of socket accepts that can made each second on a single port. But even if that would be solved or if you used something more low-level like, say nginx, you'd still be limited of how fast you can accept(). Try using ApacheBench and hit one server with keep-alive on and off respectively. Even nginx, which I would say is a performant web server, show quite some discrepancies between having to reconnect each time and having a permanent connection. I file this under the penalty of having to set up and tear down a socket each time. Something that only can be avoided by doing less reconnecting. I could of course be wrong but these are the findings from doing performance testing and discussing it on the Netty mailing list. But I'd gladly accept suggestions on how to improve this situation for our service.