4 ms·

You're right, this is fascinating stuff. Like I said, we had not yet received a pull request for the Vert.x test, but presumably we will get one before Round 5

• bilbo0s 14y ago
  Sorry (for the tardy response)... minor emergency came up. Anyway, the relevant part of the manual is: http://vertx.io/manual.html#concurrency http://vertx.io/manual.html#concurrency Also discovered you are running the equivalent of a single threaded environment anyway. (8 workers on an 8 core machine) Any reason for that??? Maybe more confusion about Vert.x concurrency??? If so, here is the relevant documentation on worker verticles: http://vertx.io/manual.html#worker-verticles http://vertx.io/manual.html#worker-verticles Long story short, you dedicated the entire machine (8 cores) to the equivalent of database connection management, (the persistor). Very little of the machine, (whatever is context switched in effectively), is dedicated to request handling. Try something a bit more fair... like 4 workers and 4 web request handlers. Also, I was going through the Node.js tests and I had a question... do you guys do any clustering for the Node tests at all? Or are these results from the tests run on a single Node? Sorry for all the questions, just want to make all of the tests do the same thing across all of the frameworks so that when you guys run it again, we can use that data here in a more meaningful fashion. For us, it is useful.
  • bhauer 14y ago
    We used 8 Vert.x verticles on i7 because there are 8 HT cores and our understanding is that the "best practice" for Vert.x is to create a number of verticles equal to the server's core count [1]. Obviously we would be happy to hear from Vert.x experts about an more idealized configuration. Admittedly, we have not spent a great deal of time attempting to tune Vert.x and, like I said earlier, we have not received a pull request [2]. In all tests, the database test is allocating a greater amount of effort to database connection management (in totality: the handling of connections, statements, queries, and result sets) versus request handling. This is not unique to Vert.x. The reason some frameworks' database tests achieve nearly 50% as many requests per second on i7 versus the pure JSON test is simply that at the ~210k rps range for the JSON tests, we are running into a Gigabit Ethernet wall (which I have commented about elsewhere). If we had 10 GBE, the JSON test results on i7 would be even higher. (Also see comments elsewhere about our intent to normalize, to a degree, the response header requirements since the variation observed is attributable to response-headers [3].) Yes, the node.js tests are running with the cluster module [4]. Thanks for the comments. We have received great feedback in the previous rounds and this round received even more attention so there have been some more good questions. Unfortunately, there has also been some rehashing, which indicates we're not doing a great job of explaining to people how each environment is configured (linking to the repository only goes so far). That said, we also continue to receive some fantastic pull requests. Thanks to everyone who has helped out! [1] http://vertxproject.wordpress.com/2012/05/09/vert-x-vs-node-js-simple-http-benchmarks/#comment-69 http://vertxproject.wordpress.com/2012/05/09/vert-x-vs-node-... [2] https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/vertx https://github.com/TechEmpower/FrameworkBenchmarks/tree/mast... [3] https://github.com/TechEmpower/FrameworkBenchmarks/issues/113 https://github.com/TechEmpower/FrameworkBenchmarks/issues/11... [4] https://news.ycombinator.com/item?id=5645598 https://news.ycombinator.com/item?id=5645598