13 ms·
600K concurrent HTTP connections with Clojure and http-kit
- leoh 14y agoThis looks really cool. But can Apache and other frameworks accomplish this?
- cgag 14y agoHere are some benchmarks (that I'm not really fit to judge the quality of) that have been going around the Clojure mailing list for comparison: https://github.com/ptaoussanis/clojure-web-server-benchmarks https://github.com/ptaoussanis/clojure-web-server-benchmarks
- ptaoussanis 14y agoBenchmarks author here. I did my best to get representative numbers, but good benchmarking is tough and this isn't my area of expertise. There's almost certainly errors and room for improvement (suggestions very welcome!). In particular, I'm pretty sure nginx could be tuned here to lead. BTW I also ran benchmarks against a more realistic EC2 environment with proportionally similar results (EC2 is slow but its effect across servers was consistent).
- rorrr 14y ago1) Notice that in his tests 97% of these connections don't do anything, just idle. He maxes out at 18764 req/sec. If you google around, Apache and Nginx can do more than that on a beefy server. 2) Notice that they are "keep-alived", coming from the same IP, so not truly separate connections. 3) Keep in mind that 600K concurrent connections cannot possibly do anything useful at the same time for many reasons (CPU, bandwidth, server I/O), so they are not truly concurrent. 4) Max concurrent connections are also limited by the OS, and that limit is much much lower than 600K by default: http://serverfault.com/questions/10852/what-limits-the-maximum-number-of-connections-on-a-linux-server http://serverfault.com/questions/10852/what-limits-the-maxim...
- programnature 14y agobut is this an apple-to-apple comparison to the node.js results referenced in the post?
- codeka 14y agoIt seems to me it's not because the Node.js test used a remote machine to drive the connections whereas this one is driving the connections from the same machine. Running the server and test harness on the same machine bypasses a whole lot of networking stack. Also, it seems the limitation in Node.js is the maximum heap size of around 1.4GB [1]. This test used a heap size of 3GB which is just not possible in Node (until the limitation is removed, anyway). [1]: http://blog.caustik.com/2012/04/10/node-js-w250k-concurrent-connections/ http://blog.caustik.com/2012/04/10/node-js-w250k-concurrent-...
- shenedu 14y agoauthor here. Yes, Clojure can use more than 1.4G of heap, and can use threads. http-kit is multithreaded (one just for IO, others for computing response). A strength of Clojure?
- notimetorelax 14y agoFrankly as somebody pointed out http-kit is written mostly in Java, so I would say that you're showing strength of JVM.
- shenedu 14y agoYes, JVM is strong. Clojure inherit it.
- shenedu 14y agoauthor here. > Notice that in his tests 97% of these connections don't do anything, just idle. He maxes out at 18764 req/sec. Yes, just testing how many concurrent connection can be held. When the 600k are held, ab confirms that it can do about 31405.53 per seconds, the http body is 1024bytes. > Notice that they are "keep-alived", coming from the same IP, so not truly separate connections Not from the same ip, from many ips: 192.168.1.200~230 > Keep in mind that 600K concurrent connections cannot possibly do anything useful at the same time for many reasons (CPU, bandwidth, server I/O), so they are not truly concurren They send a request every 5s~30s to server, and wait for response
- mbell 14y agoMaybe I'm not understanding what is going on here but it appears the author's client is communicating over a socket on the same machine as the server. The author is seeing insanely high numbers because she/he is bypassing the entire TCP/IP stack.
- ptaoussanis 14y agoI believe this test was run locally for simplicity (and so people can reproduce it easily). As you've pointed out, that's a pretty artificial context and can obviously have a big absolute effect on the numbers. I will say that in my limited experience these numbers are proportionally representative of what you can get in a real-world environment. Even testing locally, getting concurrency numbers like this is tough or impossible with many (most?) servers. So, yes, it's an artificial test - but I think the results are interesting when taken in the correct context.
- javajosh 14y agoRunning locally doesn't mean you bypass TCP/IP.
- mbell 14y agoWhile true, I'm suspect given this setup that some layer isn't just dropping to a unix socket when it notices localhost. Most notably, 600k active real TCP connections in the kernel would use somewhere around 6GB of memory assuming an average of ~10kB of memory per socket for the R/W buffers and other data. EDIT: Thats just the server side, double to include the client sockets. Most attempts I've see at this number of real TCP connections required a lot more tweaking of kernel TCP settings to achieve.
- justincormack 14y agoHe has 16G of RAM so may have not tuned this, and not noticed.
- alexkus 14y agoFor large number (1M+) of connections (doing websockets or long polling) to replicate the functionality of a COMET server I've been playing with rolling my own TCP handling via libnetfilter_queue as I simple don't need the ~10kB of r/w buffers on each socket. Linux (as far as I'm aware) doesn't allow tuning of r/w buffer sizes on a per interface basis otherwise I'd have one interface for the COMET server with drastically reduced r/w buffer sizes and the remaining interfaces with 'normal' TCP r/w buffer sizes to ensure the other things running on that host run without problem.
- ptaoussanis 14y agoFeng (http-kit's author) should be around shortly if anyone has any questions. In the meantime, you can also check out http://http-kit.org http://http-kit.org for more info. That page is a work-in-progress so please excuse any errors. We were actually planning to post to HN later this week; seems someone beat us to the punch :-)
- egeozcan 14y agoDon't get me wrong, I really like what you are doing but there are a lot of spelling and grammar errors in your web site. I'm not a native speaker (I also make a lot of errors) but maybe you can get some help from one? Not a big deal but just wanted to let you know. Makes sense if you want to get popular =) I also have a question: Do you plan to make any comparison tests with Compojure (https://github.com/weavejester/compojure https://github.com/weavejester/compojure) and lib-noir?
- ptaoussanis 14y agoThe library author is Chinese, so English isn't his native language. We'll be cleaning up typos soon - this post caught us by surprise. As for Compojure, etc.- those are libraries that operate on top of a Ring web server. Jetty is the default, http-kit is a drop-in replacement. So basically, you'd use both http-kit and whatever other libraries you normally would (like Compojure). I swapped out a production Jetty+Compojure app to use http-kit+Compojure by changing ~20 lines of code. Hope that makes sense?
- egeozcan 14y agoYes it does, thanks for the clarification.
- deleted 14y ago[deleted]
- Legion 14y agoDepending on a browser's screen width, the "Fork me on Github" bar's clickable box can almost completely cover the "Blogs" menu item.
- shenedu 14y agoHey, http-kit's author here. willing to answer any questions.
- danielrhodes 14y agoAdd an endpoint with a poorly optimized SQL query: 20 concurrent connections. I find these benchmarks to not be very valuable. Typically the bottlenecks are in the database, not in the web server/code. Even when it is in the code, optimizations only become truly valuable when you are at scale or doing something either horribly wrong or algorithmically unique.
- tlrobinson 14y agoIn an asynchronous/event driven server slow SQL queries won't limit your concurrency, just latency. Memory consumption is what limits concurrency. CPU and bandwidth limits throughput. That said, I agree it's not a particularly useful benchmark.
- fernandezpablo 14y ago> In an asynchronous/event driven server slow SQL queries won't limit your concurrency Since this is on the JVM, SQL almost always means JDBC and JDBC is blocking so yes, SQL will in fact limit your concurrency.
- shenedu 14y agoyes, SQL is blocking, JVM also have threads. The http-kit is asynchronous(epoll on linux), the API for programmers is sync, which is more familiar.
- pygy_ 14y agoAlternatively, you can use a message queue or a REST service, and whatever lightweight, async database frontend.
- fernandezpablo 14y agoThat would just move the blocking code elsewhere, and now you have one extra framework / message queue on your stack (free complexity)
- dotborg 14y agoOnce You give a little bit more of real code into those requests your JVM will die from continous GC. It's simple math :)
- deleted 14y ago[deleted]
- shenedu 14y agoAuthor here. The server is open source, on github: https://github.com/http-kit/http-kit https://github.com/http-kit/http-kit Test test code is also on github: https://github.com/http-kit/scale-clojure-web-app https://github.com/http-kit/scale-clojure-web-app Suggestions very welcome!
- stiff 14y agoThe server itself seems to be written in Java: https://github.com/http-kit/http-kit https://github.com/http-kit/http-kit I wonder what the factors were for not writing it in Clojure if Clojure is the target platform.
- shenedu 14y agoauthor here: Java part: 1. Java NIO's performance is amazing: event driven r/w bytes 2. maintain state machine when parsing HTTP from bytes buffer need many local variables, I get used to do it in a C style code. Clojure: I like this language. It's brilliant. So I write a fast http server/client for it. Clojure is also write in JAVA, great interoperation
- nqzero 14y agocan the http-kit server be used from java directly ? last i looked, all the async java webservers seemed limited to 1000s of connections, which largely defeats the purpose. i'm willing to give up the servlet spec if needed
- shenedu 14y agoHey, http-kit can be used to from java directly: https://github.com/http-kit/http-kit/blob/master/test/java/org/httpkit/server/MultiThreadHttpServerTest.java https://github.com/http-kit/http-kit/blob/master/test/java/o... Not recommended! The API is very lowlevel. Maybe you can try to tweak the max allowed open file to a larger value. The default is about 1024, that how you just get 1000(I guess). Jetty is quite good at concurrency, you can double check it. Why not try Clojure? Your web dev's productivity increase by few times instantly.
- nqzero 14y agolooks good. my application (database-ish) is in java and i haven't worked on any bindings yet. just trying to put together a demo that shows off the concurrency
- 14y ago
- zwischenzug 14y agoSimilar article here, going to 11: http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-1 http://www.metabrew.com/article/a-million-user-comet-applica...
- jadc 14y ago+ 1 for the C1M comet application The article provides a lot more in-depth information on the subject and more experimental data (across multiple machines - not only localhost).
- dschiptsov 14y agoWhat is memory usage per connection? How much unnecessary data copying is going on? What is latency? What happens under the real load of, say, a thousand concurrent TCP connections together with few thousand of back-end/other data-sources pending calls? What will happen to memory usage and latency when simple setup above will serve simplest remote requests (which means stalled connections, re-tranmitions, etc) for 8 hours? 24 hours?
- shenedu 14y agohttp-kit needs a few kilobytes of memory (buffer for parsing HTTP request, maintain state, etc) for a connection. The thread model used by http-kit: a dedicated thread(server-loop), only doing events IO and parsing, when done, queued the request for a thread poll to take it, thread pool compute the response, queue it for the server-loop thread to write it back to client. Since epoll and kqueue's readiness selection is O(1), idle connections does not hurt latency at all.
- ptaoussanis 14y agoMemory use is a few KB per connection. Latency is good (same ballpark as nginx in my tests with most workloads). Can't comment on data copying. This is a new server so obviously nowhere near battle-tested yet, but anecdotally: Both the author and I are running/testing small/medium sized deployments that are performing quite reliably. It's much too soon to say anything conclusively, but I would say the early data I've seen is promising.
- billiob 14y agoThis reminds me of http://blog.whatsapp.com/index.php/2012/01/1-million-is-so-2011/ http://blog.whatsapp.com/index.php/2012/01/1-million-is-so-2... where they handled 2 millions concurrent tcp connections.
- z3phyr 14y agoAnybody heard about joxa? Think about erlang beam features on a clojure with scheme like simplicity.
- jlward4th 14y agoI've created the same test app with Play 2.1 RC2 and Scala: https://github.com/jamesward/scale-play-web-app https://github.com/jamesward/scale-play-web-app On my laptop I have half as many cores as the poster and am getting about half the performance.