4 ms·

Cowboy measures its latency in microseconds, what bottlenecks are you running into specifically?

RobertKerans 8y ago
I assume perf comment is in relation to AWS Lambda, for which the Elixir example they've put up is severely below par perf wise
- StreamBright 8y ago
  I am talking about raw HTTP request handling performance. Elixir/Cowboy/Plug is in the middle range of web servers. Again, it is good enough for most use cases.
  - RobertKerans 8y ago
    Ah, fair enough, apologies for misconstruing. Definitely good enough for a very wide range of use cases; it's not a language built for speed and power anyway.
StreamBright 8y ago
Not really, you can't have a microsecond latency talking over a network. I understand localhost microbenchmarks are nice but real life scenarios are much better. The setup for the test: - provision node A for being the server - provision node B for being the client - open X (16..16000) connections from node A and use http pipelining start to send requests to node B I use wrk2 as the test client it is pretty amazing and looking at the latency distribution graphs. Tools that clear winners of performance: - https://github.com/valyala/fasthttp https://github.com/valyala/fasthttp - https://www.rapidoid.org https://www.rapidoid.org Elixir/Cowboy/Plug is in the middle range, kind of like what Techempower[1] guys saw during their tests. [1] https://www.techempower.com/blog/2018/10/30/framework-benchmarks-round-17/ https://www.techempower.com/blog/2018/10/30/framework-benchm...
- pmarreck 8y ago
  Wrk2 is not fast enough and will thus give spurious results since the bottleneck is the client. The only measuring tool that is fast enough to accurately measure Phoenix performance is something like Tsung (which is also an Erlang app...)
  - StreamBright 8y ago
    I am not sure what you are talking about. wrk2 can put out 7M req/s on a 16 core box. That is way beyond Phoenix's performance on the same HW type. wrk2 is widely used and accepted performance measurement tool. Again, you mentioned microseconds latency which means you are talking about localhost microbenchmarking. That is irrelevant from the production workload point of view. I have saturated network links successfully with wrk2 which is the definition of fast enough. Interestingly there was a thread on HN previously which tools are used for HTTP perf testing: >>> - Wrk: https://github.com/wg/wrk https://github.com/wg/wrk - Fastest tool in the universe. About 25x faster than Locust. 3x faster than Jmeter. Scriptable in Lua. Drawbacks are limited output options/reporting and a scripting API that is callback-based, so painful to use for scripting user scenario flows. https://news.ycombinator.com/item?id=15738967 https://news.ycombinator.com/item?id=15738967 https://news.ycombinator.com/item?id=15733910 https://news.ycombinator.com/item?id=15733910
    - pmarreck 8y ago
      alright. this looks interesting. I'll have to dive back into this space and see what's changed, I'm probably out of the loop. What's the diff between "wrk" and "wrk2"?

I assume perf comment is in relation to AWS Lambda, for which the Elixir example they've put up is severely below par perf wise

StreamBright 8y ago
I am talking about raw HTTP request handling performance. Elixir/Cowboy/Plug is in the middle range of web servers. Again, it is good enough for most use cases.
- RobertKerans 8y ago
  Ah, fair enough, apologies for misconstruing. Definitely good enough for a very wide range of use cases; it's not a language built for speed and power anyway.

StreamBright 8y ago

Not really, you can't have a microsecond latency talking over a network. I understand localhost microbenchmarks are nice but real life scenarios are much better. The setup for the test: - provision node A for being the server - provision node B for being the client - open X (16..16000) connections from node A and use http pipelining start to send requests to node B I use wrk2 as the test client it is pretty amazing and looking at the latency distribution graphs. Tools that clear winners of performance: - https://github.com/valyala/fasthttp https://github.com/valyala/fasthttp - https://www.rapidoid.org https://www.rapidoid.org Elixir/Cowboy/Plug is in the middle range, kind of like what Techempower[1] guys saw during their tests. [1] https://www.techempower.com/blog/2018/10/30/framework-benchmarks-round-17/ https://www.techempower.com/blog/2018/10/30/framework-benchm...

pmarreck 8y ago
Wrk2 is not fast enough and will thus give spurious results since the bottleneck is the client. The only measuring tool that is fast enough to accurately measure Phoenix performance is something like Tsung (which is also an Erlang app...)
- StreamBright 8y ago
  I am not sure what you are talking about. wrk2 can put out 7M req/s on a 16 core box. That is way beyond Phoenix's performance on the same HW type. wrk2 is widely used and accepted performance measurement tool. Again, you mentioned microseconds latency which means you are talking about localhost microbenchmarking. That is irrelevant from the production workload point of view. I have saturated network links successfully with wrk2 which is the definition of fast enough. Interestingly there was a thread on HN previously which tools are used for HTTP perf testing: >>> - Wrk: https://github.com/wg/wrk https://github.com/wg/wrk - Fastest tool in the universe. About 25x faster than Locust. 3x faster than Jmeter. Scriptable in Lua. Drawbacks are limited output options/reporting and a scripting API that is callback-based, so painful to use for scripting user scenario flows. https://news.ycombinator.com/item?id=15738967 https://news.ycombinator.com/item?id=15738967 https://news.ycombinator.com/item?id=15733910 https://news.ycombinator.com/item?id=15733910
  - pmarreck 8y ago
    alright. this looks interesting. I'll have to dive back into this space and see what's changed, I'm probably out of the loop. What's the diff between "wrk" and "wrk2"?