4 ms·

15k/rps is on the low side for Java, so I found the post to be a little strange. The netty library can handle 10x that without batting an eyelid. We do much m

15k/rps is on the low side for Java, so I found the post to be a little strange. The netty library can handle 10x that without batting an eyelid. We do much more than 15k/s on API nodes with hardly any attention paid to optimisation around traffic. Just to pick a random stack overflow post, as a little anecdata to go with anecdote https://stackoverflow.com/questions/44708704/netty-as-high-performant-http-server-to-handle-2-3-million-requests-sec https://stackoverflow.com/questions/44708704/netty-as-high-p... has someone who's complaining they were only hitting 50k POST/sec with Netty.

no_wizard 3y ago
Maybe they wanted to share their experience with Vert.x and it is just poor form to focus on the 15K a second?
gmokki 3y ago
We did 10k rps almost 20 years ago on pair of Sun x4100 servers. Dual core Opteron@2,4GHz and Java 1.4. And that was with storing data to spinning rust (async) and business logic using custom scripting language.
- masklinn 3y ago
  C10k was considered a challenge 20 years ago. This was the reference c10k hardware: > You can buy a 1000MHz machine with 2 gigabytes of RAM and an 1000Mbit/sec Ethernet card for $1200 or so.
  - jsd1982 3y ago
    IIRC, C10k was about maintaining 10k parallel connections, not requests per seconds.
coffeebeqn 3y ago
These benchmarks are a little pointless. 15k/s is either amazing or poor or average depending on what actual work the server does. If it’s just a glorified CDN as in many benchmarks then it’s not that great(?)
jerrygenser 3y ago
Most of the ability to get throughput is based on not blocking a request on network or other IO and also not doing writes in band of request. I think it's less about the web server choice and more about the architecture around pre-computing and caching certain results in memory and then asynchronously writing via a queue.
- rcme 3y ago
  For certain web servers that utilize an event loop instead of threading, it becomes very easy to accidentally block the event loop with synchronous work, so the choice of server does matter.
  - jerrygenser 3y ago
    Yes it's easy to block the event loop. I agree that it matters in that you're much more likely or even able to hit this throughput by using an event loop runtime. In other words, having 32/64 concurrent threads you won't get the throughput you want so agree there. However, I'll elaborate on my point and why it's not just "use event loop". If you block the event loop you can get catastrophic behavior, like your entire server acts as if it's a single threaded synchronous runtime. However, even with not blocking the event loop, you need to have optimal IO __given a coroutine__. If you are waiting a bunch of ms for multiple network calls and also a db write within a coroutine, your latency is going to go up and you might not hit your latency requirements. I was specifying that the architecture was taking these concepts into account and getting low latency on each request in a high throughput way by not blocking on writes, precaching info in server memory, etc. If they had to go to a db and/or do other calculations per query instead of pre-caching regardless of 32/64 or using an event loop, it likely would have either created another bottleneck or the web server would bottleneck and they would not hit their RPS
- 10000truths 3y ago
  You can maintain high throughput while performing in-band I/O as long as the requests are pipelined and the I/O is batched. What gets affected is the request latency.

no_wizard 3y ago

Maybe they wanted to share their experience with Vert.x and it is just poor form to focus on the 15K a second?

gmokki 3y ago

We did 10k rps almost 20 years ago on pair of Sun x4100 servers. Dual core Opteron@2,4GHz and Java 1.4. And that was with storing data to spinning rust (async) and business logic using custom scripting language.

masklinn 3y ago
C10k was considered a challenge 20 years ago. This was the reference c10k hardware: > You can buy a 1000MHz machine with 2 gigabytes of RAM and an 1000Mbit/sec Ethernet card for $1200 or so.
- jsd1982 3y ago
  IIRC, C10k was about maintaining 10k parallel connections, not requests per seconds.

coffeebeqn 3y ago

These benchmarks are a little pointless. 15k/s is either amazing or poor or average depending on what actual work the server does. If it’s just a glorified CDN as in many benchmarks then it’s not that great(?)

jerrygenser 3y ago

Most of the ability to get throughput is based on not blocking a request on network or other IO and also not doing writes in band of request. I think it's less about the web server choice and more about the architecture around pre-computing and caching certain results in memory and then asynchronously writing via a queue.

rcme 3y ago
For certain web servers that utilize an event loop instead of threading, it becomes very easy to accidentally block the event loop with synchronous work, so the choice of server does matter.
- jerrygenser 3y ago
  Yes it's easy to block the event loop. I agree that it matters in that you're much more likely or even able to hit this throughput by using an event loop runtime. In other words, having 32/64 concurrent threads you won't get the throughput you want so agree there. However, I'll elaborate on my point and why it's not just "use event loop". If you block the event loop you can get catastrophic behavior, like your entire server acts as if it's a single threaded synchronous runtime. However, even with not blocking the event loop, you need to have optimal IO __given a coroutine__. If you are waiting a bunch of ms for multiple network calls and also a db write within a coroutine, your latency is going to go up and you might not hit your latency requirements. I was specifying that the architecture was taking these concepts into account and getting low latency on each request in a high throughput way by not blocking on writes, precaching info in server memory, etc. If they had to go to a db and/or do other calculations per query instead of pre-caching regardless of 32/64 or using an event loop, it likely would have either created another bottleneck or the web server would bottleneck and they would not hit their RPS
10000truths 3y ago
You can maintain high throughput while performing in-band I/O as long as the requests are pipelined and the I/O is batched. What gets affected is the request latency.