3 ms·

What would decent metrics be to compare the performance of Nginx and Caddy?

by tmpz22 4y ago

francislavoie 4y ago
It's not really possible to compare the performance of webservers apples to apples. Really, you need to do your own benchmarking to determine which solution is best for you. But keep in mind that the web server is rarely the bottleneck, usually your app and database IO are where it takes the most time.
- pachico 4y ago
  Indeed, this is why those hello world benchmarks are just noise.
- password4321 4y ago
  What is this I'm reading here, the voice of reason?! The last frontier for these types of comparisons is unusual circumstances like embedded platforms with limited resources.
- KronisLV 4y ago
  > It's not really possible to compare the performance of webservers apples to apples. I'd say that it's possible, just not always easy, due to how many different configurations there can be for any given workload - the same servers might be used to achieve the approximately same configuration in different ways. However, if you took a common enough use case or workload, such as reverse proxy with SSL and gzip compression for $FOO resource types, serving files for $BAR application and proxying $BAZ API, then it should definitely be possible. You would just need some meaningful real-world test instead of a purely synthetic benchmark, otherwise you'll probably test something slightly different than what your server will be doing in practice. The other option is just to settle on some bit of common enough functionality and try to increase your sample size as much as possible. For example, some attempts have been made by OpenBenchmarking and at least give a vague idea of the performance of some servers: Nginx: https://openbenchmarking.org/test/pts/nginx https://openbenchmarking.org/test/pts/nginx Apache:https://openbenchmarking.org/test/pts/apache https://openbenchmarking.org/test/pts/apache I'm mentioning this because while not everybody has the time to reproduce a given setup in any number of technologies, even similar enough tests in a controlled environment can produce meaningful information, at least to let you infer what orders of magnitude you're working with. For something concerning programming languages themselves and their frameworks, one just needs to look at what TechEmpower is trying to do: https://www.techempower.com/benchmarks/#section=data-r21 https://www.techempower.com/benchmarks/#section=data-r21 > Really, you need to do your own benchmarking to determine which solution is best for you. But keep in mind that the web server is rarely the bottleneck, usually your app and database IO are where it takes the most time. This is well said, though. In general, I'm inclined to agree: most of the time, the performance of most web servers can be described as "good enough". The exception to this might be using your application servers (e.g. Tomcat) as a web server and running into situations where serving static assets (that might be baked into your application) would slow down because of API calls being slower and processing them digging into comparatively more conservative HTTP thread limits for the whole thing. Then again, personally I'd argue that you should have one of the popular web servers in front of your applications (and typically serving static assets) to act as an ingress in most cases, but I've seen some interesting things over the years.
  - francislavoie 4y ago
    Regarding that last point, that's often done with the X-Accel-Redirect header; backend can decide which file to serve, responds with the path to that file as that header, then the webserver can serve that file if that response header is there. Caddy can do this btw, with response handlers in the reverse_proxy directive. This pattern can be used to serve uploaded files stored in a non-public location, after the backend authenticated the request etc.
    - KronisLV 4y ago
      > This pattern can be used to serve uploaded files stored in a non-public location, after the backend authenticated the request etc. That's a perfectly valid use case! Packaging a front end application (e.g. React/Angular/Vue) inside of a .jar file and making Tomcat or something else serve it, as a part of a larger Java application, though? Perhaps less of an optimal solution than just using Apache/Nginx/Caddy for it, outside of really wanting to be able to deploy everything as a single package.

What would decent metrics be to compare the performance of Nginx and Caddy?

by tmpz22 4y ago

What would decent metrics be to compare the performance of Nginx and Caddy?

It's not really possible to compare the performance of webservers apples to apples. Really, you need to do your own benchmarking to determine which solution is best for you. But keep in mind that the web server is rarely the bottleneck, usually your app and database IO are where it takes the most time.

pachico 4y ago
Indeed, this is why those hello world benchmarks are just noise.
password4321 4y ago
What is this I'm reading here, the voice of reason?! The last frontier for these types of comparisons is unusual circumstances like embedded platforms with limited resources.
KronisLV 4y ago
> It's not really possible to compare the performance of webservers apples to apples. I'd say that it's possible, just not always easy, due to how many different configurations there can be for any given workload - the same servers might be used to achieve the approximately same configuration in different ways. However, if you took a common enough use case or workload, such as reverse proxy with SSL and gzip compression for $FOO resource types, serving files for $BAR application and proxying $BAZ API, then it should definitely be possible. You would just need some meaningful real-world test instead of a purely synthetic benchmark, otherwise you'll probably test something slightly different than what your server will be doing in practice. The other option is just to settle on some bit of common enough functionality and try to increase your sample size as much as possible. For example, some attempts have been made by OpenBenchmarking and at least give a vague idea of the performance of some servers: Nginx: https://openbenchmarking.org/test/pts/nginx https://openbenchmarking.org/test/pts/nginx Apache:https://openbenchmarking.org/test/pts/apache https://openbenchmarking.org/test/pts/apache I'm mentioning this because while not everybody has the time to reproduce a given setup in any number of technologies, even similar enough tests in a controlled environment can produce meaningful information, at least to let you infer what orders of magnitude you're working with. For something concerning programming languages themselves and their frameworks, one just needs to look at what TechEmpower is trying to do: https://www.techempower.com/benchmarks/#section=data-r21 https://www.techempower.com/benchmarks/#section=data-r21 > Really, you need to do your own benchmarking to determine which solution is best for you. But keep in mind that the web server is rarely the bottleneck, usually your app and database IO are where it takes the most time. This is well said, though. In general, I'm inclined to agree: most of the time, the performance of most web servers can be described as "good enough". The exception to this might be using your application servers (e.g. Tomcat) as a web server and running into situations where serving static assets (that might be baked into your application) would slow down because of API calls being slower and processing them digging into comparatively more conservative HTTP thread limits for the whole thing. Then again, personally I'd argue that you should have one of the popular web servers in front of your applications (and typically serving static assets) to act as an ingress in most cases, but I've seen some interesting things over the years.
- francislavoie 4y ago
  Regarding that last point, that's often done with the X-Accel-Redirect header; backend can decide which file to serve, responds with the path to that file as that header, then the webserver can serve that file if that response header is there. Caddy can do this btw, with response handlers in the reverse_proxy directive. This pattern can be used to serve uploaded files stored in a non-public location, after the backend authenticated the request etc.
  - KronisLV 4y ago
    > This pattern can be used to serve uploaded files stored in a non-public location, after the backend authenticated the request etc. That's a perfectly valid use case! Packaging a front end application (e.g. React/Angular/Vue) inside of a .jar file and making Tomcat or something else serve it, as a part of a larger Java application, though? Perhaps less of an optimal solution than just using Apache/Nginx/Caddy for it, outside of really wanting to be able to deploy everything as a single package.