6 ms·

C#/ASP.NET is the fastest web framework now: https://www.techempower.com/benchmarks/#section=test&runid=8ca46892-e46c-4088-9443-05722ad6f7fb&hw=ph&test=plainte

C#/ASP.NET is the fastest web framework now: https://www.techempower.com/benchmarks/#section=test&runid=8ca46892-e46c-4088-9443-05722ad6f7fb&hw=ph&test=plaintext https://www.techempower.com/benchmarks/#section=test&runid=8... 7.000.000 requests per second Even GO can only achieve 4.500.000 million requests per secnod being a low-level language, in opposite to high-level C#.

sgtnoodle 6y ago
The #1 project on that list appears to be implemented in rust? What does GO being "low-level" have to do with performance of serving requests? I'd imagine its bottlenecks would be due to something fairly arbitrary, like its garbage collection or how it represents strings or something.
nodamage 6y ago
That 7 million requests per second is achieved by writing a hard coded plain text HTTP response string directly to the client... it is so far disconnected from any real world use case that the number is basically meaningless.
- arcturus17 6y ago
  It's still 8th in the composite benchmark. And the criticism you're leveling would affect the entire benchmark design, rather than a particular framework score, no? [1] https://www.techempower.com/benchmarks/#section=data-r20&hw=ph&test=composite https://www.techempower.com/benchmarks/#section=data-r20&hw=...
  - BiteCode_dev 6y ago
    Yes but getting a good ranking at a useless benchmark if not very meaningful. It's cool, but I wouldn't use it to make any serious decision.
  - nodamage 6y ago
    > And the criticism you're leveling would affect the entire benchmark design, rather than a particular framework score, no? Indeed. The problem is that many of the scores in the top 100 are really misleading because no one building a web app would implement things that way. There is still some value in the lower down benchmarks but you have to basically read the underlying source to determine if the implementation is remotely realistic or not. For starters I would ignore anything classified as "platform" which is described as: Platform, meaning a raw server (not actually a framework at all). Good luck! You're going to need it. For C# in particular I would only consider the mvc variants as realistic. Edit: I looked into the "asp.net core" composite score a bit more, it looks like those benchmarks are based on the aforementioned "platform" implementations for each test. I actually think this score is even more misleading than the individual benchmarks. At least the individual benchmarks show you the difference between "aspcore" (platform), "aspcore-mw" (middleware-only), and "aspcore-mvc" (full framework with routing). Here are recalculated composite scores based on the more realistic implementations (aspcore-mvc, aspcore-mvc-ado-pg, aspcore-mvc-dap-pg, aspcore-mvc-ef-pg): ASP.NET Core MVC with ADO.NET (raw SQL): 3029 ASP.NET Core MVC with Dapper: 2591 ASP.NET Core MVC with Entity Framework: 2195 Compared to Flask's 468 or Django's 280 it's still significantly faster, but not to the same extreme you might think at first glance at the chart.
    - arcturus17 6y ago
      Yea I've seen that they do provide ORM benchmarks in the detailed tests. I still find it very impressive - as a full-featured framework it ranks well above the rest of its peers, despite the ORM. I am writing Django backends nowadays, but like I've mentioned elsewhere, I love its full-batteries approach and maturity, but I'm not very happy about it's meager async capabilities, and I think I'd prefer something with strong typing... I've always been put off by Microsoft's lock-in but it seems that's changed so at least I'd put the .net as a contender for side project in the near future.
qw 6y ago
C# is only at number 3 in your list. Both Java and Rust are above it in the list. It's also a very "artificial" benchmark and real world code will give different results (if you have static content, just put it in a CDN and don't worry) Other benchmarks from the same site: - JSON Serialization: C# is number 34 - Single query: C# is number 23 - Fortunes: C# is number 7
- nodamage 6y ago
  It needs to be emphasized how artificial these benchmarks really are. Here is the source for the Fortunes C# benchmark: https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/CSharp/aspnetcore/PlatformBenchmarks/BenchmarkApplication.Fortunes.cs https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast... There's no routing or templating, it just writes a bunch of strings. No one would build an actual web app this way. The only C# benchmarks that are remotely realistic are the mvc variants, starting with aspcore-mvc-ado-pg at number 79.
  - infensus 6y ago
    https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Project-Information-Framework-Tests-Overview#general-test-requirements https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Proj... Routing is required, but generally the rules allow things to be "reasonable" and "acceptable", which lets all these weird implementations through. Honestly, they should remove the "Implementation approach" column, because basically every implementation is marked as "realistic", making it meaningless.
    - nodamage 6y ago
      The "routing" implementation is more hard coded string matching: https://github.com/TechEmpower/FrameworkBenchmarks/blob/5b0e00e64b36deb46183f3c5edaa612e086c47f2/frameworks/CSharp/aspnetcore/PlatformBenchmarks/BenchmarkApplication.cs#L78 https://github.com/TechEmpower/FrameworkBenchmarks/blob/5b0e... I mean I guess technically that's routing but it's not remotely realistic. Only the mvc variants use the actual framework's routing system.
EvilEy3 6y ago
Go being low-level has nothing to do with performance. They deliberately keep feature set on this level, they're not making some kind of trade-off. Also, high performance C# is so low level that you might as well write C++. Or you think you will use EF, LINQ and have 7 millions rps?
- arcturus17 6y ago
  > Or you think you will use EF, LINQ and have 7 millions rps? I'm imagining not, but it's still comparatively faster than most, if not all, "full-batteries frameworks", right? I'm assuming the use of ORMs and such is more or less uniform in the comparison (eg, if they don't use EF for .net, they don't use Django ORM either) The overall performance of .net across these benchmarks really catches my attention like it does GP... I'm looking for a full-batteries framework based on a strongly typed language and never in my life I thought I'd say this, but it might be time to give .net / C# a whirl? I've heard really good things about the dev experience from people here on HN, F# is a really cool bonus, and the fact that it looks at least comparatively performant could be the icing on the cake.
  - akra 6y ago
    A framework that you can stay high level if you want and optimise when the rare but exceptional case arises on a particular route without splitting the process/resorting to C++ interop (with its own performance issues), or the risk of needing to rewrite due to performance to me is the big selling point. In a previous life I got some significant latency benefits by doing just that. I think its probably only gotten better with .NET 5.
Thaxll 6y ago
Hmm no they have the same speed with gnet ( Go ): https://www.techempower.com/benchmarks/#hw=ph&test=plaintext https://www.techempower.com/benchmarks/#hw=ph&test=plaintext 7M req/sec both of them. ( all those benchmarks are sort of useless anyway )
Ambix 6y ago
Why do enyone needs framework to print zillion Hello Worlds to the client? It's more interesting to see results of high-load DB tests, for example: https://www.techempower.com/benchmarks/#section=data-r20&hw=ph&test=query https://www.techempower.com/benchmarks/#section=data-r20&hw=...
- akra 6y ago
  Depends on your use case. I've written quite a few backends that didn't use a database and there are a number of cases for the pure HTTP benchmark. Pass through proxies with injected behaviour, in memory key-value data storage APIs, memory mapped files, etc. IMO not every shop offloads the state to its DB. Once you add the database you are really testing the DB driver - and that adds a lot more variance to the test. Maybe its just the DB driver for that particular DB? A fast "Hello World" benchmark implies that the HTTP/transport layers of the framework are very fast. That's your base and the lower bound to your best performance potential. As a real world example if .NET ASP NET Core has the best request/response benchmark and its better than say nginx (a popular reverse proxy) it might be better to have all your gateways using a reverse proxy implemented with that as its base instead. Over your whole network depending on your scale that could be a big cost and latency saving measure. I wouldn't be surprised if Microsoft or it's community have started writing one.

sgtnoodle 6y ago

The #1 project on that list appears to be implemented in rust? What does GO being "low-level" have to do with performance of serving requests? I'd imagine its bottlenecks would be due to something fairly arbitrary, like its garbage collection or how it represents strings or something.

nodamage 6y ago

That 7 million requests per second is achieved by writing a hard coded plain text HTTP response string directly to the client... it is so far disconnected from any real world use case that the number is basically meaningless.

arcturus17 6y ago
It's still 8th in the composite benchmark. And the criticism you're leveling would affect the entire benchmark design, rather than a particular framework score, no? [1] https://www.techempower.com/benchmarks/#section=data-r20&hw=ph&test=composite https://www.techempower.com/benchmarks/#section=data-r20&hw=...
- BiteCode_dev 6y ago
  Yes but getting a good ranking at a useless benchmark if not very meaningful. It's cool, but I wouldn't use it to make any serious decision.
- nodamage 6y ago
  > And the criticism you're leveling would affect the entire benchmark design, rather than a particular framework score, no? Indeed. The problem is that many of the scores in the top 100 are really misleading because no one building a web app would implement things that way. There is still some value in the lower down benchmarks but you have to basically read the underlying source to determine if the implementation is remotely realistic or not. For starters I would ignore anything classified as "platform" which is described as: Platform, meaning a raw server (not actually a framework at all). Good luck! You're going to need it. For C# in particular I would only consider the mvc variants as realistic. Edit: I looked into the "asp.net core" composite score a bit more, it looks like those benchmarks are based on the aforementioned "platform" implementations for each test. I actually think this score is even more misleading than the individual benchmarks. At least the individual benchmarks show you the difference between "aspcore" (platform), "aspcore-mw" (middleware-only), and "aspcore-mvc" (full framework with routing). Here are recalculated composite scores based on the more realistic implementations (aspcore-mvc, aspcore-mvc-ado-pg, aspcore-mvc-dap-pg, aspcore-mvc-ef-pg): ASP.NET Core MVC with ADO.NET (raw SQL): 3029 ASP.NET Core MVC with Dapper: 2591 ASP.NET Core MVC with Entity Framework: 2195 Compared to Flask's 468 or Django's 280 it's still significantly faster, but not to the same extreme you might think at first glance at the chart.
  - arcturus17 6y ago
    Yea I've seen that they do provide ORM benchmarks in the detailed tests. I still find it very impressive - as a full-featured framework it ranks well above the rest of its peers, despite the ORM. I am writing Django backends nowadays, but like I've mentioned elsewhere, I love its full-batteries approach and maturity, but I'm not very happy about it's meager async capabilities, and I think I'd prefer something with strong typing... I've always been put off by Microsoft's lock-in but it seems that's changed so at least I'd put the .net as a contender for side project in the near future.

qw 6y ago

C# is only at number 3 in your list. Both Java and Rust are above it in the list. It's also a very "artificial" benchmark and real world code will give different results (if you have static content, just put it in a CDN and don't worry) Other benchmarks from the same site: - JSON Serialization: C# is number 34 - Single query: C# is number 23 - Fortunes: C# is number 7

nodamage 6y ago
It needs to be emphasized how artificial these benchmarks really are. Here is the source for the Fortunes C# benchmark: https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/CSharp/aspnetcore/PlatformBenchmarks/BenchmarkApplication.Fortunes.cs https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast... There's no routing or templating, it just writes a bunch of strings. No one would build an actual web app this way. The only C# benchmarks that are remotely realistic are the mvc variants, starting with aspcore-mvc-ado-pg at number 79.
- infensus 6y ago
  https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Project-Information-Framework-Tests-Overview#general-test-requirements https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Proj... Routing is required, but generally the rules allow things to be "reasonable" and "acceptable", which lets all these weird implementations through. Honestly, they should remove the "Implementation approach" column, because basically every implementation is marked as "realistic", making it meaningless.
  - nodamage 6y ago
    The "routing" implementation is more hard coded string matching: https://github.com/TechEmpower/FrameworkBenchmarks/blob/5b0e00e64b36deb46183f3c5edaa612e086c47f2/frameworks/CSharp/aspnetcore/PlatformBenchmarks/BenchmarkApplication.cs#L78 https://github.com/TechEmpower/FrameworkBenchmarks/blob/5b0e... I mean I guess technically that's routing but it's not remotely realistic. Only the mvc variants use the actual framework's routing system.

EvilEy3 6y ago

Go being low-level has nothing to do with performance. They deliberately keep feature set on this level, they're not making some kind of trade-off. Also, high performance C# is so low level that you might as well write C++. Or you think you will use EF, LINQ and have 7 millions rps?

arcturus17 6y ago
> Or you think you will use EF, LINQ and have 7 millions rps? I'm imagining not, but it's still comparatively faster than most, if not all, "full-batteries frameworks", right? I'm assuming the use of ORMs and such is more or less uniform in the comparison (eg, if they don't use EF for .net, they don't use Django ORM either) The overall performance of .net across these benchmarks really catches my attention like it does GP... I'm looking for a full-batteries framework based on a strongly typed language and never in my life I thought I'd say this, but it might be time to give .net / C# a whirl? I've heard really good things about the dev experience from people here on HN, F# is a really cool bonus, and the fact that it looks at least comparatively performant could be the icing on the cake.
- akra 6y ago
  A framework that you can stay high level if you want and optimise when the rare but exceptional case arises on a particular route without splitting the process/resorting to C++ interop (with its own performance issues), or the risk of needing to rewrite due to performance to me is the big selling point. In a previous life I got some significant latency benefits by doing just that. I think its probably only gotten better with .NET 5.

Thaxll 6y ago

Hmm no they have the same speed with gnet ( Go ): https://www.techempower.com/benchmarks/#hw=ph&test=plaintext https://www.techempower.com/benchmarks/#hw=ph&test=plaintext 7M req/sec both of them. ( all those benchmarks are sort of useless anyway )

Ambix 6y ago

Why do enyone needs framework to print zillion Hello Worlds to the client? It's more interesting to see results of high-load DB tests, for example: https://www.techempower.com/benchmarks/#section=data-r20&hw=ph&test=query https://www.techempower.com/benchmarks/#section=data-r20&hw=...

akra 6y ago
Depends on your use case. I've written quite a few backends that didn't use a database and there are a number of cases for the pure HTTP benchmark. Pass through proxies with injected behaviour, in memory key-value data storage APIs, memory mapped files, etc. IMO not every shop offloads the state to its DB. Once you add the database you are really testing the DB driver - and that adds a lot more variance to the test. Maybe its just the DB driver for that particular DB? A fast "Hello World" benchmark implies that the HTTP/transport layers of the framework are very fast. That's your base and the lower bound to your best performance potential. As a real world example if .NET ASP NET Core has the best request/response benchmark and its better than say nginx (a popular reverse proxy) it might be better to have all your gateways using a reverse proxy implemented with that as its base instead. Over your whole network depending on your scale that could be a big cost and latency saving measure. I wouldn't be surprised if Microsoft or it's community have started writing one.