3 ms·
The addition of the "realistic" vs "synthetic" benchmarks is very welcome, with "realistic" being the default. Hopefully this could help get things to a better
by tfparty2 6y ago
The addition of the "realistic" vs "synthetic" benchmarks is very welcome, with "realistic" being the default. Hopefully this could help get things to a better state.
Still questionable as to what is allowed in "realistic" though. Aspnet core using a "rawdb" adapter doesn't seem like what I would interperet as "realistic".
Perhaps there needs to be a "marketed" category, for use as the docs describe. For aspnet core, this would include using entity framework.
Not much point in splitting the categories if it isn't a uniformly split thing.
It'd be nice to see benchmarks where:
• Everything was implemented as documented in the framework's tutorials.
• Realistic things were included end-to-end (payload validation, csrf (if applicable), etc).
Still seems like a whole basket of fruit comparisons here.
- dangerboysteve 6y agoThe hardware physical/cloud toggle is also interesting.
- nodamage 6y ago> Aspnet core using a "rawdb" adapter doesn't seem like what I would interperet as "realistic". Why not? I don't think using raw SQL instead of an ORM is particularly uncommon. The "platform" benchmarks (aspcore-rhtx-pg, aspcore-ado-pg) that appear to skip the ASP.NET Core framework entirely and write raw HTTP responses are definitely not realistic though. I'm going to assume that these are mislabeled because this filter is new and not all the benchmarks have been updated yet.
- tfparty2 6y agoI'll agree that raw database queries aren't unheard of, but EF and Dapper are much more suggested (and IMO "realistic") database paths for "asp.net core" additionally, I'm not quite sure that this is the recommended way to return database results: https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/CSharp/aspnetcore/Benchmarks/Middleware/SingleQueryRawMiddleware.cs#L33 https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast... What I believe is more "realistic" is to: • Use the framework as described: https://docs.microsoft.com/en-us/aspnet/core/tutorials/first-web-api?view=aspnetcore-3.1&tabs=visual-studio#return-values https://docs.microsoft.com/en-us/aspnet/core/tutorials/first... To be fair, I'm picking on aspnet core because I like it so much (via F#). The framework is great, but this gaming of public benchmarks (techempower, benchmarksgame) by "breaking the same rules as everyone else" rather than investing time into making these results more useful is disappointing.
- darrenkopp 6y agoI'd say I do about 40% raw ado and 60% ORM overall. I think raw ado is completely valid in a performance benchmark because you usually step down to raw ado for that very reason: optimizing performance.
- brightball 6y agoIf you take the time to dive into the source on may of these, you'll end up seeing so many strange things.