4 ms·
Except they obstruct issues and pull requests which would fix artificially hamstrung implementations, rather than actively seeking implementations to keep them
by pekk 13y ago
Except they obstruct issues and pull requests which would fix artificially hamstrung implementations, rather than actively seeking implementations to keep them honest. The benchmarks then represent a fantasy world where completely normal optimizations are disallowed.
The point of saying "send us a pull request" is just PR and it seems to be working.
- RyanZAG 13y agoAFAIK the only optimization they disallow is the use of caching results. It's clear why they do this: if you allow caching, you're just testing memory fetch speed and all your results across frameworks will probably look about the same. The idea of the benchmark is to actually do the JSON serialization/database access/etc. I believe they have plans to add additional tests which will allow for caching and that kind of thing. I don't think that removing caching across all frameworks equally is creating 'artificially hamstrung implementations' as all the frameworks would benefit from caching.