5 ms·

Oracle does the same thing. As a community, we need to punish companies that engage in these types of behaviors.

brightball 8y ago
I believe Microsoft does the same with SQL Server. https://www.reddit.com/r/programming/comments/2mhpwp/postgresql_vs_ms_sql_server_a_comparison_of_two/ https://www.reddit.com/r/programming/comments/2mhpwp/postgre...
- gerdesj 8y ago
  Ta for the link. That is quite a discussion. The first thread is priceless (it turns out that grauenwolf isn't quite as authoritative as they first appear - fancy!) Never underestimate how keen people can become about something they have paid £1,000s for. How on earth could a DBMS like MySQL/MariaDB or PostgreSQL possibly compare to the mighty Oracle money generator or MSSQL ... errrr ... money generator. Just in case you wonder whether open source DBMSs should be taken seriously, try reading this: https://postgresweekly.com/ https://postgresweekly.com/ and this: https://mariadb.com/resources/blog https://mariadb.com/resources/blog
dbcurtis 8y ago
Here is the thing, at least in the case of database software, the people that are competent to set up a benchmark correctly are fairly rare. The people that have the budget for the hardware are even more rare. The last thing the world needs is for the web to be polluted by so many incompetent "benchmarks" performed by the clueless that you can't find the authoritative information that you need to do your job. Take a wild guess about how much memory and how many disk spindles it takes to get a TPC benchmark on a modern server to be CPU limited. Take a guess as to how many weeks of engineering time it takes to perform and validate the benchmark after getting a pile of hardware delivered to your lab. The average blogger writing 3 posts a week ain't gonna do it.
- zimbatm 8y ago
  There is always a well sounding rationale attached to censorship. It also prevents meaningful benchmarks to be published.
  - walterbell 8y ago
    And it's a self-fulfilling prophecy if we don't allow inexperienced benchmarkers to learn how to run meaningful tests.
  - kjeetgill 8y ago
    I'm on the fence about the benchmarks but I think this is a generic broad brush to sweep away the parent's comment given the real concerns they've presented and you've chose to not address. Do you really thing any agreement that involves silence is censorship NDAs included? Sure, it's easy to simply say: sucks to be you! Especially to "community bullies" like Oracle. Even just between semi-competing open source projects benchmarks can be so bad they border on libelous.
    - harry8 8y ago
      Oracle has just enough budget to hire someone knowledgeable to engage in the discussion, pointing out the issues and their fixes, possibly even linking the benchmarks that were (in their view) properly performed. One good engineer. Compare with the legal fees required to draft such an appalling, scared and knowingly guilty clause as "you can't talk about how our product performs despite having paid for it." This really needs to be treated with total derision, and utter contempt. It's an approach that has earned that response for it's sheer laziness before even beginning to measure it against any ethical standards. Oracle are just an awful company in every dimension. This, is just like most everything else they do in that it reinforces the reputation they earned for themselves.
- jenscow 8y ago
  Well, most times I'd like to know how a badly configured X compares with badly configured Y, on <random hardware> - because that's usually how things are run.
kevarh 8y ago
I worked for a competitor and we had similar verbiage. It's there to prevent "XXX is a 10 million times faster than YYY." We actually gave everyone permission if they both asked and allowed us to review their methodology to make sure they weren't doing anything horrifically wrong (benchmarking is hard).

I believe Microsoft does the same with SQL Server. https://www.reddit.com/r/programming/comments/2mhpwp/postgresql_vs_ms_sql_server_a_comparison_of_two/ https://www.reddit.com/r/programming/comments/2mhpwp/postgre...

gerdesj 8y ago
Ta for the link. That is quite a discussion. The first thread is priceless (it turns out that grauenwolf isn't quite as authoritative as they first appear - fancy!) Never underestimate how keen people can become about something they have paid £1,000s for. How on earth could a DBMS like MySQL/MariaDB or PostgreSQL possibly compare to the mighty Oracle money generator or MSSQL ... errrr ... money generator. Just in case you wonder whether open source DBMSs should be taken seriously, try reading this: https://postgresweekly.com/ https://postgresweekly.com/ and this: https://mariadb.com/resources/blog https://mariadb.com/resources/blog

dbcurtis 8y ago

Here is the thing, at least in the case of database software, the people that are competent to set up a benchmark correctly are fairly rare. The people that have the budget for the hardware are even more rare. The last thing the world needs is for the web to be polluted by so many incompetent "benchmarks" performed by the clueless that you can't find the authoritative information that you need to do your job. Take a wild guess about how much memory and how many disk spindles it takes to get a TPC benchmark on a modern server to be CPU limited. Take a guess as to how many weeks of engineering time it takes to perform and validate the benchmark after getting a pile of hardware delivered to your lab. The average blogger writing 3 posts a week ain't gonna do it.

zimbatm 8y ago
There is always a well sounding rationale attached to censorship. It also prevents meaningful benchmarks to be published.
- walterbell 8y ago
  And it's a self-fulfilling prophecy if we don't allow inexperienced benchmarkers to learn how to run meaningful tests.
- kjeetgill 8y ago
  I'm on the fence about the benchmarks but I think this is a generic broad brush to sweep away the parent's comment given the real concerns they've presented and you've chose to not address. Do you really thing any agreement that involves silence is censorship NDAs included? Sure, it's easy to simply say: sucks to be you! Especially to "community bullies" like Oracle. Even just between semi-competing open source projects benchmarks can be so bad they border on libelous.
  - harry8 8y ago
    Oracle has just enough budget to hire someone knowledgeable to engage in the discussion, pointing out the issues and their fixes, possibly even linking the benchmarks that were (in their view) properly performed. One good engineer. Compare with the legal fees required to draft such an appalling, scared and knowingly guilty clause as "you can't talk about how our product performs despite having paid for it." This really needs to be treated with total derision, and utter contempt. It's an approach that has earned that response for it's sheer laziness before even beginning to measure it against any ethical standards. Oracle are just an awful company in every dimension. This, is just like most everything else they do in that it reinforces the reputation they earned for themselves.
jenscow 8y ago
Well, most times I'd like to know how a badly configured X compares with badly configured Y, on <random hardware> - because that's usually how things are run.

kevarh 8y ago

I worked for a competitor and we had similar verbiage. It's there to prevent "XXX is a 10 million times faster than YYY." We actually gave everyone permission if they both asked and allowed us to review their methodology to make sure they weren't doing anything horrifically wrong (benchmarking is hard).