3 ms·

I'm not familiar with k, and I'm not understanding this whole "no benchmarks" thing. So it's a proprietary lang? What stops benchmarks being run? Just how ridi

I'm not familiar with k, and I'm not understanding this whole "no benchmarks" thing. So it's a proprietary lang? What stops benchmarks being run? Just how ridiculous are the licenses? Why has seemingly nobody run them anyway?

eatonphil 5y ago
I think the article answers all your questions.
tsimionescu 5y ago
Not sure about K, but I know for example that MS didn't allow publishing performance benchmarks of .NET (not sure of this changed with .NET core).
- eatonphil 5y ago
  Both Oracle and MS SQL disallow this without approval. https://stackoverflow.com/a/12116865/1507139 https://stackoverflow.com/a/12116865/1507139
geocar 5y ago
> I'm not understanding this whole "no benchmarks" thing. Back in the 1980s, there was a paper published[1] that compared a number of database systems, and Oracle felt this paper harmed them. Perhaps if the authors knew Oracle better, they could have made more favourable benchmarks, they would argue, and whilst perhaps DeWitt et al could agree to that, they may point out they had not intended to harm Oracle, and being researchers and not marketers were given no notice or expectation that such a review could harm Oracle. And so for the avoidance of doubt, Oracle, Microsoft, and many others (probably including KX, although I've never checked) insist in their licensing agreements that they be able to approve benchmarks, ostensibly so that they have an opportunity to correct any misunderstandings the author might have about the software before. [1]: http://pages.cs.wisc.edu/~dewitt/includes/benchmarking/vldb83.pdf http://pages.cs.wisc.edu/~dewitt/includes/benchmarking/vldb8...
- q-big 5y ago
  > Back in the 1980s, there was a paper published[1] that compared a number of database systems, and Oracle felt this paper harmed them. This is rather an argument why the benchmark code should be made public so that other people can check it and vendors can post modified versions if they feel treated unfairly.
- benibela 5y ago
  These benchmark things reminds me of an anecdote I saw today in the D group: >Back in the 80's (!) computer magazines regularly ran C compiler benchmark results. (At one time I counted 30 C compilers available for the PC.) >I took a class at Standford on compilers, which included a lot of info on data flow analysis. I decided to implement it. Data flow analysis optimizations basically deleted the benchmark code, because it didn't do anything (lots of dead assignments). This compiler was released as Datalight Optimum C. >The article writer concluded that the compiler had a giant bug in it, because it ran the benchmarks impossibly fast, because it deleted the dead code. Instead of being lauded as the only C compiler on the PC that did global data flow analysis, it was labeled as a buggy piece of crap. >By the time we dug ourselves out of that PR disaster, other compilers had implemented it, too. >Ironically, these days DMD gets repeatedly charged with not doing data flow analysis, and clang is assumed to have invented data flow analysis. >Just goes to show the power of marketing

eatonphil 5y ago

I think the article answers all your questions.

tsimionescu 5y ago

Not sure about K, but I know for example that MS didn't allow publishing performance benchmarks of .NET (not sure of this changed with .NET core).

eatonphil 5y ago
Both Oracle and MS SQL disallow this without approval. https://stackoverflow.com/a/12116865/1507139 https://stackoverflow.com/a/12116865/1507139

geocar 5y ago

> I'm not understanding this whole "no benchmarks" thing. Back in the 1980s, there was a paper published[1] that compared a number of database systems, and Oracle felt this paper harmed them. Perhaps if the authors knew Oracle better, they could have made more favourable benchmarks, they would argue, and whilst perhaps DeWitt et al could agree to that, they may point out they had not intended to harm Oracle, and being researchers and not marketers were given no notice or expectation that such a review could harm Oracle. And so for the avoidance of doubt, Oracle, Microsoft, and many others (probably including KX, although I've never checked) insist in their licensing agreements that they be able to approve benchmarks, ostensibly so that they have an opportunity to correct any misunderstandings the author might have about the software before. [1]: http://pages.cs.wisc.edu/~dewitt/includes/benchmarking/vldb83.pdf http://pages.cs.wisc.edu/~dewitt/includes/benchmarking/vldb8...

q-big 5y ago
> Back in the 1980s, there was a paper published[1] that compared a number of database systems, and Oracle felt this paper harmed them. This is rather an argument why the benchmark code should be made public so that other people can check it and vendors can post modified versions if they feel treated unfairly.
benibela 5y ago
These benchmark things reminds me of an anecdote I saw today in the D group: >Back in the 80's (!) computer magazines regularly ran C compiler benchmark results. (At one time I counted 30 C compilers available for the PC.) >I took a class at Standford on compilers, which included a lot of info on data flow analysis. I decided to implement it. Data flow analysis optimizations basically deleted the benchmark code, because it didn't do anything (lots of dead assignments). This compiler was released as Datalight Optimum C. >The article writer concluded that the compiler had a giant bug in it, because it ran the benchmarks impossibly fast, because it deleted the dead code. Instead of being lauded as the only C compiler on the PC that did global data flow analysis, it was labeled as a buggy piece of crap. >By the time we dug ourselves out of that PR disaster, other compilers had implemented it, too. >Ironically, these days DMD gets repeatedly charged with not doing data flow analysis, and clang is assumed to have invented data flow analysis. >Just goes to show the power of marketing