6 ms·

The original investigation reports are a pretty good read. (links at the bottom of this article: http://www.chessvibes.com/reports/rybka-disqualified-and-banned

by jsnell 15y ago

The original investigation reports are a pretty good read. (links at the bottom of this article: http://www.chessvibes.com/reports/rybka-disqualified-and-banned-from-world-computer-chess-championships/ http://www.chessvibes.com/reports/rybka-disqualified-and-ban... )

dalke 15y ago
I looked at http://www.chessvibes.com/plaatjes/rybkaevidence/ZW_Rybka_Fruit.pdf http://www.chessvibes.com/plaatjes/rybkaevidence/ZW_Rybka_Fr... . It happens to be the one mentioned as biased evidence in this chessbase article, though I picked it because it was a PDF and not an RTF file. This is the the one which has a side-by-side comparison of what appears to be two pieces of code. One of which is "static const int KnightBackRankOpening = 0;" for (sq = 0; sq < 64; sq++) { P(piece,sq,Opening) += KnightRank[square_rank(sq)] * KnightBackRankOpening; } That appears to damning, but as the chessbase article points out, there's no source for the Rybka case, only reverse engineering. Since KnightBackRankOpening is "static const int 0" (can you really reverse engineer a "const" from machine code?), you would expect any half-decent optimizing compiler to remove that whole chunk of code. In other words, this seemingly indicting code is a façade; a functionally equivalent implementation constructed to maximize similarity. Granted, the PDF does say "The code shown here is simply the functional equivalent" and "Fruit and Rybka have functionally identical code here too," I looked at the code and the biggest similarity is that both code snippets are rendered in the same style. I don't see any copying. I see different tunings (eg, different weight parameters), and even though both tools are working in the same data representation and algorithm space, I see different implementations of those algorithms. I find the chessbase article to be much more convincing than the results of the original investigation. What would convince me otherwise is the same analysis of other modern chess programs, to show that they don't use the same approach.
- jsnell 15y ago
  In this case there is not even any reverse engineering as such. These are values that IIUC Fruit computes at startup while Rybka has them precomputed. What that section was showing was merely that these tables were most likely computed using the code from Fruit, but with scaled and (automatically?) adjusted weights. So you're totally correct in that displaying the hypothetical source code in such situation was questionable. It's not a horribly strong bit of evidence, and as an isolated incident, this would hardly be interesting. But as one more piece in a pattern of persistent copying, more so. In particular, it's worth noting that e.g. the code from earlier versions of Fruit would require larger changes than just tweaking these weights to generate the same output tables, making it pretty unlikely that this is just a case of convergent evolution. So if the PST tables are weak, what's the stronger evidence? First of all, there's the circumstances of the rebirth of the project: - Older versions of Rybka were indisputably based on Crafty. (Including replicating some harmless bugs in exactly the same places as a certain version of Crafty, making it possible to pinpoint the point of plagiarism fairly closely to a specific Crafty version). - After Fruit was released, suddenly Rybka loses all the similarity to Crafty, and acquires similarities to Fruit. This coincides with: ++ Rybka becoming a dramatically stronger player. ++ Rybka losing features that it previously had in support code. ++ Rybka gaining new idiosyncracies in support code, matching those of Fruity. ++ Rybka losing some game playing / evaluation features that it (and Crafty) previously had but Fruit didn't. ++ Rybka suddenly acquiring an evaluation feature set that is a very close match to those of Fruity. Very close, as in much closer than any unrelated engines are to each other, than previous versions of Rybka to the then current versions, or that version of Rybka to much later versions. ++ Rybka suddenly acquiring new fairly arbitary data structures that match those of the same specific version of Fruit, while not matching those of other Fruit version (e.g. the hash structure). And probably other things, it's been a while since I read the papers. It's very surprising to me that somebody could read through all of the evidence and believe there's no foul play going on. It seems totally unreasonable to assume that he first wrote from scratch a chess engine that managed to very exactly replicate the foibles of a top open source engine, then threw all of that code out and rewrote from scratch a new engine that happened to replicate the idiosyncracies of a completely unrelated and stronger open source engine. It seems much more believe that substantial parts of the code of Rybka were copied verbatim from Fruit, and that other substantial parts were ported over to use a different structure in a way that's not really showing a lot of creativity. Did Rajlich also substantially improve on the code? Clearly at some point he must have, it did eventually become a much stronger program. It's hard to say how much of this original innovation there was in Rybka 1/2, which as I understand are the particularly contentious versions. Did Rybka later evolve beyond the Fruity origings? Almost certainly so. Does any of that excuse either violating the GPL or claiming that it was all his own work, at most "inspired" by others? Clearly not.
  - dalke 15y ago
    It's very strange then that both bits of code have a loop which add a lot of 0s together. When you make inferences at this level, you must validate your methods of comparison. That PDF document does not show how other chess programs do the same feature, so it's very hard to tell if that's a natural way to implement the same approach or if it's a copyright violation. My knowledge of this is based solely on the chessbase article. It shows one method of fingerprinting chess programs which suggsts that Rybka 1.0b and Streika 2.0 are the closest matches to each other, and somewhat close to Naum 3.1/4.2, while Fruit, Toga, Onno, and Loop are close to each other. Was the analysis incorrect? Rybka 3 and 4.1 are much less similar. What analysis show that Rybka was "a very close match to those of Fruity. Very close, as in much closer than any unrelated engines are to each other"? The dendrogram clearly shows Rybka 1.0 closer to Naum than to Fruit. Clearly it would be appropriate to compare to the similar methods of Naum. I don't understand your statement "It seems much more believe that substantial parts of the code of Rybka were copied verbatim from Fruit". Quite clearly OpenOffice looks and feels like Microsoft Office, but just as clearly, the code was not copied verbatim from MS Office. Quite clearly Linux 1.0 was written with knowledge of how MINIX works, along with information from Sun and POSIX; I myself learned about OS design from Tannenbaum's MINIX textbook. But there's no credible claim that that Linux was copied verbatim from MINIX. Do you think Torvalds could have developed an OS from scratch, without any knowledge of how OSes work and without any standards? People do write systems from scratch which are bug compatible. DR-DOS 5 was bug-compatible to MS-DOS. That was for commercial reasons. Researchers will implement someone else's algorithm so they have baseline comparison for alternate algorithm development. Free software people will rewrite software which isn't free. People who follow the kata development style will reimplement code just to get the feel for how that version is done. So no, I have no problems in believing that someone would start with one method, do a deep analysis of another program, implement the ideas (but not copy the code!) and end up with similar results. While it may not be perfect clean-room style, it's still not in violation of the GPL. If there's a GPL compliance issue, then where is the evidence of copyright infringement? All I've seen is examples of algorithm reimplementation. Not only that, but I gather that there's a lot of reverse-engineering going on; how much of the other proprietary chess programs are similarly "inspired" by deep inspection of other implementations?

by jsnell 15y ago

dalke 15y ago

I looked at http://www.chessvibes.com/plaatjes/rybkaevidence/ZW_Rybka_Fruit.pdf http://www.chessvibes.com/plaatjes/rybkaevidence/ZW_Rybka_Fr... . It happens to be the one mentioned as biased evidence in this chessbase article, though I picked it because it was a PDF and not an RTF file. This is the the one which has a side-by-side comparison of what appears to be two pieces of code. One of which is "static const int KnightBackRankOpening = 0;" for (sq = 0; sq < 64; sq++) { P(piece,sq,Opening) += KnightRank[square_rank(sq)] * KnightBackRankOpening; } That appears to damning, but as the chessbase article points out, there's no source for the Rybka case, only reverse engineering. Since KnightBackRankOpening is "static const int 0" (can you really reverse engineer a "const" from machine code?), you would expect any half-decent optimizing compiler to remove that whole chunk of code. In other words, this seemingly indicting code is a façade; a functionally equivalent implementation constructed to maximize similarity. Granted, the PDF does say "The code shown here is simply the functional equivalent" and "Fruit and Rybka have functionally identical code here too," I looked at the code and the biggest similarity is that both code snippets are rendered in the same style. I don't see any copying. I see different tunings (eg, different weight parameters), and even though both tools are working in the same data representation and algorithm space, I see different implementations of those algorithms. I find the chessbase article to be much more convincing than the results of the original investigation. What would convince me otherwise is the same analysis of other modern chess programs, to show that they don't use the same approach.

jsnell 15y ago
In this case there is not even any reverse engineering as such. These are values that IIUC Fruit computes at startup while Rybka has them precomputed. What that section was showing was merely that these tables were most likely computed using the code from Fruit, but with scaled and (automatically?) adjusted weights. So you're totally correct in that displaying the hypothetical source code in such situation was questionable. It's not a horribly strong bit of evidence, and as an isolated incident, this would hardly be interesting. But as one more piece in a pattern of persistent copying, more so. In particular, it's worth noting that e.g. the code from earlier versions of Fruit would require larger changes than just tweaking these weights to generate the same output tables, making it pretty unlikely that this is just a case of convergent evolution. So if the PST tables are weak, what's the stronger evidence? First of all, there's the circumstances of the rebirth of the project: - Older versions of Rybka were indisputably based on Crafty. (Including replicating some harmless bugs in exactly the same places as a certain version of Crafty, making it possible to pinpoint the point of plagiarism fairly closely to a specific Crafty version). - After Fruit was released, suddenly Rybka loses all the similarity to Crafty, and acquires similarities to Fruit. This coincides with: ++ Rybka becoming a dramatically stronger player. ++ Rybka losing features that it previously had in support code. ++ Rybka gaining new idiosyncracies in support code, matching those of Fruity. ++ Rybka losing some game playing / evaluation features that it (and Crafty) previously had but Fruit didn't. ++ Rybka suddenly acquiring an evaluation feature set that is a very close match to those of Fruity. Very close, as in much closer than any unrelated engines are to each other, than previous versions of Rybka to the then current versions, or that version of Rybka to much later versions. ++ Rybka suddenly acquiring new fairly arbitary data structures that match those of the same specific version of Fruit, while not matching those of other Fruit version (e.g. the hash structure). And probably other things, it's been a while since I read the papers. It's very surprising to me that somebody could read through all of the evidence and believe there's no foul play going on. It seems totally unreasonable to assume that he first wrote from scratch a chess engine that managed to very exactly replicate the foibles of a top open source engine, then threw all of that code out and rewrote from scratch a new engine that happened to replicate the idiosyncracies of a completely unrelated and stronger open source engine. It seems much more believe that substantial parts of the code of Rybka were copied verbatim from Fruit, and that other substantial parts were ported over to use a different structure in a way that's not really showing a lot of creativity. Did Rajlich also substantially improve on the code? Clearly at some point he must have, it did eventually become a much stronger program. It's hard to say how much of this original innovation there was in Rybka 1/2, which as I understand are the particularly contentious versions. Did Rybka later evolve beyond the Fruity origings? Almost certainly so. Does any of that excuse either violating the GPL or claiming that it was all his own work, at most "inspired" by others? Clearly not.
- dalke 15y ago
  It's very strange then that both bits of code have a loop which add a lot of 0s together. When you make inferences at this level, you must validate your methods of comparison. That PDF document does not show how other chess programs do the same feature, so it's very hard to tell if that's a natural way to implement the same approach or if it's a copyright violation. My knowledge of this is based solely on the chessbase article. It shows one method of fingerprinting chess programs which suggsts that Rybka 1.0b and Streika 2.0 are the closest matches to each other, and somewhat close to Naum 3.1/4.2, while Fruit, Toga, Onno, and Loop are close to each other. Was the analysis incorrect? Rybka 3 and 4.1 are much less similar. What analysis show that Rybka was "a very close match to those of Fruity. Very close, as in much closer than any unrelated engines are to each other"? The dendrogram clearly shows Rybka 1.0 closer to Naum than to Fruit. Clearly it would be appropriate to compare to the similar methods of Naum. I don't understand your statement "It seems much more believe that substantial parts of the code of Rybka were copied verbatim from Fruit". Quite clearly OpenOffice looks and feels like Microsoft Office, but just as clearly, the code was not copied verbatim from MS Office. Quite clearly Linux 1.0 was written with knowledge of how MINIX works, along with information from Sun and POSIX; I myself learned about OS design from Tannenbaum's MINIX textbook. But there's no credible claim that that Linux was copied verbatim from MINIX. Do you think Torvalds could have developed an OS from scratch, without any knowledge of how OSes work and without any standards? People do write systems from scratch which are bug compatible. DR-DOS 5 was bug-compatible to MS-DOS. That was for commercial reasons. Researchers will implement someone else's algorithm so they have baseline comparison for alternate algorithm development. Free software people will rewrite software which isn't free. People who follow the kata development style will reimplement code just to get the feel for how that version is done. So no, I have no problems in believing that someone would start with one method, do a deep analysis of another program, implement the ideas (but not copy the code!) and end up with similar results. While it may not be perfect clean-room style, it's still not in violation of the GPL. If there's a GPL compliance issue, then where is the evidence of copyright infringement? All I've seen is examples of algorithm reimplementation. Not only that, but I gather that there's a lot of reverse-engineering going on; how much of the other proprietary chess programs are similarly "inspired" by deep inspection of other implementations?