5 ms·

Sorry. A peer-reviewed research paper in computer science is supposed to be an original contribution to knowledge meeting the usual criteria of new, correct, a

Sorry. A peer-reviewed research paper in computer science is supposed to be an original contribution to knowledge meeting the usual criteria of new, correct, and significant. Okay, how do we present knowledge? Text? Yes. Text with math as in, say, a book on math or mathematical physics? Yes. Code? Nope! Sorry 'bout that! Yes, coming from the programming side, many programmers have accepted that code, especially with mnemonic identifers, actually is meaningful and a substitute for good text possibly with good math. Sorry, guys: In principle, it just ain't. In simple terms, the code is meaningless; code with mnemonic identifer names and careful indenting is still meaningless and at best a puzzle problem to solve to identify the intended meaning. Sorry 'bout that. Yes, often in practice, with enough context from outside the code, can get by with just mnemonic names and pretty printing. Still, it's just ain't knowledge. Really, for being knowledge, the code should be like the math in, say, a physics text, that is, surrounded by text. The math does not, Not, NOT stand alone and by itself is just meaningless. E.g.. F = ma is no more meaningful than a = bc. For either equation, the meaning has to come from surrounding text where the equation becomes just a nice abbreviation for what is in the text. So, for code, the comments play the role of text in a physics book and the code plays the role of the math in a physics book. As in a physics book, the text (comments) are MORE important than the math (code). Again, code just AIN'T knowledge. Sorry guys.

spacemanaki 15y ago
"Really, for being knowledge, the code should be like the math in, say, a physics text, that is, surrounded by text..." You kind of set yourself up for the rebuttal here. I don't think many people would suggest that presenting code devoid of context would be useful. Usually, really good writing about code, whether in papers or in books, is written in a kind of literate style, a la Knuth. Even if it's not formally done using literate tools, in many cases the code is accompanied and surrounded by text, just like mathematics. This isn't some kind of pissing match between code and math, as you seem to want to make it out to be. Someone publishing a CS paper presumably has some interest in communicating something to the reader. The OP is just suggesting that for some subset of the readership, code might aid in that goal.
- NY_Entrepreneur 15y ago
  A published paper is supposed to present 'knowledge' in a clear and solid way. We know how to do that in, say, a physics text in a natural language, e.g., English, with text and some math. So, now there's 'code' that is neither text nor math. So, in communicating knowledge, we don't know what role the code has. Fundamentally, the role is not promising: The code alone doesn't mean anything; mnemonic identifier names do not make a fundamental difference; with a lot of explanatory comments we are still a very long way from good examples such as a physics text. In practice, programmers are often forced to look at code and discover its 'meaning', but that unfortunate situation does not mean that code should be in a published paper. In simple terms, there are two cases of 'code' here: (1) It is common to have a dozen or half-dozen lines of, say, 'pseudo-code' in a figure in a paper. The figure can help, but the paper should be plenty clear and solid without the figure. E.g., there was a long theme in much of the best in math that the writing should be so clear that no figures are needed; or, the content should not be in 'figures'. But, I'm 'liberal' and can agree that intuition is important, that often there is an appropriate 'figure', that the reader should not always be expected to draw their own, either in their mind or on paper, and that, similarly, a few lines of pseudo-code can be helpful. Still, the pseudo-code is not the solid, clear content of the paper. (2) In practice, 'code' tends to grow quickly from dozens of lines to KLOC. Putting such code in a paper would be a bit much! There is a real danger in computing: In nearly universal computing practice in writing and documentation, the content is not all in just text or math but in mnemonic identifiers and code, and that danger is real and a big step backwards in science. While it is clear where the danger comes from in computing practice, still the results don't belong in journals. You mentioned Knuth's 'literate programming', i.e., his shot back at 'structured programming'! For the code itself, I was always in favor of his 'literate' idea. Although TeX is my main source of high quality word whacking, I never actually got through enough of his Web, Weave, and Tangle to write 'literate' code. The best I do is just use a good text editor. In our AI project at Yorktown Heights, I tried to get people interested in literate programming and Knuth's tools, but people called it just 'pretty printing' and laughed.
sophacles 15y ago
I think you are missing the point. The article is not about "give us the code instead of a paper", but instead about "give us the code as a supplement to the paper". The basic argument is that sometimes text alone is not sufficient to explain - a point you concede to by mentioning that the formulae exist in textbooks along with the description. The reason they include those formulae in physics books is because the text is one way of describing something, and the math explains the relationships. Both have their strong points. Take for instance many protocol standards: they come with a reference implementation, because sometimes the text is harder to understand when you can't compare it to code which actually does what is being described.
- NY_Entrepreneur 15y ago
  I was trying to be short and clear. Your "The article is not about 'give us the code instead of a paper', but instead about 'give us the code as a supplement to the paper'." is an appropriate interpretation of the paper. But I can't have much sympathy for that interpretation because the code can be so darned long and obscure. Some 'pseudo-code' of, say, a dozen lines as a 'figure' is okay. In nearly all cases, any code very close to something that actually runs in practice would tend to be too darned long. The 'knowledge' of the paper is supposed to be clear just from the paper. Then the code is supposed to be 'routine'. If the main body of the paper does not make the knowledge clear, then including a few hundred lines of code has little promise of improving the exposition. There's a good history supporting my point in books by Knuth, Ullman, Sedgewick, the paper Ronald Fagin, Jurg Nievergelt, Nicholas Pippenger, H. Raymond Strong, 'Extendible hashing—a fast access method for dynamic files', "ACM Transactions on Database Systems", ISSN 0362-5915, Volume 4, Issue 3, September 1979, Pages: 315 - 344. and more. For your "The basic argument is that sometimes text alone is not sufficient to explain - a point you concede to by mentioning that the formulae exist in textbooks along with the description." I believe you are asking too much of the equations, not a lot too much but too much. In simple terms, even pure math is, usually is, at best is, is supposed to be written in complete sentences in a natural language, say, English. E.g., in a physics book describing Newton's second law, we discuss force, mass, and acceleration, explain that force is a vector F, mass is a scalar m, and acceleration is a vector a, say that force is the product of mass and acceleration, e.g., F = ma So, by the time get to the equation, almost don't need it. Of course, not ALL writers write this way! A long sequence of algebraic derivations is supposed to have a start much like what I gave for F = ma and after that is just algebraic derivations, often with not much need for more text. For "Take for instance many protocol standards: they come with a reference implementation, because sometimes the text is harder to understand when you can't compare it to code which actually does what is being described." yes, there is a practical challenge there. Finally the Unix world decided to 'go empirical' and have 'code runoffs' or whatever they were called where people just got together to see if the actual code, say, the TCP/IP stack with SSL, would actually appear to work. Of course, that empirical approach was always risky because it is a long way from a mathematical proof of correctness or even a solid engineering document. And we've seen problems, e.g., with security in DNS. For more, the AI world decided to f'get about 'theory' or even usual scientific documentation and make the criterion just 'does the code appear to work'. This, too, has proven to be risky and, of course, is a big step back in documentation quality from good applied math, mathematical physics, and engineering of past decades. E.g., how the heck to know that the Manhattan Project was on track before the first test? Sure: There were a LOT of derivations and calculations; that is, the project didn't rely on just the empirical approach. Indeed, the uranium bomb was dropped on Japan as the first 'test'; the engineering was so solid there was no doubt it would explode. Sure, there is a big role for testing, but, still, the 'knowledge' in a journal is supposed to be able to stand on its own without much help from detailed documentation on code, electronic circuits, mechanical mechanisms, or chemical reactions. In particular, in experimental science, the published paper is supposed to be clear enough to permit others to reproduce the experiment, and to do so without detailed photographs and drawings of apparatus. When I was an undergrad, one summer I worked in the lab of a famous scientist. He explained to me how a scientist is supposed to keep a good notebook and really good records. Then if another scientist wants to check details or reproduce results, sending the notebooks and data is supposed to be an obligation, and the materials are expected to be handled with great care and then returned. So, yes, at times, in practice, there can be more that is relevant than just the published paper. Still, communicating the 'knowledge' of a research paper with a few KLOC of code is a bit much!
bchjam 15y ago
sounds like a good argument for literate programming to me

spacemanaki 15y ago

"Really, for being knowledge, the code should be like the math in, say, a physics text, that is, surrounded by text..." You kind of set yourself up for the rebuttal here. I don't think many people would suggest that presenting code devoid of context would be useful. Usually, really good writing about code, whether in papers or in books, is written in a kind of literate style, a la Knuth. Even if it's not formally done using literate tools, in many cases the code is accompanied and surrounded by text, just like mathematics. This isn't some kind of pissing match between code and math, as you seem to want to make it out to be. Someone publishing a CS paper presumably has some interest in communicating something to the reader. The OP is just suggesting that for some subset of the readership, code might aid in that goal.

NY_Entrepreneur 15y ago
A published paper is supposed to present 'knowledge' in a clear and solid way. We know how to do that in, say, a physics text in a natural language, e.g., English, with text and some math. So, now there's 'code' that is neither text nor math. So, in communicating knowledge, we don't know what role the code has. Fundamentally, the role is not promising: The code alone doesn't mean anything; mnemonic identifier names do not make a fundamental difference; with a lot of explanatory comments we are still a very long way from good examples such as a physics text. In practice, programmers are often forced to look at code and discover its 'meaning', but that unfortunate situation does not mean that code should be in a published paper. In simple terms, there are two cases of 'code' here: (1) It is common to have a dozen or half-dozen lines of, say, 'pseudo-code' in a figure in a paper. The figure can help, but the paper should be plenty clear and solid without the figure. E.g., there was a long theme in much of the best in math that the writing should be so clear that no figures are needed; or, the content should not be in 'figures'. But, I'm 'liberal' and can agree that intuition is important, that often there is an appropriate 'figure', that the reader should not always be expected to draw their own, either in their mind or on paper, and that, similarly, a few lines of pseudo-code can be helpful. Still, the pseudo-code is not the solid, clear content of the paper. (2) In practice, 'code' tends to grow quickly from dozens of lines to KLOC. Putting such code in a paper would be a bit much! There is a real danger in computing: In nearly universal computing practice in writing and documentation, the content is not all in just text or math but in mnemonic identifiers and code, and that danger is real and a big step backwards in science. While it is clear where the danger comes from in computing practice, still the results don't belong in journals. You mentioned Knuth's 'literate programming', i.e., his shot back at 'structured programming'! For the code itself, I was always in favor of his 'literate' idea. Although TeX is my main source of high quality word whacking, I never actually got through enough of his Web, Weave, and Tangle to write 'literate' code. The best I do is just use a good text editor. In our AI project at Yorktown Heights, I tried to get people interested in literate programming and Knuth's tools, but people called it just 'pretty printing' and laughed.

sophacles 15y ago

I think you are missing the point. The article is not about "give us the code instead of a paper", but instead about "give us the code as a supplement to the paper". The basic argument is that sometimes text alone is not sufficient to explain - a point you concede to by mentioning that the formulae exist in textbooks along with the description. The reason they include those formulae in physics books is because the text is one way of describing something, and the math explains the relationships. Both have their strong points. Take for instance many protocol standards: they come with a reference implementation, because sometimes the text is harder to understand when you can't compare it to code which actually does what is being described.

NY_Entrepreneur 15y ago
I was trying to be short and clear. Your "The article is not about 'give us the code instead of a paper', but instead about 'give us the code as a supplement to the paper'." is an appropriate interpretation of the paper. But I can't have much sympathy for that interpretation because the code can be so darned long and obscure. Some 'pseudo-code' of, say, a dozen lines as a 'figure' is okay. In nearly all cases, any code very close to something that actually runs in practice would tend to be too darned long. The 'knowledge' of the paper is supposed to be clear just from the paper. Then the code is supposed to be 'routine'. If the main body of the paper does not make the knowledge clear, then including a few hundred lines of code has little promise of improving the exposition. There's a good history supporting my point in books by Knuth, Ullman, Sedgewick, the paper Ronald Fagin, Jurg Nievergelt, Nicholas Pippenger, H. Raymond Strong, 'Extendible hashing—a fast access method for dynamic files', "ACM Transactions on Database Systems", ISSN 0362-5915, Volume 4, Issue 3, September 1979, Pages: 315 - 344. and more. For your "The basic argument is that sometimes text alone is not sufficient to explain - a point you concede to by mentioning that the formulae exist in textbooks along with the description." I believe you are asking too much of the equations, not a lot too much but too much. In simple terms, even pure math is, usually is, at best is, is supposed to be written in complete sentences in a natural language, say, English. E.g., in a physics book describing Newton's second law, we discuss force, mass, and acceleration, explain that force is a vector F, mass is a scalar m, and acceleration is a vector a, say that force is the product of mass and acceleration, e.g., F = ma So, by the time get to the equation, almost don't need it. Of course, not ALL writers write this way! A long sequence of algebraic derivations is supposed to have a start much like what I gave for F = ma and after that is just algebraic derivations, often with not much need for more text. For "Take for instance many protocol standards: they come with a reference implementation, because sometimes the text is harder to understand when you can't compare it to code which actually does what is being described." yes, there is a practical challenge there. Finally the Unix world decided to 'go empirical' and have 'code runoffs' or whatever they were called where people just got together to see if the actual code, say, the TCP/IP stack with SSL, would actually appear to work. Of course, that empirical approach was always risky because it is a long way from a mathematical proof of correctness or even a solid engineering document. And we've seen problems, e.g., with security in DNS. For more, the AI world decided to f'get about 'theory' or even usual scientific documentation and make the criterion just 'does the code appear to work'. This, too, has proven to be risky and, of course, is a big step back in documentation quality from good applied math, mathematical physics, and engineering of past decades. E.g., how the heck to know that the Manhattan Project was on track before the first test? Sure: There were a LOT of derivations and calculations; that is, the project didn't rely on just the empirical approach. Indeed, the uranium bomb was dropped on Japan as the first 'test'; the engineering was so solid there was no doubt it would explode. Sure, there is a big role for testing, but, still, the 'knowledge' in a journal is supposed to be able to stand on its own without much help from detailed documentation on code, electronic circuits, mechanical mechanisms, or chemical reactions. In particular, in experimental science, the published paper is supposed to be clear enough to permit others to reproduce the experiment, and to do so without detailed photographs and drawings of apparatus. When I was an undergrad, one summer I worked in the lab of a famous scientist. He explained to me how a scientist is supposed to keep a good notebook and really good records. Then if another scientist wants to check details or reproduce results, sending the notebooks and data is supposed to be an obligation, and the materials are expected to be handled with great care and then returned. So, yes, at times, in practice, there can be more that is relevant than just the published paper. Still, communicating the 'knowledge' of a research paper with a few KLOC of code is a bit much!

bchjam 15y ago

sounds like a good argument for literate programming to me