7 ms·

What's Wrong with CS Research

32 points by byrneseyeview 19y ago

dfranke 19y ago
I've never seen criticism of Haskell that was at once so well-informed and so stupendously misguided. It gets under my skin even more since every other point made in the article had my total sympathy. Now I'm going to have to somehow find the time to write a detailed rebuttal.
- mencius 19y ago
  I think this is my favorite response ever to anything I've ever written. I look forward to your rebuttal - if you could post a link to it in the Blogger comments, that would be ideal...
  - dfranke 19y ago
    Will do.
- amichail 19y ago
  I'm certainly no fan of Haskell. But I do like Scala very much and it did come out of academia.
amichail 19y ago
Related posts: http://weblog.fortnow.com/2006/07/science-and-art-of-computation.html http://weblog.fortnow.com/2006/07/science-and-art-of-computa... http://www.paulgraham.com/hp.html http://www.paulgraham.com/hp.html
mangodrunk 19y ago
>churn out meaningless solutions to irrelevant problems. That can be said of most of the advancements in math, science, and computer science. At a time, there was no need for zero, calendars, calculus, square root of negative one, etc. What the author calls irrelevant today will be tomorrow's great achievement. Most of Turing's work was irrelevant at the time, other than his work on cryptography. I don't think it is proper for great minds to worry about trivial things like enterprise Java code or what it takes to build a social networking website. This stuff is simple, that's why we can outsource it to other countries that don't have the resources to invest in the future developments that are needed for progress. Also, the whole article is based on the author's opinions, which isn't bad for that fact (actually the article was entertaining to say the least), but they aren't backed by something in any form.
- portLAN 19y ago
  I think the notion that "we have great minds!" in combination with, "we outsource the simple stuff to other countries" is rather a bit ungallant. It's a classic trap, overestimating one's self and underestimating others. The reason outsourcing is used is because of a favorable exchange rate due to historical inertia, even as we nudge it closer to the midway point every time we engage other countries. Going by population alone, their complement of great minds surely outnumbers ours.
  - mangodrunk 19y ago
    I agree with you and didn't intend to say that we're smarter. I was just saying that it is easier to outsource simple programming problems than the required investment for tough problems. The pool of programmers that can take a specification and imeplement it is larger and doesn't require to be in direct contact with marketers, but the person who comes up with the spec has to be able to talk with the stakeholders.
- plinkplonk 19y ago
  Indeed. One weak point in the essay (rant?) would seem to be the underlying assumption that the output of all CS research has to be usable by Joe Average Programmer to be counted as "good". Physics wouldn't have progressed much if all research results had to be immediately usable by kids playing with Lego blocks. Most so called "programmers" are just fitting blocks together without any deep understanding. Just look at all the horrendous VB code out there.
  - mencius 19y ago
    I wouldn't say it has to be usable by Joe Average Programmer, but if it's not - and it will obviously never be - it should have to explain why. And it shouldn't be described in any context, whether paper, grant proposal, or university press release, as anything else. C, for example, should never come within a mile of Joe Average Programmer. But the world needs C, it needs the Linux kernel, etc, etc. For other things, this is not so clear. You have to be able to judge the prospects of research programs. Otherwise, how do you separate the dry holes from the promising ones? There is an infinite set of dry holes in the world, and any of them is happy to accept all the funding you can give it.
Tichy 19y ago
The reservations seem rather unqualified indeed. A lot of strawman argumentation to begin with (singling out specific subjects of CS the authors deems unworthy). But even about PCC I don't think the author has the right notion. It is not the same as type checking. As an example, at my old university department they created a proof of the euclidian algorithm and extracted a program from it that would compute the gcd. That way you have more than a syntactically correct program, you have a proof that the code actually gives you the gcd. And yes, it was very academic, but still. Also, if CS is so useless, how come most leaders in the Netflix challenge seem to be CS departments?
- mencius 19y ago
  I think what you are describing is program verification, not proof-carrying code. In proof-carrying code, of course, the proposition you are proving may not be associated with the source code as an invariant. But if it isn't, why isn't it? It makes no sense to have one language for the invariant and another for the program. For example, one use of PCC is to prove propositions about a bit of machine code, such as a packet filter. When you create an environment that can state propositions about machine code, you have created a typed assembler, whether you like it or not. Most leaders in the Netflix challenge are (at) CS departments because CS departments are full of smart programmers.
  - Tichy 19y ago
    But if CS sucks and the programmers are so smart, why do they stay at the CS departments?
    - mencius 19y ago
      "If you're so smart, why aren't you rich?"
- mencius 19y ago
  Also, when you're examining something that dubs itself "science," pointing out a spot of routine dishonesty, however small and irrelevant, is hardly indicating a "strawman." Shades of the [Curate's egg](http://en.wikipedia.org/wiki/Curate's_egg http://en.wikipedia.org/wiki/Curate's_egg). I'm sure most of the entries on Enron's balance sheet were perfectly accurate, as well.
jsnx 19y ago
This piece is both well informed -- there are certainly no factual errors -- and vitriolic beyond belief. What is anyone supposed to take away from the author's attack on partial evaluation, for example? The author seems to suggest -- but does not say -- that function composition would be better handled by writing even more functions and passing around lists of function pointers.
- Goladus 19y ago
  I agree, that's why I'm going to have to come back and read it again when I have the energy to filter out all the strong opinions.
- mencius 19y ago
  In the rare cases that a programmer actually needs (dynamic) function composition, yes. For very understandable reasons, most of the examples you see of higher-order FP languages tend to stress very intricate and elegant algorithms. But what is 99.999% of the actual functional programming that is actually done in the world? SQL and Web template transforms. About as intricate as a brick. The result of having these incredibly high-powered, high-strung thoroughbred FP languages is that no one even thinks of these mundane tasks as cases of functional programming. But of course they are. FP is the natural generalization of declarative programming. The real tragedy of Haskell is that, if the people who designed Haskell had actually been working on, say, XML and XSLT, XML and XSLT would not suck nearly so much. I repeat that in almost all cases, dynamic construction of functions is unnecessary and can be eliminated. Otherwise, yes: write even more functions and pass around lists of function pointers. But you're right that I could probably make the same point in a much less vitriolic way. Blame acid reflux.
  - steveOfAR 19y ago
    > f the people who designed Haskell had actually been working on, say, XML and XSLT, XML and XSLT You might want to check out SXML / SXPATH/ SXSLT. These definitely don't suck, in fact I'd consider them nothing less than "XML processing done right". They are created by a well known type-system wizard (Oleg) with a fondness for Haskell.
Goladus 19y ago
I'm skeptical (but can't refute without actually researching it) of the assertions about academic advancement vs. free market advancement. I think the fact that academic research may not have any immediately obvious practical application is an advantage in the long run. In any case, that post was very interesting. I bookmarked it and will have to revisit it sometime.
menloparkbum 19y ago
The author chooses to rant about CS research being patronage that wastes US tax dollars. The examples he chooses don't support his argument very well. He doesn't like Haskell or Caml because they sprung from academia. Haskell was designed mostly by a couple Scots and the little funding it received came from Microsoft Research. Caml comes from IRNIA in France. No US tax dollars were wasted in either case. He claims that Python wasn't invented by a programming language researcher. Guido Van Rossum was employed as a programming language researcher (working on ABC) by a Dutch government research lab, and then by NIST and CRNI in the USA. NIST and CRNI are US government funded research organizations. Using the author's train of thought, the development Python WAS partially funded with US tax dollars.
- mencius 19y ago
  If you don't know the difference between what Guido van Rossum does, and what Andrew Appel, Luca Cardelli, Zhong Shao, etc, etc, do, you are clearly not a PL researcher! I am not a budget analyst. What bothers me is that I think a field I admire is being mismanaged, not that it is driving the US (or any other country) bankrupt. The US has worse problems than a little bit of science pork.
  - nostrademons 19y ago
    The thing is - you're essentially distinguishing Guy Steele and Guido van Rossum from Andrew Appel and Luca Cardelli by success. Steele and van Rossum created languages used by lots of people, and so they're "system software designers", while Appel and Cardelli created languages used by small communities, and so they're "programming language researchers". If you set up your definitions like that, it's tautological to claim that programming language research doesn't yield any useful stuff, because everybody who creates useful stuff gets taken out of the PL research category. The big question is: can you detect, a priori, which research projects will be successful? For example, would you have funded Dave Moon's research at the AI lab? How about L. Peter Deutsch? Ricky Greenblatt? Kernighan and Ritchie? Tony Hoare? Dijkstra? Knuth? James Gosling? Matz? The irony is that no matter who you consider relevant, you're almost always wrong a decade later. Moon was brilliant at the AI lab, and his work was the foundation for most of the 1980s AI boom. But it's largely irrelevant today. Matz and Guido were relatively unknown throughout the 1990s, but they're suddenly critical now, because technology has shifted towards dynamic languages. Hoare spent most of his life working on arcane topics like proof-carrying code, but think of how many bugs have been found because of assertions. What if the next big development in computing comes out of type theory? Would you then add Luca Cardelli, Benjamin Pierce, and Phillip Wadler to your "system software designers" category? (Heck, Wadler has already made important contributions to Java 1.5 generics.) The university system is built upon the premise that you can't tell what basic research will be successful beforehand, and so you better fund all of it (or as much as you can, at least ;-)) and wait for some entrepreneur to sift through it and figure out what's commercially viable. You seem to think that all that random chance is unnecessary, and you can pick the winners beforehand and shut down the rest. If you're right, there's a very lucrative career waiting for you as a venture capitalist, where you get to bet Other People's Money on the future system software designers. (BTW, I met a girl at a party who's in a swing dancing class with Guy Steele, and he calls himself a Programming Language Designer. So maybe your title for him is a bit off the mark...)
jmzachary 19y ago
This isn't an agreement or disagreement with the essay (I tend to lament the current sorry state of CS research but not necessarily for the reasons stated in this essay). But I would like to throw out that (a) before there was chemistry and physics, there was alchemy and that was hardly based on the scientific method, (b) computer science is becoming more ingrained in hard sciences via simulation and visualization, and (c) there is no law that says the scientific method can't be adapted to include information as a first-class primitive like stars, particles, energy, and inclined planes (among other things). Maybe (a) and (c) are the same point and we are muddling our way through as informatic alchemists at this stage of the field in the sense of CS being defined by the scientific method. As a CS PhD, I like the essay and appreciate that it raises points to think about. I never really thought about academics as bureaucrats in that way but, as a former academic who left the hallowed halls for the reasons you describe much better than I could formulate, I think I agree with you on that point.
cowmoo99 19y ago
I used to be one of those people who came into a CS undergrad program, complaining about the barrage of math we had to learn and the very little detailed real-world programming we had to do. However, after having taken PL, automata, crypto, algorithms and AI, I see the point of having concrete rigor of proof and abstraction about your code. However, what the author says about CS being a badly managed field, by being a mix-bag of math and science rings true to me. For instance, I have found out when I was a student sysadmin for my school, one of my CS professor who does research in typed-system does not know Perl. It behooves me that someone involved in CS, particularly research of programming languages, do not have a fundamental understanding of the practical application of their field. Someone earlier pointed out Church-Turing thesis being completely useless at the time of their publication, the same argument could be made for the Crick-Watson DNA publication (in their original publication, Watson only suggested that DNA being a double-helix). But the difference between these truly remarkable scientific discoveries with the average Joe academic pumping out crap each year, is that these theories were even then, foreseeable to have impact. No, I don't buy the "oh, theory is an art; let the academics work for knowledge's sake." Recent advances in technology are as much driven by industry as much by academia, see Bioinformatics (Celera), see high performance computing (Aerospace companies), see AI/data-mining (Spam Bayesian filters, Amazon, Netflix). I read recently a quote that declared "form is liberating" - just as architects are forced to design their pieces by both the constraints of material and economics; I would rather design my software and theory in the domain of application.
randomhack 19y ago
well .. some pieces of the article are not even wrong

32 points by byrneseyeview 19y ago

I've never seen criticism of Haskell that was at once so well-informed and so stupendously misguided. It gets under my skin even more since every other point made in the article had my total sympathy. Now I'm going to have to somehow find the time to write a detailed rebuttal.

mencius 19y ago
I think this is my favorite response ever to anything I've ever written. I look forward to your rebuttal - if you could post a link to it in the Blogger comments, that would be ideal...
- dfranke 19y ago
  Will do.
amichail 19y ago
I'm certainly no fan of Haskell. But I do like Scala very much and it did come out of academia.

amichail 19y ago

Related posts: http://weblog.fortnow.com/2006/07/science-and-art-of-computation.html http://weblog.fortnow.com/2006/07/science-and-art-of-computa... http://www.paulgraham.com/hp.html http://www.paulgraham.com/hp.html

mangodrunk 19y ago

>churn out meaningless solutions to irrelevant problems. That can be said of most of the advancements in math, science, and computer science. At a time, there was no need for zero, calendars, calculus, square root of negative one, etc. What the author calls irrelevant today will be tomorrow's great achievement. Most of Turing's work was irrelevant at the time, other than his work on cryptography. I don't think it is proper for great minds to worry about trivial things like enterprise Java code or what it takes to build a social networking website. This stuff is simple, that's why we can outsource it to other countries that don't have the resources to invest in the future developments that are needed for progress. Also, the whole article is based on the author's opinions, which isn't bad for that fact (actually the article was entertaining to say the least), but they aren't backed by something in any form.

portLAN 19y ago
I think the notion that "we have great minds!" in combination with, "we outsource the simple stuff to other countries" is rather a bit ungallant. It's a classic trap, overestimating one's self and underestimating others. The reason outsourcing is used is because of a favorable exchange rate due to historical inertia, even as we nudge it closer to the midway point every time we engage other countries. Going by population alone, their complement of great minds surely outnumbers ours.
- mangodrunk 19y ago
  I agree with you and didn't intend to say that we're smarter. I was just saying that it is easier to outsource simple programming problems than the required investment for tough problems. The pool of programmers that can take a specification and imeplement it is larger and doesn't require to be in direct contact with marketers, but the person who comes up with the spec has to be able to talk with the stakeholders.
plinkplonk 19y ago
Indeed. One weak point in the essay (rant?) would seem to be the underlying assumption that the output of all CS research has to be usable by Joe Average Programmer to be counted as "good". Physics wouldn't have progressed much if all research results had to be immediately usable by kids playing with Lego blocks. Most so called "programmers" are just fitting blocks together without any deep understanding. Just look at all the horrendous VB code out there.
- mencius 19y ago
  I wouldn't say it has to be usable by Joe Average Programmer, but if it's not - and it will obviously never be - it should have to explain why. And it shouldn't be described in any context, whether paper, grant proposal, or university press release, as anything else. C, for example, should never come within a mile of Joe Average Programmer. But the world needs C, it needs the Linux kernel, etc, etc. For other things, this is not so clear. You have to be able to judge the prospects of research programs. Otherwise, how do you separate the dry holes from the promising ones? There is an infinite set of dry holes in the world, and any of them is happy to accept all the funding you can give it.

Tichy 19y ago

The reservations seem rather unqualified indeed. A lot of strawman argumentation to begin with (singling out specific subjects of CS the authors deems unworthy). But even about PCC I don't think the author has the right notion. It is not the same as type checking. As an example, at my old university department they created a proof of the euclidian algorithm and extracted a program from it that would compute the gcd. That way you have more than a syntactically correct program, you have a proof that the code actually gives you the gcd. And yes, it was very academic, but still. Also, if CS is so useless, how come most leaders in the Netflix challenge seem to be CS departments?

mencius 19y ago
I think what you are describing is program verification, not proof-carrying code. In proof-carrying code, of course, the proposition you are proving may not be associated with the source code as an invariant. But if it isn't, why isn't it? It makes no sense to have one language for the invariant and another for the program. For example, one use of PCC is to prove propositions about a bit of machine code, such as a packet filter. When you create an environment that can state propositions about machine code, you have created a typed assembler, whether you like it or not. Most leaders in the Netflix challenge are (at) CS departments because CS departments are full of smart programmers.
- Tichy 19y ago
  But if CS sucks and the programmers are so smart, why do they stay at the CS departments?
  - mencius 19y ago
    "If you're so smart, why aren't you rich?"
mencius 19y ago
Also, when you're examining something that dubs itself "science," pointing out a spot of routine dishonesty, however small and irrelevant, is hardly indicating a "strawman." Shades of the [Curate's egg](http://en.wikipedia.org/wiki/Curate's_egg http://en.wikipedia.org/wiki/Curate's_egg). I'm sure most of the entries on Enron's balance sheet were perfectly accurate, as well.

jsnx 19y ago

This piece is both well informed -- there are certainly no factual errors -- and vitriolic beyond belief. What is anyone supposed to take away from the author's attack on partial evaluation, for example? The author seems to suggest -- but does not say -- that function composition would be better handled by writing even more functions and passing around lists of function pointers.

Goladus 19y ago
I agree, that's why I'm going to have to come back and read it again when I have the energy to filter out all the strong opinions.
mencius 19y ago
In the rare cases that a programmer actually needs (dynamic) function composition, yes. For very understandable reasons, most of the examples you see of higher-order FP languages tend to stress very intricate and elegant algorithms. But what is 99.999% of the actual functional programming that is actually done in the world? SQL and Web template transforms. About as intricate as a brick. The result of having these incredibly high-powered, high-strung thoroughbred FP languages is that no one even thinks of these mundane tasks as cases of functional programming. But of course they are. FP is the natural generalization of declarative programming. The real tragedy of Haskell is that, if the people who designed Haskell had actually been working on, say, XML and XSLT, XML and XSLT would not suck nearly so much. I repeat that in almost all cases, dynamic construction of functions is unnecessary and can be eliminated. Otherwise, yes: write even more functions and pass around lists of function pointers. But you're right that I could probably make the same point in a much less vitriolic way. Blame acid reflux.
- steveOfAR 19y ago
  > f the people who designed Haskell had actually been working on, say, XML and XSLT, XML and XSLT You might want to check out SXML / SXPATH/ SXSLT. These definitely don't suck, in fact I'd consider them nothing less than "XML processing done right". They are created by a well known type-system wizard (Oleg) with a fondness for Haskell.

Goladus 19y ago

I'm skeptical (but can't refute without actually researching it) of the assertions about academic advancement vs. free market advancement. I think the fact that academic research may not have any immediately obvious practical application is an advantage in the long run. In any case, that post was very interesting. I bookmarked it and will have to revisit it sometime.

menloparkbum 19y ago

The author chooses to rant about CS research being patronage that wastes US tax dollars. The examples he chooses don't support his argument very well. He doesn't like Haskell or Caml because they sprung from academia. Haskell was designed mostly by a couple Scots and the little funding it received came from Microsoft Research. Caml comes from IRNIA in France. No US tax dollars were wasted in either case. He claims that Python wasn't invented by a programming language researcher. Guido Van Rossum was employed as a programming language researcher (working on ABC) by a Dutch government research lab, and then by NIST and CRNI in the USA. NIST and CRNI are US government funded research organizations. Using the author's train of thought, the development Python WAS partially funded with US tax dollars.

mencius 19y ago
If you don't know the difference between what Guido van Rossum does, and what Andrew Appel, Luca Cardelli, Zhong Shao, etc, etc, do, you are clearly not a PL researcher! I am not a budget analyst. What bothers me is that I think a field I admire is being mismanaged, not that it is driving the US (or any other country) bankrupt. The US has worse problems than a little bit of science pork.
- nostrademons 19y ago
  The thing is - you're essentially distinguishing Guy Steele and Guido van Rossum from Andrew Appel and Luca Cardelli by success. Steele and van Rossum created languages used by lots of people, and so they're "system software designers", while Appel and Cardelli created languages used by small communities, and so they're "programming language researchers". If you set up your definitions like that, it's tautological to claim that programming language research doesn't yield any useful stuff, because everybody who creates useful stuff gets taken out of the PL research category. The big question is: can you detect, a priori, which research projects will be successful? For example, would you have funded Dave Moon's research at the AI lab? How about L. Peter Deutsch? Ricky Greenblatt? Kernighan and Ritchie? Tony Hoare? Dijkstra? Knuth? James Gosling? Matz? The irony is that no matter who you consider relevant, you're almost always wrong a decade later. Moon was brilliant at the AI lab, and his work was the foundation for most of the 1980s AI boom. But it's largely irrelevant today. Matz and Guido were relatively unknown throughout the 1990s, but they're suddenly critical now, because technology has shifted towards dynamic languages. Hoare spent most of his life working on arcane topics like proof-carrying code, but think of how many bugs have been found because of assertions. What if the next big development in computing comes out of type theory? Would you then add Luca Cardelli, Benjamin Pierce, and Phillip Wadler to your "system software designers" category? (Heck, Wadler has already made important contributions to Java 1.5 generics.) The university system is built upon the premise that you can't tell what basic research will be successful beforehand, and so you better fund all of it (or as much as you can, at least ;-)) and wait for some entrepreneur to sift through it and figure out what's commercially viable. You seem to think that all that random chance is unnecessary, and you can pick the winners beforehand and shut down the rest. If you're right, there's a very lucrative career waiting for you as a venture capitalist, where you get to bet Other People's Money on the future system software designers. (BTW, I met a girl at a party who's in a swing dancing class with Guy Steele, and he calls himself a Programming Language Designer. So maybe your title for him is a bit off the mark...)

jmzachary 19y ago

This isn't an agreement or disagreement with the essay (I tend to lament the current sorry state of CS research but not necessarily for the reasons stated in this essay). But I would like to throw out that (a) before there was chemistry and physics, there was alchemy and that was hardly based on the scientific method, (b) computer science is becoming more ingrained in hard sciences via simulation and visualization, and (c) there is no law that says the scientific method can't be adapted to include information as a first-class primitive like stars, particles, energy, and inclined planes (among other things). Maybe (a) and (c) are the same point and we are muddling our way through as informatic alchemists at this stage of the field in the sense of CS being defined by the scientific method. As a CS PhD, I like the essay and appreciate that it raises points to think about. I never really thought about academics as bureaucrats in that way but, as a former academic who left the hallowed halls for the reasons you describe much better than I could formulate, I think I agree with you on that point.

cowmoo99 19y ago

I used to be one of those people who came into a CS undergrad program, complaining about the barrage of math we had to learn and the very little detailed real-world programming we had to do. However, after having taken PL, automata, crypto, algorithms and AI, I see the point of having concrete rigor of proof and abstraction about your code. However, what the author says about CS being a badly managed field, by being a mix-bag of math and science rings true to me. For instance, I have found out when I was a student sysadmin for my school, one of my CS professor who does research in typed-system does not know Perl. It behooves me that someone involved in CS, particularly research of programming languages, do not have a fundamental understanding of the practical application of their field. Someone earlier pointed out Church-Turing thesis being completely useless at the time of their publication, the same argument could be made for the Crick-Watson DNA publication (in their original publication, Watson only suggested that DNA being a double-helix). But the difference between these truly remarkable scientific discoveries with the average Joe academic pumping out crap each year, is that these theories were even then, foreseeable to have impact. No, I don't buy the "oh, theory is an art; let the academics work for knowledge's sake." Recent advances in technology are as much driven by industry as much by academia, see Bioinformatics (Celera), see high performance computing (Aerospace companies), see AI/data-mining (Spam Bayesian filters, Amazon, Netflix). I read recently a quote that declared "form is liberating" - just as architects are forced to design their pieces by both the constraints of material and economics; I would rather design my software and theory in the domain of application.

randomhack 19y ago

well .. some pieces of the article are not even wrong