10 ms·

Talk about head in the sand. CS is subsuming stats (not really) because computers are a, if not the, major driving force begind statistical advances. L1 regulat

Talk about head in the sand. CS is subsuming stats (not really) because computers are a, if not the, major driving force begind statistical advances. L1 regulation became a fad in stats because computational optimization could efficiently solve this non-smooth optimization problem. Engineers are now advancing algorithms and moving it to real world use, while statisticians are still re-searching their field and finding applications using existing algorithms. There is nothing wrong with application, but this means that they are not at the forefront of research and more importantly they have to wait for other people to solve their problems because they are no capable of inventing novel algorithms. When the world at the large relies on computers for every little thing and on computer scientists for a lot of scientific advances, is it really surprising that computer scientists are more prominant? And computer scientists have plenty of rigor. NIPs conferrence is an AI/neuroscience conference that is now prestigious in stats as well. Some machine learning experts lack mathematical foundations but statisticians are not really good mathematicians either. Being able to regurgitate stat theorems and prove convergence in expection under two dozen conditions (no kidding; read average JASA papers) are not the hallmark of analytic rigor. Average stats Ph.D., even from top programs, lack thorough understanding of measure theory. Just look at standard stat textbooks; examples are aplenty (using statistical software), but proofs are nowher to be found. It is amazing how many statisticians are still using pseudo-inverses in numeric computation. The statistical education is a problem. The biggest factor being the very software this article is promoting: R. It is an atrocious monstrocity. Perhaps statisticians can live with it. Fine, but don't expect good programmers coming out of using R for five years. Python would be far better. In the end, statisticians are not professional programmers, the same way accountants are not. I don't see too many accountants fretting CS is subsuming their field. Let us not forget, most statistical methods have their roots in other science and engineering fields, and they are not particularly bothered.

1971genocide 12y ago
I am just a undergrad maths student but here is my experience: I learnt more about bayesian statistics in my Machine Learning course than I did from my statistics module, reason ? The stat book/course is the most dull nonsense I ever came across even number theory made more sense to me than statistics, the whole point of the book/course was to proof some really abstract axiom which no one understood why they were doing it. Compare that to the process of writing a spam filter from scratch in MATLAB. A mathematical axiomatic system is only useful if you are solving some real problem using it, otherwise good luck living in the matrix. I have learnt that is far easier for mathematics to create garbage than to create useful ideas. The reason why I think stats sucks so much is because its a cultural problem in the community, If you google "how to write spam filter" I am sure someone made a wikihow with pictures to explain bayesian stats even to a snail. The reason is the computer science community constantly accepts their ignorance, even if a proof/program is trivial they will make sure to explain it. In the mathematics world no such thing exists. The big assumptions are "we leave it to the reader to proof this trivial fact" No Mr Smarty Pants, pls assume the reader is a 17 year girl who cannot send an email ( if you want to attract young people please understand their mindset first ). /end rant
- GFK_of_xmaspast 12y ago
  Don't advertise your ignorance.
- capnrefsmmat 12y ago
  Which stats book did you use for which course? Intro stats is typically taught abysmally, because it's been dragged down by eighty years of pre-computing historical stats.
  - 1971genocide 12y ago
    I was lucky to have studied in a british curriculum high school The books I used were these: http://www.amazon.co.uk/Edexcel-Level-Modular-Mathematics-Statistics/dp/0435519123 http://www.amazon.co.uk/Edexcel-Level-Modular-Mathematics-St... These books were amazing. they were about 100 pages with lots of color and worked example. They go from S1-S6. What I find amazing is I have seen some insane greek syndrome books explaining the same topics as these ""kiddee" colored books. However My statistics education stopped after college ( exception being khan academy videos ), I have replaced all my stats modules with CS modules in university, I have hacked the stats modules I was forced to do by mugging up formulas and memorizing exam techniques :( so you can see why this stats prof complaining ticked me off as he thinks students haven't tried and are acting in ignorance.
- eli_gottlieb 12y ago
  >The reason is the computer science community constantly accepts their ignorance, even if a proof/program is trivial they will make sure to explain it. In the mathematics world no such thing exists. The big assumptions are "we leave it to the reader to proof this trivial fact" No Mr Smarty Pants, pls assume the reader is a 17 year girl who cannot send an email ( if you want to attract young people please understand their mindset first ). Tell it preacher! Also, if you still want a Bayesian stats book (which covers plenty of the material from a Frequentist intro stats course, too), William Bolstad's Introduction to Bayesian Statistics is as clear as anything I've ever read.
  - 12y ago
michaelochurch 12y ago
It is amazing how many statisticians are still using pseudo-inverses in numeric computation. What is specifically wrong with the pseudo-inverse? Is it that eigenvalues close to zero (that might actually be zero, but are not due to FP error) get magnified immensely? Or is it the general principle that inverting a matrix is usually a bad idea, for reasons that John D. Cook elaborates quite well? R. It is an atrocious monstrocity. Perhaps statisticians can live with it. Fine, but don't expect good programmers coming out of using R for five years. Python would be far better. Python has great libraries but I could see Clojure overtaking it as the data science language in the next 10-15 years, and Haskell has some obvious strengths for production systems (strong, static typing).
- zaphar 12y ago
  I would actually bet on Julia before Clojure as the next big data science language.
- east2west 12y ago
  The following quote is from "Matrix analysis and applied linear algebra" by Carl D Meyer, p 423-424. "Generalized inverses are useful in formulating theoretical statements as those above, ut, just as in the case of the ordinary inverse, generalized inverses are not practical computational tools. In addition to being computationally inefficient, serious numerical problems result from the fact that A+ needs not be a continuous function of the entries of A. ... Not only is A+(x) discontinuous in the sense that lim_{x->0}A+(x)!=A+(0), but it is discontinuous in the worst way because as A(x) comes closer to A(0) the matrix A+(x) moves farther away from A+(0)." Here A+ denotes Moore-Penrose pseudo-inverse and A(x) = [1, 0; 0, x] in Matlab notation (or Numpy? I am very rusty on Matlab). And in case you are wondering why the generalized inverse is plural, there are different kinds of generalized inverses, Drazin inverse being a well defined alternative.
  - thearn4 12y ago
    Pseudo-inverse (and other direct methods) do have a lot going against them, especially as problems scale. In practice, iterative methods introduce their own challenges as well though. Optimal algorithm selection is highly problem dependant, and often hinges on knowing how to generate an appropriate preconditioner. GMRES is still pretty much my favorite general method for inverse problems, but it comes with plenty of nuances that you really don't see when using a direct method on a tiny problem that you can bank on being well-conditioned. It's hard to say how often those kinds of problems exist in the real world (in my case: exceedingly rare), but they are still out there.
dredmorbius 12y ago
I'd be interested in more about your beefs on R. Note that its origins are the earlier language 'S', part of the original AT&T UNIX suite, and seen as an adjunct to the data knife of awk. Later you'd likely have paired Perl, Python, or Ruby, and likely a pseudo-database (sqlight, say), along with some stats and graphing libraries or tools. My own stats background includes some very basic SPSS, as well as SAS and R, and graphics under gnuplot. I've always found R tremendously difficult to grok, though I've known a number of statisticians I respected (at least as statisticians, if not as programmers) who found R exceptionally useful and powerful. My understanding is that its real value is as a matrix manipulation language, which may account for both its obscure syntax and appeal among hard-core stats geeks. But still: your criticisms?
- east2west 12y ago
  OK, to be fair, R is a reasonable language for statisticians and others. I guess R and S are different enough from usual programming languages that I may be biased against them. But I did spent five years working on bioinformatics and statistical genetics using it, and I couldn't wait to get away from it when I did. I will list my personal peeves. 1. R's syntax has problems. This is well documented on the web. I recall : has wrong precedence. 2. My biggest problem is R's lack of basic abstraction facility. Beyond functions there is nothing good. There are three object-oriented programming systems, none of which are simple or straightforward. It has first-class functions and lexical closure so at least basic functional programming should have been possible and fun. But R functions' have a weird parameter passing mechanism which scared me away from investigating deep. In the end I used RCPP to write large codebase and expose an R interface. And I like to delve deep into things. I was learning template metaprogramming when I was working on bioinformatics. 3. CRAN has way too many low quality packages, but far more importantly almost all packages' documentation consist of use-cases plus reference. To this day I don't have a good mental picture of genomicRange (a foundational bioinformatics package in Bioconductor for NGS data analysis) data structure and how to manipulate it (one main data structure, I believe). This is annoying at best and infuriating at best. 4. R, more than any other language I know, is an implementation-defined language. The language reference manual frankly admits fuzzy rules for some of the more advanced features. Python, which has been recently criticised for similar problems, is clear at the basic language features level. I don't want to this discussion to degenerate into a flamewar. Programming language preference is a highly personal thing and I respect people who use R, even admire programmers who create awsome packages. ggplot2 and Plyr are some of the best packages I have ever used, and their documentation is superb. I just don't believe R is a good language for statistical students if they hope to move on to professional programming in other languages.
thearn4 12y ago
> It is amazing how many statisticians are still using pseudo-inverses in numeric computation I'm not entirely sure what you mean - pseudoinverse/SVD methods definitely have their place.
kiyoto 12y ago
Here is a math major's perspective (I also have a C.S. master's from Stanford, a decent school for computer science). The real reason statistics is losing to machine learning is M&M: money and marketing. Money is a huge factor when kids choose majors in college. Many of them have student loans to pay off, and for many of them, getting a high-paying job post college is a serious consideration. In this regard, statistics is a great major, but computer science is flat-out ridiculous. When big tech companies are offering a Stanford graduate with only a few summers of programming internships under the belt for 150k/year, naturally a lot of kids are lured into computer science. Then, once they start studying computer science, they discover this thing called machine learning. While I do think there is a difference in emphasis between stats and machine learning, the fundamentals are same, except the nomenclature sounds much cooler in machine learning. Nonparametric inference sounds esoteric and cryptic, but unsupervised learning sounds futuristic and cool. The biggest problem with both statistics and machine learning education, I would say, is their lack of emphasis on mathematical foundations. When I was in college, CS229 (Introduction to Machine Learning) was touted to be the hardest class at Stanford. Having helped my friends wade through CS229 problems (a lot of which comes down to wading through linear algebra and multi-variable differential calculus), I do not think this is remotely true: CS229 is hard because it attracts students who do not have the requisite mathematical maturity to learn statistics/machine learning in a serious way (I am not even talking about the real-analytic foundation of probability and calculus but rudimentary linear algebra and chain rules). Also, I do think CS229@Stanford is a great class that brings theory and practice together =) As for R/Python/Matlab/etc., I will let the zealots argue what's best. To me, they are like statistics and machine learning: similar with different emphasis.
- wwweston 12y ago
  > The biggest problem with both statistics and machine learning education, I would say, is their lack of emphasis on mathematical foundations. Any recommendations for someone who'd like to pick up sound (if modest) foundations?
  - GFK_of_xmaspast 12y ago
    Hogg & Craig used to be one of the standard texts.
enupten 12y ago
If I were Kramer, I would dub thee Anti-Statite.
eruditely 12y ago
I fail to believe that computer science graduates also know measure theory.
- eli_gottlieb 12y ago
  Except that computer science graduates are not supposed to know measure theory, whereas statisticians ought to know how measure theory underlies probability theory underlies statistics.

1971genocide 12y ago

I am just a undergrad maths student but here is my experience: I learnt more about bayesian statistics in my Machine Learning course than I did from my statistics module, reason ? The stat book/course is the most dull nonsense I ever came across even number theory made more sense to me than statistics, the whole point of the book/course was to proof some really abstract axiom which no one understood why they were doing it. Compare that to the process of writing a spam filter from scratch in MATLAB. A mathematical axiomatic system is only useful if you are solving some real problem using it, otherwise good luck living in the matrix. I have learnt that is far easier for mathematics to create garbage than to create useful ideas. The reason why I think stats sucks so much is because its a cultural problem in the community, If you google "how to write spam filter" I am sure someone made a wikihow with pictures to explain bayesian stats even to a snail. The reason is the computer science community constantly accepts their ignorance, even if a proof/program is trivial they will make sure to explain it. In the mathematics world no such thing exists. The big assumptions are "we leave it to the reader to proof this trivial fact" No Mr Smarty Pants, pls assume the reader is a 17 year girl who cannot send an email ( if you want to attract young people please understand their mindset first ). /end rant

GFK_of_xmaspast 12y ago
Don't advertise your ignorance.
capnrefsmmat 12y ago
Which stats book did you use for which course? Intro stats is typically taught abysmally, because it's been dragged down by eighty years of pre-computing historical stats.
- 1971genocide 12y ago
  I was lucky to have studied in a british curriculum high school The books I used were these: http://www.amazon.co.uk/Edexcel-Level-Modular-Mathematics-Statistics/dp/0435519123 http://www.amazon.co.uk/Edexcel-Level-Modular-Mathematics-St... These books were amazing. they were about 100 pages with lots of color and worked example. They go from S1-S6. What I find amazing is I have seen some insane greek syndrome books explaining the same topics as these ""kiddee" colored books. However My statistics education stopped after college ( exception being khan academy videos ), I have replaced all my stats modules with CS modules in university, I have hacked the stats modules I was forced to do by mugging up formulas and memorizing exam techniques :( so you can see why this stats prof complaining ticked me off as he thinks students haven't tried and are acting in ignorance.
eli_gottlieb 12y ago
>The reason is the computer science community constantly accepts their ignorance, even if a proof/program is trivial they will make sure to explain it. In the mathematics world no such thing exists. The big assumptions are "we leave it to the reader to proof this trivial fact" No Mr Smarty Pants, pls assume the reader is a 17 year girl who cannot send an email ( if you want to attract young people please understand their mindset first ). Tell it preacher! Also, if you still want a Bayesian stats book (which covers plenty of the material from a Frequentist intro stats course, too), William Bolstad's Introduction to Bayesian Statistics is as clear as anything I've ever read.
- 12y ago

michaelochurch 12y ago

It is amazing how many statisticians are still using pseudo-inverses in numeric computation. What is specifically wrong with the pseudo-inverse? Is it that eigenvalues close to zero (that might actually be zero, but are not due to FP error) get magnified immensely? Or is it the general principle that inverting a matrix is usually a bad idea, for reasons that John D. Cook elaborates quite well? R. It is an atrocious monstrocity. Perhaps statisticians can live with it. Fine, but don't expect good programmers coming out of using R for five years. Python would be far better. Python has great libraries but I could see Clojure overtaking it as the data science language in the next 10-15 years, and Haskell has some obvious strengths for production systems (strong, static typing).

zaphar 12y ago
I would actually bet on Julia before Clojure as the next big data science language.
east2west 12y ago
The following quote is from "Matrix analysis and applied linear algebra" by Carl D Meyer, p 423-424. "Generalized inverses are useful in formulating theoretical statements as those above, ut, just as in the case of the ordinary inverse, generalized inverses are not practical computational tools. In addition to being computationally inefficient, serious numerical problems result from the fact that A+ needs not be a continuous function of the entries of A. ... Not only is A+(x) discontinuous in the sense that lim_{x->0}A+(x)!=A+(0), but it is discontinuous in the worst way because as A(x) comes closer to A(0) the matrix A+(x) moves farther away from A+(0)." Here A+ denotes Moore-Penrose pseudo-inverse and A(x) = [1, 0; 0, x] in Matlab notation (or Numpy? I am very rusty on Matlab). And in case you are wondering why the generalized inverse is plural, there are different kinds of generalized inverses, Drazin inverse being a well defined alternative.
- thearn4 12y ago
  Pseudo-inverse (and other direct methods) do have a lot going against them, especially as problems scale. In practice, iterative methods introduce their own challenges as well though. Optimal algorithm selection is highly problem dependant, and often hinges on knowing how to generate an appropriate preconditioner. GMRES is still pretty much my favorite general method for inverse problems, but it comes with plenty of nuances that you really don't see when using a direct method on a tiny problem that you can bank on being well-conditioned. It's hard to say how often those kinds of problems exist in the real world (in my case: exceedingly rare), but they are still out there.

dredmorbius 12y ago

I'd be interested in more about your beefs on R. Note that its origins are the earlier language 'S', part of the original AT&T UNIX suite, and seen as an adjunct to the data knife of awk. Later you'd likely have paired Perl, Python, or Ruby, and likely a pseudo-database (sqlight, say), along with some stats and graphing libraries or tools. My own stats background includes some very basic SPSS, as well as SAS and R, and graphics under gnuplot. I've always found R tremendously difficult to grok, though I've known a number of statisticians I respected (at least as statisticians, if not as programmers) who found R exceptionally useful and powerful. My understanding is that its real value is as a matrix manipulation language, which may account for both its obscure syntax and appeal among hard-core stats geeks. But still: your criticisms?

east2west 12y ago
OK, to be fair, R is a reasonable language for statisticians and others. I guess R and S are different enough from usual programming languages that I may be biased against them. But I did spent five years working on bioinformatics and statistical genetics using it, and I couldn't wait to get away from it when I did. I will list my personal peeves. 1. R's syntax has problems. This is well documented on the web. I recall : has wrong precedence. 2. My biggest problem is R's lack of basic abstraction facility. Beyond functions there is nothing good. There are three object-oriented programming systems, none of which are simple or straightforward. It has first-class functions and lexical closure so at least basic functional programming should have been possible and fun. But R functions' have a weird parameter passing mechanism which scared me away from investigating deep. In the end I used RCPP to write large codebase and expose an R interface. And I like to delve deep into things. I was learning template metaprogramming when I was working on bioinformatics. 3. CRAN has way too many low quality packages, but far more importantly almost all packages' documentation consist of use-cases plus reference. To this day I don't have a good mental picture of genomicRange (a foundational bioinformatics package in Bioconductor for NGS data analysis) data structure and how to manipulate it (one main data structure, I believe). This is annoying at best and infuriating at best. 4. R, more than any other language I know, is an implementation-defined language. The language reference manual frankly admits fuzzy rules for some of the more advanced features. Python, which has been recently criticised for similar problems, is clear at the basic language features level. I don't want to this discussion to degenerate into a flamewar. Programming language preference is a highly personal thing and I respect people who use R, even admire programmers who create awsome packages. ggplot2 and Plyr are some of the best packages I have ever used, and their documentation is superb. I just don't believe R is a good language for statistical students if they hope to move on to professional programming in other languages.

thearn4 12y ago

> It is amazing how many statisticians are still using pseudo-inverses in numeric computation I'm not entirely sure what you mean - pseudoinverse/SVD methods definitely have their place.

kiyoto 12y ago

Here is a math major's perspective (I also have a C.S. master's from Stanford, a decent school for computer science). The real reason statistics is losing to machine learning is M&M: money and marketing. Money is a huge factor when kids choose majors in college. Many of them have student loans to pay off, and for many of them, getting a high-paying job post college is a serious consideration. In this regard, statistics is a great major, but computer science is flat-out ridiculous. When big tech companies are offering a Stanford graduate with only a few summers of programming internships under the belt for 150k/year, naturally a lot of kids are lured into computer science. Then, once they start studying computer science, they discover this thing called machine learning. While I do think there is a difference in emphasis between stats and machine learning, the fundamentals are same, except the nomenclature sounds much cooler in machine learning. Nonparametric inference sounds esoteric and cryptic, but unsupervised learning sounds futuristic and cool. The biggest problem with both statistics and machine learning education, I would say, is their lack of emphasis on mathematical foundations. When I was in college, CS229 (Introduction to Machine Learning) was touted to be the hardest class at Stanford. Having helped my friends wade through CS229 problems (a lot of which comes down to wading through linear algebra and multi-variable differential calculus), I do not think this is remotely true: CS229 is hard because it attracts students who do not have the requisite mathematical maturity to learn statistics/machine learning in a serious way (I am not even talking about the real-analytic foundation of probability and calculus but rudimentary linear algebra and chain rules). Also, I do think CS229@Stanford is a great class that brings theory and practice together =) As for R/Python/Matlab/etc., I will let the zealots argue what's best. To me, they are like statistics and machine learning: similar with different emphasis.

wwweston 12y ago
> The biggest problem with both statistics and machine learning education, I would say, is their lack of emphasis on mathematical foundations. Any recommendations for someone who'd like to pick up sound (if modest) foundations?
- GFK_of_xmaspast 12y ago
  Hogg & Craig used to be one of the standard texts.

enupten 12y ago

If I were Kramer, I would dub thee Anti-Statite.

eruditely 12y ago

I fail to believe that computer science graduates also know measure theory.

eli_gottlieb 12y ago
Except that computer science graduates are not supposed to know measure theory, whereas statisticians ought to know how measure theory underlies probability theory underlies statistics.