4 ms·

>While R is the de facto standard for performing statistical analysis, it has quite a high learning curve What? R has a ridiculously low learning curve. I reme

by rm999 14y ago

>While R is the de facto standard for performing statistical analysis, it has quite a high learning curve What? R has a ridiculously low learning curve. I remember literally the first time I used R I loaded up a dataset and had a histogram and qqplot within 5 minutes and 3 or 4 lines of code. Just figuring out what libraries I would need to do that in python (and installing them) would probably take me at least 30 minutes. I think it's still highly debatable if Python is the way to go for general data science, especially if you're spending a lot of time analyzing data. R is more mature, but the tides are steadily moving in python's direction.

minimize_me 14y ago
Depends on the analysis lifecycle I think - I hate using Matlab for writing any sort of large experimental framework in, especially if it has to interact with other systems, but for quick and dirty experiments, plotting, or parallelizing totally independent for-loops eg. for a grid-search over some model hyper-parameters, it's great (the latter is literally changing "for" to "parfor". After you fork out for the Parallel Computing Toolbox of course.) But then a lot of code in research is written in the style of use-code-for-a-week->get a graph->curve is higher than their curve?->publication. When I need to write something more substantive and reusable though, Python forever.
ahi 14y ago
Some things, maybe most things that a beginner wants to do are easy, but beyond the basics, R has a cliff for a learning curve. It's a complex language with some questionable design choices.
- rm999 14y ago
  I agree with you about questionable design choices, I really don't like developing in R. But if you have some data and you want to know more about that data (data analysis), R + a small handful of packages is the best free environment that I know of. It's not just the basics, this can extend into fairly complex operations on your data (not to mention producing very pretty visualizations). Data analysis is one of the most important steps in data science, so I think it's worth keeping R around.
clearspandex 14y ago
Jonathan here, Co-Founder of Zipfian Academy. While it is easy to get up and running quite easily in R for simple analysis, it is a complex language that takes years to master. I recommend learning Python for the aspiring scientist because of its breadth of applicability. While R is probably better for statistical analysis than Python (every language has its specific domain where it shines), across the entire domain of tasks a data scientist must perform, I feel that Python provides the best aggregate utility. Also, as the comments below highlight, actual statistical analysis is but a small part of the data pipeline. Python has great facilities for interacting with data stores/sources in addition to being a powerful tool to clean and munge data. >When I think about, in my data programming related work, I'd say about 5% is doing analysis or executing statistical routines. And 95% of my time is spent on finding, cleaning, and properly normalizing data. I hope the post doesn't downplay the importance of R to statistical analysis, it is a mature language with a great community surrounding it. The toolset of a data scientist is probably one of the most heterogeneous out there and necessitates learning and using many different abstractions. For such a new (and hard to define) subject, I think dialogue is crucial to constructively advance the field. I would love to hear suggestions from the HN community about how to train the next generation of data scientist, what aspiring data scientists want to learn (or find difficult to learn), and how we can build a great data community.

by rm999 14y ago

minimize_me 14y ago

Depends on the analysis lifecycle I think - I hate using Matlab for writing any sort of large experimental framework in, especially if it has to interact with other systems, but for quick and dirty experiments, plotting, or parallelizing totally independent for-loops eg. for a grid-search over some model hyper-parameters, it's great (the latter is literally changing "for" to "parfor". After you fork out for the Parallel Computing Toolbox of course.) But then a lot of code in research is written in the style of use-code-for-a-week->get a graph->curve is higher than their curve?->publication. When I need to write something more substantive and reusable though, Python forever.

ahi 14y ago

Some things, maybe most things that a beginner wants to do are easy, but beyond the basics, R has a cliff for a learning curve. It's a complex language with some questionable design choices.

rm999 14y ago
I agree with you about questionable design choices, I really don't like developing in R. But if you have some data and you want to know more about that data (data analysis), R + a small handful of packages is the best free environment that I know of. It's not just the basics, this can extend into fairly complex operations on your data (not to mention producing very pretty visualizations). Data analysis is one of the most important steps in data science, so I think it's worth keeping R around.

clearspandex 14y ago

Jonathan here, Co-Founder of Zipfian Academy. While it is easy to get up and running quite easily in R for simple analysis, it is a complex language that takes years to master. I recommend learning Python for the aspiring scientist because of its breadth of applicability. While R is probably better for statistical analysis than Python (every language has its specific domain where it shines), across the entire domain of tasks a data scientist must perform, I feel that Python provides the best aggregate utility. Also, as the comments below highlight, actual statistical analysis is but a small part of the data pipeline. Python has great facilities for interacting with data stores/sources in addition to being a powerful tool to clean and munge data. >When I think about, in my data programming related work, I'd say about 5% is doing analysis or executing statistical routines. And 95% of my time is spent on finding, cleaning, and properly normalizing data. I hope the post doesn't downplay the importance of R to statistical analysis, it is a mature language with a great community surrounding it. The toolset of a data scientist is probably one of the most heterogeneous out there and necessitates learning and using many different abstractions. For such a new (and hard to define) subject, I think dialogue is crucial to constructively advance the field. I would love to hear suggestions from the HN community about how to train the next generation of data scientist, what aspiring data scientists want to learn (or find difficult to learn), and how we can build a great data community.