11 ms·
R Passes SAS, but Python Leaves Them Both Behind
- NumberSix 10y agoThere is a question regarding the definition of "data science jobs." The author explains his methodology in a lengthy report http://r4stats.com/articles/how-to-search-for-data-science-jobs/ http://r4stats.com/articles/how-to-search-for-data-science-j... The issue is what is "data science" really? In what respect is it different from traditional statistics and data analysis and not just a new buzzphrase? Probably many jobs using SAS could be considered "data science" but don't use the specific buzz words and phrases that the author specifies in his methodology to identify "data science" jobs. Thus, the headline that "R Passes SAS" could be inaccurate, except in the sense that R is more popular among statistics and data analysis jobs that use "data science" buzzwords and phrases.
- jl6 10y agoSAS is also a huge iceberg, without much in the way of open source culture that lends itself to visibility. A lot of SAS happens behind closed doors at megacorps.
- R_haterade 10y agoAlso notable that so many new shops are foregoing it entirely. SAS, and its price tag, is a holdover from the days when 'analytics' was an afterthought for companies looking to maximize profit. Now that much less mature companies are realizing the value of 'analytics' (I hate that word) SAS's cost doesn't really make sense.
- halflings 10y agoWhat word do you prefer to 'analytics', if we are talking about taking data and analysing it to extract insights? This does not have to involve machine learning, and "pattern recognition" is an academic term (and might be confusing for laymen).
- R_haterade 10y agoDefinitely agreed. But the word analytics means too many different things to too many different people at this point. I'm in favor of more words to differentiate each part.
- xapata 10y agoDifferent types of analysis: - descriptive analysis (clustering, summary statistics, etc.) - predictive analysis / forecasting - optimization - automation I suppose that last one involves engineering as well.
- R_haterade 10y agoWe try to do this as much as possible where I work. It's... kinda working.
- deleted 10y ago[deleted]
- xapata 10y agoWhy not just "analysis"? Business people seem determined to invent new jargon when our current vocabulary is sufficient.
- R_haterade 10y agoThe euphemism treadmill makes my raises bigger.
- bigger_cheese 10y agoMy company uses SAS extensively a large part of that is legacy (for the same reason we have a heap of Fortan code still). SAS has been in use here since the 80's. I have gone as far as porting some (non performance sensitive). C++ models into SAS because people in my org understand the language better. I don't actually hate SAS and I'm very productive using it but at the same time I do feel that not knowing R limits my opportunities if I ever want to change jobs and work for an outside organisation. We don't strictly use SAS for analytics. A big part of SAS we use is the "BI" side I don't know if that acronym is still in vogue but I'm talking about ad-hoc querying reporting etc. The kind of stuff one step above what you'd use a spreadsheet for if that makes sense. I think where SAS excels is they have made it very easy for non experts to be productive with it. Kind of similar to MATLAB in engineering world if people are familiar with that. A lot of non statiscians and non programmers use it inside my work (my background is engineering). Accountants, Managers, mechanical engineers etc are all pretty capable of using Enterprise Guide to run adhoc queries and generate reports and the like. The only other similar tool I'm aware of is IBM's Cognos. We used to use both packages (as well as Microsoft Access) but about 10 years ago the business agreed to standardize around SAS. I've heard there is a similar tool in the R world to Enterprise Guide (R-Studio I think???) but I'm not all that familiar with it I've heard it behaves more like an IDE rather then a drag and drop way to construct queries, graphs/reports etc. If anyone has made the transition from SAS (or Cognos) to R (especially for a large org) I'd be keen to hear what tools you'd recommend and how the business found it?
- R_haterade 10y agoRstudio is an IDE. If you want a gui for R, check out Rattle. My shop is going the opposite direction, from R/excel to SAS+Cognos. I'm not happy about that decision, but the pay is good and the problems are still interesting.
- StClaire 10y agoEven some of the mega corps want to get away from SAS. A friend of mine who does data analytics at the power company told me they want to bring on more Python developers despite their regulators wanting things done in SAS.
- tom_b 10y agoNice data munging out of Indeed.com here. The article author gives a detailed description of searching Indeed in a write-up linked from the original article as well (http://r4stats.com/articles/how-to-search-for-data-science-jobs/ http://r4stats.com/articles/how-to-search-for-data-science-j...). Just playing around with the search terms from that second linked article is also interesting - it would appear that many terms ("machine learning", "data science", "predictive modeling", some others) show that Amazon has the largest number of job listings from a single company - for "machine learning" Amazon shows 1706 listings out of 12499 or almost 14% of all listings . . . The way Amazon also pops out in other data science term searches is also interesting - at least in their job listings, Amazon seems to really be attempting to slurp up candidates with deeper data and stats skills. For some time I have been somewhat cynical about data science. My impression has been that much of what has been pushed as data science jobs is thinly veiled data reporting gigs (just plain old business intelligence). While I still think data science is over-hyped, I think I need to reconsider just how critical it will be as a knowledge base or skill set. While there may not be a large number of deep learning jobs out there, the expectation that a data hacker can be expected to perform a linear or logistic regression against a set of gathered and cleaned data may be closer to fizz buzz than I previously assumed. I am teaching an introductory programming class (using Python) this semester and students are definitely focused on data science as a career track.
- sixhobbits 10y agoI'm surprised that the author combines "the C languages" saying that most adverts that mention any of C/C++/C# mention all three. In my experience there is a large difference between companies searching for C# developers and those searching for C/C++. After blurring this distinction he concludes that R and Python are "very different languages" while I consider them to be largely overlapping.
- brogrammernot 10y agoAgreed. So far in my search, C# leans toward Microsoft shops seeking C#/.NET whereas C/C++ has been companies searching for embedded software roles.
- metaobject 10y agoMostly agree, but I've also seen a fair bit of C/C++ skills associated with jobs involving *nix development environments. I've rarely seen C++ associated with embedded jobs (even though I've read that it can certainly be used if care is taken to avoid things like dynamic dispatch, etc)
- charles-salvia 10y agoC++ is very big in finance and game development.
- Arizhel 10y agoC++ is used in avionics.
- xapata 10y agoR and Python have some similarities, but saying a they are largely overlapping is a bit of an exaggeration.
- cwyers 10y agoI hate these sort of comparisons. You've got R, Python, SAS... okay, those are sort of similar. Then you've got Java and "C, C++ or C#," and man, including C# with C/C++ is... fraught. Then you've got Hadoop, Spark, Hive... okay, those are all kind of different from what we've had so far. Now you've got Tableu and RapidMiner. Uh. In the second chart, "Microsoft" is included as a keyword. Okay. It's just... comparing apples, oranges, bananas, grapes and pears. What's it supposed to tell us?
- dredmorbius 10y ago"Microsoft was a difficult search since it appears in data science ads that mention other Microsoft products such as Windows or SQL Server. To eliminate such over-counting, I treated Microsoft different from the rest by including product names such as Azure Machine Learning and Microsoft Cognitive Toolkit. So there’s a good chance I went from over-emphasizing Microsoft to under-emphasizing it with only 157 jobs." Read the methodology, Luke.
- cwyers 10y agoOkay, but what does that mean? Microsoft Cognitive Toolkit is like Tensorflow, which is included as an item on the list. Azure Machine Learning is something you can script with R or Python, and allows you to create APIs for predictive use on Azure. What good does lumping those together do? And what does it tell us that some things that are basically an R/Python library are less popular than R or Python themselves? It's this weird, uneven mix of things. Some are programming languages, some are libraries or frameworks, some are end-user products like Tableau.
- nl 10y agoThis is pretty much what data science in the real world is like. Define the question you are interested in (in this case, a somewhat reasonable attempt to compare R/Python/SAS) and the put other things in blobs with a note that says this is what this blob is. Enjoy.
- 10y ago
- rm999 10y agoI'm glad to see the quick convergence on Python. I've evaluated Python every ~2 years since 2006 for "data science" tasks (machine learning, statistics, data munging, and visualization). I'd argue that Python only properly covered this full data science stack 1.5-2 years ago. R covered this stack adequately probably around 2011-2012. Matlab had this before 2005. What makes Python a superior language to Matlab and R is the ease of software development. It's an easy, pleasant language to work in, and I trust it for production tasks (I've written production R code and it's fairly hard to read and fragile). What's even better is data science is moving 100% into Python 3 (from 2) by 2020: http://www.python3statement.org/ http://www.python3statement.org/
- nerdponx 10y agoHow do you feel about Julia? Data analysis in Python has plenty of warts.
- acmj 10y agoJulia is great, but IMO, it needs to reach 1.0 first before making a splash. It would be frustrating if you found your tools developed on v0.4 stopped working on v0.5 released a year later. API/ABI stability is critical to the adoption of new programming languages/libraries.
- Buttons840 10y agoI've heard they plan to release Julia 1.0 this year. So keep an eye on it.
- blauditore 10y agoI once worked on a piece of software communicating with a SAS instance (doing live decision management), but everything about it seemed sketchy. No one using it really liked it, and its internals always appeared like a blackbox to me. Also, either it is terribly engineered or the devs working with it were just bad - we wanted a JSON-based REST API, but they said that it's "not possible with SAS", so we fell back to badly organized HTTP calls with XML. Does anybody have some insights about internal quality and code "health" in SAS?
- R_haterade 10y agoAnecdotal, but our SAS decision support environment went down Friday. I can report back after the post-mortem if you like.
- blauditore 10y agoYeah, would surely be interesting.
- R_haterade 10y agoNothing exciting to report. It was a resource-sizing problem and they didn't adequately separate prod and dev environments. Someone choked the dev and the prod went down with it.
- blauditore 10y agoThanks anyway for the insight!
- bigger_cheese 10y agoFrom experience I've found the "core" parts SAS Base, SAS GRAPH, OR, ETS etc are very stable. other parts we have had issues with in the past - Stored processes, LSF for scheduled jobs. SAS tech support have been pretty good though I believe my workplace pays a lot for the privilege of being able to contact them direct. One thing I have found is you can usually throw something together in a pretty "hackish" manner but typically there is a better and more optimal method of doing it which will be much more stable. Sometimes it doesn't hurt to ask tech support "What is the recommended method for doing..." Code which abuses the SAS macro language is especially notorious and a good candiate for asking this. A quick google tells me first class JSON support is pretty new (dec 2016). http://blogs.sas.com/content/sasdummy/2016/12/02/json-libname-engine-sas/ http://blogs.sas.com/content/sasdummy/2016/12/02/json-libnam... I don't know much about this all the A2A messaging is XML based in my org.
- confounded 10y agoThis is a sample of non-data-science job postings, which mention buzzwords like "big data". Java is not a popular language for data analysis.
- Xcelerate 10y agoI recently finished grad school and accepted a data scientist job a couple weeks ago in part because the job description mentioned expertise with Julia as one of the preferred qualifications (it's rare to see a listing that mentions Julia). It's the language I used for the bulk of my research over the last four years, and has been improving rapidly since it was released. I like Julia a lot more than Python, and I hope it continues rising in popularity. I think once it hits v1.0, we'll begin to see a lot more companies adopting its usage for data science, statistics, and machine learning.
- fnord123 10y ago> I think once it hits v1.0, we'll begin to see a lot more companies adopting its usage for data science, statistics, and machine learning. I don't. It doesn't have the incumbency of R, the use in other areas of programming of Python, or a company actively marketing it like MATLAB. It's not 5x or 10x or whatever good enough than the alternatives to assert itself in the playing field. If it means you get your work done using it, be all means use it. But I think it will stay around clojure levels of use in data science, statistics, and machine learning.
- whyrt12 10y agoOnce you can compile a julia app- front end, back and probabilistic prog/ML to web assembly and have it run in browser and mobile, it will skyrocket in popularity.
- fnord123 10y agoWhy? The only current benefit of Julia afaict is the tracing jit. If you run the tracing jit on web assembly then it's giving up most of its performance benefits. And Python could be built on web assembly as well. But who knows. Weird things seem to become popular despite all the negative points.
- whyrt12 10y agoIt does not have a tracing JIT, and its speed is by far not the only benefit. See link below It can precompile very fast code before runtime. Python will require an interpreter and or hefty runtime. https://discourse.julialang.org/t/julia-motivation-why-werent-numpy-scipy-numba-good-enough/2236/10?u=mikeinnes https://discourse.julialang.org/t/julia-motivation-why-weren...
- robertk 10y agoPython suuuucks.
- llukas 10y agoThis attempt to eli5 could be improved.
- IndianAstronaut 10y agoR's data analysis libraries are still far ahead of Python. Dplyr, Shiny, etc. So much stuff is still in built in R. Places where I still use R is its easy to use statistical functions, handling large amounts of missing data, etc.