4 ms·

I consider myself a pretty experienced developer/software engineer/whatever. Decade into my career, started in iOS in Obj-C, learned Swift along the way, event

by auto 5y ago

I consider myself a pretty experienced developer/software engineer/whatever. Decade into my career, started in iOS in Obj-C, learned Swift along the way, eventually migrated to the backend with Java/SQL, and finally found myself where I really wanted to be, embedded doing C/C++ work for a household name. That said, about halfway through my master's about two years ago, I found myself in an intro to data mining course that was sold as an "we will teach you R". I had heard non-programmer math friends talk about what they had accomplished in R, and was excited to dive in. Now, it didn't help that the class ended up being heavy on the statistics side (which despite a math/cs double major, stats was never my thing), but the actually learning R part was 99% left as an exercise to us alongside of the classwork required. I can say without a doubt, learning R is the worst programming experience I've ever had. Our assignments would give some high level direction on which libraries to use, but getting the right libraries setup and in the environment was just an absolute nightmare. All I remember from that class is hours every week googling unreadable python pukes from R studio (because apparently everything data mining/ML related in R is actually just python), and then spending an hour or less actually doing the statistics work. I feel bad because I feel like I was setup to not be able to give it a fair chance, but if that's what non-programmer math types are subjected to when told "you need to do some programming for your job", I can understand the apprehension.

CapmCrackaWaka 5y ago
> All I remember from that class is hours every week googling unreadable python pukes from R studio (because apparently everything data mining/ML related in R is actually just python) I’m curious what libraries you were using, I’ve had to go into the source of quite a few popular libraries and I don’t think I’ve ever encountered Python. Lots of C and it’s derivatives, lots of Stan and FORTRAN, but I don’t think I’ve seen Python yet.
- auto 5y ago
  Just going back and looking through some of the homework, here's what I'm seeing imported: readr, caret, lattice, ggplot2, RColorBrewer, mlbench, ElemStatLearn, klaR, dplyr, arules, arulesViz, tensorflow
  - malshe 5y ago
    In this list only tensorflow requires Python
    - BoiledCabbage 5y ago
      And I haven't used it, but there is Torch for R as the alternative which isn't supposed to have any dependency on Python. https://torch.mlverse.org/ https://torch.mlverse.org/
  - disgruntledphd2 5y ago
    Yeah tensor flow is python, and is a nightmare to work with. That's not Rs fault though
temp8964 5y ago
What you described is a common phenomenon in stats / data mining / psychometrics / econometrics. They all need students to use certain programing languages, such as SAS / Stata / R / Python, but they don't really spend time to teach those programing languages. I guess this could also be the same for MATLAB in math?
- pacbard 5y ago
  The problem is that there are no incentives in learning how to program for people in those fields. Most people just get their code to "work" (i.e., output the analyses that they want) without really wanting to know how it works. Most of the time code is passed from grad student to grad student and modified to make it work for the specific analysis. As a result, you get Frankenstein code that somewhat works but that is good enough for writing a results section of a paper. There are people that know how to code but those are few and in-between. Usually they are pushed out of academic positions because there are very few ways to fund work to develop scientific code.
gonzo41 5y ago
I pretty much had the same experience with R. I've never been able to get really productive with it as a software developer. I feel like I know too much and can't break from old habbits. It's very academic which I feel really holds it back from software devs and also captures non developers in it's web. Whilst it's certainly got some runs on the board. I think having data science folk work in more standard languages would actually make supporting them and their needs easier.
- notagoodidea 5y ago
  Funny, I'll never say that R itself is very academic. The environment maybe, the users sure but the language is very much an mutated Lisp with vectorized operation. Same for the python stuff, I never hit that problem working a lot with it but most of the time when I wanted to check a lib, I landed in C++ aka I am not sure to understand how to read the code. What in R made it feel academic for you?
  - haihaibye 5y ago
    For one, R has a built in citation() function. Secondly, it seems optimised for producing one off figures and results, not production systems generally desired by industry.
beforeolives 5y ago
> because apparently everything data mining/ML related in R is actually just python Everything? I'm just wondering what you've been doing exactly. I know that Tensorflow in R is just a wrapper on top of Python, not sure what else. If you're doing any deep learning, then going straight to Python is certainly much better than using R. For most other things it's not quite as clear of a decision.
- _Wintermute 5y ago
  Hilariously there's an argparse library for R, which has python as a dependency.
malshe 5y ago
> because apparently everything data mining/ML related in R is actually just python I think it is apparent only to you. The most popular package for ML in R is `caret` and it has nothing to do with Python. Similarly, `mlr` also has no Python. In fact, except for tensorflow and keras, I can't think of any major ML package that needs Python. Even torch package which brings pytorch to R doesn't need Python (https://cran.r-project.org/web/packages/torch/index.html https://cran.r-project.org/web/packages/torch/index.html). What confuses me even more is that you were learning statistics but using R packages that use Python as backend. In my experience, almost all the new statistics and econometrics methods are first released as R packages by the researchers. Can you name any data mining R packages that you used that required Python? I am really curious to know.
warlog 5y ago
Dataframes...python has R to thank...too bad they suck compared to R.

by auto 5y ago

CapmCrackaWaka 5y ago

> All I remember from that class is hours every week googling unreadable python pukes from R studio (because apparently everything data mining/ML related in R is actually just python) I’m curious what libraries you were using, I’ve had to go into the source of quite a few popular libraries and I don’t think I’ve ever encountered Python. Lots of C and it’s derivatives, lots of Stan and FORTRAN, but I don’t think I’ve seen Python yet.

auto 5y ago
Just going back and looking through some of the homework, here's what I'm seeing imported: readr, caret, lattice, ggplot2, RColorBrewer, mlbench, ElemStatLearn, klaR, dplyr, arules, arulesViz, tensorflow
- malshe 5y ago
  In this list only tensorflow requires Python
  - BoiledCabbage 5y ago
    And I haven't used it, but there is Torch for R as the alternative which isn't supposed to have any dependency on Python. https://torch.mlverse.org/ https://torch.mlverse.org/
- disgruntledphd2 5y ago
  Yeah tensor flow is python, and is a nightmare to work with. That's not Rs fault though

temp8964 5y ago

What you described is a common phenomenon in stats / data mining / psychometrics / econometrics. They all need students to use certain programing languages, such as SAS / Stata / R / Python, but they don't really spend time to teach those programing languages. I guess this could also be the same for MATLAB in math?

pacbard 5y ago
The problem is that there are no incentives in learning how to program for people in those fields. Most people just get their code to "work" (i.e., output the analyses that they want) without really wanting to know how it works. Most of the time code is passed from grad student to grad student and modified to make it work for the specific analysis. As a result, you get Frankenstein code that somewhat works but that is good enough for writing a results section of a paper. There are people that know how to code but those are few and in-between. Usually they are pushed out of academic positions because there are very few ways to fund work to develop scientific code.

gonzo41 5y ago

I pretty much had the same experience with R. I've never been able to get really productive with it as a software developer. I feel like I know too much and can't break from old habbits. It's very academic which I feel really holds it back from software devs and also captures non developers in it's web. Whilst it's certainly got some runs on the board. I think having data science folk work in more standard languages would actually make supporting them and their needs easier.

notagoodidea 5y ago
Funny, I'll never say that R itself is very academic. The environment maybe, the users sure but the language is very much an mutated Lisp with vectorized operation. Same for the python stuff, I never hit that problem working a lot with it but most of the time when I wanted to check a lib, I landed in C++ aka I am not sure to understand how to read the code. What in R made it feel academic for you?
- haihaibye 5y ago
  For one, R has a built in citation() function. Secondly, it seems optimised for producing one off figures and results, not production systems generally desired by industry.

beforeolives 5y ago

> because apparently everything data mining/ML related in R is actually just python Everything? I'm just wondering what you've been doing exactly. I know that Tensorflow in R is just a wrapper on top of Python, not sure what else. If you're doing any deep learning, then going straight to Python is certainly much better than using R. For most other things it's not quite as clear of a decision.

_Wintermute 5y ago
Hilariously there's an argparse library for R, which has python as a dependency.

malshe 5y ago

> because apparently everything data mining/ML related in R is actually just python I think it is apparent only to you. The most popular package for ML in R is `caret` and it has nothing to do with Python. Similarly, `mlr` also has no Python. In fact, except for tensorflow and keras, I can't think of any major ML package that needs Python. Even torch package which brings pytorch to R doesn't need Python (https://cran.r-project.org/web/packages/torch/index.html https://cran.r-project.org/web/packages/torch/index.html). What confuses me even more is that you were learning statistics but using R packages that use Python as backend. In my experience, almost all the new statistics and econometrics methods are first released as R packages by the researchers. Can you name any data mining R packages that you used that required Python? I am really curious to know.

warlog 5y ago

Dataframes...python has R to thank...too bad they suck compared to R.