7 ms·

Data science seems, for now, to be what software engineering was supposed to be: a career where you choose your own tools and problems (with some constraints) a

Data science seems, for now, to be what software engineering was supposed to be: a career where you choose your own tools and problems (with some constraints) and that gives you the freedom to move about in different industries instead of being stuck to one in the way that most programmers now are. In many organizations, data scientists are full-time programmers but who get the dibs on the most interesting projects. I identify as a data scientist as code-word for "no-hire if the work's not interesting". There's plenty of hard engineering (in addition to traditional data science, where statistical intuition is more important) in data science. There are plenty of data scientists working on OS hacks, compilers, and other "hard engineering" topics. The difference and advantage for a data scientist is that your boss doesn't think he could do your job if he wanted to. If your title shows that you actually know math, you're not "just a code monkey".

kamaal 13y ago
>>There are plenty of data scientists working on OS hacks, compilers, and other "hard engineering" topics. Can you give some examples on this? Seems very interesting.
- michaelochurch 13y ago
  Nothing comes off the top of my head, and it's not like they get to specialize in compilers. They mostly end up doing one-off hacks to make an existing algorithm more performant. What's fun about machine learning is that it touches so many other parts of computer science. You could be at a high level writing DSLs in Clojure to make it possible for statisticians to specify their models directly, or you could go to the low level and write GPU code. The general rule is that if your boss thinks he can do your job, you lose. If he doesn't think that, you win. When you're a data scientist, your odds are much higher of coming out in the second category.
  - christopheraden 13y ago
    To your last point, it depends on what role you currently serve to your company. If you're among statisticians, you make yourself valuable by knowing more about computer science and programming than the rest of the group. If you're amongst engineers (probably more common than the former--at least on HN), knowing probability and statistics gives you that edge.
- christopheraden 13y ago
  I encourage you to peruse the Github page of the R celebrity Hadley Wickham. While trained as a statistician and employed as a professor of statistics at Rice, he writes quite a few R packages that deal with more traditional development roles (unit testing and debugging, not to mention Plyr and GGplot): https://github.com/hadley?tab=repositories https://github.com/hadley?tab=repositories Also, Duncan Temple Lang, another professor of statistics, has done some really amazing work with R compilers and CUDA interfaces. https://github.com/duncantl?tab=repositories https://github.com/duncantl?tab=repositories http://www.omegahat.org/ http://www.omegahat.org/
- sedachv 13y ago
  Rob Zinkov (https://github.com/zaxtax https://github.com/zaxtax) is a first-rate hacker. He has a popular ML blog (http://zinkov.com/ http://zinkov.com/) and organizes the Los Angeles machine learning users group.
rm999 13y ago
I've seen you write something similar to this before, but I don't agree. Counterexample: a good friend of mine specialized in computer graphics where he did a lot of math-heavy research. I specialized in machine learning where I also did a lot of math-heavy research. We are both full-time programmers who choose our tools and flexibly work on interesting projects, but he would never get hired as a "data scientist", whereas I did. I think anyone who has specialized enough in something that's useful to a company can find a good, flexible job (e.g. my graphics friend and me). The fact that the term "data scientist" is so ubiquitous simply means it's cooler/more useful than other specializations right now. Some companies may get away with abusing the title because it's so vague, but it does mean something to companies that actually know they need one.

kamaal 13y ago

>>There are plenty of data scientists working on OS hacks, compilers, and other "hard engineering" topics. Can you give some examples on this? Seems very interesting.

michaelochurch 13y ago
Nothing comes off the top of my head, and it's not like they get to specialize in compilers. They mostly end up doing one-off hacks to make an existing algorithm more performant. What's fun about machine learning is that it touches so many other parts of computer science. You could be at a high level writing DSLs in Clojure to make it possible for statisticians to specify their models directly, or you could go to the low level and write GPU code. The general rule is that if your boss thinks he can do your job, you lose. If he doesn't think that, you win. When you're a data scientist, your odds are much higher of coming out in the second category.
- christopheraden 13y ago
  To your last point, it depends on what role you currently serve to your company. If you're among statisticians, you make yourself valuable by knowing more about computer science and programming than the rest of the group. If you're amongst engineers (probably more common than the former--at least on HN), knowing probability and statistics gives you that edge.
christopheraden 13y ago
I encourage you to peruse the Github page of the R celebrity Hadley Wickham. While trained as a statistician and employed as a professor of statistics at Rice, he writes quite a few R packages that deal with more traditional development roles (unit testing and debugging, not to mention Plyr and GGplot): https://github.com/hadley?tab=repositories https://github.com/hadley?tab=repositories Also, Duncan Temple Lang, another professor of statistics, has done some really amazing work with R compilers and CUDA interfaces. https://github.com/duncantl?tab=repositories https://github.com/duncantl?tab=repositories http://www.omegahat.org/ http://www.omegahat.org/
sedachv 13y ago
Rob Zinkov (https://github.com/zaxtax https://github.com/zaxtax) is a first-rate hacker. He has a popular ML blog (http://zinkov.com/ http://zinkov.com/) and organizes the Los Angeles machine learning users group.

rm999 13y ago

I've seen you write something similar to this before, but I don't agree. Counterexample: a good friend of mine specialized in computer graphics where he did a lot of math-heavy research. I specialized in machine learning where I also did a lot of math-heavy research. We are both full-time programmers who choose our tools and flexibly work on interesting projects, but he would never get hired as a "data scientist", whereas I did. I think anyone who has specialized enough in something that's useful to a company can find a good, flexible job (e.g. my graphics friend and me). The fact that the term "data scientist" is so ubiquitous simply means it's cooler/more useful than other specializations right now. Some companies may get away with abusing the title because it's so vague, but it does mean something to companies that actually know they need one.