13 ms·

Mathematicians becoming data scientists

237 points by heinrichf 10y ago

nrdwavexe 10y ago
Look, this is evil. I don't want to hear any complaints about my calling a spade a spade. The Nazis did it: evil. Google does it, and it is evil, and they go out of their way to say WE ARE NOT EVIL and guess what: it is evil. Why is it evil? You have a bunch of people in corporate america who are FORCING an information technology posture on the American people that the American people HATE because it is HORRIBLE and EVIL. Next, these EVIL people go around saying what they are doing is "data science" and start blaming mathematicians. This is absolutely 100% evil. Mathematicians have zero interest in this. Pointing the finger at mathematicians and calling them "data scientists" is intolerable stupidity and the people who do it should expect to get their fingers cut off by mathematicians who don't like to be pointed at by IDIOTS.
- ianai 10y ago
  Care to elaborate on specific points that you consider evil? There's a lot to discuss and I think you make an interesting claim.
- M_Grey 10y ago
  ...You said "evil" a lot, but what exactly is the evil part? That's a hell of a claim to make, and then only support with hyperbole and bluster.
heinrichf 10y ago
Note: this is the blog of Jordan Ellenberg (https://en.wikipedia.org/wiki/Jordan_Ellenberg https://en.wikipedia.org/wiki/Jordan_Ellenberg).
- curiousgal 10y ago
  I've just started reading How Not to Be Wrong: The Power of Mathematical Thinking by him. I enjoy these synchronicities.
- deleted 10y ago
  [deleted]
M_Grey 10y ago
•Can you walk away from a problem when the solution is “good enough,” are you able to switch between tasks or problems with relative ease? Are you OK with simple solutions to problems that could have more complicated solutions, but only with rapidly diminishing returns? Translation: Can you be a good assembly line worker, devoid of pride or curiosity and just do the damned job? If so, great, we can use you! If not, please return to academia. Lovely stuff.
- denzil_correa 10y ago
  A lot of things in the industry are built via hacks where good enough is good enough. The goal is not to satiate curiosity but to generate business value. This does not equate to lack of pride, just a different goal.
  - M_Grey 10y ago
    Yeah, like the IoT. I'm sorry, but we've reached the point where that level of blithe pursuit of a "different goal" is becoming destructive on a broad scale.
- hyperbovine 10y ago
  A lot of successful academics I know (including mathematicians) operate on similar principles. Not everybody embraces the Gowers/Perelman/Zhang model, nor should they.
- laretluval 10y ago
  That is not the intended meaning at all. It can be a great point of pride and skill to find a solution that is as good as possible subject to complexity and time constraints that do not exist in academia.
  - M_Grey 10y ago
    And how often does that pride last once you see the application that work is put to? At least if you do good math, you can take pride in that.
denzil_correa 10y ago
A lot of this can also generally apply to a Computer Science Ph.D graduates who would like to understand what working in the industry entails.
11thEarlOfMar 10y ago
I've been encouraging my daughter, a statistics major, to pursue data science by including Python/R in her studies and then possibly heading back for an MBA. But not sure if an MBA would be a benefit. Thoughts from actual data scientists?
- ecesena 10y ago
  Personally I think a PhD would be better than an MBA if she really wants to do data science, but generally speaking I'm for as more edu as possible, and I'm sure she'll benefit from an MBA too.
  - laughfactory 10y ago
    The PhD will only be useful in a small (but growing) subset of data science jobs. Their are data scientists who develop new algorithms and techniques, and those who apply the. For application, the PhD is probably overkill and extensive experience with a bachelor's, or a master's is better. For theory the PhD can't be beat, of course. I say the opportunity costs are high, and should be carefully weighed, because I have my bachelor's and am making more than a data scientist friend of mine who has his PhD. The vast majority of what he learned in the PhD program he hasn't found useful in the real world (yet, anyways). So it's all about what problems you want to work on. Personally I thrive in the application arena of data science, while others thrive in the R&D data science arena. If you like one you probably won't like the other.
    - tgokh 10y ago
      It may not necessarily be the best option for the OP's daughter, but it is possible to do a PhD that uses data science as a tool rather than the focus. There's a lot of interesting work going on in academia using data science in a variety of fields (like medicine) related to predictive analytics (predicting hospital readmission, predicting therapy outcomes), time series analysis (analyzing EKGs, and other vital sign data) etc. A masters would likely allow you to work on similar projects, but if the goal is a PhD, there are certainly application rather than theory focused research groups across the country
diimdeep 10y ago
>. A big thing about the transition to tech is that you possibly start communicating with people who don’t really know what a vector space is. Be ready to have those conversations. Honestly ask yourself if you’re OK with having those conversations. This is so snobbish. What here is even think about ?
- pmiller2 10y ago
  It's not snobbish at all. The CS/developer version is "someone who doesn't know what static typing is," or, maybe even "someone who doesn't know what computers are and are not capable of."
- cottonseed 10y ago
  This is the wrong comparison. Read the second footnote. An intern is a developer in training. The post is talking about mathematicians (who spent a decade or more learning a highly technical field) going to work in an environment where, largely, their peers, boss and consumers of their work do not know, have no interest in and and will never learn the most basic concepts of their field.
  - pavlov 10y ago
    I suspect the majority of programmers work in that kind of environments...? The startups and Googles of this world are the exception when it comes to employer tech-savviness.
    - Stasis5001 10y ago
      When you get a PhD, you mainly only take classes in your department for the whole time, whereas in undergrad you still have classes in other fields. This usually extends beyond coworkers into friend circles too. This isn't math specific, but more generally to grad school. Second, programmers at companies almost never work in isolation from other programmers, and in most open office environments are close enough they could touch another one from their desk. On the other hand, a startup with five to ten engineers may be hiring you as their first data scientist, and bigger companies may be putting you on an embedded team, with the nearest data scientist a hallway or floor away. And this isn't a big deal in terms of teamwork, but most data scientists don't have a PhD in mathematics, so if you find higher level mathematical ideas and notation to be the most efficient way for you to think about a problem, your colleagues may not. That's also not a math specific thing though -- an economist, statistician, mathematician, computer scientist, physicist, etc. are all going to have slightly different ways they think about things internally.
stared 10y ago
I had two blog posts exactly on that topic, math/phys -> data science: - why: http://p.migdal.pl/2015/12/14/sci-to-data-sci.html http://p.migdal.pl/2015/12/14/sci-to-data-sci.html (on academia vs industry) - how: http://p.migdal.pl/2016/03/15/data-science-intro-for-math-phys-background.html http://p.migdal.pl/2016/03/15/data-science-intro-for-math-ph... (also got reprinted at KDnuggets)
- heinrichf 10y ago
  These posts were insightful, thank you for writing them.
- kurmouk 10y ago
  Thanks for sharing this. Especially for someone who's thinking about getting a data scientist job.
ykler 10y ago
I was a PhD student in mathematical logic, and I know a couple people who got PhDs in this area and became data scientists. One thing about this field (and other areas of math) is that almost everyone in it feels mediocre. It's like 10% of the people are 10x better than the other 90%, 1% are 10x better than the next 9%, .1% are 10x better than the next .9%, etc. The top two people in the field seem to be notably better than everyone else. Another thing is that the problems are incredibly arcane. They have no connection to practical concerns or even often other parts of math, and also you can't really explain what you are working on to someone who works in a different area of mathematical logic (unless maybe the person is extremely good). So you have a lot of people getting PhDs who are very, very smart by any normal measure but who would be doing mediocre and arcane work in mathematical logic. Part of the appeal of working in data science or working for the NSA or something is that it is somewhat down-to-earth and people will think you are really smart even if you are at the bottom of the top .01%. (And you can do really great work at this level -- I don't mean to say it is all about caring what other people think.) It is a little weird that the blog post is so negative in a way.
- george_ciobanu 10y ago
  A lot of these "arcane" topics have long-term horizons. Agree with everything else but the whole point of research is to probe the space of possibilities (theoretical and practical) to find major ideas. Most of them fail but that's part of the exploratory process - it's expected.
  - glial 10y ago
    It's expected, but as someone doing research it's super discouraging. I can certainly see the appeal of moving away of basic research with a very long term time horizon (fine for science, but frustrating for individuals) and towards a field with the potential of more immediate impact.
- deleted 10y ago
  [deleted]
- rcar 10y ago
uihsdfjkh 10y ago
What about the other way around? I'm from an engineering background, going in the direction of data scientist. Sometimes I find that my math skills could be stronger, and I try to read up on things when I encounter them, but still it sometimes feels like there is an infinite amount to learn. Maybe I could use some more systematic approach to it. Anyone else who has walked this path, and could come with some useful advice/resources?
- tmccrmck 10y ago
  Are you talking about self-studying mathematics or going back into higher education? I can give recommendations on books for the former but not the latter.
  - bsg75 10y ago
    Self study interest here.
  - praveenster 10y ago
    Yes, I am also interested in going the self study route with MOOCs but there is way too much information and resources that it is hard to figure out how/where to start. It would definitely be helpful if you could provide recommendations on books.
- digitalzombie 10y ago
  I got a bachelor in CS and went industry for 6 years. I'm coming back for master in applied statistic for data science. Just take statistic, especially Multivariate for big data (big data for statisticians is huge predictors not petabyte of observations). Math people can do so much in data science. I think statistic is better for a non math person and it's much better suited for data. Since statistic is all about data.
- sdenton4 10y ago
  Stats and linear algebra are what you want... On the linear algebra front, you get some understanding from your first course, but the more you internalize it by meeting the ideas in different contexts, the more useful it will be. A decent amount of higher math is turning things into almost-linear problems, and then trying to sort out the parts which don't quite fit...
olivierva 10y ago
Slightly off topic: They still can't build proper software. Academics (including mathematicians) are notoriously bad at writhing production grade software. This leads to handovers of 'proof of concepts' to seasoned software developer team who than struggle with the (often complex) mathematics/science behind it. Imho universities should give a bit more attention on how to write quality software; a bit of test driven development and continuous integration is not that hard and would massively improve the quality of the software written by scientists.
- pryelluw 10y ago
  Ive been involved in projects where this was an issue. The main problem was that operations were not abstracted away from the types or boilerplate.
- JustSomeNobody 10y ago
  As a developer, I assure you that even some developers can't write production quality software. Especially some enterprise developers. They either over engineer and never ship or under engineer and ship crap.
- contravariant 10y ago
  Then again, those are things that can easily be thought. The harder parts of programming such as choosing the right abstraction, or coming up with the right approach for solving a problem, or even formulating a problem, are more of mathematical nature.
- emmanuel_1234 10y ago
  This is precisely why I dread hiring academic-only profile. Note: I am myself from academia, but I was lucky enough to specialized in CS-related field (NLP). When I wanted to leave academia, Data Science was not a thing where I live and I had to start again at a junior dev position (i.e.: it was that, starving or staying in academia). Now, I work with a lot of people way smarter than I am, who are mostly useless because they can hardly prototype their stuff in Python or run an SQL query. And they'd expect to only work on the best, cleaned and formatted dataset and only do high-end maths on those. Reality hits hard, we're losing money paying them and they're wasting their time not doing what they like. Add to that the frustration / jealousy that this creates. In that regard, I like that famous definition for Data Scientist: "A programmer that know more about statistics than most programmers, or a statistician that knows more about programming than most statisticians". And don't get me started on the general repulsion for understanding the basics of how a business runs from academia. Data Science is all about application.
nerdherfer 10y ago
One side of this that bugs the crap out of me: I have a PhD in math and, after getting sick of teaching mediocre students with a palpable aversion to mathematics, I spent a couple of years applying for data science jobs and getting very few responses. The few interviews I got seemed to go well, but I had no offers and gave up on leaving academics. I have experience in software development (C++), I have a portfolio of statistics/data analysis projects in R/RStudio, but what I think is toxic is having spent so many years in academics; I finished my Phd in 2003. Being an over-40 academic feels like an insurmountable liability, even though I have many of the skills that people complain about being in short supply. And another thing: get off my lawn, you rowdy kids! :|
- deleted 10y ago
  [deleted]
- ordinaryperson 10y ago
  It's never too late to switch. I'd suggest figuring out which niche of data science most aligns with your personal interests and start there. Hiring managers don't discriminate based on "too many years in academia", it's more what's the reason they should hire YOU over candidate y? Diehard animal lover? I bet the Sierra Club would love a statistician who could help them quantify how many elephants are being poached in Africa per year. Interested in sports? Strava gets millions of data points a day, I'm sure they're looking for help finding signals in the noise. Can't get enough of financial markets? Finance jobs all over the place for quants. If your profile is too generic they'll see no good reason to hire you, but if you're passionate about X it can really work in your favor. The job search is depressing but never give up hope.
  - deleted 10y ago
    [deleted]
  - theparanoid 10y ago
    As an over 40yr old, yes it's probably too late to switch. After 30 it was a shock to me getting turned down for jobs when I aced the technical interviews.
wolfkill 10y ago
This post mostly describes me. I am a PhD student in applied mathematics and returned school to get my PhD precisely because I wanted to get a job outside of academics, most likely as a data scientist or scientific programmer. 2.5 years into my program I still struggle to see beyond the degree but am currently looking at non-academic internship opportunities as a way to start to untangle myself from academics. I also spend a significant amount of my time trying to teach my self good programming practices that I don't get from mathematics (how to write clean, adaptable code, choosing appropriate design patterns and writing generic code without over-abstracting, writing readable documentation and using version control, etc.) To those of you who have left academics do you have any other suggestions? How about for marketing myself as a programming mathematician when competing against CS and engineering students for internships/jobs?
- IndianAstronaut 10y ago
  Take a look into data engineering. A lot of data engineering jobs require you to implement mathematical models at scale.
mrcactu5 10y ago
I remember asking why i was rejected from Palantir and vaguely recall the phrase "too mathematical". After years of trying I have been unable to make the transition from Mathematics into Data Science and have since shifted my attention to other things.
bayesian_horse 10y ago
I guess it is easier to get into data science as a math graduate. I'm trying without a degree, and haven't succeeded yet.
uptownfunk 10y ago
I interview probably two to three people a week for a DS job. It's a really difficult role to hire for. I'm basically looking for mathematicians with some knowledge of stats who also enjoy coding and follow best practices in coding (i.e. They didn't just pick it up and hack something that works but care enough to document and structure so others can follow it at a minimum) but we also need the MBA component as well, a large part of this industry is how do you take what a DS does and deploy it in a business and ensure the business derives value from it. But I think coming at it from a mathematical background puts you in a much stronger footing than the cs guys who try to learn the math later. I rarely find strong candidates without a formal mathematical background.
- kriro 10y ago
  Interesting, thanks. It obviously depends a lot on the job but I would have thought that a good math background (which typically CS grads should have) would be enough for many applications. Especially since you mention the business problem aspect. I think that you can actually get quite far in DS (at least in deep learning) with rather "basic math". Unless you're innovating at the bleeding edge it's usually enough to reproduce existing ideas and applying them to interesting problems is the real key. So my hiring would have focused on CS grads with a natural interest in math, ideally very rigorous/meticulous.
  - laichzeit0 10y ago
    It makes me almost feel sad for the people that followed the "why bother doing a CS degree, you can learn to code without one" trend that probably peaked and is now maybe beginning to die a slow death as the market is flooded with people who can write code. If they ever want to transition into DS they're going to have to skill up and do the equivalent of undergrad CS math curriculum (discrete, calculus, linear algebra, math stats, etc.), which let's be honest, you cannot pick this up in any meaningful way in a "few months" of after hours/weekend study like you can when learning to program or learning some new "framework"; either that or just remain another dime-a-dozen code-cutter. It's sad because if you just did the 4 years of computer science and all the math and the stats that go with it you'd be so close to pivoting right now if you wanted to.
godmodus 10y ago
A few fellow students graduated with a BA and had their aplications rejected by most companies due to a lack of experience. Theory is nice. But Experience trumps all. Esp when you think it mostly boils down to PCA/DBSCAN, regression and scatter matrices. A month of Python/R/D3 on some initial data like pubmed/twitter or any toy dataset would go a looong way.
- kriro 10y ago
  I think doing well on Kaggle competitions would already be a good enough indicator for smart companies so it might be a good idea to have student teams do those on the side. Might be a decent idea to inform students that these exist. I feel like this applies even more for other degrees like econ(ometrics) (since I feel like a good math grad should not have trouble getting any data science job).
happy-go-lucky 10y ago
There is positive correlation between applied math and programming. I see expertise in these areas coupled with a knack of getting a point across as a potential career opportunity. No wonder academics in some parts of Asia have, for generations, seen math as “evergreen.”
happy-go-lucky 10y ago
If you see programming as a tool for solving domain-specific problems, then domain experts who can exploit the tool obviously have an edge over programmers who rely on inputs from these experts when developing domain-specific applications.
visarga 10y ago
> Do you like modeling complex problems mathematically? Do you enjoy kicking the ball towards the gate and scoring? Then you might be good for Football.
known 10y ago
I think MBA adds more value to your career in globalization
xamuel 10y ago
As a mathematician who successfully made the leap, I think the biggest requirement is ability to swallow your pride and "lower" yourself to seriously study professional coding. There's too much temptation to hack something in Python or whatever, google as you go, etc. What I did was sit down and practically memorize entire programming manuals. Many academics would refuse to do something so plebeian.
shas3 10y ago
An earnest question I've heard from people trying to switch is: what's a good benchmark for testing where you stand in terms of coding skills? How do you calibrate and measure your production-quality coding skills in Scala, C, etc. when you have spent all your life in academia? I feel Kaggle is sufficient for the exploratory part of data science. But Kaggle's relevance to testing and honing "production quality" code-writing skills is sometimes minimal. In the larger context of programming and software engineering, scientific programming is fairly easy to code-up (though they are harder to conceptualize and understand mathematically). The coding part of non-CS/non-CE/non-some-parts-of-EE academia is pretty much mostly scientific programming. Yet, production quality code is seldom just scientific programming.

237 points by heinrichf 10y ago

Look, this is evil. I don't want to hear any complaints about my calling a spade a spade. The Nazis did it: evil. Google does it, and it is evil, and they go out of their way to say WE ARE NOT EVIL and guess what: it is evil. Why is it evil? You have a bunch of people in corporate america who are FORCING an information technology posture on the American people that the American people HATE because it is HORRIBLE and EVIL. Next, these EVIL people go around saying what they are doing is "data science" and start blaming mathematicians. This is absolutely 100% evil. Mathematicians have zero interest in this. Pointing the finger at mathematicians and calling them "data scientists" is intolerable stupidity and the people who do it should expect to get their fingers cut off by mathematicians who don't like to be pointed at by IDIOTS.

ianai 10y ago
Care to elaborate on specific points that you consider evil? There's a lot to discuss and I think you make an interesting claim.
M_Grey 10y ago
...You said "evil" a lot, but what exactly is the evil part? That's a hell of a claim to make, and then only support with hyperbole and bluster.

heinrichf 10y ago

Note: this is the blog of Jordan Ellenberg (https://en.wikipedia.org/wiki/Jordan_Ellenberg https://en.wikipedia.org/wiki/Jordan_Ellenberg).

curiousgal 10y ago
I've just started reading How Not to Be Wrong: The Power of Mathematical Thinking by him. I enjoy these synchronicities.
deleted 10y ago
[deleted]

M_Grey 10y ago

•Can you walk away from a problem when the solution is “good enough,” are you able to switch between tasks or problems with relative ease? Are you OK with simple solutions to problems that could have more complicated solutions, but only with rapidly diminishing returns? Translation: Can you be a good assembly line worker, devoid of pride or curiosity and just do the damned job? If so, great, we can use you! If not, please return to academia. Lovely stuff.

denzil_correa 10y ago
A lot of things in the industry are built via hacks where good enough is good enough. The goal is not to satiate curiosity but to generate business value. This does not equate to lack of pride, just a different goal.
- M_Grey 10y ago
  Yeah, like the IoT. I'm sorry, but we've reached the point where that level of blithe pursuit of a "different goal" is becoming destructive on a broad scale.
hyperbovine 10y ago
A lot of successful academics I know (including mathematicians) operate on similar principles. Not everybody embraces the Gowers/Perelman/Zhang model, nor should they.
laretluval 10y ago
That is not the intended meaning at all. It can be a great point of pride and skill to find a solution that is as good as possible subject to complexity and time constraints that do not exist in academia.
- M_Grey 10y ago
  And how often does that pride last once you see the application that work is put to? At least if you do good math, you can take pride in that.

denzil_correa 10y ago

A lot of this can also generally apply to a Computer Science Ph.D graduates who would like to understand what working in the industry entails.

11thEarlOfMar 10y ago

I've been encouraging my daughter, a statistics major, to pursue data science by including Python/R in her studies and then possibly heading back for an MBA. But not sure if an MBA would be a benefit. Thoughts from actual data scientists?

ecesena 10y ago
Personally I think a PhD would be better than an MBA if she really wants to do data science, but generally speaking I'm for as more edu as possible, and I'm sure she'll benefit from an MBA too.
- laughfactory 10y ago
  The PhD will only be useful in a small (but growing) subset of data science jobs. Their are data scientists who develop new algorithms and techniques, and those who apply the. For application, the PhD is probably overkill and extensive experience with a bachelor's, or a master's is better. For theory the PhD can't be beat, of course. I say the opportunity costs are high, and should be carefully weighed, because I have my bachelor's and am making more than a data scientist friend of mine who has his PhD. The vast majority of what he learned in the PhD program he hasn't found useful in the real world (yet, anyways). So it's all about what problems you want to work on. Personally I thrive in the application arena of data science, while others thrive in the R&D data science arena. If you like one you probably won't like the other.
  - tgokh 10y ago
    It may not necessarily be the best option for the OP's daughter, but it is possible to do a PhD that uses data science as a tool rather than the focus. There's a lot of interesting work going on in academia using data science in a variety of fields (like medicine) related to predictive analytics (predicting hospital readmission, predicting therapy outcomes), time series analysis (analyzing EKGs, and other vital sign data) etc. A masters would likely allow you to work on similar projects, but if the goal is a PhD, there are certainly application rather than theory focused research groups across the country

diimdeep 10y ago

>. A big thing about the transition to tech is that you possibly start communicating with people who don’t really know what a vector space is. Be ready to have those conversations. Honestly ask yourself if you’re OK with having those conversations. This is so snobbish. What here is even think about ?

pmiller2 10y ago
It's not snobbish at all. The CS/developer version is "someone who doesn't know what static typing is," or, maybe even "someone who doesn't know what computers are and are not capable of."
cottonseed 10y ago
This is the wrong comparison. Read the second footnote. An intern is a developer in training. The post is talking about mathematicians (who spent a decade or more learning a highly technical field) going to work in an environment where, largely, their peers, boss and consumers of their work do not know, have no interest in and and will never learn the most basic concepts of their field.
- pavlov 10y ago
  I suspect the majority of programmers work in that kind of environments...? The startups and Googles of this world are the exception when it comes to employer tech-savviness.
  - Stasis5001 10y ago
    When you get a PhD, you mainly only take classes in your department for the whole time, whereas in undergrad you still have classes in other fields. This usually extends beyond coworkers into friend circles too. This isn't math specific, but more generally to grad school. Second, programmers at companies almost never work in isolation from other programmers, and in most open office environments are close enough they could touch another one from their desk. On the other hand, a startup with five to ten engineers may be hiring you as their first data scientist, and bigger companies may be putting you on an embedded team, with the nearest data scientist a hallway or floor away. And this isn't a big deal in terms of teamwork, but most data scientists don't have a PhD in mathematics, so if you find higher level mathematical ideas and notation to be the most efficient way for you to think about a problem, your colleagues may not. That's also not a math specific thing though -- an economist, statistician, mathematician, computer scientist, physicist, etc. are all going to have slightly different ways they think about things internally.

stared 10y ago

I had two blog posts exactly on that topic, math/phys -> data science: - why: http://p.migdal.pl/2015/12/14/sci-to-data-sci.html http://p.migdal.pl/2015/12/14/sci-to-data-sci.html (on academia vs industry) - how: http://p.migdal.pl/2016/03/15/data-science-intro-for-math-phys-background.html http://p.migdal.pl/2016/03/15/data-science-intro-for-math-ph... (also got reprinted at KDnuggets)

heinrichf 10y ago
These posts were insightful, thank you for writing them.
kurmouk 10y ago
Thanks for sharing this. Especially for someone who's thinking about getting a data scientist job.

ykler 10y ago

I was a PhD student in mathematical logic, and I know a couple people who got PhDs in this area and became data scientists. One thing about this field (and other areas of math) is that almost everyone in it feels mediocre. It's like 10% of the people are 10x better than the other 90%, 1% are 10x better than the next 9%, .1% are 10x better than the next .9%, etc. The top two people in the field seem to be notably better than everyone else. Another thing is that the problems are incredibly arcane. They have no connection to practical concerns or even often other parts of math, and also you can't really explain what you are working on to someone who works in a different area of mathematical logic (unless maybe the person is extremely good). So you have a lot of people getting PhDs who are very, very smart by any normal measure but who would be doing mediocre and arcane work in mathematical logic. Part of the appeal of working in data science or working for the NSA or something is that it is somewhat down-to-earth and people will think you are really smart even if you are at the bottom of the top .01%. (And you can do really great work at this level -- I don't mean to say it is all about caring what other people think.) It is a little weird that the blog post is so negative in a way.

george_ciobanu 10y ago
A lot of these "arcane" topics have long-term horizons. Agree with everything else but the whole point of research is to probe the space of possibilities (theoretical and practical) to find major ideas. Most of them fail but that's part of the exploratory process - it's expected.
- glial 10y ago
  It's expected, but as someone doing research it's super discouraging. I can certainly see the appeal of moving away of basic research with a very long term time horizon (fine for science, but frustrating for individuals) and towards a field with the potential of more immediate impact.
deleted 10y ago
[deleted]
rcar 10y ago

uihsdfjkh 10y ago

What about the other way around? I'm from an engineering background, going in the direction of data scientist. Sometimes I find that my math skills could be stronger, and I try to read up on things when I encounter them, but still it sometimes feels like there is an infinite amount to learn. Maybe I could use some more systematic approach to it. Anyone else who has walked this path, and could come with some useful advice/resources?

tmccrmck 10y ago
Are you talking about self-studying mathematics or going back into higher education? I can give recommendations on books for the former but not the latter.
- bsg75 10y ago
  Self study interest here.
- praveenster 10y ago
  Yes, I am also interested in going the self study route with MOOCs but there is way too much information and resources that it is hard to figure out how/where to start. It would definitely be helpful if you could provide recommendations on books.
digitalzombie 10y ago
I got a bachelor in CS and went industry for 6 years. I'm coming back for master in applied statistic for data science. Just take statistic, especially Multivariate for big data (big data for statisticians is huge predictors not petabyte of observations). Math people can do so much in data science. I think statistic is better for a non math person and it's much better suited for data. Since statistic is all about data.
sdenton4 10y ago
Stats and linear algebra are what you want... On the linear algebra front, you get some understanding from your first course, but the more you internalize it by meeting the ideas in different contexts, the more useful it will be. A decent amount of higher math is turning things into almost-linear problems, and then trying to sort out the parts which don't quite fit...

olivierva 10y ago

Slightly off topic: They still can't build proper software. Academics (including mathematicians) are notoriously bad at writhing production grade software. This leads to handovers of 'proof of concepts' to seasoned software developer team who than struggle with the (often complex) mathematics/science behind it. Imho universities should give a bit more attention on how to write quality software; a bit of test driven development and continuous integration is not that hard and would massively improve the quality of the software written by scientists.

pryelluw 10y ago
Ive been involved in projects where this was an issue. The main problem was that operations were not abstracted away from the types or boilerplate.
JustSomeNobody 10y ago
As a developer, I assure you that even some developers can't write production quality software. Especially some enterprise developers. They either over engineer and never ship or under engineer and ship crap.
contravariant 10y ago
Then again, those are things that can easily be thought. The harder parts of programming such as choosing the right abstraction, or coming up with the right approach for solving a problem, or even formulating a problem, are more of mathematical nature.
emmanuel_1234 10y ago
This is precisely why I dread hiring academic-only profile. Note: I am myself from academia, but I was lucky enough to specialized in CS-related field (NLP). When I wanted to leave academia, Data Science was not a thing where I live and I had to start again at a junior dev position (i.e.: it was that, starving or staying in academia). Now, I work with a lot of people way smarter than I am, who are mostly useless because they can hardly prototype their stuff in Python or run an SQL query. And they'd expect to only work on the best, cleaned and formatted dataset and only do high-end maths on those. Reality hits hard, we're losing money paying them and they're wasting their time not doing what they like. Add to that the frustration / jealousy that this creates. In that regard, I like that famous definition for Data Scientist: "A programmer that know more about statistics than most programmers, or a statistician that knows more about programming than most statisticians". And don't get me started on the general repulsion for understanding the basics of how a business runs from academia. Data Science is all about application.

nerdherfer 10y ago

One side of this that bugs the crap out of me: I have a PhD in math and, after getting sick of teaching mediocre students with a palpable aversion to mathematics, I spent a couple of years applying for data science jobs and getting very few responses. The few interviews I got seemed to go well, but I had no offers and gave up on leaving academics. I have experience in software development (C++), I have a portfolio of statistics/data analysis projects in R/RStudio, but what I think is toxic is having spent so many years in academics; I finished my Phd in 2003. Being an over-40 academic feels like an insurmountable liability, even though I have many of the skills that people complain about being in short supply. And another thing: get off my lawn, you rowdy kids! :|

deleted 10y ago
[deleted]
ordinaryperson 10y ago
It's never too late to switch. I'd suggest figuring out which niche of data science most aligns with your personal interests and start there. Hiring managers don't discriminate based on "too many years in academia", it's more what's the reason they should hire YOU over candidate y? Diehard animal lover? I bet the Sierra Club would love a statistician who could help them quantify how many elephants are being poached in Africa per year. Interested in sports? Strava gets millions of data points a day, I'm sure they're looking for help finding signals in the noise. Can't get enough of financial markets? Finance jobs all over the place for quants. If your profile is too generic they'll see no good reason to hire you, but if you're passionate about X it can really work in your favor. The job search is depressing but never give up hope.
- deleted 10y ago
  [deleted]
- theparanoid 10y ago
  As an over 40yr old, yes it's probably too late to switch. After 30 it was a shock to me getting turned down for jobs when I aced the technical interviews.

wolfkill 10y ago

This post mostly describes me. I am a PhD student in applied mathematics and returned school to get my PhD precisely because I wanted to get a job outside of academics, most likely as a data scientist or scientific programmer. 2.5 years into my program I still struggle to see beyond the degree but am currently looking at non-academic internship opportunities as a way to start to untangle myself from academics. I also spend a significant amount of my time trying to teach my self good programming practices that I don't get from mathematics (how to write clean, adaptable code, choosing appropriate design patterns and writing generic code without over-abstracting, writing readable documentation and using version control, etc.) To those of you who have left academics do you have any other suggestions? How about for marketing myself as a programming mathematician when competing against CS and engineering students for internships/jobs?

IndianAstronaut 10y ago
Take a look into data engineering. A lot of data engineering jobs require you to implement mathematical models at scale.

mrcactu5 10y ago

I remember asking why i was rejected from Palantir and vaguely recall the phrase "too mathematical". After years of trying I have been unable to make the transition from Mathematics into Data Science and have since shifted my attention to other things.

bayesian_horse 10y ago

I guess it is easier to get into data science as a math graduate. I'm trying without a degree, and haven't succeeded yet.

uptownfunk 10y ago

I interview probably two to three people a week for a DS job. It's a really difficult role to hire for. I'm basically looking for mathematicians with some knowledge of stats who also enjoy coding and follow best practices in coding (i.e. They didn't just pick it up and hack something that works but care enough to document and structure so others can follow it at a minimum) but we also need the MBA component as well, a large part of this industry is how do you take what a DS does and deploy it in a business and ensure the business derives value from it. But I think coming at it from a mathematical background puts you in a much stronger footing than the cs guys who try to learn the math later. I rarely find strong candidates without a formal mathematical background.

kriro 10y ago
Interesting, thanks. It obviously depends a lot on the job but I would have thought that a good math background (which typically CS grads should have) would be enough for many applications. Especially since you mention the business problem aspect. I think that you can actually get quite far in DS (at least in deep learning) with rather "basic math". Unless you're innovating at the bleeding edge it's usually enough to reproduce existing ideas and applying them to interesting problems is the real key. So my hiring would have focused on CS grads with a natural interest in math, ideally very rigorous/meticulous.
- laichzeit0 10y ago
  It makes me almost feel sad for the people that followed the "why bother doing a CS degree, you can learn to code without one" trend that probably peaked and is now maybe beginning to die a slow death as the market is flooded with people who can write code. If they ever want to transition into DS they're going to have to skill up and do the equivalent of undergrad CS math curriculum (discrete, calculus, linear algebra, math stats, etc.), which let's be honest, you cannot pick this up in any meaningful way in a "few months" of after hours/weekend study like you can when learning to program or learning some new "framework"; either that or just remain another dime-a-dozen code-cutter. It's sad because if you just did the 4 years of computer science and all the math and the stats that go with it you'd be so close to pivoting right now if you wanted to.

godmodus 10y ago

A few fellow students graduated with a BA and had their aplications rejected by most companies due to a lack of experience. Theory is nice. But Experience trumps all. Esp when you think it mostly boils down to PCA/DBSCAN, regression and scatter matrices. A month of Python/R/D3 on some initial data like pubmed/twitter or any toy dataset would go a looong way.

kriro 10y ago
I think doing well on Kaggle competitions would already be a good enough indicator for smart companies so it might be a good idea to have student teams do those on the side. Might be a decent idea to inform students that these exist. I feel like this applies even more for other degrees like econ(ometrics) (since I feel like a good math grad should not have trouble getting any data science job).

happy-go-lucky 10y ago

There is positive correlation between applied math and programming. I see expertise in these areas coupled with a knack of getting a point across as a potential career opportunity. No wonder academics in some parts of Asia have, for generations, seen math as “evergreen.”

happy-go-lucky 10y ago

If you see programming as a tool for solving domain-specific problems, then domain experts who can exploit the tool obviously have an edge over programmers who rely on inputs from these experts when developing domain-specific applications.

visarga 10y ago

> Do you like modeling complex problems mathematically? Do you enjoy kicking the ball towards the gate and scoring? Then you might be good for Football.

known 10y ago

I think MBA adds more value to your career in globalization

xamuel 10y ago

As a mathematician who successfully made the leap, I think the biggest requirement is ability to swallow your pride and "lower" yourself to seriously study professional coding. There's too much temptation to hack something in Python or whatever, google as you go, etc. What I did was sit down and practically memorize entire programming manuals. Many academics would refuse to do something so plebeian.

shas3 10y ago

An earnest question I've heard from people trying to switch is: what's a good benchmark for testing where you stand in terms of coding skills? How do you calibrate and measure your production-quality coding skills in Scala, C, etc. when you have spent all your life in academia? I feel Kaggle is sufficient for the exploratory part of data science. But Kaggle's relevance to testing and honing "production quality" code-writing skills is sometimes minimal. In the larger context of programming and software engineering, scientific programming is fairly easy to code-up (though they are harder to conceptualize and understand mathematically). The coding part of non-CS/non-CE/non-some-parts-of-EE academia is pretty much mostly scientific programming. Yet, production quality code is seldom just scientific programming.