3 ms·

Thank you for this comment. I joined a non-tech company as their first data scientist, and am now tasked with building up the data science efforts of the compa

Thank you for this comment. I joined a non-tech company as their first data scientist, and am now tasked with building up the data science efforts of the company. Your comment is really ringing true, though I've found the company here to be generally great to work with in terms of their flexibility. Funny enough, it took me months of working with the IT department to get a VM with 8-cores and 64 GB of RAM, or the ability to spin up AWS instances (none of the analysts at the company had ever asked for that). I'm happy with it though, it's a fascinating challenge and I'll admit at times I feel a bit in over my head. In my experience, many companies are wanting to hire a lot more data scientists (and paying them six-figure salaries to boot), when what they really are looking for are what has traditionally been called a "data analyst" position. I.E., the entire sum of the job is Excel reporting and basic statistics, but instead wrapped in some "Machine Learning" marketing language. As opposed to more long-term focus on what business problems can be solved by carefully cleaning data, building models, testing predictive power, and then deploying these models into production. Which obviously takes a lot more time, a lot more effort focusing on a single problem, and doesn't have nearly as many immediate deliverables that higher-ups can open their Outlook inbox and see. Naturally they then wonder why they are paying such a premium for someone who doesn't bring as many deliverables to the table as their 60-70k analysts. And then the data scientist wonders why he's spending most of his time sending Excel attachments. Definitely more of a problem in non-tech companies.

kthejoker2 9y ago
I work for a large consulting firm that does both of these activities (data engineer/analyst delivering rapid value + long-term data scientist / architects engineering fully integrated solutions connecting ML to business process) and I can say without a doubt we get much higher feedback from the the former than the latter. Most businesses think small. I'm not even sure it's about risk/reward, either. I think people really don't have vision especially when it comes to technology and more specifically data. The next generation is coming, though.
- apohn 9y ago
  IMO every Data Scientist needs to be able to quickly identify a win and deliver quick value, even in a longer term ML project. Iterate and get something out. Most people in large businesses cannot do any data ETL or build a dashboard, so if you can do that in the first few weeks of a project you've already got a deliverable/win. Having some expertise with BI applications (e.g. Tableau) has been a major strength for showing value.
apohn 9y ago
In my experience, many companies are wanting to hire a lot more data scientists (and paying them six-figure salaries to boot), when what they really are looking for are what has traditionally been called a "data analyst" position. I.E., the entire sum of the job is Excel reporting and basic statistics, but instead wrapped in some "Machine Learning" marketing language. 100% Agree. Some of the execs I reported into were obsessed with the idea of being a startup within a large organization, and part of this was the assumption that one of the things startups did was use data better than big companies. To them, there was some hidden value in their data that their Business Analysts did not have the skills to unlock. Data Scientists are perceived as the magic key to unlock this value, even when what the business is asking for doesn't make sense or isn't feasible in the short-term. One of the small ways we delivered value was to write small ETL scripts that fed into Tableau dashboards. People loved the Dashboards because it cut down on so much Excel stuff that was done weekly/monthly/quarterly, etc. The company did have a large IT org with a BI team, but they were always busy and took months to do anything. So for us to do ETL+Tableau in a few days was seen as a miracle. Plus, Tableau has an Excel export :) Of course, ETL+Tableau wasn't enough to retain the team.

kthejoker2 9y ago

I work for a large consulting firm that does both of these activities (data engineer/analyst delivering rapid value + long-term data scientist / architects engineering fully integrated solutions connecting ML to business process) and I can say without a doubt we get much higher feedback from the the former than the latter. Most businesses think small. I'm not even sure it's about risk/reward, either. I think people really don't have vision especially when it comes to technology and more specifically data. The next generation is coming, though.

apohn 9y ago
IMO every Data Scientist needs to be able to quickly identify a win and deliver quick value, even in a longer term ML project. Iterate and get something out. Most people in large businesses cannot do any data ETL or build a dashboard, so if you can do that in the first few weeks of a project you've already got a deliverable/win. Having some expertise with BI applications (e.g. Tableau) has been a major strength for showing value.

apohn 9y ago

In my experience, many companies are wanting to hire a lot more data scientists (and paying them six-figure salaries to boot), when what they really are looking for are what has traditionally been called a "data analyst" position. I.E., the entire sum of the job is Excel reporting and basic statistics, but instead wrapped in some "Machine Learning" marketing language. 100% Agree. Some of the execs I reported into were obsessed with the idea of being a startup within a large organization, and part of this was the assumption that one of the things startups did was use data better than big companies. To them, there was some hidden value in their data that their Business Analysts did not have the skills to unlock. Data Scientists are perceived as the magic key to unlock this value, even when what the business is asking for doesn't make sense or isn't feasible in the short-term. One of the small ways we delivered value was to write small ETL scripts that fed into Tableau dashboards. People loved the Dashboards because it cut down on so much Excel stuff that was done weekly/monthly/quarterly, etc. The company did have a large IT org with a BI team, but they were always busy and took months to do anything. So for us to do ETL+Tableau in a few days was seen as a miracle. Plus, Tableau has an Excel export :) Of course, ETL+Tableau wasn't enough to retain the team.