4 ms·

I'm a data scientist with a stats background. I'm also a self-taught programmer (going on 15 yrs of programming in various languages). There are very few data s

by nrjames 10y ago

I'm a data scientist with a stats background. I'm also a self-taught programmer (going on 15 yrs of programming in various languages). There are very few data science jobs that I am aware of where somebody will hand you a tidy data set and ask you to start creating models. For those jobs, it's great to go straight from stats/math to data science, as long as you learn a bit of R and some SAS along the way. Most data science jobs that I know of involve a lot of data engineering. That is to say, you need to understand where the data originates, how to get it through an API or via other scripts, how to design and implement a reasonable (often relational) model of it and put it into a database, how to clean it, how to query it into nice and clean output files, and then how to model it. You also need to know how to visualize it effectively and use it to tell a business story. I would not be able to do my job at all if I didn't know Python, R, and JavaScript well (and know my way around various Linux flavors). The modeling is fun, but it comes at the end of a long pipeline requiring a lot of skills that are more engineering oriented. Rarely do my tasks sound like "model this weekly and give me the result." Often, they sound like, "I need you to pull together data from these 5 sources, model it, and produce a weekly report showing these derived KPIs. And I need to be able to access it in a web browser so that I can send links to colleagues. And it needs to be secure. And generating a report across an arbitrary date range needs to take less than a minute." By the way, that is a request that I've gotten at three different jobs. To give you a sense of what this looks like: most recently, I wrote Python scripts to harvest and ETL the data into a Postgres database (running on Google Cloud) and a BigQuery table, then wrote a Flask app to accept the arbitrary report requests and query the database, run models, store computed results in a local SQLite database for fast future retrieval... finally producing dynamic Reveal.js slide decks available through the Flask app. That's a long-winded way of saying that I strongly suggest that your daughter get some practical experience with the data engineering side of data science, preferably using Python for fetching, manipulating, storing, cleaning, and preparing data. It's the most flexible tool for the job and it easily the most important tool in my data science toolkit.

nrjames 10y ago
Replying to myself here just to throw out an idea for an easy project that is representative of the type of work a data scientist might be expected to do. Register as a Twitter dev (free) and get the tokens, etc. needed to access the public API. Pick a topic of interest -- hockey, for example, or whatever floats your boat -- and write scripts to harvest all tweets coming off of the public API related to the topic. Design a relational schema for the tweets and push them into a SQLite database. I suggest SQLite because it's ubiquitous and has a low barrier to entry. Something like MongoDB also works well for dumping everything straight off of the API. You can then have another script pull out of MongoDB and push into SQLite, for example. Once you're collecting tweets, storing them, etc... try some unsupervised clustering; k-means, for example, or a decision tree. If you feel up to it, go through 1000 or so of the tweets manually and label them according to some target variable of interest to the project. Then, use that labeled data set to run supervised models. Maybe start with a binary target variable and run a simple logistic regression. Then, visualize the data. There are a lot of ways to go about this, but I suggest trying to use something JavaScript based, such as D3 or p5.js, since it allows you to create interactive web-based visuals. Create a public GitHub repo and push work to it as you progress through the project. When done, use GitHub pages to put a summary of the project online. Twitter data is great because there are tons of variables. It's horrifying because it's like reading the refuse of language, littered with abbreviations, emoji, and other weird characters. However, it's the terrible part of it that makes it great for learning. There are other similar public APIs that would accommodate similar projects. Having a few self-initiated projects similar to this under your belt will really help you when it comes time to apply to graduate school or to jobs. If nothing else, it will give you something to talk about in interviews.
11thEarlOfMar 10y ago
Thank you! As mentioned separately, I suggested a data science boot camp after getting the Stats BS, work a couple of years as a data scientist, then go back for the MBA. My thinking RE the MBA was to give a sense of the business value of the analyses she's performing, and, to make it easier to promote her to executive positions. Maybe that's old school thinking, I know that the MBA in general gets a mixed reception these days, but we can look more closely at it after she's out of school.
- nrjames 10y ago
  To me, an MBA is more about business strategy. Knowing the relevance of data to the operations of a business is vital to being an effective data scientist. Knowing when to make the call to lay off 500 people is the job of somebody with an MBA. Data scientists prepare and present the evidence in the language of business. There is crossover, of course. What many data scientists (myself included) find is that they often are excluded from meetings and conversations that provide the context for the analysis they are doing. I always tell my supervisors that it is very helpful for me to sit in on as many business strategy meetings as possible, just to listen, because it builds context around the work that I do and helps keep me properly focused. On the flip side, many involved in business strategy do not understand the nuances of analysis performed to support their business questions. There can be many reasons for that, from not being directly involved in the analysis to being excluded from data science meetings. Effective business analysis and data science requires trust between the players and that trust is built through showing an ability to deliver focused, relevant, and accurate results that support decision making. As somebody who likes to write code and dislikes sitting in tons of meetings, I tend to avoid climbing the career ladder to management positions. I'm happy in a Senior Data Scientist position. At my last job, I was being groomed for management and I never got to do any actual data science work. It was boring as hell. I made a lateral move to a different company so that I could be more hands on and work in an industry that is more fun. Now, I get to write code and build and run models every day. I also get to present the results and have a trusting relationship with the managers. As your daughter finishes school and gets some job experience, don't be too quick to suggest routes leading to business strategy and management. While it's the "top of the career ladder" at many companies, so to speak, it's not for everybody.

by nrjames 10y ago

nrjames 10y ago

Replying to myself here just to throw out an idea for an easy project that is representative of the type of work a data scientist might be expected to do. Register as a Twitter dev (free) and get the tokens, etc. needed to access the public API. Pick a topic of interest -- hockey, for example, or whatever floats your boat -- and write scripts to harvest all tweets coming off of the public API related to the topic. Design a relational schema for the tweets and push them into a SQLite database. I suggest SQLite because it's ubiquitous and has a low barrier to entry. Something like MongoDB also works well for dumping everything straight off of the API. You can then have another script pull out of MongoDB and push into SQLite, for example. Once you're collecting tweets, storing them, etc... try some unsupervised clustering; k-means, for example, or a decision tree. If you feel up to it, go through 1000 or so of the tweets manually and label them according to some target variable of interest to the project. Then, use that labeled data set to run supervised models. Maybe start with a binary target variable and run a simple logistic regression. Then, visualize the data. There are a lot of ways to go about this, but I suggest trying to use something JavaScript based, such as D3 or p5.js, since it allows you to create interactive web-based visuals. Create a public GitHub repo and push work to it as you progress through the project. When done, use GitHub pages to put a summary of the project online. Twitter data is great because there are tons of variables. It's horrifying because it's like reading the refuse of language, littered with abbreviations, emoji, and other weird characters. However, it's the terrible part of it that makes it great for learning. There are other similar public APIs that would accommodate similar projects. Having a few self-initiated projects similar to this under your belt will really help you when it comes time to apply to graduate school or to jobs. If nothing else, it will give you something to talk about in interviews.

11thEarlOfMar 10y ago

Thank you! As mentioned separately, I suggested a data science boot camp after getting the Stats BS, work a couple of years as a data scientist, then go back for the MBA. My thinking RE the MBA was to give a sense of the business value of the analyses she's performing, and, to make it easier to promote her to executive positions. Maybe that's old school thinking, I know that the MBA in general gets a mixed reception these days, but we can look more closely at it after she's out of school.

nrjames 10y ago
To me, an MBA is more about business strategy. Knowing the relevance of data to the operations of a business is vital to being an effective data scientist. Knowing when to make the call to lay off 500 people is the job of somebody with an MBA. Data scientists prepare and present the evidence in the language of business. There is crossover, of course. What many data scientists (myself included) find is that they often are excluded from meetings and conversations that provide the context for the analysis they are doing. I always tell my supervisors that it is very helpful for me to sit in on as many business strategy meetings as possible, just to listen, because it builds context around the work that I do and helps keep me properly focused. On the flip side, many involved in business strategy do not understand the nuances of analysis performed to support their business questions. There can be many reasons for that, from not being directly involved in the analysis to being excluded from data science meetings. Effective business analysis and data science requires trust between the players and that trust is built through showing an ability to deliver focused, relevant, and accurate results that support decision making. As somebody who likes to write code and dislikes sitting in tons of meetings, I tend to avoid climbing the career ladder to management positions. I'm happy in a Senior Data Scientist position. At my last job, I was being groomed for management and I never got to do any actual data science work. It was boring as hell. I made a lateral move to a different company so that I could be more hands on and work in an industry that is more fun. Now, I get to write code and build and run models every day. I also get to present the results and have a trusting relationship with the managers. As your daughter finishes school and gets some job experience, don't be too quick to suggest routes leading to business strategy and management. While it's the "top of the career ladder" at many companies, so to speak, it's not for everybody.