2 ms·

Totally realistic. It'll depend on what the company is, and what they're looking for (and hopefully, this will match what they should be looking for). While pe

by IanCal 10y ago

Totally realistic. It'll depend on what the company is, and what they're looking for (and hopefully, this will match what they should be looking for). While people can often focus on applying the latest deep learning thought-vector approach to their BIG DATA, there's an enormous gulf between the common condition of data and this aspiration. You don't need PhD level stats and machine learning to apply the things that many companies actually could benefit from. Storing, maintaining and managing the data properly is a start. Then working with the people in the business to get insights from the data they have. Often simple aggregations and visualisations can provide enormous benefit. Being able to show correlations, and sometimes even just being able to show how noisy things are can be important. Big questions: How is reality different from what we think? How are these differences important for the company? That might be a correlation that we don't expect, or a lack of one we think is there. Part of a next step might be to become more "pro-active" and design new experiments to answer questions that can't quite be answered yet due to a lack of data. Beyond that you're heading towards bringing a new feature into a product. Does your current company have some data? Do they spend a lot of time emailing spreadsheets around, or have salesforce or a proper database? See if you can make something useful for your work that's based on an analysis of that data (and possibly do some work to smooth those workflows of passing data around).

mswen 10y ago
IanCal - I upvoted you but let me comment for extra emphasis. I remember coming out of PhD level quantitative social science studies in academia back in the late 90s where I was using K-means clustering, Factor analysis, multiple linear regression, ANOVA and more. When I moved into marketing research and dealing with company data it was shocking and disheartening how little of those skills I could actually deploy. Data quality, data management and just the cost of capturing relevant data was so high that we were reduced to much simpler analyses. Over time I came to appreciate exactly what you were saying. Many companies can be helped by fairly simple analyses. Fast forward - data is getting much cheaper and now all these years later my more advanced stats skills seem to actually matter. But even in this apparent abundance ... data management and data quality assurance is often lacking or significantly underfunded and simple aggregations and analyses still make the most difference in many organizations.
- Declanomous 10y ago
  I went into marketing from a biology background, and I felt the transition was very easy. Collecting data in biology, especially on an ecosystem scale, is extremely difficult, and your data has a lot of noise in it. How the data is collected and processed is almost more important than how the data was analyzed. Collecting and processing data is going to be extremely time consuming and expensive regardless of what you do, but you can make it easier on yourself through smart experimental design. P-values are still widely used for analysis in biology, to the chagrin of most statisticians. I don't think it is as much of a problem as it comes across as though. Bayesian reasoning is inherently included in the scientific process as part of the experimental design stage. The reliance on p-values becomes a problem when journalists report on research findings in a single paper that is "significant" because of p-values. The professionals in the field are capable of balanced analysis of p-value based research, but we end up with issues like anti-vaxxers when research is presented outside the field of professionals. Of course there are still ways the process can allow incorrect research to present itself. For one thing, journals are not interested in publishing studies that do not have "significant" results, which means that bad research can stick around for far too long. I personally think that the solution to these problems is open data, since so much of the research depends on how the data was manipulated and 'cleaned' prior to analysis.

by IanCal 10y ago

mswen 10y ago

IanCal - I upvoted you but let me comment for extra emphasis. I remember coming out of PhD level quantitative social science studies in academia back in the late 90s where I was using K-means clustering, Factor analysis, multiple linear regression, ANOVA and more. When I moved into marketing research and dealing with company data it was shocking and disheartening how little of those skills I could actually deploy. Data quality, data management and just the cost of capturing relevant data was so high that we were reduced to much simpler analyses. Over time I came to appreciate exactly what you were saying. Many companies can be helped by fairly simple analyses. Fast forward - data is getting much cheaper and now all these years later my more advanced stats skills seem to actually matter. But even in this apparent abundance ... data management and data quality assurance is often lacking or significantly underfunded and simple aggregations and analyses still make the most difference in many organizations.

Declanomous 10y ago
I went into marketing from a biology background, and I felt the transition was very easy. Collecting data in biology, especially on an ecosystem scale, is extremely difficult, and your data has a lot of noise in it. How the data is collected and processed is almost more important than how the data was analyzed. Collecting and processing data is going to be extremely time consuming and expensive regardless of what you do, but you can make it easier on yourself through smart experimental design. P-values are still widely used for analysis in biology, to the chagrin of most statisticians. I don't think it is as much of a problem as it comes across as though. Bayesian reasoning is inherently included in the scientific process as part of the experimental design stage. The reliance on p-values becomes a problem when journalists report on research findings in a single paper that is "significant" because of p-values. The professionals in the field are capable of balanced analysis of p-value based research, but we end up with issues like anti-vaxxers when research is presented outside the field of professionals. Of course there are still ways the process can allow incorrect research to present itself. For one thing, journals are not interested in publishing studies that do not have "significant" results, which means that bad research can stick around for far too long. I personally think that the solution to these problems is open data, since so much of the research depends on how the data was manipulated and 'cleaned' prior to analysis.