3 ms·

You are a data wrangler? Perhaps a guide would do; noobs like me have no perspective on what to learn and how to learn it. I mean, it wasn't even a year ago tha

by iamjeff 10y ago

You are a data wrangler? Perhaps a guide would do; noobs like me have no perspective on what to learn and how to learn it. I mean, it wasn't even a year ago that I was convinced that I could go from 0 to 100 data science-wise in under a year: I wanted to learn it all. It took me the better part of a year to realize that I had wasted innumerable hours devising a curriculum and timelines that were plain dumb. A practical guide could have spared me a lot of hurt and while I cannot at this moment compensate you (or the community for that matter), I am sure that opportunities will certainly arise for me to pay my debt. Would love to see a guide from you- it would come with the added advantage that you would be accessible to the brilliant HN community. I would give away a limb to see such a discussion go down: what to learn and where to learn it from (a lot of folks, I imagine, would not mind recommendations for openly accessible material; I know I wouldn't mind that)? how fast should you expect to go/move/learn? time commitments? tools and frameworks? motivation hacks? where would I go to find remote jobs? what level of proficiency should I achieve in the first sprint?

minimaxir 10y ago
The best way to learn how to wrangle data is practice, especially outside of academic settings, where the example data is not necessarily reflective of real-world data. Helping others wrangle data is one of the reasons I publish my Jupyter notebooks open-sourced. A few examples my data wrangling with R: Processing Stack Overflow Developer data: https://github.com/minimaxir/stack-overflow-survey/blob/master/stack_overflow_dev_survey.ipynb https://github.com/minimaxir/stack-overflow-survey/blob/mast... Identifying related Reddit Subreddits: https://github.com/minimaxir/subreddit-related/blob/master/find_related_subreddits.ipynb https://github.com/minimaxir/subreddit-related/blob/master/f... Determining correlation between genders of lead actors of movies on box office revenue: https://github.com/minimaxir/movie-gender/blob/master/movie_gender.ipynb https://github.com/minimaxir/movie-gender/blob/master/movie_...
- iamjeff 10y ago
  The "Analysis of Lead Gender and Box Office" project put a smile in my face because it looks like so much fun- that is something that I would like to know how to do in the coming months. Thank you for open sourcing the notebooks and the recommendation to practice then demo on real data.
- nl 10y ago
  This. I'd add that Kaggle is very good for the "other end" of data science: they generally have pretty clean data, and clear problem descriptions. In real life the data is never clean and the problems are rarely known in advance.
  - benhamner 10y ago
    We also are growing https://www.kaggle.com/datasets https://www.kaggle.com/datasets, which won't necessarily have clean data, clear problem statements, and a well-defined task.

by iamjeff 10y ago

minimaxir 10y ago

The best way to learn how to wrangle data is practice, especially outside of academic settings, where the example data is not necessarily reflective of real-world data. Helping others wrangle data is one of the reasons I publish my Jupyter notebooks open-sourced. A few examples my data wrangling with R: Processing Stack Overflow Developer data: https://github.com/minimaxir/stack-overflow-survey/blob/master/stack_overflow_dev_survey.ipynb https://github.com/minimaxir/stack-overflow-survey/blob/mast... Identifying related Reddit Subreddits: https://github.com/minimaxir/subreddit-related/blob/master/find_related_subreddits.ipynb https://github.com/minimaxir/subreddit-related/blob/master/f... Determining correlation between genders of lead actors of movies on box office revenue: https://github.com/minimaxir/movie-gender/blob/master/movie_gender.ipynb https://github.com/minimaxir/movie-gender/blob/master/movie_...

iamjeff 10y ago
The "Analysis of Lead Gender and Box Office" project put a smile in my face because it looks like so much fun- that is something that I would like to know how to do in the coming months. Thank you for open sourcing the notebooks and the recommendation to practice then demo on real data.
nl 10y ago
This. I'd add that Kaggle is very good for the "other end" of data science: they generally have pretty clean data, and clear problem descriptions. In real life the data is never clean and the problems are rarely known in advance.
- benhamner 10y ago
  We also are growing https://www.kaggle.com/datasets https://www.kaggle.com/datasets, which won't necessarily have clean data, clear problem statements, and a well-defined task.