4 ms·
Since you already know R/STATA/Linux/SQL, you're already way ahead of most people. You'll find that a lot of the specific technical stuff is project/industry de
by chaps 5y ago
Since you already know R/STATA/Linux/SQL, you're already way ahead of most people. You'll find that a lot of the specific technical stuff is project/industry dependent and since you're just starting in the field, think of any hard requirements for a specific technology as a warning sign. Good employers will give you room to learn their niche technicals.
The thing about data sciencey stuff is that the data you're working with will often be extraordinarily messy. It's not uncommon to realize two weeks into a project that you made a terrible cleanup assumption on day one... then have to run hours and hours of mostly-hand-executed ETL work again. And then again when you realize you forgot that you ran a unix one-liner on a random input file, but didn't write it down anywhere.
So, one of the most important things to learn off the bat is learning how to clean data programmatically, specifically with the goal of making sure that your cleanup is repeatable at any step of the way. You want to be able to get to a point where you feel confident that you can mostly trivially recover after deleting all cache/temp files, tables, etc. Makefiles are great for this.
You'll save a lot of time in the long run if you can get good at this.
- mateo411 5y agoSo, basically data engineering. If you want to be good at data science you should at least be a mediocre data engineer.