Along the same line, I find the hardest thing to get good at for junior data-sciency people is how to tell if your model sucks. Too easy to trick yourself into

Along the same line, I find the hardest thing to get good at for junior data-sciency people is how to tell if your model sucks. Too easy to trick yourself into thinking you have a good model. I think this is an essential, foundational skill for modern data science, not unlike some knowledge of R or Python, but an order of magnitude or two harder to learn. A lot of the difficulty is in lack of good educational material. I also think this is significantly more important than a deep understanding of various modeling approaches; you can learn that as you go, so long as you know if what you have now is "good enough" or if you need a fancier approach. Unfortunately you can't just throw k-fold CV around, average some fit metrics, and call it a day.