I guess I just see the level of "scientific soundness" as a spectrum. For some work (peer reviewed journals, work you hope other people to be able to reproduce

I guess I just see the level of "scientific soundness" as a spectrum. For some work (peer reviewed journals, work you hope other people to be able to reproduce, etc.) you need a higher standard. I find that higher standard requires a significant amount more time, effort, and ultimately prevents me from accomplishing what I need to. It depends on the application and how you're using the result. Most of the time, for me, it just doesn't matter. It may be worth noting that much of what i'm talking about is for personal understanding (e.g., baseball statistics) or is not mission critical and will never see the light of day. Sometimes good enough is just that. I do appreciate the thoughtful response you wrote. But often my decision is this: do I spend 2 weeks trying to understand a technique, or do I spend 1 day using it, understanding that my analysis has limitations. When you only have 1 day, my choice becomes "do what seems like it will work" or "do nothing." I agree that more complex techniques (neural nets, etc.) may need more understanding. But because they do, they are also inaccessible to me. Maybe I can sum it up like this: it's not always necessary to do an analyses in a completely scientifically sound manner, if I can answer the questions to my satisfaction. Especially relevant when the other option is to not get anything done, and get stuck in a textbook trying to understand complex theory. It's bit me once in a while, but that doesn't matter in what I'm doing. (And its usually pretty easy to recover from.) Also worth a mention that I would not call myself a data scientist, nor do I operate in that role. It's all tradeoffs, all the way down. I just happen to choose mine differently than someone who might call themselves a data scientist or statistician.