3 ms·
The trick is to realize the act of verifying the accuracy of the data "is" the science. I come from High energy physics background. People think the point is to
by euix 7y ago
The trick is to realize the act of verifying the accuracy of the data "is" the science. I come from High energy physics background. People think the point is to find sexy new particles or make ground breaking discoveries. No the real goal is to understand the inner workings of every part of your data acquisition pipeline (from the hardware detector to the software analysis) so well that you eliminate all possible other explanations when you spot a deviation. That then is your discovery.
I was a DS for a couple years in NYC area. My advise is the same as in science, treat the act of verification as the end goal itself. If you understand your experimental apparatus, in this case, how the DB is populated then you understand all subsequent of the "physics". Anything you can't understand should be carefully quantified as random and systematic errors.
Sorry I don't have any magic bullet or standard recipe for how to go about it. Every case is different.
- lost_sqler 7y agoThat does help, thank you. I will look into making the verification a core part of the process.