8 ms·
enterprise is rapidly approaching a data quality crisis where they have all these data warehouses but the final analytic artifacts end up being garbage and unus
by dustingetz 6y ago
enterprise is rapidly approaching a data quality crisis where they have all these data warehouses but the final analytic artifacts end up being garbage and unusable for data science ... you will be hearing a lot more of this in the 2020s
- kqr 6y agoI'd argue that is a completely orthogonal problem. Business have extracted useful metrics out of their "eventually consistent" operations ever since operational research was invented. That companies have collected more data than they can pay for processing of is a separate issue, I think.
- majormajor 6y agoA lot of this isn't related to data processing tools at all, but is a sort of downstream affect of the predominant "bugs are cheap" mentality of today. The less guarantees of correctness on your daily/weekly/whatever releases, the messier your downstream data is gonna be. Monday's data is partially missing due to a bug in the client; Tuesday's data is weird/nonrepresentative because of a server bug that caused 5% of sessions to get disconnected; Wednesday's data is good; Thursday's data is good but was a release day and the feature changed so it means different stuff...
- delusional 6y agoI don't think that has as much to do with eventual consistency as with the old school system design of "the UI is a database editor, here are your plaintext fields" that still permeates a lot of businesses today.