3 ms·
How and where data gets scrubbed will have significant consequences. While clean data is more valuable than dirty data (the mass of raw data collected by sensor
by SemiTom 8y ago
How and where data gets scrubbed will have significant consequences. While clean data is more valuable than dirty data (the mass of raw data collected by sensors), there is hidden value in those large masses of data. They can show broad trends and patterns that are not obvious in clean data, which is the modern equivalent of not seeing the forest for the trees https://semiengineering.com/data-vs-physics/ https://semiengineering.com/data-vs-physics/
- jmatthews 8y agoThis may be a bit elementary for this crowd but regarding the balance of data cost vs capturing the most significant features. We use a simple decision tree as a significance cluster and optimize data munging around these clusters. On some levels it is anti-diversity but given real world constraints it has yielded the best results. Any thoughts or links regarding this topic would be appreciated.