This seems to be a growing issue in the industry as we go more and more data driven. I don't think most devs seem to realise that just throwing data into a dat

This seems to be a growing issue in the industry as we go more and more data driven. I don't think most devs seem to realise that just throwing data into a data store, without constraints, integrity checks of any of that other boring stuff, actually makes the data useless. Try doing analytics against data that can be anything. You can't. Mixing numbers with strings, duplicating fields or having them under varying naming strategies, all just mean you have lots of data that you cannot use. One of the pet peeves I see data scientists having is lots of data, with no organisation, as they can't use it. If you want to collect data now that's fine. Collect it in it's original format and label what that format is and where it came from. Then make a proper data store with a proper schema and work on importing the data from those sourcs when you have this foundation sorted out.