> Even though it is expensive/slow for a consumer to figure out how to use data, it is even more expensive/slow to have producers try to continuously anticipate

> Even though it is expensive/slow for a consumer to figure out how to use data, it is even more expensive/slow to have producers try to continuously anticipate the needs of all possible current and future consumers, rather than focus on their system and have actual consumers figure out how to use the subset of data they need as it changes. I agree completely on the speed impact but I would go further and say this is basically impossible. I have worked on data everywhere from small to massive enterprises and the only model I have ever see work at a large enterprise is for source systems to just produce data in a raw-ish format that comes out of their system and leave it to the people who need to consume that data to figure out how to massage it into whatever format they need. The reasons I think it's not possible for upstream to anticipate are: 1) Analytic usecases multiply as systems do and as users start consuming data. As soon as you start using data to address problems in the org, you see more and more questions you would like to use data to address. There is no way for a producer system to forsee all the possible analytic questions people would like to answer using their data, especially as data analysis is a creative endeavour and peoples' imagination is pretty amazing (especially taken collectively). 2) As data sources multiply, the analytical possibilities of combining these data sources explode quadratically. There is no way for any single upstream system to anticipate how this will go. What upstream systems need to do is produce all the data they can from their system and let users do what they do when combining. You can make some sensible guidelines available (eg "document what the columns mean", "keep up to date estimates of volumes to expect", "have a data sample/staging system/test instance for people to integrate against", "put your goddam timestamps into UTC wherever possible" etc) that aren't too onerous for upstream maintainers to follow if they want to be good citizens.