While I wish there were good principled methods for determining these choices, isn't a meaningful limitation that often when you first start writing the data, y

While I wish there were good principled methods for determining these choices, isn't a meaningful limitation that often when you first start writing the data, you typically don't know all of the ways it will eventually be used? Sometimes, the team tasked with making data available for various forms of bulk processing has to guess at what teams and mandates might exist years from when they are setting up the data lake, or even what compliance requirements will arise. E.g. as different areas introduce data protection laws, perhaps your datalake which could previously assume that records are written but not deleted has to support queries to find PII values associated with a person who has issued a delete request, and you're forced to rewrite lots of files. Or a downstream team starts using vector DBs and wants to assemble datasets by doing queries for records which match any of 100k IDs, which have associated vectors in regions of interest, and you need something that's not a single scan but also not a large number of point lookups. Etc, etc. How do you have a principled method for optimizing for unknown future use cases? Does it make sense to talk about a probability distribution over future queries?