3 ms·
What happens when a consumer does not understand the producer’s domain and misuses the data? As we load metadata with semantics of their own, we just kick the c
by cgio 3y ago
What happens when a consumer does not understand the producer’s domain and misuses the data? As we load metadata with semantics of their own, we just kick the can down the road. Metadata is itself semantics about data. In many regulated environments, as producer you are responsible for downstream aggregations etc. I think the idea works great in prototyping, but similar to data mesh, ownership is poorly defined in the presence of aggregate domains and it’s more tuned to operational point to point interfaces.
- martypitt 3y ago> What happens when a consumer does not understand the producer’s domain and misuses the data? That's a problem with or without Taxi/Orbital. Today, it's generally left to consumers to pick the fields that look right, and hope they get it right. That's a risky approach. And, by leaving it to consumers, you run the risk on every new integration. I think by asking producers to annotate their attributes with a strongly defined semantic contract, you reduce the risk of consumers getting it wrong. Producers understand their data much better than consumers, so are better informed on how to map their attributes to a set of semantic contracts that consumers can leverage.