3 ms·
There is a way to splitting data. The service which owns the data always does the writes and others who need reads on that data can store replicas. ofc the comp
by ngc248 3y ago
There is a way to splitting data. The service which owns the data always does the writes and others who need reads on that data can store replicas. ofc the complication then will be in replicating data, but this will enable services to massively scale and eliminate SPOFs
- code_biologist 3y agoThe other part of this pattern you mention (replicating data) seems solved in many cases by the data warehouse patterns of the last few years. Stuff it all in Snowflake or BigQuery then readers can query as they see fit. Query engines like Trino can paper over data storage heterogeneity / breakdowns in centralization. I'm not a fan of the "lakehouse" terminology, but it is the thing. There are downsides to coupling more loosely with data consumers, but it keeps service owners moving without wasting time vending data.
- amne 3y agoAs long as you're not replicating the data model also which means you're now replicating the business logic that go along with the data model. An ETL approach from the start will save you a lot of headache.