3 ms·
Avoiding rebuilding the table is definitely possible for certain schema changes like adding a new field. But supporting data type changes without rebuilding is
by ovaistariq 4y ago
Avoiding rebuilding the table is definitely possible for certain schema changes like adding a new field.
But supporting data type changes without rebuilding is not ideal. It will lead to data quality issues and complexity on the application. Integer -> String example is simple. But what about String -> integer, how are the consumers of data supposed to handle the situation where the field in some records has a string value and in some has an integer value? They will have to add type checking which complicates each of these consumers.
Then some of the downstream consumers such as data warehouses depend on strict data validations. We went through this problem at Uber- I blogged about it here https://www.uber.com/blog/dbevents-ingestion-framework/ https://www.uber.com/blog/dbevents-ingestion-framework/
- olivermuty 4y agoWhat they are saying is that if you grab the value it will be made into String (as you expect it to be if you locked and did it all) with a JIT migration as a part of the fetch operation. Solving the exact problem you stipulate.