4 ms·
The process is more complicated under the hood. I’m describing it from the POV of an engineer that wants to update a field — from that POV, it really is that si
by bcherny 5y ago
The process is more complicated under the hood. I’m describing it from the POV of an engineer that wants to update a field — from that POV, it really is that simple.
- DaiPlusPlus 5y ago> The process is more complicated under the hood. Right, but that's exactly what I'm asking about :)
- bcherny 5y agoSorry, I must have misunderstood the question! :)
- RandomBK 5y agoThe key is in making sure every step-wise change is backwards-compatible. Creating a new field, double-writing, and backfilling are all relatively easy tasks in a distributed environment. Individual nodes can take their time catching up to the latest schema. After a day or two, the engineer can verify that all data have been correctly migrated and double-written to the new ent/assoc. They then turn off double-writing to begin deprecating the old assoc. This can be done in an A/B test to ensure no major breakages. Once the double-writing is fully turned off, the engineer checks one last time that all the data have been migrated. They check that the read/write rate to the old ent/assoc is 0, which then confirms that the old field is now safe to delete. Overall, the trick is to think of the migration as a series of safe, backwards-compatible, easily-distributed steps, rather than a single atomic operation.