4 ms·
Since this is a in house project, in some ways your organization have already failed. They shouldn't drop someone new onto leading "the big migration", it shoul
by Too 2y ago
Since this is a in house project, in some ways your organization have already failed. They shouldn't drop someone new onto leading "the big migration", it should have been built into the process of developing V2 from the beginning, delivered in increments rather than a big bang.
Now that's too late and you just have to do the best of the situation. Lots of good advice has already been given in the thread, here are a few more.
- Remove as much data (and features) from v1 as you possibly can, before you start migrating (obviously after backing up everything). There will always be edge cases of strangely formatted data stored, that got inserted before data validator X was implemented 8 years ago. These outliers will throw grit into your migration machinery. Do you really need those audit logs from a decade ago? No. Do you really need obscure feature that was only used for compatibility with windows 98? No. Just remove it.
Working with smaller data sets not only removes outliers but makes data size more manageable. Snapshot of the db will fit on a developer workstation. Running the migration with short downtime. Rolling back the migration with short downtime. Compare a database that fits in memory vs a distributed monster, could mean the difference of seconds to hours and once you reach that threshold, a rollback can be scary.
- Which brings us to the next topic. Avoid an atomic switchover of data. You will always find some errors when you swap and that puts you in a sweaty position if you should quickfix or rollback. Rolling back data is a lot more difficult than rolling back applications, which can pressure you into quick fixes, that may not even be possible due to some oversight when developing V2.
If you can, stop writing into V1, only use it for reading. Gradually move more and more write-paths into V2 until there are none left. Then, gradually remove read-paths from V1 as you move the old data over into V2 (or remove it).
- When transitioning the application gradually like this, you may end up with some corner of the system that never get completely migrated. This is fine. Maybe you realize V1 was better than V2 after all in this particular situation. Maybe the cost outweighs the benefit. Don't see this as a failure. Always reevaluate what is the best solution now and avoid the sunken cost fallacy.
- Finally, make sure the migration get priority from the whole org. If V1 never get phased out, others will start delivering new features there, forcing you to take two steps forward and one step back all the time, cleaning up new features to migrate into v2. Making sure there is only one project should be everyone's priority.