4 ms·

Considered doing the migration via Liquibase? I dont know that it supports parallelism in the migration process but you could do most of the rest in it (pre/pos

by gregw2 2y ago

Considered doing the migration via Liquibase? I dont know that it supports parallelism in the migration process but you could do most of the rest in it (pre/post migration scripts, rollbacks, test cases (within precondition changesets), etc... Not a real programming language though; its xml config files or annotated SQL scripts...

YorickPeterse 2y ago
I hadn't heard of Liquibase until now. Taking a brief look, it doesn't seem like it would be suitable though. For large scale migrations you really need to be able to express arbitrary logic ("migrate A to B but only if file X on the file system contains Y"), which you can't do with something like SQL or XML.
- svieira 2y ago
  The great thing is, Liquibase even allows you to run arbitrary programs as migrations using `customChange`: https://docs.liquibase.com/change-types/custom-change.html https://docs.liquibase.com/change-types/custom-change.html Though you can get a long way with just specifying the appropriate contexts before you kick off your migrations (and tagging your changesets with those context tags as well): https://docs.liquibase.com/concepts/changelogs/attributes/contexts.html https://docs.liquibase.com/concepts/changelogs/attributes/co...
gelatocar 2y ago
I found that liquibase doesn't really support rollbacks, particularly with mysql as you can't do transactions for schema updates, if the migration fails in the middle of an update it just gets left in a half-updated state.
- a8m 2y ago
  I'm glad to see I'm not the ~only one bothered by this issue. This has been, and still is, one of the challenges in managing MySQL schemas. That's one of the reasons we implemented "migrate down" differently than other tools. I'm not here to promote my blog post, but if you are interested in seeing how we tackled this in Atlas, you can read more here: https://atlasgo.io/blog/2024/04/01/migrate-down https://atlasgo.io/blog/2024/04/01/migrate-down
  - deleted 2y ago
    [deleted]
  - gregw2 2y ago
    Liquibase does support rollback DDL with MySQL; I used it. I put each DDL in a Liquibase changeset with a corresponding rollback DDL I constructed by hand. If the Liquibase changeset failed, I could run the rollback for all the steps after the "top" of my wish-I-could-put-them-in-a-MySQL-transaction operations. But you are right MySQL itself doesn't support transactions for DDL and that is true whatever tool you use. It is true that if you put multiple SQL operations in a single Liquibase changeset that are not transactional you can't reliably do rollbacks like the above. It is also true that constructing an inverse rollback SQL for each changeset SQL by hand takes time and effort particularly to ensure sufficient testing, and the business/technical value of actually doing that coding+testing may or may not be worth it depending on your situation/use-case.

by gregw2 2y ago

YorickPeterse 2y ago

I hadn't heard of Liquibase until now. Taking a brief look, it doesn't seem like it would be suitable though. For large scale migrations you really need to be able to express arbitrary logic ("migrate A to B but only if file X on the file system contains Y"), which you can't do with something like SQL or XML.

svieira 2y ago
The great thing is, Liquibase even allows you to run arbitrary programs as migrations using `customChange`: https://docs.liquibase.com/change-types/custom-change.html https://docs.liquibase.com/change-types/custom-change.html Though you can get a long way with just specifying the appropriate contexts before you kick off your migrations (and tagging your changesets with those context tags as well): https://docs.liquibase.com/concepts/changelogs/attributes/contexts.html https://docs.liquibase.com/concepts/changelogs/attributes/co...

gelatocar 2y ago

I found that liquibase doesn't really support rollbacks, particularly with mysql as you can't do transactions for schema updates, if the migration fails in the middle of an update it just gets left in a half-updated state.

a8m 2y ago
I'm glad to see I'm not the ~only one bothered by this issue. This has been, and still is, one of the challenges in managing MySQL schemas. That's one of the reasons we implemented "migrate down" differently than other tools. I'm not here to promote my blog post, but if you are interested in seeing how we tackled this in Atlas, you can read more here: https://atlasgo.io/blog/2024/04/01/migrate-down https://atlasgo.io/blog/2024/04/01/migrate-down
- deleted 2y ago
  [deleted]
- gregw2 2y ago
  Liquibase does support rollback DDL with MySQL; I used it. I put each DDL in a Liquibase changeset with a corresponding rollback DDL I constructed by hand. If the Liquibase changeset failed, I could run the rollback for all the steps after the "top" of my wish-I-could-put-them-in-a-MySQL-transaction operations. But you are right MySQL itself doesn't support transactions for DDL and that is true whatever tool you use. It is true that if you put multiple SQL operations in a single Liquibase changeset that are not transactional you can't reliably do rollbacks like the above. It is also true that constructing an inverse rollback SQL for each changeset SQL by hand takes time and effort particularly to ensure sufficient testing, and the business/technical value of actually doing that coding+testing may or may not be worth it depending on your situation/use-case.