4 ms·

Yes, we need to move away from the notion of a migration "chain", it's not a very useful way to think about schema changes. Most people have an ideal schema in

Yes, we need to move away from the notion of a migration "chain", it's not a very useful way to think about schema changes. Most people have an ideal schema in mind - they should be free to simply craft this directly, with tooling to make the changes necessary to evolve the schema accordingly. I wrote a tool for this exactly this purpose, and there are an increasing number of similar tools to support working this way. Another common problem is migration tooling tied to an ORM as rails/django do (it looks like refinery avoids this, which is good). Working with migrations directly on the database level means you can support multiple (or zero) ORMs and apps with less lock-in, support more database features, and confirm the structure of the database directly.

bradstewart 7y ago
> with tooling to make the changes necessary to evolve the schema accordingly Isn't that exactly what most migration tools do? How do you get from schema version 1 to schema version 2 to schema version N without a "chain" of transformations? different from a "migration 'chain'"? While Rails' Migrations are part of/use parts of ActiveRecord, Nothing about them forces you to use AR (or any other ORM) for application code. Heck, I've used Rails Migrations in Node.js projects because they work so well.
- djrobstep 7y ago
  Yeah most migration tools work this way, although refinery doesn't. My preferred alternative to reconstructing prod from a historical chain of changes is to compare the preferred state to the production state directly. In most cases you can generate the script you need completely automatically from a diff tool. Direct comparison also allows you to directly test that your changes have been applied correctly and that prod schema explicitly matches. If it's possible to use rails migrations without an ORM that's great, although that's always going to be secondary to the ORM use case. Django migrations are extremely heavily coupled to the ORM, and it's almost impossible to use one without the other.
  - irishsultan 7y ago
    > In most cases you can generate the script you need completely automatically from a diff tool. I'd like to know what most cases is here, a diff tool isn't going to know whether adding a column and removing a column is a rename (assuming the right types), or whether they really are different columns. A diff tool isn't going to know what the value needs to be for your new non-NULL, non-DEFAULT column (nor is it going to know that for a NULL column existing data does need a value, but that new data needs a different value, if there is even a singular value that it can use).
  - bradstewart 7y ago
    Generate a script from the diff of _what_ exactly, the old and new schemas? How did you get to the "new" schema without some sort of migration? SQLAlchemy does something like what I think you're describing (and is tightly coupled with the application-level ORM): you update attributes on the model classes, and then it generates a migration for you. But I've always had to manually edit those migrations by hand. The Rails "migration first, model later" method feels much cleaner to me, but maybe that's just because I learned it that way. Migrations are one of the most dangerous things I run on prod, so I'm always interested in better/safer ways of doing them.
    - djrobstep 7y ago
      Yeah you compare your intended state for prod to what you currently have on prod, and generate the differences. It's still a "migration", the difference is it's generated from a direct comparison to production rather than a crude reconstruction of the database's history. Checking production directly is much more robust (more testable, changes made outside the context of the migration tooling are no problem) and simpler because it you don't need to maintain a historical migration chain - there's only ever one migration script to consider. And using a diff tool directly on the database means you can support database features that aren't directly supported by your ORM.

bradstewart 7y ago

> with tooling to make the changes necessary to evolve the schema accordingly Isn't that exactly what most migration tools do? How do you get from schema version 1 to schema version 2 to schema version N without a "chain" of transformations? different from a "migration 'chain'"? While Rails' Migrations are part of/use parts of ActiveRecord, Nothing about them forces you to use AR (or any other ORM) for application code. Heck, I've used Rails Migrations in Node.js projects because they work so well.

djrobstep 7y ago
Yeah most migration tools work this way, although refinery doesn't. My preferred alternative to reconstructing prod from a historical chain of changes is to compare the preferred state to the production state directly. In most cases you can generate the script you need completely automatically from a diff tool. Direct comparison also allows you to directly test that your changes have been applied correctly and that prod schema explicitly matches. If it's possible to use rails migrations without an ORM that's great, although that's always going to be secondary to the ORM use case. Django migrations are extremely heavily coupled to the ORM, and it's almost impossible to use one without the other.
- irishsultan 7y ago
  > In most cases you can generate the script you need completely automatically from a diff tool. I'd like to know what most cases is here, a diff tool isn't going to know whether adding a column and removing a column is a rename (assuming the right types), or whether they really are different columns. A diff tool isn't going to know what the value needs to be for your new non-NULL, non-DEFAULT column (nor is it going to know that for a NULL column existing data does need a value, but that new data needs a different value, if there is even a singular value that it can use).
- bradstewart 7y ago
  Generate a script from the diff of _what_ exactly, the old and new schemas? How did you get to the "new" schema without some sort of migration? SQLAlchemy does something like what I think you're describing (and is tightly coupled with the application-level ORM): you update attributes on the model classes, and then it generates a migration for you. But I've always had to manually edit those migrations by hand. The Rails "migration first, model later" method feels much cleaner to me, but maybe that's just because I learned it that way. Migrations are one of the most dangerous things I run on prod, so I'm always interested in better/safer ways of doing them.
  - djrobstep 7y ago
    Yeah you compare your intended state for prod to what you currently have on prod, and generate the differences. It's still a "migration", the difference is it's generated from a direct comparison to production rather than a crude reconstruction of the database's history. Checking production directly is much more robust (more testable, changes made outside the context of the migration tooling are no problem) and simpler because it you don't need to maintain a historical migration chain - there's only ever one migration script to consider. And using a diff tool directly on the database means you can support database features that aren't directly supported by your ORM.