4 ms·

Oh please, tell me how I'm misinformed? If you need a migration library because you added a column to your database, then you have a serious problem. If you n

by tinix 9y ago

Oh please, tell me how I'm misinformed? If you need a migration library because you added a column to your database, then you have a serious problem. If you need a migration library because you don't know how to write SQL, then you have a serious problem. If your data structure is truly changing that often, then you shouldn't be using a SQL database in the first place, and you have a serious problem.

orf 9y ago
Again, FUD. If you really need me to spell out why then I will, but I doubt it will change your viewpoint. Let me spell it out for you: You build your awesome [app] for [client]. You get your schema bang on first time and it's all working absolutely fine. Everyone is happy. Then [client] comes to you and says "[app] is awesome, but we need [x]". You think, and [x] needs some database modifications. Maybe it's a new column to store something, or perhaps it's a column that now shouldn't be nullable. Maybe it's a new table entirely. Perhaps what you need is already implemented in library [y], which needs database modifications of some kind like it's own tables. The possibilities are endless. Great. So how do we do this. Obviously we need to run some SQL on the database. So you change all your code, press deploy, and really quickly log onto the production server and smash out some artisanal SQL to modify the database before any requests hit the server. Everyone is happy! You quickly write an email all your co-workers to do these migrations locally so they can run the app. Oh no, but wait, you forgot to do operation Z! Now your app and everyone elses is broken. You resolve to ensure this doesn't happen again by writing a .sql file for each migration and running them in sequence. Great, you can even check them into your VCS and share them with your co-workers. Awesome! You can even make them part of your CI build (you do that, right?). You do another deploy, but nuts, something went wrong! You need to roll a migration back! But you only added forward migrations in your .sql files! Ok, so lets add [migration]-backwards.sql and [migration]-forwards.sql from now on. Awesome! So you've got a forwards and backwards migrations. But... wait, we need to run some Python code to do something as part of the migration. Ok... lets add a [migration]-{forwards}.py as well. Fantastic. But then you start to install package [z] because it does exactly what the client wants and you realize the benefits of using well tested, widely used libraries in your systems and not writing everything by hand. This requires some database modifications. The author of this package has the migrations it needs but it's in it's own home-grown format and it's only for MySQL and Oracle! Nuts, ok, so you translate them by hand and add them to your migrations. But you made a mistake somewhere in the translation and everything breaks. You fix it by hand. Congratulations. You've just written a shit version of Django migrations that everyone hates and that doesn't really work. tl;dr: schema changes are a natural part of development. You, me, and everyone else working on any kind development project has done them. They happen, this is a fact. If you need to: 1. Share the migrations in your team 2. Run them as part of your CI build 3. Use third party migrations for packages (including built in contenttypes or other core Django tables) 4. Have them in a format that works for any supported database 5. Have rollback and the ability to execute arbitrary code as part of the migration Then you can either write your own shitty version or use a well supported, built in system that handles the complexity for you. This is not a bad thing. I can't comprehend your viewpoint, which mostly boils down to "these damn kids are on my lawn", so I can only imagine your setup is some ad-hoc .sql files that you run by hand. This is a bad thing.
- tinix 9y ago
  You are setting up a comical hypothetical situation in which the developer is blundering through their job in a mindless inept manner. I'll just stop you right there. No amount of best practices, process, or tooling will fix an incompetent developer who is part of a disorganized team that are working on a pile of bad architecture.
allover 9y ago
Okay, so you changed the DB schema in your 'dev' environment to add a column. 1. In what manner/format do you define or commit the change to VCS so it is in sync with the corresponding code change (that uses the new column)? 2. What happens when a teammate comes to deploy that schema change to production? 3. What happens when we need to roll back production to some previous commit? The need to rollback may/may-not relate to the schema change (not suggesting you 'got it wrong'). If you can answer 1, 2, 3 by some convention ... you've created a migration system that should probably be codified.
- tinix 9y ago
  1. Change model code to add the new field, test, and commit it to version control. Then it's merged to staging, where it's tested in a production-like environment with a recent production data snapshot. After that's approved, it goes to master/production servers. 2. git pull then restart some wsgi/worker processes 3. git checkout to desired revision and restart some wsgi/worker processes It's perfectly fine to have models comprised of only a subset of the data they interact with. The models do not have to reflect all fields that exist in the database. The models only reflect information that is necessary for the application in which said models are used. Basically, if you clobber your database, no amount of migration tooling or process will help you. Ever. It will just get in the way, to be honest, and will prevent you from doing things properly in the first place.
  - allover 9y ago
    You missed the point of all those questions (e.g. in 2 you don't mention how the schema change gets applied to the prod database). Unfortunately I'm frankly out of patience explaining any further, so we'll have to leave it there.
    - tinix 9y ago
      There is no schema change, that's the point. You just don't get it... but that's okay.
dang 9y ago
Please don't get involved in technical flamewars on HN (or other flamewars). As agitation goes up, information goes down, and these discussions turn into back-and-forth spats that benefit no one, including the combatants.

by tinix 9y ago

orf 9y ago

Again, FUD. If you really need me to spell out why then I will, but I doubt it will change your viewpoint. Let me spell it out for you: You build your awesome [app] for [client]. You get your schema bang on first time and it's all working absolutely fine. Everyone is happy. Then [client] comes to you and says "[app] is awesome, but we need [x]". You think, and [x] needs some database modifications. Maybe it's a new column to store something, or perhaps it's a column that now shouldn't be nullable. Maybe it's a new table entirely. Perhaps what you need is already implemented in library [y], which needs database modifications of some kind like it's own tables. The possibilities are endless. Great. So how do we do this. Obviously we need to run some SQL on the database. So you change all your code, press deploy, and really quickly log onto the production server and smash out some artisanal SQL to modify the database before any requests hit the server. Everyone is happy! You quickly write an email all your co-workers to do these migrations locally so they can run the app. Oh no, but wait, you forgot to do operation Z! Now your app and everyone elses is broken. You resolve to ensure this doesn't happen again by writing a .sql file for each migration and running them in sequence. Great, you can even check them into your VCS and share them with your co-workers. Awesome! You can even make them part of your CI build (you do that, right?). You do another deploy, but nuts, something went wrong! You need to roll a migration back! But you only added forward migrations in your .sql files! Ok, so lets add [migration]-backwards.sql and [migration]-forwards.sql from now on. Awesome! So you've got a forwards and backwards migrations. But... wait, we need to run some Python code to do something as part of the migration. Ok... lets add a [migration]-{forwards}.py as well. Fantastic. But then you start to install package [z] because it does exactly what the client wants and you realize the benefits of using well tested, widely used libraries in your systems and not writing everything by hand. This requires some database modifications. The author of this package has the migrations it needs but it's in it's own home-grown format and it's only for MySQL and Oracle! Nuts, ok, so you translate them by hand and add them to your migrations. But you made a mistake somewhere in the translation and everything breaks. You fix it by hand. Congratulations. You've just written a shit version of Django migrations that everyone hates and that doesn't really work. tl;dr: schema changes are a natural part of development. You, me, and everyone else working on any kind development project has done them. They happen, this is a fact. If you need to: 1. Share the migrations in your team 2. Run them as part of your CI build 3. Use third party migrations for packages (including built in contenttypes or other core Django tables) 4. Have them in a format that works for any supported database 5. Have rollback and the ability to execute arbitrary code as part of the migration Then you can either write your own shitty version or use a well supported, built in system that handles the complexity for you. This is not a bad thing. I can't comprehend your viewpoint, which mostly boils down to "these damn kids are on my lawn", so I can only imagine your setup is some ad-hoc .sql files that you run by hand. This is a bad thing.

tinix 9y ago
You are setting up a comical hypothetical situation in which the developer is blundering through their job in a mindless inept manner. I'll just stop you right there. No amount of best practices, process, or tooling will fix an incompetent developer who is part of a disorganized team that are working on a pile of bad architecture.

allover 9y ago

Okay, so you changed the DB schema in your 'dev' environment to add a column. 1. In what manner/format do you define or commit the change to VCS so it is in sync with the corresponding code change (that uses the new column)? 2. What happens when a teammate comes to deploy that schema change to production? 3. What happens when we need to roll back production to some previous commit? The need to rollback may/may-not relate to the schema change (not suggesting you 'got it wrong'). If you can answer 1, 2, 3 by some convention ... you've created a migration system that should probably be codified.

tinix 9y ago
1. Change model code to add the new field, test, and commit it to version control. Then it's merged to staging, where it's tested in a production-like environment with a recent production data snapshot. After that's approved, it goes to master/production servers. 2. git pull then restart some wsgi/worker processes 3. git checkout to desired revision and restart some wsgi/worker processes It's perfectly fine to have models comprised of only a subset of the data they interact with. The models do not have to reflect all fields that exist in the database. The models only reflect information that is necessary for the application in which said models are used. Basically, if you clobber your database, no amount of migration tooling or process will help you. Ever. It will just get in the way, to be honest, and will prevent you from doing things properly in the first place.
- allover 9y ago
  You missed the point of all those questions (e.g. in 2 you don't mention how the schema change gets applied to the prod database). Unfortunately I'm frankly out of patience explaining any further, so we'll have to leave it there.
  - tinix 9y ago
    There is no schema change, that's the point. You just don't get it... but that's okay.

dang 9y ago

Please don't get involved in technical flamewars on HN (or other flamewars). As agitation goes up, information goes down, and these discussions turn into back-and-forth spats that benefit no one, including the combatants.