6 ms·

> I'm not sure why you put quotes around migrations as if it's some alien, obscure or weird feature. If you've never written a web application that needs migrat

by tinix 9y ago

> I'm not sure why you put quotes around migrations as if it's some alien, obscure or weird feature. If you've never written a web application that needs migrations then you're not writing the kinds of applications (or indeed any 'serious' application) that would benefit from Django IMO. I have been programming for almost 15 years, and have never once needed to use migrations. I have worked on projects for multiple years with multiple developers and both small and large teams, and grew and scaled them, massively overhauled schemas, etc... and still, have never needed migrations. Have I ever had to add a column to a database? Absolutely, have I ever needed a massively overcomplicated "migration" tool? Hell no, because I put more than 5 minutes of thought into my application logic and data structures before I even started writing code or designing the database schema. I've single-handedly built, and maintained with a team, applications that did millions of dollars of revenue per day, with 10s of thousands of users per minute normally, with 100s of thousands per minute during peak load and used various styles of SQL data stores with between 20 and 50 tables on some projects (sounds pretty serious to me)... and still never once had a need for migrations. My point is, if you need to lean on migrations even remotely often, you're doing something very very wrong. I think, though, the type of developer using Django for its large pool of third-party extensions isn't really the type of developer who puts a lot of thought into what they are doing, though. Maybe I'll catch flak for this, but it's pretty true in my experience. It's the same as some front-end JS devs who sling various frameworks and libraries together, and end up with a pile of unmaintainable mess in the end... Django is used to sling backend web apps together, fast, and needing migrations is evidence of that. However, if one is doing proper clean development and following some simple best practices, an automated and complex migration should never be needed in the first place.

tobltobs 9y ago
You get your DB model 100% correct the first time, always? Even years later your DB model is able to accommodate all those always changing business requirements?
- tinix 9y ago
  Yeah, pretty much. Haven't had a problem yet where I needed automated migrations, and I have been doing this for years upon years upon years. Like I said, if you're leaving heavily on migrations, you're doing something wrong.
  - collyw 9y ago
    I'd like to know what I am doing wrong, as I use migrations all the time. The option is change the database later (and use migrations) or write a whole ton of crappy code to avoid using them. I have tried both ways and usually migrations work better.
    - tinix 9y ago
      I dunno what to tell you without specific details about your application and environment in general. Clearly, you need a better system architecture if you're having to do migrations constantly.
      - tobltobs 9y ago
        I always thought a data model should be as agnostic to the system architecture as possible. Insofar I don't understand how a system architecture can help to avoid data model migrations.
        tinix 9y ago
        Data structure, both storage, and in-memory, are very much a part of architecture.
      - collyw 9y ago
        So requirements are never added / changed in your systems? Thats usually why I need migrations.
  - rhizome 9y ago
    Haven't had a problem yet where I needed automated migrations, and I have been doing this for years upon years upon years. Just out of curiosity, have you had to maintain/modify any of these apps over these years and years? If so, without any schema changes?
- mythrwy 9y ago
  If you get your DB model 100% correct the first time I want to work for your clients!
asdfgadsfgasfdg 9y ago
How do you add columns to your db? Do your requirements never change? I am fully capable of doing everything done with migrations with SQL - but I always use migrations (even for custom sql outside of the ORM's capability). Why? because then we have a record of the schema state at any given moment (that is in sync with the application code at that point) and how it got to where it is now.
- tinix 9y ago
  If you're leaning on migrations often, with SQL, then I think you should not be using SQL, obviously (or you're just doing something wrong) Or, one should use a SQL database that allows more abstract data types (like JSON blobs in Postgres)
  - afarrell 9y ago
    So where is your data model enforced if not in your tables?
    - tinix 9y ago
      > So where is your data model enforced if not in your tables? It's enforced... ...in the models. :P
  - jnbiche 9y ago
    And how do you keep your database schema under version control? Edit: Do all you downvoters not keep track of your schema changes as you develop your application (via migrations, or even sql files), so you can easily roll back changes if needed?
    - tinix 9y ago
      The models live in version control. It's that simple. If your data structure is changing so often in such a way that you need to keep a history, then I'm afraid you have some fundamental problems that no amount of tooling or process can solve. You will not be able to scale your development efforts with this type of constantly changing data model, even with a migration library. If these are truly problems that you or your team has, then you should hire someone to help you organize your application, data, and workflow in a more effective manner.
orf 9y ago
> I have been programming for almost 15 years, and have never once needed to use migrations. > Have I ever had to add a column to a database? Absolutely, ... then you've used a migration. A statement like "I've never once had to use a migration" is so obviously incorrect that it's comical. Your contradiction two lines down makes it more so. The rest of your comment is fairly misinformed about a few things, full of FUD, misses the point of Django migrations entirely and seems desperate to paint any use of migrations as the result of being a "bad programmer" rather than changing requirements or a natural part of development. Seeing as you've used migrations, you're a bad programmer as well. As such I'm not going to bother writing a reply to the rest of it like I usually would.
- tinix 9y ago
  Oh please, tell me how I'm misinformed? If you need a migration library because you added a column to your database, then you have a serious problem. If you need a migration library because you don't know how to write SQL, then you have a serious problem. If your data structure is truly changing that often, then you shouldn't be using a SQL database in the first place, and you have a serious problem.
  - orf 9y ago
    Again, FUD. If you really need me to spell out why then I will, but I doubt it will change your viewpoint. Let me spell it out for you: You build your awesome [app] for [client]. You get your schema bang on first time and it's all working absolutely fine. Everyone is happy. Then [client] comes to you and says "[app] is awesome, but we need [x]". You think, and [x] needs some database modifications. Maybe it's a new column to store something, or perhaps it's a column that now shouldn't be nullable. Maybe it's a new table entirely. Perhaps what you need is already implemented in library [y], which needs database modifications of some kind like it's own tables. The possibilities are endless. Great. So how do we do this. Obviously we need to run some SQL on the database. So you change all your code, press deploy, and really quickly log onto the production server and smash out some artisanal SQL to modify the database before any requests hit the server. Everyone is happy! You quickly write an email all your co-workers to do these migrations locally so they can run the app. Oh no, but wait, you forgot to do operation Z! Now your app and everyone elses is broken. You resolve to ensure this doesn't happen again by writing a .sql file for each migration and running them in sequence. Great, you can even check them into your VCS and share them with your co-workers. Awesome! You can even make them part of your CI build (you do that, right?). You do another deploy, but nuts, something went wrong! You need to roll a migration back! But you only added forward migrations in your .sql files! Ok, so lets add [migration]-backwards.sql and [migration]-forwards.sql from now on. Awesome! So you've got a forwards and backwards migrations. But... wait, we need to run some Python code to do something as part of the migration. Ok... lets add a [migration]-{forwards}.py as well. Fantastic. But then you start to install package [z] because it does exactly what the client wants and you realize the benefits of using well tested, widely used libraries in your systems and not writing everything by hand. This requires some database modifications. The author of this package has the migrations it needs but it's in it's own home-grown format and it's only for MySQL and Oracle! Nuts, ok, so you translate them by hand and add them to your migrations. But you made a mistake somewhere in the translation and everything breaks. You fix it by hand. Congratulations. You've just written a shit version of Django migrations that everyone hates and that doesn't really work. tl;dr: schema changes are a natural part of development. You, me, and everyone else working on any kind development project has done them. They happen, this is a fact. If you need to: 1. Share the migrations in your team 2. Run them as part of your CI build 3. Use third party migrations for packages (including built in contenttypes or other core Django tables) 4. Have them in a format that works for any supported database 5. Have rollback and the ability to execute arbitrary code as part of the migration Then you can either write your own shitty version or use a well supported, built in system that handles the complexity for you. This is not a bad thing. I can't comprehend your viewpoint, which mostly boils down to "these damn kids are on my lawn", so I can only imagine your setup is some ad-hoc .sql files that you run by hand. This is a bad thing.
arctangent 9y ago
You've caught a lot of flak for your statement. With the knowledge I have now it would easy for me to say "and rightly so", but I think I can understand where you're coming from. Django didn't always have schema migrations built in. I think they appeared in version 1.7 about 2 and a bit years ago. Before then there were separate add-on apps that could do migrations for you. When Django added built-in support for migrations I still didn't use them for a while because at the time I was primarily a solo developer and wanted to have full control over what happened in the DB. And because I was a solo developer it didn't matter. So in that sense you're comment is in some ways correct. However, in recent times I've started to work as part of a team. And that's where migrations really start to shine, because there might be several different people making app changes that result in DB schema changes. Django stores migration files with information about which other migrations each one is dependent upon. Without that kind of migrations system it would simply be impossible for developers working on combinations of different branches of a development tree to build a working version of the code.

by tinix 9y ago

tobltobs 9y ago

You get your DB model 100% correct the first time, always? Even years later your DB model is able to accommodate all those always changing business requirements?

tinix 9y ago
Yeah, pretty much. Haven't had a problem yet where I needed automated migrations, and I have been doing this for years upon years upon years. Like I said, if you're leaving heavily on migrations, you're doing something wrong.
- collyw 9y ago
  I'd like to know what I am doing wrong, as I use migrations all the time. The option is change the database later (and use migrations) or write a whole ton of crappy code to avoid using them. I have tried both ways and usually migrations work better.
  - tinix 9y ago
    I dunno what to tell you without specific details about your application and environment in general. Clearly, you need a better system architecture if you're having to do migrations constantly.
    - tobltobs 9y ago
      I always thought a data model should be as agnostic to the system architecture as possible. Insofar I don't understand how a system architecture can help to avoid data model migrations.
      - tinix 9y ago
        Data structure, both storage, and in-memory, are very much a part of architecture.
    - collyw 9y ago
      So requirements are never added / changed in your systems? Thats usually why I need migrations.
- rhizome 9y ago
  Haven't had a problem yet where I needed automated migrations, and I have been doing this for years upon years upon years. Just out of curiosity, have you had to maintain/modify any of these apps over these years and years? If so, without any schema changes?
mythrwy 9y ago
If you get your DB model 100% correct the first time I want to work for your clients!

asdfgadsfgasfdg 9y ago

How do you add columns to your db? Do your requirements never change? I am fully capable of doing everything done with migrations with SQL - but I always use migrations (even for custom sql outside of the ORM's capability). Why? because then we have a record of the schema state at any given moment (that is in sync with the application code at that point) and how it got to where it is now.

tinix 9y ago
If you're leaning on migrations often, with SQL, then I think you should not be using SQL, obviously (or you're just doing something wrong) Or, one should use a SQL database that allows more abstract data types (like JSON blobs in Postgres)
- afarrell 9y ago
  So where is your data model enforced if not in your tables?
  - tinix 9y ago
    > So where is your data model enforced if not in your tables? It's enforced... ...in the models. :P
- jnbiche 9y ago
  And how do you keep your database schema under version control? Edit: Do all you downvoters not keep track of your schema changes as you develop your application (via migrations, or even sql files), so you can easily roll back changes if needed?
  - tinix 9y ago
    The models live in version control. It's that simple. If your data structure is changing so often in such a way that you need to keep a history, then I'm afraid you have some fundamental problems that no amount of tooling or process can solve. You will not be able to scale your development efforts with this type of constantly changing data model, even with a migration library. If these are truly problems that you or your team has, then you should hire someone to help you organize your application, data, and workflow in a more effective manner.

orf 9y ago

> I have been programming for almost 15 years, and have never once needed to use migrations. > Have I ever had to add a column to a database? Absolutely, ... then you've used a migration. A statement like "I've never once had to use a migration" is so obviously incorrect that it's comical. Your contradiction two lines down makes it more so. The rest of your comment is fairly misinformed about a few things, full of FUD, misses the point of Django migrations entirely and seems desperate to paint any use of migrations as the result of being a "bad programmer" rather than changing requirements or a natural part of development. Seeing as you've used migrations, you're a bad programmer as well. As such I'm not going to bother writing a reply to the rest of it like I usually would.

tinix 9y ago
Oh please, tell me how I'm misinformed? If you need a migration library because you added a column to your database, then you have a serious problem. If you need a migration library because you don't know how to write SQL, then you have a serious problem. If your data structure is truly changing that often, then you shouldn't be using a SQL database in the first place, and you have a serious problem.
- orf 9y ago
  Again, FUD. If you really need me to spell out why then I will, but I doubt it will change your viewpoint. Let me spell it out for you: You build your awesome [app] for [client]. You get your schema bang on first time and it's all working absolutely fine. Everyone is happy. Then [client] comes to you and says "[app] is awesome, but we need [x]". You think, and [x] needs some database modifications. Maybe it's a new column to store something, or perhaps it's a column that now shouldn't be nullable. Maybe it's a new table entirely. Perhaps what you need is already implemented in library [y], which needs database modifications of some kind like it's own tables. The possibilities are endless. Great. So how do we do this. Obviously we need to run some SQL on the database. So you change all your code, press deploy, and really quickly log onto the production server and smash out some artisanal SQL to modify the database before any requests hit the server. Everyone is happy! You quickly write an email all your co-workers to do these migrations locally so they can run the app. Oh no, but wait, you forgot to do operation Z! Now your app and everyone elses is broken. You resolve to ensure this doesn't happen again by writing a .sql file for each migration and running them in sequence. Great, you can even check them into your VCS and share them with your co-workers. Awesome! You can even make them part of your CI build (you do that, right?). You do another deploy, but nuts, something went wrong! You need to roll a migration back! But you only added forward migrations in your .sql files! Ok, so lets add [migration]-backwards.sql and [migration]-forwards.sql from now on. Awesome! So you've got a forwards and backwards migrations. But... wait, we need to run some Python code to do something as part of the migration. Ok... lets add a [migration]-{forwards}.py as well. Fantastic. But then you start to install package [z] because it does exactly what the client wants and you realize the benefits of using well tested, widely used libraries in your systems and not writing everything by hand. This requires some database modifications. The author of this package has the migrations it needs but it's in it's own home-grown format and it's only for MySQL and Oracle! Nuts, ok, so you translate them by hand and add them to your migrations. But you made a mistake somewhere in the translation and everything breaks. You fix it by hand. Congratulations. You've just written a shit version of Django migrations that everyone hates and that doesn't really work. tl;dr: schema changes are a natural part of development. You, me, and everyone else working on any kind development project has done them. They happen, this is a fact. If you need to: 1. Share the migrations in your team 2. Run them as part of your CI build 3. Use third party migrations for packages (including built in contenttypes or other core Django tables) 4. Have them in a format that works for any supported database 5. Have rollback and the ability to execute arbitrary code as part of the migration Then you can either write your own shitty version or use a well supported, built in system that handles the complexity for you. This is not a bad thing. I can't comprehend your viewpoint, which mostly boils down to "these damn kids are on my lawn", so I can only imagine your setup is some ad-hoc .sql files that you run by hand. This is a bad thing.

arctangent 9y ago

You've caught a lot of flak for your statement. With the knowledge I have now it would easy for me to say "and rightly so", but I think I can understand where you're coming from. Django didn't always have schema migrations built in. I think they appeared in version 1.7 about 2 and a bit years ago. Before then there were separate add-on apps that could do migrations for you. When Django added built-in support for migrations I still didn't use them for a while because at the time I was primarily a solo developer and wanted to have full control over what happened in the DB. And because I was a solo developer it didn't matter. So in that sense you're comment is in some ways correct. However, in recent times I've started to work as part of a team. And that's where migrations really start to shine, because there might be several different people making app changes that result in DB schema changes. Django stores migration files with information about which other migrations each one is dependent upon. Without that kind of migrations system it would simply be impossible for developers working on combinations of different branches of a development tree to build a working version of the code.