4 ms·

SQL migrations may seem tedious, but the process of planning your data model and having rigid definitions for your data types saves you huge amounts of time in

SQL migrations may seem tedious, but the process of planning your data model and having rigid definitions for your data types saves you huge amounts of time in the long run. Your reasoning seems too black and white, most the of the time, you can't know your data model ahead of time, that's why prototypes are so useful, so just using your language data structures or some less rigid datastore lets you get quicker to your data model.

paulmd 10y ago
Having rigid definitions for data types doesn't necessarily mean the datastore is rigid. It just means that you know what type of data goes in each column. Like it or not, if you change your schema that change has to be handled somewhere - either you update the old data to the new schema (do this), or you handle old versions of the schema in the application (fundamentally non-scalable IMO). It doesn't matter whether they're values in a JSON object or columns in a relational DB. Flyway is a fantastic tool for making changes to relational schemas. Your code is versioned, and you can just keep the schema migration scripts right next to it. If you need to upgrade an old schema, it just walks through the migration scripts until it's up-to-date. If there is some portion of your model that really, truly, honestly consists of unstructured data, you can just put that in a JSON column.
- gjolund 10y ago
  I also really like the sqitch project for migration management. http://sqitch.org/ http://sqitch.org/
- Scarbutt 10y ago
  That is why I mentioned prototypes, they are a quick messy hack that is not meant to scale, its purpose its to validate your ideas and decisions, what is going to become your MVP, which IMO is far more important than getting the perfect stack/tooling from the start.
  - gjolund 10y ago
    We obviously value very different things in regards to software development. You really seem to favor initial velocity and rapid iteration. Fair, I can totally understand why. I favor planning, stability, and limiting thrash due to bad/uninformed decision making. In my experience the rush to market only pans out when you plan on having a large influx of funding to rebuild your stack. Otherwise you are stuck holding a mess of code and trying to scale ideas that never should have gone to production in the first place. There is a whole other side to the industry that demands provably correct solutions, and will not tolerate the "hacker culture". Not because of any petty reasons like work hours or perks, but because some applications require a level of accuracy that your average growth hacker can't meet.
wvenable 10y ago
This sounds so logical, is repeated often, and is fundamentally incorrect. Just because you are defining a data model doesn't mean you can't change it. There's nothing easier. Undefined schemas are actually worse under change than defined ones.
- prodigal_erik 10y ago
  Like I've argued before, you have a schema. Your app cares about the location and format of the data. You're only deciding whether or not you want to write down your schema and use it to catch mistakes.
  - wvenable 10y ago
    Like I've argued before, with NoSQL you have many schemas -- one for every change (unless you delete all the data after every schema change). In SQL, you have one schema -- which is much easier to work with even when you're just doing early spec development.

paulmd 10y ago

Having rigid definitions for data types doesn't necessarily mean the datastore is rigid. It just means that you know what type of data goes in each column. Like it or not, if you change your schema that change has to be handled somewhere - either you update the old data to the new schema (do this), or you handle old versions of the schema in the application (fundamentally non-scalable IMO). It doesn't matter whether they're values in a JSON object or columns in a relational DB. Flyway is a fantastic tool for making changes to relational schemas. Your code is versioned, and you can just keep the schema migration scripts right next to it. If you need to upgrade an old schema, it just walks through the migration scripts until it's up-to-date. If there is some portion of your model that really, truly, honestly consists of unstructured data, you can just put that in a JSON column.

gjolund 10y ago
I also really like the sqitch project for migration management. http://sqitch.org/ http://sqitch.org/
Scarbutt 10y ago
That is why I mentioned prototypes, they are a quick messy hack that is not meant to scale, its purpose its to validate your ideas and decisions, what is going to become your MVP, which IMO is far more important than getting the perfect stack/tooling from the start.
- gjolund 10y ago
  We obviously value very different things in regards to software development. You really seem to favor initial velocity and rapid iteration. Fair, I can totally understand why. I favor planning, stability, and limiting thrash due to bad/uninformed decision making. In my experience the rush to market only pans out when you plan on having a large influx of funding to rebuild your stack. Otherwise you are stuck holding a mess of code and trying to scale ideas that never should have gone to production in the first place. There is a whole other side to the industry that demands provably correct solutions, and will not tolerate the "hacker culture". Not because of any petty reasons like work hours or perks, but because some applications require a level of accuracy that your average growth hacker can't meet.

wvenable 10y ago

This sounds so logical, is repeated often, and is fundamentally incorrect. Just because you are defining a data model doesn't mean you can't change it. There's nothing easier. Undefined schemas are actually worse under change than defined ones.

prodigal_erik 10y ago
Like I've argued before, you have a schema. Your app cares about the location and format of the data. You're only deciding whether or not you want to write down your schema and use it to catch mistakes.
- wvenable 10y ago
  Like I've argued before, with NoSQL you have many schemas -- one for every change (unless you delete all the data after every schema change). In SQL, you have one schema -- which is much easier to work with even when you're just doing early spec development.