3 ms·

Writing data migrations that fail safely is tricky, and requires some extra care. Temp tables, adding new columns instead of modifying, feature flags, etc. all

Writing data migrations that fail safely is tricky, and requires some extra care. Temp tables, adding new columns instead of modifying, feature flags, etc. all help with this. Basically, you want to run stuff in production with less risk. On the AWS/CloudFormation side, "Cattle, not pets" is the motto[0]. Terraform specifically is really good at capturing the desired vs current state of your infra, and showing you what those changes will be before you apply them. Point being - as long as your automation is captured in Infra-as-code, you shouldn't care too much if something gets destroyed. [0]- http://cloudscaling.com/blog/cloud-computing/the-history-of-pets-vs-cattle/ http://cloudscaling.com/blog/cloud-computing/the-history-of-... [1] - https://www.terraform.io/ https://www.terraform.io/

k__ 6y ago
Are data migrations easier with MongoDB/DynamoDB? They don't really have a schema.
- solumos 6y ago
  Yeah, just shovel the data in. No schema, no problem. The tradeoff is that if you're actually relying on the structure of that data, you're introducing risk since its changes aren't being intentionally managed at the DB level.
  - k__ 6y ago
    How do you work around that with databases that don't have a (formal) schema?
    - solumos 6y ago
      You dont. Use DBs with schemas for structured data.
- siliconwrath 6y ago
  With document DBs you still have "schemas", they just won't be explicit at the database layer. If you rename a field your app uses, you will end up having to either rename that field in every document, or punt on migrating and instead ensure that your app can handle either field name when reading. While the latter may sound easier (and often is easier when you're starting out and iterating quickly), you will soon start to feel the pain of managing dozens of different versions of your "schema" at the app layer. I've always found it easier to only have to deal with a single schema at a time in any situation where you can't just regenerate all records on the fly, like a cache, and for which you don't care about what the specific schema was at a point in time. If you actually do care about what the schema was at any point in time at the app layer, your app WILL need to account for any version of the data—but if you need this you're accepting the tradeoffs of the additional complexity. Definitely more nuanced than "no migrations = easier" though.

k__ 6y ago

Are data migrations easier with MongoDB/DynamoDB? They don't really have a schema.

solumos 6y ago
Yeah, just shovel the data in. No schema, no problem. The tradeoff is that if you're actually relying on the structure of that data, you're introducing risk since its changes aren't being intentionally managed at the DB level.
- k__ 6y ago
  How do you work around that with databases that don't have a (formal) schema?
  - solumos 6y ago
    You dont. Use DBs with schemas for structured data.
siliconwrath 6y ago
With document DBs you still have "schemas", they just won't be explicit at the database layer. If you rename a field your app uses, you will end up having to either rename that field in every document, or punt on migrating and instead ensure that your app can handle either field name when reading. While the latter may sound easier (and often is easier when you're starting out and iterating quickly), you will soon start to feel the pain of managing dozens of different versions of your "schema" at the app layer. I've always found it easier to only have to deal with a single schema at a time in any situation where you can't just regenerate all records on the fly, like a cache, and for which you don't care about what the specific schema was at a point in time. If you actually do care about what the schema was at any point in time at the app layer, your app WILL need to account for any version of the data—but if you need this you're accepting the tradeoffs of the additional complexity. Definitely more nuanced than "no migrations = easier" though.