3 ms·
can you break up your migrations into smaller chunks that can be run incrementally? There is also: AWS Database Migration Service https://aws.amazon.com/dms/
by andrewfromx 4y ago
can you break up your migrations into smaller chunks that can be run incrementally?
There is also:
AWS Database Migration Service
https://aws.amazon.com/dms/ https://aws.amazon.com/dms/
Azure Database Migration Service
https://azure.microsoft.com/en-au/products/database-migration/ https://azure.microsoft.com/en-au/products/database-migratio...
Liquibase CI/CD for Databases
https://resources.liquibase.com/ci-cd-database https://resources.liquibase.com/ci-cd-database
In PostgreSQL, you can use the "CONCURRENTLY" keyword with the "ALTER TABLE" statement to perform certain operations without locking the table and without requiring downtime.
ALTER TABLE table_name ADD INDEX CONCURRENTLY index_name (column_name);
DynamoDB offers a feature called "Online Indexing" that allows you to create or delete a global secondary index (GSI) without any downtime to your application. This feature allows you to add, remove, or modify an index on a table without any impact on read or write traffic to the table.
aws dynamodb update-table --table-name table_name --attribute-definitions AttributeName=attribute_name,AttributeType=S --global-secondary-index-updates '[{"Create":{"IndexName":"index_name","KeySchema":[{"AttributeName":"attribute_name","KeyType":"HASH"}],"Projection":{"ProjectionType":"ALL"},"ProvisionedThroughput":{"ReadCapacityUnits":5,"WriteCapacityUnits":5}}}]' --online-index-upgrade True
aws dynamodb update-table --table-name table_name --global-secondary-index-updates '[{"Delete":{"IndexName":"index_name"}}]' --online-index-upgrade True
Elasticsearch offers a feature called "Zero Downtime Indexing" that allows you to create, update or delete an index without any downtime or interruption to the search functionality of the index.
To create an index with zero downtime in Elasticsearch, you can use the "alias" feature
PUT /new_index
{
"aliases": {
"my_search_alias": {}
}
}
After indexing data into the new index, you can switch the alias to the new index:
POST /_aliases
{
"actions": [
{
"add": {
"index": "new_index",
"alias": "my_search_alias"
}
},
{
"remove": {
"index": "old_index",
"alias": "my_search_alias"
}
}
]
}
- edweis 4y agoUsing the alias is very handy indeed. I think generally everything you mentioned and linked is the best practice. I am amazed that it still need to be implemented for each of us and there is no third-party to "fix and forget" data migration.