4 ms·
I think the key thing is data interchange so you can get off of product X when you have business reasons to do so without running into a huge data migration pro
by mattbillenstein 10y ago
I think the key thing is data interchange so you can get off of product X when you have business reasons to do so without running into a huge data migration project.
Standard SQL databases you can use pretty much the same tooling as if you were running it yourself. Cloud storage, you can download/upload to a new thing, but it may take some time and cost some bandwidth. BigQuery/Redshift you should be storing your data as .json.gz on S3/GCS as a backup anyway, so an export shouldn't be needed -- you'd just import the cloud storage data into another system.
I did a smallish BigTable dump a couple months ago -- kinda a PITA to do -- I don't recommend it; maybe I was doing it wrong, I'm not that familiar with hbase and it was a one-off thing.
I also did a rather small scale data migration out of app engine (~100GB) awhile back -- total PITA -- datastore dumps are protobuf, so you need to build special tooling to import the data into another database.
And generally, I don't use the other services to avoid lock-in. DNS round-robin+nginx for load balancing and ssl termination, Saltstack/Ansible deploying to Ubuntu VMs for config management and deployment. I run the databases myself so I can run it on any cloud I want -- or localhost; having a full dev environment you can create and destroy on your own machine using the same tools that you deploy to is really nice. This also enables doing things like running dev workloads on cheaper clouds (Linode) while running production in a VPC on AWS.