17 ms·
AWS Database Migration Service
- dantiberian 11y agoThe truly interesting thing to me about this product is the pricing. Competitors in this space sell software starting at ~$5,000 all the way up into 5, and I presume 6 figures. AWS is giving away their software and only charging for the hardware to run this migration software on. The catch is that you need to be migrating to AWS resources which you are paying for. I'm sure this tool will have less features than the more established competitors, but for customers that fit the sweet spot, it will be very attractive. Disclosure: I know some of the people at http://www.dbvisit.com/ http://www.dbvisit.com/, a database migration company, and competitor to this.
- Mahn 11y agoWell it's in Amazon's best interest to get people to use and migrate to their infrastructure at the end of the day, so it's no surprise that a service that helps you move to AWS is competitively priced.
- tehalex 11y agoYou can migrate in our out of AWS, but one or both of your endpoints needs to be on AWS.
- LeoPanthera 11y agoI wonder if you could realistically do a two stage migration, X -> Amazon -> Y
- tbrock 11y agoAmazon probably wouldn't care if you did. They still get paid to shuffle the bits.
- Artemis2 11y agoWhat they really want is keep you on RDS, not just get paid for the migration bandwidth.
- ProblemFactory 11y agoSure. But by forcing you to go through RDS even if you migrate X->RDS->Y, they are halfway there. In the process you need to set up an AWS account, create a RDS database, import your data into it... and perhaps you stop there despite the initial plans. Or use RDS for your next project, since you are already more familiar with it.
- swehner 11y agoThe lock-in is where the catch is right.
- matt_wulfeck 11y agoThere's not a whole lot of lock-in using an rds instance. It's MySQL.
- nkozyra 11y ago... with automatically managed updates, backups, monitoring, auto-scaling, auto-replication ... The managed services are a huge part of the lock-in with AWS services, because even though they generally cost more than the competitors, they can often come out equal after accounting for those aggregate hours lost to doing those things.
- rubiquity 11y agoThat's what you call "paying for a service." Lock-In is an entirely different matter.
- nkozyra 11y agoLock-in happens when migrating would be expensive, too. And leaving a managed service like this could introduce substantial cost.
- Johnny555 11y agoWhen the vendor lets you export your data in a form that you can easily import into your own copy of the exact same software, I fail to see how that is "lock-in". If the vendor increase the cost of the service or changes other terms in a way that's unfavorable to you, you can easily leave, that's the opposite of lock-in. You can't really accuse a vendor of lock in just because his product is priced lower than it would cost you to do it yourself.
- dsavinkov 11y agoI wonder, if it will be possible for me to "rollback" back to my "plain" database server if level of service does not satisfy me.
- toomuchtodo 11y agoNot unless you're ready to try to create a physical host replica, and then fail over to it as a master during downtime.
- aidos 11y agoIs it possible to run a read replica outside of RDS these days?
- deleted 11y ago[deleted]
- IgorPartola 11y agoI don't see why not. Replication is just a command stream over a network. You can secure it over the daabsse's built in TLS support or over VPN. The only question is bandwidth and latency between your DC and AWS's.
- aidos 11y agoI believe AWS stop you from connecting to anything outside your RDS subnet.
- IgorPartola 11y agoThen get you outside DB server via a VPN.
- aidos 11y agoUnless something has changed, that's the problem. When I last looked RDS was a sandboxed environment. You can't ssh to it or connect it to a VPN.
- drewpc 11y agoLooks like this could be a better way to migrate between major versions of PostgreSQL on RDS than rolling your own.
- znep 11y agoIf you didn't catch it, AWS now supports upgrades between so-called "major versions" (eg. 9.3 -> 9.4). https://aws.amazon.com/about-aws/whats-new/2015/11/rds-postgresql-now-supports-point-and-click-upgrade-from-postgresql-93-to-94/ https://aws.amazon.com/about-aws/whats-new/2015/11/rds-postg...
- univerio 11y agoWhen 9.5 lands in RDS, they will inevitably take six months to roll out the major version upgrade to 9.5 feature, during which time this can be very useful.
- akurilin 11y agoFolks in the know claim that in-place major version upgrades are unadvisable, as it's not a super heavily tested area of postgres development and in the worst case scenario you might end up with unwanted side-effects in the data that can go uncaught for a while.
- drewpc 11y agoI didn't catch that, thanks for point it out. Like another commenter said: this seems to be an in-place upgrade which is not advisable for major PostgreSQL versions. Additionally, it looks like this involves restarting the cluster (downtime). The Database Migration Service looks like it wouldn't require downtime.
- eistrati 11y agoFinally! This is one of the most wanted functionality that we've waited since AWS re:Invent announcement!!!
- znep 11y agoThis has been in beta for a while, the key thing lacking for our uses, at least with postgres, was full support for DDL. The lack of "true" replication into postgres RDS instances is one of the things blocking us from fully using it.
- williesleg 11y agoHow do you back all this stuff up?
- josephidziorek 11y agoWhy is Amazon Redshift not a target for this service?
- georgewfraser 11y agoSyncing to redshift for reporting is a somewhat different use case - you don't always want to sync everything, you want to sync in larger batches, the "migration" never ends. There are several startups that will do this for you, including mine (Fivetran).
- shrikant 11y ago..or a source. I've deployed a Redshift-based analytics application, and we're discovering operationally that -- for some workloads -- we would be better served by a Postgres RDS instance instead. It would be nice if I could one-click (or few-click) migrate everything from our Redshift instance to RDS.
- jrgoj 11y agoWhat are my chances of a relatively successful SQL Server migration with several hundred tables and complex views to PostgreSQL using their migration tool?
- manigandham 11y agoI tried this, SQL Server isnt supported yet... its just not even in the dropdown options. However it is listed in the marketing and product overview text so it seems like it's coming. EDIT: looks like it's there now. will try it out tonight then.
- jrgoj 11y agoGreat! Let me know how it goes. It will be a couple of days before I can try it. Hopefully I can migrate just the schema and cherry pick some relatively static tables. We'll see.
- ams6110 11y agoPostgres and SQL Server have different concurrency semantics, unlikely that you won't have subtle bugs unless your updates are very straightforward.
- jschmitz28 11y agoOne feature that I wish was supported in RDS is to migrate from a MySQL engine to Aurora engine without having to create new database instance at all. It looks like Amazon DMS makes this process easier, but since Aurora is fully compatible with MySQL, I don't see why I have to create a new database instance and point my application to talk to the new server when Amazon DMS could do this behind the scenes on the one I already have. Or am I allowed to set the source and target databases to be the one and the same?
- Xorlev 11y agoYou can snapshot your RDS database and restore it as an Aurora DB. Only caveat is it has to be MySQL 5.6. http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Aurora.Migrate.html http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Aurora...
- jschmitz28 11y agoNot quite the same, unless I'm misunderstanding. I'm talking about an online migration on a single RDS instance with an engine switch handled fully behind the scenes. The link provided doesn't seem to handle data changes since the snapshot, and also requires a switch in the application to point to the new instance which has been restored via snapshot. DMS gets closer than the above link, I think, since it will handle replication onto the target database for you. But, for two fully compatible database engines, it seems like you should be able to make the switch without having to fuss with replication configuration or spinning up a new instance at all.
- ec109685 11y agoThe on disk format for Aurua is likely different.
- Negative1 11y agoHearing a few mixed things and the page doesn't make it clear so can someone clear it up for me, please; if you wanted to, could you migrate _away_ from AWS using this service?
- webdestroya 11y agoYes, you could migrate away. As long as either the source or the target is within AWS, you can use this.
- obisw4n 11y agoDoes anyone know how the MySQL migration & replication works under the hood? How is Amazon doing all this remotely via just a DB connector?
- gedrap 11y agoTypical MySQL replication works when slave is listening to changes on master's binlog. Which, essentially, is a log of all operations performed on the dataset. I did some quick googling and apparently it can be accessed quite easily: http://dev.mysql.com/doc/refman/5.7/en/mysqlbinlog.html http://dev.mysql.com/doc/refman/5.7/en/mysqlbinlog.html so I'd assume that this is how they are doing the replication, or something very similar. As for migration, don't know, you could export the database and listen to binlog but that will lock table for a bit, depending on the database size. But maybe that's acceptable. Would be curious to hear from folks with more DBA experience :)
- grapehut 11y agoOne of the biggest problems I've had with RDS (postgres specifically) is not being able to do incremental offsite backups. I wonder if the database migration service can help with that, as they say the process is "reliable" and "automatically restarts the process and continues the migration from where it was halted"
- deleted 11y ago[deleted]
- darkr 11y agoYou can, you just need to implement it yourself in EC2 (other compute platforms are available) with trigger-based replication, using something like bucardo or londiste. https://aws.amazon.com/blogs/aws/rds-postgres-read-replicas/ https://aws.amazon.com/blogs/aws/rds-postgres-read-replicas/
- Narkov 11y agoThe "Price and Availability" section doesn't seem to mention anything about the price?!
- brianwawok 11y agoFree!
- Narkov 11y agoNot free! They have now updated the post to include pricing details.
- orillian 11y ago"Because you only pay for the compute resources used during the migration process, a terabyte-sized database can be migrated for as little as $3." The above is quoted from an email I received from them regarding the service this afternoon. Note: using continuous replication and adding logging Dashboards to your links will also increase the cost. I seem to recall seeing the dashboards are $3 a piece after the first 3 or something silly like that and then data costs on top of that. My cost so far don't match that though. I'm guess-estimating my DMS cost are going to be about $30 if the $6.48 I've spent in the last 7 days during setup and configuration time averages out.