20 ms·
Amazon is the new IBM. Knock yourself out and jump into the AWS ecosystem. In a few years down the line, you'll understand that you've lost the leverage you had
by thelastidiot 8y ago
Amazon is the new IBM. Knock yourself out and jump into the AWS ecosystem. In a few years down the line, you'll understand that you've lost the leverage you had to potentially take your public cloud business somewhere else when you have so many dependencies on Amazon tech. Basic principles from my view: don't adopt anything but standard EC2/S3 services and create diversity not only in your teams but in your infrastructure policies.
- potle 8y agowhat are some alternative that one can use for rds postgresql and sqs?
- deleted 8y ago[deleted]
- josephruscio 8y agoFor Aurora/RDS Postgres there's https://www.citusdata.com/ https://www.citusdata.com/ or https://www.heroku.com/postgres https://www.heroku.com/postgres
- 013a 8y agoThere probably isn't a lot of fear with using RDS. They just do management for you, it isn't any fundamentally different technology that would cause heartache during a migration. Just stay away from Aurora if you care about this. SQS: There's AWS MQ, which is based on ActiveMQ and supports AMQP. If you're going with a more CNCF-focused stack and want to use NATS, I'm not aware of any hosted options.
- RhodesianHunter 8y agoYou don't even need to stay away from Aurora ince it's MySql/Postgres SQL API compatible. You would just want to stay away from Aurora specific features like the one in OP.
- koolba 8y agoFeatures like database rewind aren’t something that would be bound to your core app structure either. While you could structure business process’s around it, it’s more of a shit hits the fan restore scenario, not a daily or weekly action. Replicating it on a different platform could also be fine with a combination of logical and physical backups.
- cookiecaper 8y agoPoint-in-time database restores are a best practice that is provided by maintaining database write logs for the timeframe that you expect to have point-in-time restoration for. You don't have to use Aurora to get them, Aurora just has the clicky buttons to make it a clicky-button matter. Any serious DBA should know how to do this without Amazon's platform wrapping it up in a GUI.
- awgupta 8y agoWe've had point in time restore for quite some time. Backtrack is different. It moves you to a different point using the same instance. Since we don't do destructive writes to blocks (it is log-structured storage), we can simply mark a portion of the log as "ignored". It is a server feature, not a UI enhancement.
- d33 8y agoThere's Comcast Message Bus which is self-hosted API-compatible SQS, but I don't yet know how good it is: https://github.com/Comcast/cmb https://github.com/Comcast/cmb
- gfodor 8y agoThis is good advice -- but now that AWS has so many services, it's an extremely complex question. There are path dependencies: adopting an AWS service today may accelerate your product sufficiently to get it to the point of traction and sustainability, where it would have failed otherwise. I think it's important to have a good framework for making these decisions. As you mentioned, EC2 and S3 are on one end of the spectrum: low switching costs and coupling, but also low in terms of differentiation at this point. Whereas other AWS services that are more bleeding edge could actually be the foundation of a temporary competitive advantage for your product, but today would introduce deep coupling of your service with AWS. The trick is basically landing somewhere where you've made good bets: the ideal scenario is that you have a good chance of having an escape plan if/when the situation warrants it, while not missing out on the opportunity to take advantage of the services that your competitors (like, for example, those who are on the "EC2/S3 only" methodology you mention) may be forced to build themselves out of fear. Unfortunately the best way to do this is to be able to predict the future: which services on AWS have high value and high longevity, and will have similar, cheap offerings on their competitors and/or via OSS in the coming years? S3 is an example of a good bet: if you adopted S3 when it first arrived, it may have felt like unnecessary couping with AWS, but it was a huge accelerant if your business relied upon large, reliable file storage. (In fact many startups likely today exist because they were able to bootstrap off of S3.) S3 was a good, long term bet -- now, you can drop in an OSS replacement that is API-compatible in a worst-case scenario. Other services on AWS have come-and-gone through their hayday without going through that transition, because they were either (in retrospect) transitional technology or basically "the wrong thing." So, it comes down to predicting :) A couple of things that increase probability an AWS service is more destined for long term success + commoditzation (in my view): - Built on open protocols/standards - Simplicity in APIs and feature set - Versatility of use-cases - Deeper in the stack, vs user-facing - No existing open source alternative with traction - Backchannel confirmation that Amazon itself is using it :) For example, DynamoDB when it first came out was a huge "no" for me personally -- it was a very quirky design, with complex, bespoke concepts around indexing, with lots of good open source projects offering alternatives, and I heard Amazon didn't trust it internally :) Whereas other services like Redshift ticked all the right boxes out of the gate. So there's some hope of making good bets here, but sometimes you do need to throw caution to the wind if something extremely compelling comes along that feels like it could be a shot-in-the-arm to your product despite not having most of these attributes. Also, FWIW since I like calling my shots: I think the current iteration of AWS server-less tech (while a necessary first step towards understanding the design space, and a technical marvel imho) is probably not a good long term bet as a foundational technology for a product you would like to de-risk in this aspect, but we'll see :)
- erikpukinskis 8y agoHow hard is it to get off Aurora? The article said it’s mysql compatible.
- assface 8y agoAFAIK It uses the MySQL frontend with a heavily modified InnoDB engine that writes to their shared-disk storage.
- paulddraper 8y agoMySQL and PostgreSQL compatible. I've used both in production, with all sorts of advanced features, and (to my surprise) I've never yet found a difference. I would have a hard time leaving Aurora though. Not because of normal "vendor lock-in", but because of the operational worry-free scaling, backups, replicas, point-in-time restores etc. that it offers. Basically, I'm "stuck" on Aurora just because no one else offers MySQL/PostgreSQL databases like that.
- RhodesianHunter 8y agoYou know, I'm totally cool with that. Given the ten-ish years I've been using AWS everything has gotten progressively cheaper and more feature rich. I see no signs of that stopping. I'll happily take on the risk of that theoretical future wherein for some reason I need to unwind from AWS, and in turn continue to save countless hours of my and my engineers' time not spent tackling solved problems, and focusing on our core business instead.
- toomuchtodo 8y agoI’m okay with you being cool with it because where there’s chaos, there’s money to be made. People will eventually move out of AWS (no lead lasts forever), as the cycle goes, and someone will get paid to orchestrate that. I encourage my competitors to marry themselves to a vendor’s proprietary stack. Source: 18 years in tech. Have done my share of unwinding from vendors
- CoolAndComposed 8y agoPerfection is the enemy of good. You have to have a product before improving it. Blah blah blah. God forbid you tie yourself to that git ecosystem. eyeroll
- deleted 8y ago[deleted]
- lccarrasco 8y agoYou can't compare a FOSS like git to a complete stack of proprietary services over which you have zero control or knowledge of how it works under the hood.
- scarface74 8y agoRight because if you trust FOSS you never have to worry about a 2 year old vulnerabilities in how they implemented SSL....
- debt 8y agoIt's really hilarious to me that some new amazing technology comes out and people immediately are like "how can we not use this?" The right approach would be to open an text file and start writing all your hardware drivers, an operating system, a database system, application services etc. so that you'll never ever be locked into anything, and you'll always be 100% in control of everything at all times. Shit, at that point you'll also need to make sure you fabricate your own chips, lay your own pipes, invent your own transfer protocol, etc. as that's the only way to be absolutely certain you won't get locked in. But then at that point, you've created a prison of your own demise.
- spookthesunset 8y agoThat is the thing that the “lock into a vendor” people don’t understand. You build your own stuff and you can lock yourself into your own stuff and it can be incredibly hard to get out. I’ve seen it happen time and time again. Companies write their own frameworks, build systems, hardware provisioning systems, analytics systems, you name it... all to avoid “lock in”. All of that home brew stuff usually sucks. Almost all of it becomes abandonware that one or two people in the company know how to change. Why? Cause all that stuff has nothing to do with how the business delivers value. All of it should have been provided by third party packages. But now the company got big and they are locked into shitty homebrew garbage that would take a massive political and engineering effort to get out of. Moral: it is just as easy to lock yourself into your own garbage as it is to lock yourself into a third party. Leave stuff that doesn’t add value to your business to people whose business it is to build that stuff. Beware of doing everything in-house.
- hinkley 8y agoI hate lock-in and I’ve seen the same problems you discuss here. Nobody likes to feel powerless or over a barrel. The other solution is to use common components (standards or oss) and then you have nobody to rattle your saber at when it breaks. But two of those options are for the most part self-inflicted by the developers. The other one is usually inflicted by management (vendor lock-in).
- 8y ago
- EpicEng 8y ago>In a few years down the line, you'll understand that you've lost the leverage you had to potentially take your public cloud business somewhere else when you have so many dependencies on Amazon tech This is a non-issue for the vast majority of products. You do your eval and you chose what makes sense. Short of hiking prices by 1000% you'll never move anyway (this also applied to building ridiculous abstractions because "maybe we'll want to switch cloud providers someday!" No, you won't.)
- scarface74 8y agoOr "I'm going to put a facade over my database access because I don't want to be locked in to one vendor". Hardly anyone ever changes there database vendors just to save a few dollars.
- tracker1 8y agoIn 23 years I've worked on one project that moved databases. Client requirement, huge system that Oracle was the best match for. Also, one that was a licensed software that needed to support DB2, Oracle and MS-SQL to integrate with various state's agencies. The former was a huge pain, the latter was horrible to write code for. For the most part, you use what works for you and stick with it. The more risky a platform, the less you rely on features like stored procedures, etc. In the end, I'd rather have vendor lock-in than have to write complex queries three times, and other really cumbersome abstractions, slowing things down a lot! Extrapolating to services beyond the DB. In the end, the db migration was far less time/effort than maintaining an untethered software.
- zitterbewegung 8y agoI absolutely agree with you. I think we are in the position that AWS is IBM but in the time of the mainframe. That there were multiple competing mainframes and it wasn't certain who would be the winner but your best bet would be IBM. I wonder if we will be able to recognize Amazon in ten years.
- LoSboccacc 8y agoI second your opinion and add that even s3 is kinda risky. Even if there are open replacement good old network attachhed storage can cover but the most outrageous scale and it’s even more trivial as a building block to replace. RDS is fine too as long as one sticks to standard backends.
- scarface74 8y agoWell, in this case in particular, Aurora is "wire compatible" with MySQL. The only thing you have to do to move away from Aurora is migrate your data to another MySQL instance and change your connection string. In general, if you're moving to AWS and just setting up some EC2 instances and using S3 in the fear of one day in the future it may not be what you want, you are wasting money. There are much cheaper pure hosting solutions. The benefits of using AWS is all of the services they provide that keep you from doing the "undifferentiated heavy lifting".
- cookiecaper 8y ago> Well, in this case in particular, Aurora is "wire compatible" with MySQL. The only thing you have to do to move away from Aurora is migrate your data to another MySQL instance and change your connection string. Don't be fooled, the compatibility is there so that it's easy to port applications to Aurora, not so it's easy to take them out. Amazon still offers MySQL RDS. If you move to the much-more-expensive Aurora, it's generally because your MySQL RDS application is struggling, and Amazon says "Aurora knows special tricks to make the engine faster." Some management headaches, like adjusting for disk growth, are handled by Aurora transparently, which is nice, but for the most part, I saw workloads going to Aurora because they were too slow on standard MySQL. That means that instead of optimizing or fixing your application, you're relying on a single vendor's Secret Go Juice to make your application usable/performant. In such cases, unless the mainline distribution catches up in the meantime, you're going to be stuck because your stuff will be too slow on other stacks. From the risk management POV, Aurora is at best a temporary solution while the program is refactored/optimized to be performant on standard distributions.
- scarface74 8y agoMy first maxim I've been saying in comments is that "no one ever changes Infrastructure just to save a few dollars". You architect with plans to do it, but 99 Times our of 100 you will do a risk rewards calculation and realize it's not worth it. The second maxim is "no one ever got fired for buying IBM". If you're going to bet on a horse, there is always a chance that things go sideways. You might as well bet on the fastest horse with the best record. If you had a choice between IBM, DEC VAX, and Stratus VOS back in the day and you went the safe route to go with IBM, you would have been vindicated two decades later. IBM is still selling compatible systems. I could go through the trouble of staying "vendor neutral" and use tools like Consul+Vault with an AWS backend, Nomad, and Terraform (been there done that) but the time I'm wasting being vendor neutral, I can spin up equivalent services on AWS that cost less and I don't have to maintain them. Most of the things that I depend on AWS for now, I was doing the equivalent of on prem before and after AWS existed. I'm glad to not have to do that anymore and just being able to press a few buttons.
- mkirklions 8y agoDo people use.. other features? I think I used AWS for email octopus and maybe 1 other thing. Idk how I've avoided AWS for so long.
- tekism 8y agoI've avoided AWS because I'm using Azure more (not that I have a problem with AWS)
- duggan 8y agoFolks have been saying this for a decade. Actually they were saying it about EC2 and S3 a decade ago. There are plenty of companies, large and small, running very successfully on all these platforms; sabre rattling notwithstanding.
- mabbo 8y agoIt's a form a technical debt, but on a larger scale. Like all technical debt, it let's you move fast to achieve your goals quickly and get to market sooner. And like all technical debt, eventually you may find good reasons to rewrite, redesign, rethink, in order to achieve some different goals. But you might not have been around long enough to reach that stage if you hadn't taken on the debt early on.
- palisade 8y agoThat is a weird way to look at it. Companies that used IBM hardware and services became successful. If anything they became locked into success with a reliable business partner. It is odd to look at that as a negative thing. Likewise, Amazon has built a reliable and professional platform for cloud computing that is guaranteed to make your business successful for years to come. Why wouldn't you want that? What is the disadvantage really to becoming reliant on a superior architecture and support structure.
- WaylonKenning 8y agoI think you have to bet on some things. You can't just make yourself completely technology agnostic. You have to pick a development language, an IDE, a database technology, an operating system, network infrastructure etc. So then is it worth it spending momentum (dollars + thought) on that stuff, or on solving the business problem?
- ben509 8y agoArchitect your system as microservices. If AWS gets it wrong, it will likely not be bad for every part of your system simultaneously. Your migration strategy will be to stand up new services with a competing provider, and all the other services should see is a DNS change. And, really, if you're doing that from the beginning, you might find you save a lot of money by taking advantage of other providers' strengths.
- segmondy 8y agoNope, we are going to run on Kubernetes. We don't care if it's on AWS, GC, Azure, DigitalOcean. We will abstract all local services so when we move, we can extend the code base with no modification.
- nvarsj 8y agoYou're close. The real problem is Amazon may become your competitor, with their ever expanding reach into different markets. And the last thing you want to do is help your competitor succeed. I think Netflix made an incredible blunder in this regard.