8 ms·
The paradox is that if the coworking space were the cloud, it would just grow with you, and easily house a 50k-strong company, while you'd be paying the same (h
by nine_k 5y ago
The paradox is that if the coworking space were the cloud, it would just grow with you, and easily house a 50k-strong company, while you'd be paying the same (high) rate per employee.
Why would this happen? Because what the company builds is a large and growing money-making contraption right inside the coworking space, also using parts of the building for critical functions, and the door is intentionally kept small. The only way to move that contraption is to dismantle it and reassamble in a much cheaper, purpose-built hangar, rebuilding some of the critical parts along the way. During all that time, the contraption would stop making money.
That's the beauty of AWS business model: it's a no-brainer for a startup to use it, but the startup grows, more financially efficient infrastructure options become unattainable because of the very high cost of the move. This is the best-executed vendor lock-in I know.
- okareaman 5y agoThis seems like a well known trap that people would know how to avoid with a little planning, but I guess they have no one thinking that far ahead
- sneak 5y agoThere's also the argument that some of the cloud advantages (such as AWS proprietary services like SQS) simply aren't directly available as selfhosted replacements. You end up needing a whole new team to do HA/scaling for a core service. There are off the shelf things for, say, S3 and ALB which are entirely workable, but once you start getting into more complicated stuff (like S3's new consistency semantics, or SQS) then you're looking at a whole small company (at a minimum) worth of additional work. It's a non-trivial expansion, even for a large org with lots of money. You can avoid using these sorts of services to maintain flexibility/independence, but you lose out on their unique benefits. This isn't an accident. It's not like you're going to be able to selfhost an SES clone and get the same kind of deliverability percentages as AWS netblocks, no matter how many engineers you throw at the problem.
- toomuchtodo 5y agoThere are ways to engineer around this, but it’s going to depend on balancing risk management with business agility. Maybe you use Backblaze for object storage instead of S3. Maybe you use Cloudflare for CDN and Serverless at the edge. Maybe you use Mailgun or another SES competitor. Containers let you use managed k8s off the shelf anywhere. Lots of options for technologists, PMs, or CTOs to pick from.
- sneak 5y agoGraviton2? Aurora? S3 global read-after-write consistency? Glacier? SQS's ability to scale? Some of the goods you just can't get anywhere else, and they know it.
- deleted 5y ago[deleted]
- rad_gruchalski 5y agoIt can be done. Putting the right abstractions in front of these services makes it possible to migrate and replacements exist. Okay, granted, there will be complexity in self hosted infra but it is possible.
- sneak 5y agoI don't really think anyone with anything less than an 8 figure budget can get the perf/watt of Graviton2. They're literally not available for sale - AWS had them fabbed and then kept all of them to rent out.
- wmf 5y agoIt's called Ampere Altra. Maybe you should have picked a different example like Aurora...
- 5y ago
- toyg 5y ago> I guess they have no one thinking that far ahead Small businesses don't care because they are too busy surviving. Medium businesses don't care because why break what works? Large businesses don't care because their managerial careers promote short-term priorities and marquee "transformation" projects which are buzzword-driven, and cloud is the current one.
- emteycz 5y agoSo what is the problem exactly?
- jjoonathan 5y agoPaying more for less because of marketing. Life goes on. It happens all the time. The trick is to learn and be smarter next time.
- taneq 5y agoThe ones thinking that far ahead aren’t using AWS in the first place.
- abrookewood 5y agoYep, there the ones with stressed out Ops staff running around replacing hard drives at 3 AM and with managers trying to work out how they are going to pass the next security assessment because they don't handle end-to-end encryption or have sufficient controls in place.
- addicted 5y agoIs this common? My company recently shifted from on prem to cloud. And we aren’t primarily a software company (although, much like most companies, we are increasingly turning into one). We outsourced our data center needs, and I don’t remember us ever facing these middle of the night massive breakdowns (we had load balancing between a couple of data centers) and the number of ops issues hasn’t reduced a bit since the switchover. I do think it’s been easier to improve performance for non US customers, because spinning up servers in data centers across the world seems a little easier around the world, but the millions saved by self hosting would have easily paid for someone to spin us up a Singapore and UK colo servers.
- OldHand2018 5y agoIf you spent $X on a SAN rack 10 years ago, that same $X now buys you a fully-managed rack with NVMe drives and redundant, hot-swappable everything. Your Ops staff will get an email about a drive or hardware failure and the tracking number for the replacement part that’s already left the depot and can be replaced at their leisure.
- ZWoz 5y agoYou experienced badly run shop. You can have remote hands (if you don't have your own DC, but buy colocation service). You have spare disks in server. I don't know, how end-to-end encryption is linked to bare metal, those physical servers are several abstraction levels below.
- pie420 5y agoA startup worrying about long term cloud implications is like a hot dog vendor in new York worrying about how his branding will be received in India when his hot dog empire expands there. 99.99% of startups will never have to worry about migrating off cloud, and if they do, they will have succeeded beyond their wildest dreams, and will be once sequential. Nobody is losing sleep over the prospect that their startup will only be worth 22.4 billion in 17 years, but if they eschewed the cloud and built in efficiencies from the get go the company would be worth 23.7 billion. Not to mention the fact that worrying about long term things like that would take focus off the present and in tease chances of failure.
- mlindner 5y agoNitpick, but we don't hear about the companies that failed in the mid-stage because they ended up being consumed by their AWS fees.
- tw04 5y ago>This seems like a well known trap that people would know how to avoid with a little planning, but I guess they have no one thinking that far ahead But you see part of the magic trick was Amazon spent years having companies like Garnter proclaim that cloud was cheaper, and nobody could possibly do on-premises IT cheaper because of economies of scale. As a result you've got CIOs everywhere trying to make a name for themselves by driving their company to the cloud to show immense cost savings. They don't have time to be bothered by the actual financial models or real costs. By the time finance realizes what happened they'll be on to the next gig. It was honestly brilliant, the number of "cloud first" strategies that originated in board rooms filled with people that don't know the first thing about IT is kind of disgusting.
- benhurmarcel 5y agoIt can be a "trap" but also the best decision for a startup. It probably is actually.
- edogrider 5y agoWhen Companies Should Go to the Public Cloud + When they Should Return https://blog.min.io/repatriation_wave/ https://blog.min.io/repatriation_wave/
- 908B64B197 5y agoDropBox did it [0]. [0] https://techcrunch.com/2017/09/15/why-dropbox-decided-to-drop-aws-and-build-its-own-infrastructure-and-network/ https://techcrunch.com/2017/09/15/why-dropbox-decided-to-dro...
- musicale 5y ago"Why Dropbox decided to drop AWS and build its own infrastructure and network" tl;dr: to save money (and to fix issues and irritations with AWS)
- kwindla 5y agoAnd then moved back: https://aws.amazon.com/solutions/case-studies/dropbox-s3/ https://aws.amazon.com/solutions/case-studies/dropbox-s3/
- sam_lowry_ 5y agoI am sure they could negotiate excellent conditions because of their previous setup.
- nerdponx 5y agoFor analytics. I don't think the entire thing runs on AWS anymore.
- AtlasBarfed 5y agoEvery truly large corp should have inhouse options for negotiations with AWS.
- redwood 5y agoAnd stopped innovating at around that time
- gonzo41 5y agoTwitter was on a laptop, then it was in ruby, then it was in java. Companies grow and change, and if your seriously scaling, then your behavior in different environments needs to adjust. It's simply a strategic planning issue for the CIO.
- throwawayboise 5y agoIf moving to another cloud provider, or on-prem, has a "very high cost" then you're doing it wrong.
- abraxas 5y agoI don't think you ever worked on a large enough application to appreciate this complexity.
- scarface74 5y agoHave you been part of any large migration? Even if you try to stay “cloud agnostic” (no using Terraform doesn’t count. Each provisioner is cloud specific), everything takes time at scale. Not to mention data migration costs, project management, regression testing, a company of any scale is probably still partially on prem and has all sorts of interconnecting network points with the cloud provider, IAM, etc.
- throwawayboise 5y agoOf course any migration has costs. But if they are higher because of deep entanglements in specific cloud vendor services, that is a vulnerability. It's already demonstrated that the big cloud providers will cut you off with little warning if you run afoul of their political sensibilities. So the only way to use those services is in the most generic way you can, so that migrations don't cost more because of it.
- scarface74 5y agoAnd if you are using cloud just as an overpriced colo, you aren’t really getting any advantage of being in the cloud. You’re going up spend more money hosting all your own infrastructure on VMs, more time on maintenance, and does it add any business value? Do you not use IAM? Cloud specific infrastructure as code (again TF has cloud specific provisioners), permissions, compliance, training.
- zozbot234 5y ago
- JumpCrisscross 5y agoThe difference is the bottleneck in skilled cloud practitioners. There aren't that many people who can deploy these systems at scale, and they are best rewarded at the centre of the system. So the system centralizes. Enterprises would love to duplicate AWS, but they don't have the ability to each hire similar calibre talent and keep them engaged on their pet clouds.
- mistrial9 5y agoI disagree with this characterization as incomplete. In the post-SUN era there were plenty of people who could build and administer a dozen forms of server OS+hardware, but a confluence of factors caused a greater tide to move greater numbers, in another way. Some commerce did need to scale fast and easily, and AWS provided that with great strength. But I will add that the management and executive branch were constantly and relentlessly seeking to undermine the bargaining power, day-to-day operations authority, and pay of skilled engineers who could do that kind of setup. Not that it was easy or risk-free .. I did see some powerfully strained eyes and nerves on those individuals who built and ran those kind of servers at various times. But do not leave out the manipulative and gaming motives of executive management to dumb-down admin requirements, seize authority via legal means, and seek the lowest wages across borders while doing so ..
- dotcommand 5y ago> This is the best-executed vendor lock-in I know. Another good one is microsoft initially tying excel, word, access, etc to windows. And then IIS, SQL Server, etc to NT. An entire generation of businesses and managers got tied to windows this way. In a way facebook and social media is a form of "vendor lock-in". The network/community itself and your "investment/time" in facebook makes it costly for many to move to another "facebook" or give up facebook altogether. The more time you spend on facebook and the more people you bring onto facebook, the more "locked-in" you are. It's quite ingenious and I suspect facebook understood this very early on.
- deleted 5y ago[deleted]
- fuzzylightbulb 5y agoThe former is lock-in. The later example is better described as "network effects", i.e. Facebook is compelling to you because all your friends are on there, and the more of your group that engages on the platform the more valuable it is to you and the harder it is to get everyone to jump in mass to another platform.
- noduerme 5y agoHaving taken a large organization's entire stack the other way in 2016 - from dedicated metal colocation to AWS, after a series of devastating ddos attacks on our datacenter - I've remained cautious and judicious in how I use cloud services, to attempt to prevent lock-in, for this reason. I build all critical systems (including outbound mail - which requires negotiating with Amazon) on raw EC2 VMs, allocated drives and RDS images, and more or less balance workloads between them manually. This involves a bit more hassle than e.g. putting frontends on elastic beanstalk with automatic load balancing, or using buckets or any of the nicer cloud features... I basically treat AWS as a colocation facility where I don't need to get anyone on the phone to reboot a machine at 3am. I've been told I'm "misusing" or even "misunderstanding the purpose of" the wonderful services provided by AWS. But for me, the upshot is that I could probably port the entire infrastructure to any other server farm within a few hours if necessary.
- justsomeuser 5y agoI like how some of the services are very cheap (S3, Lambda), but then at some point you end up using a bunch of other complements, one of which is very expensive and makes the profit. At which point, as you say, the cost and complexity of a move is huge so you are stuck with a recurring bill. Another interesting thing is that you pay for 100% of a VM virtual CPU even if it is only utilised 40%, but the cloud is selling your other 60% capacity to another user. They have no incentive to charge you “based on CPU usage” for VM’s that are always on, even though it is possible. I think this would be a better approach than Lambda - to dynamically add more cores to your OS as you need them.
- sofixa 5y ago> Another interesting thing is that you pay for 100% of a VM virtual CPU even if it is only utilised 40%, but the cloud is selling your other 60% capacity to another user AWS don't overprovision CPUs outside of the t instance range, in which you get vCPU credits, not full vCPUs. If you rent an apartment but don't use it, is it the landlord's fault you don't? The landlord can't rent the same appartement to another client. Especially on AWS/GCP/Azure, where there are many alternatives ( Lambda, Google Cloud Run, Google App Engine, etc.) where you pay per compute used.
- justsomeuser 5y agoAt the hyper visor level, I was under the impression that many VM’s are scheduled onto the same hardware CPU core. One hardware core can be running 5 VMs at 20% capacity, and each VM owner will pay for 100% of their CPU capacity right? So in this case the landlord is renting your unused apartment to other customers then evicting them faster than you can open the door. My point is cloud is benefiting from that ability and not passing it on the customers, which they could.
- sofixa 5y agoYour impression is correct for VPS-style "clouds", like Digital Ocean and co. Its fully wrong for AWS, where a vCPU you rent == physical core/thread.
- AtlasBarfed 5y agoThe solution is an internal marketplace. Large enterprises formed from conglomeration often flub this. They often end up with a half dozen or more datacenters and ops groups. What do they do? Consolidate them, and a knife fight happens between middle management machiavellis and the one that wins is not necessarily the best on price or features. It's the one who is best at winning reporting hierarchy wars. Instead they should pit these groups against each other as internal options for hosting to various services that provide what is needed for growth, adaptation, or cost. AWS should simply be one of those options. Really it can be the prototyping phase, and then budget squeezing will move the app to other (hopefully cheaper) inhouse options. What is frustrating from my years in the big corps is that AWS was cheaper than the internal "clouds" or other relationships.