7 ms·
Amazon’s Scorpion Problem
- ForHackernews 12y ago"While we are in the middle of a massive secular shift from owned data centers to outsourced data centers" A secular shift? As opposed to a religious one?
- adorton 12y agohttp://en.wikipedia.org/wiki/Secular_variation http://en.wikipedia.org/wiki/Secular_variation
- gojomo 12y agoLike many words, secular has other senses. Per a representative definition like… http://www.merriam-webster.com/dictionary/secular http://www.merriam-webster.com/dictionary/secular …Feld is using the sense (3), and especially (3)(c), a quite common usage in economics (as a contrast with transient/cyclical changes). You might benefit from a more catholic appreciation of vocabulary. (And no, that has nothing to do with the Catholic church.)
- ghaff 12y agoSecular also has a meaning of once in a long time or once in an age. One source I looked at says that the Latin root in also the root for "century" in Romance languages. I could argue it doesn't add much to that sentence (which I don't really agree with in any case) but it's not an incorrect use of the word.
- ableal 12y agohttp://en.wikipedia.org/wiki/In_saecula_saeculorum http://en.wikipedia.org/wiki/In_saecula_saeculorum : "The saeculum in Roman antiquity was the potential maximal human lifespan, or roughly a century"
- deleted 12y ago[deleted]
- liveoneggs 12y agowhere do people get these numbers? You should move to physical hardware well before $200k/month.
- opendais 12y agoThey probably assume you have to pay for 24/7 DC tech / sysadmin types to do the racking, setup, etc. that AWS automates and are factoring it into the price. To be fair, that is part of the cost. However, unless you are really big, you will just pay remote hands which is like $100-200/hr. The number of incidents are probably way too few, even at 200k a month, for 24/7 on-site sysadmins/techs.
- __john 12y agoDo you have a source or some insight you could share as to when a company should move away from AWS? A quick google search turns up $1000 per square foot[1] for a buidling a datacenter. There's also a nice discussion on reddit[2], but there's no talk of cost. I think this is an interesting economics? problem and would like to hear more on it if anyone has the time. [1]http://www.webhostingtalk.com/showthread.php?t=920078 http://www.webhostingtalk.com/showthread.php?t=920078 [2]http://redd.it/16m9pq http://redd.it/16m9pq
- liveoneggs 12y agoI can tell you that you can rent an entire rack with power for $500-$1000/month (42U); A decent internet connection from level3 or the colo house internet for another few hundred. So even at $2k/month we've still got $180k/month to buy servers and storage (or $2M for the year). $200k/month is a lot of money.
- eli 12y agoWhat if your usage isn't evenly distributed across a month?
- Zombieball 12y ago
- iancarroll 12y agoI call bullshit on the support and the "knockoff" claim. I've always felt they are the best with support - they've even waived an accidental $120 instance for me. Yhe tools they provide me are exemplary and cannot be replaced IMO.
- toddmatthews 12y ago"AWS is not the best product at anything – most of their features are mediocre knock offs of other products." Does not list a single example.
- dasil003 12y agoElastic Transcoder is a good example. They don't even support variable bitrate, so you have to choose between huge files and video that looks good for a while until you hit some dark scenes or high contrast sequences that exhibit terrible artifacting.
- danielweber 12y agoWhat does this have to do with "scorpion"? From the title, I thought it would be about Amazon competing with its customers, like Netflix using AWS to compete with Amazon Prime, and how its inevitable that Amazon will eventually have to sting Netflix.
- ojbyrne 12y agoFrom the article: "In addition, they always end up competing on every front possible, hence the chatter about Dropbox moving away from AWS since AWS has now come out with a competitive product. It appears that it’s just not in Amazon’s nature to collaborate with others."
- wdewind 12y agoI also was confused but I think he was referring to the constant (alleged) behavior of Amazon towards his portfolio companies in which they act like partners/potential acquirers and then just build the tech themselves.
- x0x0 12y agoamazon now is actively parasitic to marketplace vendors: they find the best selling products and compete directly. At some point this has to impact willingness for small shop owners to essentially show their books to amazon, right?
- opendais 12y agoYes, but that is a MWS issue not an AWS issue.
- femto113 12y agoAnyone who makes the blanket assertion that "AWS is not the best product at anything" is failing to grasp the fundamental advantage of AWS--it is the only (and therefore best and cheapest) "full line" IAAS provider. Obviously simple value added services like Dropbox that primarily leverage just one piece of the cloud (storage) will eventually move to wherever (and everywhere) that they can get their infrastructure cheapest. When complex services like Netflix begin moving off of AWS is when I'd start worrying.
- opendais 12y agoEh? What do you consider "full line"? I'm pretty sure Netflix uses it due to the highly variable usage situation. It never makes sense to have a server sitting idle for 23 hours a day.
- opendais 12y agoI'm amused by the downvotes combined with the unwillingness to define "full line".
- femto113 12y agoI think "full line" isn't an entirely static concept in IAAS, but I'm using it here to suggest there's no other player offering the same breadth of services. I haven't surveyed other providers in detail lately, but I expect for any given competitor it wouldn't take you long to find some AWS service (http://aws.amazon.com/products/ http://aws.amazon.com/products/) that they simply don't have an equivalent for. EDIT: found someone who's made a feature grid here: http://www.planforcloud.com/pages/resources/cloud_services.html http://www.planforcloud.com/pages/resources/cloud_services.h... This doesn't even include more esoteric stuff like Redshift and Elastic Beanstalk.
- bellerocky 12y agoThe other thing that AWS has going for it is widely adopted third party support. Pretty much any software, site, tool or service that has any sort of cloud monitoring, deployment etc feature will be integrated with AWS. This not true for any of AWS's competitors. I've never seen a third-party provide support for an AWS competitor before supporting AWS. I know monitoring tools that do not support Azure, Google Cloud or Rackspace. I know libraries that don't support all of them but always support AWS. In addition most sysadmins and developers will be more likely familiar with AWS's tools, API and dashboard. They'll be more likely to already have their development environment set up with AWS command lines tools. You ever heard of someone with HP cloud tools installed [1]? I frequently browse dotfiles on GitHub and often see non-sensitive AWS credentials and aliases. I've never seen a Rackspace or Google Cloud thing in them. AWS wins on mindshare. It's also a proven system. Infrasture decisions are really important, and downtime is very bad and when you choose AWS you know at the very least that if your shit goes down because of provider issues everyone else will be down too and you can kind of get a pass (although outsourcing downtime is kind of shitty), but if you're not on AWS and your shit goes down because of the provider people will ask you why aren't you using AWS. Your customers and investors might actually be unhappy with you because of this. As for support Google also has terrible support, Rackspace's managed support is very expensive and actually probably not what a competent team would want, otherwise it is easy to get a person on the phone at Rackspace, so that is commendable. Support doesn't scale, but if you are paying AWS a lot I bet you can get support. I don't really like the mono-culture of AWS everywhere, but I feel like the points above need to be raised. A standard on APIs like OpenStack would be nice, but implementations will differ and AWS will fight this kind of change and because of their dominance getting traction will be hard. [1] this is actually a thing: https://docs.hpcloud.com/cli https://docs.hpcloud.com/cli
- andlarry 12y ago> AWS is unbelievably lousy at support. Compared to whom? Google? How many times have you seen tickets like [1] for Android, Gapps, etc? Additionally, the google cloud platform onboarding was so bad, I almost gave up a $500 credit in frustration. [1] https://productforums.google.com/forum/#!topic/gmail/HTX9cLb3E3o https://productforums.google.com/forum/#!topic/gmail/HTX9cLb...
- smacktoward 12y agoRackspace leaps to mind.
- lwhalen 12y agoRackspace's support is indeed fantastic, however their product is significantly more friction-ful to use (I'm currently at a company that has purchased a Rackspace Private Cloud, and uses it in conjunction with their public cloud). It works as advertised, kinda-sorta and with a whole lot of spit, duct-tape, and prayer, but I'd much rather use Amazon's system and deal with one support ticket every other quarter, than the weekly support ticket(s) using Rackspace's product seems to require.
- mark_l_watson 12y agoI also received the free $500 credit and decided not to use it because of an on boarding issue. On the other hand, when Amazon first released EC2, they gave me a $1000 credit to use anyway I wanted, and that I did use. I really like Google products like GMail, Blogger, and G+ and working at Google the infrastructure was more than wonderful, but for external developers the Google Cloud Platform products really could be a lot better.
- blowski 12y agoI have no idea what the OP is talking about. As far as I can understand, the author is saying Amazon (yes, Amazon) is not hard-nosed enough to win a fight with Google and Microsoft. #1 - "AWS is not the low price provider", "Being low priced is in Amazon’s nature so this will be intensely challenging to them". This is true, but as the market-leader, do they need to be? #2 - "most of their features are mediocre knock offs of other products". As @ToddMathews said here (https://news.ycombinator.com/item?id=8104364 https://news.ycombinator.com/item?id=8104364), he doesn't list any examples of these superior cloud services. I can't think of any. Sure, ELB is a knock off of HAProxy, but in that sense, Gmail is a knockoff of Postfix. #3 - "AWS is unbelievably lousy at support", "Support is just not in Amazon’s nature". Really? Compared to whom? In my experience, they have always been one of the best providers of support of any online company. #4 - "Once you are at $200k / month of spend, it’s cheaper and much more effective to build your own infrastructure". And yet many businesses spend more than $200k PCM. Are they stupid? Or is the author wrong?
- Semaphor 12y agoEspecially #3. Amazon is willing to let themselves be scammed before denying free replacement products to customers.
- opendais 12y agoThat isn't true. I've had to issue chargebacks when Amazon's vendor and Amazon couldn't come to an understanding and I never received the software product w/o a revoked key.
- Semaphor 12y agoMight also be country dependent. I'm using amazon.de (besides .com for prime instant) and Germany is known for it's strong consumer protection laws so Amazon might have beefed up their service here. But I've read some articles before from US customers having amazing experiences. edit: Sorry, didn't see "vendor" before. True, when you are using marketplace you seem to be pretty much on your own.
- mark_l_watson 12y agoThat was a strange article. Isn't it a self obvious truth that AWS is most appropriate when you need lots of computational resources in bursts? If you often need many extra servers for a short period of time, then AWS is really good for that. Another thing that AWS provides is some very handy infrastructure APIs like S3, DynamoDB, Eleastic Mapreduce, etc. For flat usage scenarious, then renting beefy VPSs or physical servers is more cost effective.
- callesgg 12y ago$200k that is insane, it is cheaper to go width own hardware long before that. The server room we have at our office have had max 5 hours of total down time since 2011 And we have moved our entire office under that period. I will try to estimate our general costs in EUR. Inital costs for servers, switches, cooling, Racks UPS 150K Replacement of servers every 3-4 years 60K/3 = 20K Internet line = 10K a year Electricity = 2K a year Consulting cost for hardware maintenance = 10K Old servers we put those in an a room a bit away and use them for backups/failsafe. That is about 50K a year. Counting in initial costs and just cause is is till almost nothing compared to 200K a month, I would say that switching to self hosting is financially beneficial when getting to about 10K a month with 3party hosting.
- imjk 12y agoWhere are your costs for labor. Is that all included in your consulting cost? If we as an advertising startup decided to migrate all that internally, I'm sure our costs would go up not only in manpower but also from the distraction of dealing with that rather than focusing on our product.
- callesgg 12y agoThe initial cost was with labor included. The labor is the consulting cost. But to be honest there is not like one has to do much width the hardware. When one hardware server fails all the virtual machines will still just continue on another hardware server. Vmware takes care of that stuff and it just works, at least in our testing of pulling the cords to the server (stuff has not failed yeat). Well if money is no object, you are definitely doing the right thing.
- eco 12y agoWithout numbers showing the actual quantities involved this cost comparison is kind of useless. Not that I doubt going with your own hardware is cheaper at those levels, it's just not a conclusion I can draw from what you've written. Also, as others said, labor costs need to be factored in. How much would a comparable setup cost on AWS per year? Once we know that we can draw conclusions.
- mark-r 12y agoDoes Google also have the "lousy at support" problem? Their DNA has always favored algorithmic approaches that scale vs. human contact which doesn't. I remember hearing horror stories when they first came on the scene.
- callesgg 12y agoI actually find that google has quite good support (where they have support) Google Apps for business for example.
- orandolabs 12y agoWe (Orando Labs, https://orandolabs.com https://orandolabs.com) feel the sting of the scorpion. Check out EnduroSync vs Cognito at http://orandolabs.wordpress.com/2014/07/12/amazon-cognito/ http://orandolabs.wordpress.com/2014/07/12/amazon-cognito/
- guiambros 12y agoI respect Brad and usually enjoy his posts, but this one is full of s... He doesn't give a single example of why "AWS is not the best product at anything". This is obviously true, but the same could be said about Google, Microsoft, or Rackspace. So what's the point? Who has a better offering? The discussion of AWS being more expensive than bare metal is valid, but that's an obvious statement. Amazon uses bare metal (albeit at scale), and adds their application stack + support. You're paying for the convenience of purchasing all services integrated from a single vendor. If your business is simple enough and you can just throw hardware and bandwidth to solve your needs, then bare metal will always be cheaper. But for more complex environments (e.g., multi-regions, global distribution, auto-scaling), or when you need to quickly scale up and down (to accomodate surge in demand, for example) bare metal is not an option. Net net, this sounded like a rant against Amazon's world domination plans, and the threat to the very existence of some of the companies on his fund. He said it best: "We used to think of Amazon as a potential acquirer for these companies, or at least a powerful strategic partner. Now we know they are just using the bait of “we want to work more closely with you” as market and product intelligence." If your business model depends on a strategic partnerships with or being acquired by Amazon, you're doing it wrong.