6 ms·
Hoo boy. Here we go. The problem with AWS reps is that they only see everything as working perfectly, with no possibility for downtime of their services. RDS i
by falcolas 11y ago
Hoo boy. Here we go. The problem with AWS reps is that they only see everything as working perfectly, with no possibility for downtime of their services.
RDS is great, but only to a certain level. You'll still need to pull it off RDS once you reach that service's capacity (much sooner than their 10m user mark). They also keep pushing Aurora, but without telling us what the tradeoffs are for the high availability. Based on the responses so far (MySQL backed by InnoDB), it appears to be based on a technology similar to Galara, which has a lot of caveats for its use, especially with multiple writers.
Don't depend on Elastic Scaling for high availability - when an AZ is having issues, the AWS API will either be down or swamped, so you want to have at least 50% extra capacity at all times, if you want high availability.
Using their scaling numbers, your costs start spiking at 10 users. Realistically, with intelligent caching (even something as simple as Nginx caching), you can easily support several thousand users just fine with a t2 style instance, either a small or micro. Splitting services onto different hosts not only increases your hosting costs, it increases the workload on your developers/admins and likeliness of failure.
DR: Don't wait until you have over a thousand users to have multiple instances in different AZs. The cost of duplicating a t2.small across an AZ is small compared to lost users or sales.
Automation: Be prepared for vendor lockin if you use Amazon's solutions. Also be prepared for their APIs being unavailable during times of high load or during AZ failures.
> Lambda [...] We’ve done away with EC2. It scales out for you and there’s no OS to manage.
The biggest problem with Lambda right now are the huge latency costs with cold lambda instances. You'll get a pretty good 95% percentile response times, but that other 5% will be off-the-chart bad.
In summary, AWS has a lot of great toys, and can absolutely be used for scaling up to silly levels. However, most who have done this degree of scaling do not do so using AWS tools.
- late2part 11y agoAgreed - they are a great solution for small teams that are growing fast and don't have predictability. But, once you have some level of predictability and scale, it makes sense to move off to something much higher performance and lower cost. Until you become a decrepit Fortune 50 company and can't manage an IT department due to bloat, and it's cheaper to outsource.
- rjsamson 11y agoCurious what you see out there that's higher performance and lower cost than AWS? In my experience it's been a great fit for small apps all the way up to large complicated applications at scale - and once your infrastructure is large enough you're buying reserved instances anyway at anywhere between a 33% and 70% discount.
- vidarh 11y agoYou can beat AWS on cost with pretty much any hosting provider (with some exceptions - e.g. Rackspace seems almost proud to be expensive). The 33% to 70% "discount" doesn't mean much when you then tie yourself into long term costs that are far more limiting than most manage hosting providers - so much for benefits of being able to scale up and down. What really kills you on AWS are the insane bandwidth prices. Buying bandwidth elsewhere is often so much cheaper than AWS that the difference in bandwidth costs alone more than finances the servers.
- semerda 11y agoHow is Netflix able to manage this so effectively and still serve ~30% of US traffic off AWS? I've heard the non-AWS folks talk of these vendor lock ins or long term costs but aren't those irrelevant in 2016+? eg. microservices to reduce the issue of vendor lock in and long term costs on infrastructure that goes out of date every 2-3 years is a poor planning indicator no?
- noir_lord 11y agoI suspect Netflix is paying something a lot closer to AWS cost price than any of us will get. TBH The cost of AWS isn't what concerns me so much as the massive vendor lock-in.
- grepory 11y agoVendor lock-in is an unavoidable cost of doing business. Even if you build literally everything yourself, which you shouldn't, you still have resources, processes, apis, automation, expertise amassed around a specific set of operating constraints. Not only that, but if you invest significantly in any single technology, migrating to another technology is always going to be an extreme effort. Having led migrations from datacenters to AWS, AWS to Digital Ocean, RabbitMQ to NSQ to SNS+SQS, etc., I can say at this point that I do not believe in vendor lock-in as a legitimate reason to disqualify any particular solution.
- LoSboccacc 11y ago> Realistically, with intelligent caching (even something as simple as Nginx caching), you can easily support several thousand users just fine with a t2 style instance agreed, the article approach to scalability is to throw silly amounts of money at the problem, instead of going for an architecture to squeeze first every bit of performance out of the app. true this approach is pretty simple and works for any kind of application, but the RDS will hit connections cap quite fast if on just throws instances at the problem. edit: yep, just noticed this comes from a Amazon Web Services Solutions Architect, of course the solution is to throw money at them
- falcolas 11y ago> of course the solution is to throw money at them Yup. They put out a white paper at one point on surviving DDOS attacks on AWS which amounted to "out-scale the attack". AKA the Wallet based DDOS.
- semi-extrinsic 11y ago> you can easily support several thousand users just fine with a t2 style instance Yep. I've recently load tested (with Locust) a Flask/uWSGI/Nginx webapp I built that does Pandas DataFrame queries based on user input and serves data computed from the query result. I put a bit of effort into profiling and optimizing the Python code^1, and I do caching in uWSGI. Running on the equivalent of a single t2.small instance, it can handle about 70,000 requests per hour, which I figure is the equivalent of a few thousand simultaneous users^2. For just serving a dynamic webpage from Flask it can handle almost a million requests per hour. ^1 (Surprisingly, a Pandas DataFrame lookup like `df[df.alpha == input]` can be almost an order of magnitude faster if you replace `df.alpha` with `df.alpha.values`.) ^2 (The data it serves is input for simulation codes which take hours to run on the user's hardware, so 30 lookups per hour is probably more than a typical user would do.) Edit: asterisk doesn't work as a footnote symbol here...
- ch8230 11y agoAgreed on Lambda latency costs. I've used it to process API calls and I noticed it can add almost half a sec to the response or sometimes even longer.
- cddotdotslash 11y agoThis is a bit of a hack workaround, but all you need to do is have the function run at least every ten minutes. So, using the scheduled task feature, just kick off an event every ten minutes that invokes the function with a custom event that you can respond to instantly within the event handler (to minimize costs). Once you set that up, the function will never scale down and you'll always get hot boot times for just a few pennies extra per month.
- gshx 11y agoThe other problem with Lambda is that you cannot keep persistent connections in a connection pool. It is after all, designed for statelessness. This can be considerable cost for doing calls to other business services (http connection pools) or infra services like databases that all maintain persistent connections.
- falcolas 11y agoAbsolutely. The overhead of re-establishing a secure DB connection for every request is hardly trivial.
- wsh91 11y agoIt would be, if it were necessary, but it's not. (Static initializers or default constructors in Java, for example.)
- falcolas 11y agoQuestion then: How do you omit the overhead of setting up a new socket and all of the SSL handshakes? I'm not concerned about the Java overhead associated with new connections, I'm concerned with the raw connectivity/handshake overhead required with new connections to the DB.
- wsh91 11y ago
- djhworld 11y agotbh if you're that concerned about the 5% of response times being affected by cold Lambdas, then maybe lambda isn't really the solution to the problem you are trying to tackle.