8 ms·

Ask HN: Do I need AWS? Or am I thinking this wrong?

• guender 3y ago
  [dead]
• gurchik 3y ago
  If you find yourself constantly reinventing the wheel when AWS offers a managed service that handles that for you, it might be time to switch to AWS. Some examples might be: writing a complicated postgres backup solution instead of using RDS Automated Backups, or running your own Kubernetes control plane instead of using EKS. But if you have an infrastructure that is easy for yourself to maintain, then spend your effort generating value elsewhere. Depending on your current provider, there may be advantages moving your existing bare metal server to EC2. For example, what happens if the drive on your server fails? You'll probably need to write a backup script and write documentation on how to restore from it. If that happened to an EC2, AWS would just restart your instance on a new host and your EBS volume would come with it, and automated snapshots can be set up with a single click. Security groups are simpler to set up than configuring a Linux firewall. Lastly, EC2 has support for automated horizontal scaling with on-demand pricing and Spot instances. But none of these are a must-have, they are conveniences, and you pay a premium cost for it. Hetzner has servers with 64 GB memory for €37 per month, and a similarly-specced EC2 will easily be 10x that cost.
  • pleoxy 3y ago
    > similarly-specced EC2 will easily be 10x that cost. I actually tried this, it's closer to 200-300x. The number of premium add on fees that appear as you build a higher performing EC2 server is just stupid. It's not so bad if you scale out horizontally instead of vertically. But that complicates the architecture when other providers don't charge crazy pricing for vertical scaling.
    • gurchik 3y ago
      Can you give some specific examples of the add-on fees when vertically scaling EC2 instances?
      • pleoxy 3y ago
        Dedicated vCPU requires a "enable dedicated vCPU" service that is $2/hr (per region). It starts billing you as soon as you enable a dedicated vCPU. I had no idea this existed till I saw the charges. Max IOPS without paying for provisioned IOPS is 3000 (gp3 stock setting). Much lower than other providers and will cap your database performance. That's combined too, 1500 read and 1500 write. Throughput is 125mb/s. Hetzner gives you 45k if memory serves, vultr gives you a ton, akamai cloud gives you at least 40k and some instances give you 125k. With throughput in the >3GB/s range. Provisioned IOPS cost thousands per month to reach these levels via io2 volume. Io2 is capped at 45k if memory serves unless you have a bare metal instance. Which is a whole new pricing tier I didn't investigate. I believe it is cheaper per IOP to go wide with gp3, complete with additional compute, than to provision io2. Which really kills the whole "go tall not wide" architecture strategy. Big price bump with dedicated vCPU encourages wide rather than tall also. You can buy a ton of compute for that $2/hr fee. Moral of the story. Don't try to go tall on Amazon.
        • gurchik 3y ago
          AWS has some complicated features that they don't really explain well. In your case, you're talking about "Dedicated vCPU" having a $2/hr fee, and this was confusing to me because I don't recall this fee existing. It turns out they have two completely different offerings for dedicated hardware: Dedicated Hosts and Dedicated Instances. The latter has the $2/hr fee, but I've only ever used the former which has no fee[1]. I'm still not exactly sure what the advantages are for the latter and why those advantages cost $2/hr. 1: When you use a Dedicated Host, you won't see a "fee" on your bill, but you are charged more. As an example, in us-east-1, an m5 Dedicated Host costs 10% more than a m5.metal AKA m5.24xlarge. It's about $0.40/hr more expensive. You'd think it would get cheaper since you're buying in bulk, but I can understand this actually maybe is not preferable for AWS because it is difficult to find a completely unused server on demand.
• movedx 3y ago
  > I'm keeping things simple, so I've got mostly Go services, pg + caching, and a svelte webapp. I deployed my Go services on a low-ish end bare metal provider, and for now it is fine. Deployments are triggered via scripts, and so far so good. Is it sexy, using all the latest and greatest tech? No, its just simple shell scripts. But it works? Don't change a thing. This is perfect. > Am I wrong to think that I could probably scale like crazy and avoid AWS completely with my stack? Why should I pay hundreds/thousands per month plus a premium for bandwidth? I'm also enjoying staying sane avoiding IAM. You're not wrong at all. Check out the hardware stack for Stack Overflow as of 2016: https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture-2016-edition/ https://nickcraver.com/blog/2016/02/17/stack-overflow-the-ar... Don't over think it. Focus on the software and its features. Focus on getting users and ramping up your MRR. Good luck.
  • dangus 3y ago
    One thing I would point out is that the simplicity of StackOverflow's hardware stack means that it would also be very simple to host in a cloud provider like AWS. I would also point out that StackOverflow isn't an incredibly good performer compared to a number of other websites. Check it out on PageSpeed Insights. I also used a couple other tools that suggest that it loads slower in some regions than others.
    • tommek4077 3y ago
      I would not give much on those gambled page speed analysis. It comes down to: is ist fast enough? The answer clearly was yes during all those years for the users.
      • movedx 3y ago
        This is exactly right.
    • sebazzz 3y ago
      > One thing I would point out is that the simplicity of StackOverflow's hardware stack means that it would also be very simple to host in a cloud provider like AWS. Yes, of course. Every cloud provider also does PaaS, but often it is also not the most cost effective option - often it is the most expensive one! The cheapest option is often the option where toy get reliant on cloud-specific sdks, serverless, etc.
• andrewstuart 3y ago
  Sounds like you've answered your own questions.
• kn100 3y ago
  You can definitely avoid the hyperscalers, but unless your ops are very simple, you'll end up paying for it with the cost of managing your infra yourself. Think about resiliency for example. What happens if the DC you're in goes down? Have you got another DC to direct traffic at? Do you need that? These sorts of questions are the ones that being in "the cloud" makes simpler to answer. I'm of the opinion that not every app needs 99.99999% uptime and not every app needs to be in the cloud though, so godspeed to you should you continue with this approach. There's no technical reason why you can't dramatically scale from this point.
• Supply5411 3y ago
  >I'm also enjoying staying sane avoiding IAM. This sticks out to me. IAM isn't crazy, and having access controls in place is going to be the very first thing you want to do when you bring someone else on board. Maybe you're not there yet, but it would be a wise time investment to understand how these work, to the point where they don't feel confusing.
  • fragmede 3y ago
    For a lifestyle business that's never going to grow into needing enterprise offerings, IAM is overkill. Like K8 would be for HN or other simple web forum.
    • Supply5411 3y ago
      I disagree. IAM is like having a house with doors that lock. It doesn't matter how big or small your house is, you're going to want the ability to restrict who can go where and do what. It's part of basic security. If they're not doing that, and their 1 server is talking to AWS resources, it means its using superadmin credentials. If that 1 server is compromised, can you see why that would be a bad idea?
      • syndicatedjelly 3y ago
        But does not using IAM mean that they are not doing anything to handle credentials securely?
        • theshrike79 3y ago
          To keep with the house analogy, not using IAM is like building your own locks for your house. It might be enough for now, but if you grow big enough to be a target it's very likely your home-spun lock won't stand up to professionals.
          • rovr138 3y ago
            I think it's figuring out where to put them or installing them yourself vs building the locks. Most things have accounts (database, servers). Are you using separate ones or a global admin?
• hnpxr 3y ago
  Changing something for the sake of changing something is meaningless. If whatever works for you works - you're fine, don't change a thing. Though, I'd comment that you don't need to go all-in in AWS, you can use EC2 instances the same way as bare metal servers from Hetzner and get some cool benefits, if you need them (and ok with 10x+ extra costs) - easy backup/snapshots, migrations, better server access and management (I like having SSM to connect to the server with MFA).
  • fragmede 3y ago
    If you really want to use AWS, AWS Lightsail is the service to look into.
    • deleted 3y ago
      [deleted]
• dangus 3y ago
  There's nothing wrong with feeling this way, but you're comparing a very small company (your own) to a much larger corporation. The appeal of AWS' "insane" bills are that they are not so "insane" when compared to the salaries required to maintain a home grown stack in-house, especially as complexity increases. It's not just application complexity, it's organizational complexity. Doing everything yourself is fine until that complexity increases and you start finding out that all your dozens or hundreds of expensive employees are spending a lot of time on home grown overhead. For example, when you use shell scripts for your deploys, you can't as easily prove things to auditors like you can with a CI/CD system that's tied in to role based access control. That sort of thing will be dealbreaker for customers who expect your company to maintain industry compliance standards. When you use a product like RDS you can tell your customers that Amazon is the one responsible for handling backups, security patching, etc. Also keep in mind that anyone with a significant amount of AWS infrastructure should be buying reservations and savings plans. The sticker price of AWS is not what corporations pay. As you grow your company, I encourage you to focus on the activities that make your business profitable and deliver value to customers. Every hour you spend upgrading a database or patching an operating system is an hour you could have been spending on developing a unique feature in your product that nobody else has. For now, I'm sure using this setup is much better than getting something too complex for your size, but that can change quickly if you're lucky enough to reach a larger scale.
  • shrubble 3y ago
    In the mid-1990s a large ISP I am aware of, rented "pizzabox" and larger Sun SPARC systems (like SPARCstation 2s; but even the larger systems ran the same OS) to corporate clients. The team of 5 sysadmins setup, installed, secured and managed 2000 such systems, mostly running SunOS 4.1.x , which didn't have nearly the automation that we have now on newer Unix based OSes. Using only the bash shell and the many scripts they wrote for managing them. Guess the profitability of that kind of setup? 1 sysadmin per 400 systems... the kind of overhead people think exists, doesn't.
    • theshrike79 3y ago
      "A large ISP" will have the money to hire a team of 5 experienced sysadmins to be 24/7 on-call to handle 2000 systems. A small to medium business won't have the 500k to 1M per year pay the sysadmins salaries. For them AWS or other managed services are the correct choice.
• evbogue 3y ago
  How simple are you willing to go here? One thought that occurred to me was a reverse ssh proxy into your phone or a small always-on computer at your home. Perhaps both?
• mlboss 3y ago
  You can choose the middle between AWS and baremetal. Something like DO apps. They provide managed deployment, db, object storage etc
• catlover76 3y ago
  If your current setup starts to feel insufficiently robust, then there are several options to consider before AWS, namely things like Heroku, Render, and Fly.io. You'll hear critical reviews about all of these, so do your research first. Personally, I have used Heroku in production at more than one place, and plan on using it again.
• smackeyacky 3y ago
  What are you doing for backups and redundancy? The main reason I use RDS and so forth is I pay AWS to do that for me. I do have a script which backs up my database to an on premise host so I can put it on a tape, but that is faintly ridiculous when AWS is so good.
• fancythat 3y ago
  Using any cloud only makes sense if a) you don't know how to setup things yourself and b) your project is so big, that salaries for ops would be higher than cloud cost. Also, be careful with b) since you still need people that know how to do stuff, otherwise, you will experience either data breach or long downtimes. What most people won't admit when recommending cloud is that, yes, cloud has all sorts of utility built into, like data replication, backups, scaling, etc. but things go wrong in cloud as well and if you don't have good knowledge of how things work, you will be in for a wild ride. IMO, it is much better to learn setup things yourself, even in non-optimized way and have some knowledge on what to do when stuff begins to break.
  • qwytw 3y ago
    > your project is so big, that salaries for ops would be higher than cloud cost. Couldn't it be the other way? On lower scale saving significant amounts of time just to save a few hundreds or thousands might not be worth it. OTH if you're paying millions to AWS hiring an extra person or two to save 20-50% (or whatever) might be a very good deal.
    • Patrickmi 3y ago
      Well kinda, there’re indeed some services can cost more hosting than cloud hosting especially when it comes to geo-location, auto scaling, preventing dead servers but there’re are companies that maintain data centers, etc it depends
    • fancythat 3y ago
      Location plays a role here. If you are in the industry where your people are paid 200k usd minimum then cloud will probably pay off, if you can pay them 1/5 of that, then it is probably another way around. Ahrefs cloud story reminds me of this in some way.
  • Saphyel 3y ago
    This is so wrong. Cloud is for DECLARE the setup, if you don't care or you don't know then avoid cloud. On-premise you can find always dodgy scripts with no copy, no backups, some people praying everyday if the service restart he'll quit, or something working but you don't know why and the worst one, everyone afraid of update anything
• koollman 3y ago
  You seem to manage fine. I do think most projects do not need AWS, and at work we run most of our infrastructure on our own servers (colo, we rent racks and buy servers). Remember to consider more annoying points that can often become hidden costs, or hidden risks: - backups (and more importantly, restores :) ) - maintenance and security (you are in charge of the lower levels too) - decent authentication, authorization, and accounting (AAA) - SLA, both 'in theory' (what does the contract with your provider say) and 'in practice' (how much would an interruption could cost you and what happens if the provider does not respect SLA at all) - how do you handle outages (ideally you want to have a plan to know what to do when the obvious things go wrong. Then for the non-obvious things) - how do you transfer knowledge to a potential new employee. Is there only one person that know how something works. (called 'bus factor') And yes, most of those also apply if you use AWS, but typically they would cover different parts or have different risks. And usually some associated product you can pay to manage that risk
• shrubble 3y ago
  You have it figured out. If you are worried about redundancy, get a second server at a different company with the server in a different part of the country, and back up to it every hour (incremental backups) with a full backup every night, or whatnot.
• alex_lav 3y ago
  What do you think would be more beneficial to your company in the short and medium term?
• wholesomepotato 3y ago
  It's good, don't touch it. If you need AWS you'll know. A low complexity org with low complexity tech stack (both good things) can serve a ton users, and don't have much benefit from huge scale, elasticity and features of AWS. In particular network transfer fees on AWS are ridiculous, IMO.
• subhro 3y ago
  Probably unpopular opinion here, but my policy is, If I can choose to not pay rent, I will choose to not pay rent.
• moltar 3y ago
  I’m biased, but I originally came from the sort of setup you have into AWS. And I’ve never looked back. The time I spent tinkering with bare metal servers is just wasted. AWS has figured it out. Yes, there’s a bit of premium you’ll pay for that. But that’s your own time saved. Only you can tell what’s your time worth. But as an engineer it’s probably a lot and every hour you spend tinkering can probably pay for the whole AWS monthly bill. For me, the strength of AWS is in scalability and reproducibility. I use AWS CDK to define infra as code. Which allows me to make changes with confidence. Test changes in other environments. Use CI/CD and pipelines even on small projects. I only needed to figure this out once. Then all of it is reused across any project from first commit to deployed infra is matter of minutes. When I’m done I can tear it down in minutes.
• px1999 3y ago
  If you can do things yourself or with your staff, you probably don't need aws. But when the cost of you doing something (either direct or opportunity cost) becomes higher than that of using aws (or another cloud provider), you should use them. Yes, you can get really really really far without needing to scale if you spend a lot of energy and effort on optimisation, but that's time you're not doing biz dev, or building out capability. And even now you probably are currently spending a lot of energy on things that are useless outside of saving $ you'd pay to Amazon. As you grow, the cost:value of doing these things yourselfor even in-house changes. If you're successful and scaling, it's often waaaaay easier and cheaper to throw $$ at a problem short term than getting engineers to actually prioritise and look at it.
  • shrubble 3y ago
    This is mostly FUD. A single 8core server made in the last 10 years, with 64GB RAM and SSD/NVME drives, can saturate a 10gb/s link even with some db lookups thrown into the mix. How many lifestyle businesses would even need more than this, aside from backups and redundancy?
    • px1999 3y ago
      Many people underestimate how much easier it is as a business to throw money at things than to actually solve possibly hard problems. "Database failover and scale to z1d.12xlarge" as a strategy is insanely cheap in comparison to spending days/weeks/months trying to prevent spikes from impacting performance. As much as folks here might not want to hear it, throwing an army of mediocre (or, ideally, decent) developers at a problem and paying through the nose for managed infrastructure often winds up being a much (much!) cheaper way to arrive at a good outcome than a smaller number of more skilled engineers with a shoestring infrastructure budget.
      • shrubble 3y ago
        With (if using Postgres) WAL and log shipping, ZFS send/receive snapshots every minute, and other dead-simple setups, a single developer can easily protect against hardware failure, have solid backups, and unless using an absolutely horrific stack, can easily handle a very large amount of business with the hardware I have mentioned. Have you ever worked with mediocre developers? Have you ever been responsible for a built-out implementation of the sort you are talking about? And, have you ever actually run intensive tasks on hardware where you owned the entirety of the system and could have visibility into the full OS? (Many VM providers oversubscribe their CPU and RAM, and, you have no visibility into memory bandwidth performance either.)
• theshrike79 3y ago
  If you're not using AWS services, there is no good reason to stay with AWS. But if you see yourself reimplementing AWS services on your own stack, you should fire up Excel and do the math which is more cost-effective. Your business is delivering product to your customers, not building and maintaining the technology to do so.
• ochronus 3y ago
  What you need to understand (at any given point in time) is: - how is the complexity & scale of your system going? Make projections - identify potential breaking points in this - figure out roughly how long it'd take to migrate to e.g. AWS - the cost of migration (not only the diff between your current costs and AWS costs, but the opportunity costs all the projects you won't do while you're migrating) - now you have your "buffer" figured out, as in, when you need to start acting.
• omeze 3y ago
  This all depends on product requirements. But the single simplest reason for choosing a cloud provider like AWS is reliability. If a region or machine goes down, how do you handle it? AWS has the best reliability among cloud providers, so by choosing them, the odds you even need to think about it are minimized. And they have answers to multi-AZ and regions. But if a few mins of downtime every few months isn’t the end if the world, by all means save yourself the trouble until it is.
• atmosx 3y ago
  How many RPS are you serving? What about caching later for assets in the svelte app?
• tekkk 3y ago
  I also considered AWS but then decided to go with Hetzner. For me it's a learning experience as well so I enjoy figuring out stuff. AWS would have cost 3-4x although I am using RDS for now. I have scripts but use Ansible to setup snapshots to spin new servers from. And Terraform to configure everything. The biggest downside as I see it, is that _if_ I had to scale up the stack there would be issues regarding auto-scaling and additional regions. However, Im not worried about that and if it happens, I probably have money to move back AWS - at least in part. So my advice is that if it works and the servers stay up - keep it. AWS is no silver bullet and moreover, if you have little to no revenue it's better to keep costs down. As a small company I think you get a little more lee-way also regarding server downtime.
• matt_heimer 3y ago
  How baremetal are we talking? Hardware dies. Can you deal with the outage?
• mindcrash 3y ago
  37 Signals is living proof nobody needs a public cloud. And from my own experience, basically what all cloud providers are relying on is ops being lazy. Set up some decent hardware in a good data center, set up a good platform on top of it, automate the hell out of it and with all current open source software you can run everything by yourself. At least, that's something I am currently actively researching because our cloud bills are going through the proverbial roof and I rather would spend that money on our own stuff, than Jeff's next house.
  • datadrivenangel 3y ago
    To be fair, 37 signals still has terabytes of data in S3 according to DHH... So I don't know if it's fair to say that they're cloudless...
    • Agingcoder 3y ago
      Terabytes of data in s3 is not a lot, and can be replicated easily on prem.
  • Spivak 3y ago
    > And from my own experience, basically what all cloud providers are relying on is ops being lazy. This makes no sense to me, I've worked jobs where the app was hosted entirely on perm and jobs where it's hosted in AWS. By far the AWS shops required more work. There's so many more limitations, random footguns, arbitrary rate limits, capacity issues, having to do "cost optimization" because it happens that what is simple and maintainable is expensive. Once you have a setup you like on-prem it will hum along until you decide to break it. S3 is the one of the 7 technical wonders of the world but I would rather run Postgres/Redis/Kafka/RabbitMQ on my own hw than the AWS managed service given the choice.
    • mindcrash 3y ago
      The premise of cloud computing is click and click ... and you magically have a fully operational database up and running without ever thinking about the important bits like scaling and storage, because that's all managed away for you (as a example). That's what I mean with that. Funny thing is that you can easily do that on your own hardware using a solution such as KubeDB just as well these days, after you get through the pain of getting Kubernetes on bare metal operational.
• sgarland 3y ago
  You are doing just fine. If you need to eventually scale or just add reliability, spend some time learning HAProxy and slap that in front of a couple of nodes. If you’re concerned about security, spend some time learning SELinux. I’m assuming you’ve already done the reasonable defaults like public key required for ssh, no root login, fail2ban, etc.
• jesterson 3y ago
  > I could probably scale like crazy and avoid AWS completely with my stack You don't need AWS for that (although they do a good job convincing you otherwise). AWS didn't invent anything. They just simplify (or make complex) systems in order to send you hefty bill.
• demarq 3y ago
  AWS re-invents the wheel so that you don't have to! You could also just use the wheel.
• sjducb 3y ago
  I help big companies reduce their cloud costs. Frequently I recommend moving Postgres and Kubernetes to on-prem. When you’re spending over $1 million per year on either of those then you’ll save a lot of money by moving to on-prem. The ideal cloud lifecycle is to iterate quickly on the cloud while you find product market fit, then when you know what you need start moving to on-prem to save money. I don’t think you’ll ever need to migrate to AWS.
  • hnGuyReaderDude 3y ago
    Is one million really the point where on prem becomes cheaper? I would think it’s much more like 10 million. Between having to pay server engineers and regional distribution and power costs I’d think it adds up to far more than 1 million. Where am I doing my math wrong?
    • Agingcoder 3y ago
      In a large company the engineers , regional distribution etc are already there and pis for by other parts of the company I think. Your math is otherwise correct.
      • sjducb 3y ago
        Exactly.
• delomoto 3y ago
  As a startup it's not about scaling. Either you have to scale than you should have the revenue or VC for doing so or you don't need it. Cloud provider is about were to spend your resources: do you want to build features which make money or do you want to manage your deploy scripts and build monitoring etc yourself.
• deleted 3y ago
  [deleted]
• mark_l_watson 3y ago
  It sounds like you are doing things right! When I pay for cloud infrastructure myself, my first stop is Hetzner because they combine low prices with good service. I do see some complaints on HN occasionally about Hetzner, so do some research. I have never had problems myself with them. I worked at Google for a while ten years ago, so out of nostalgia I sometimes use GCP but I watch my spend.
• vichle 3y ago
  You pay a premium for - Lower chance of fatal bugs/downtime - More reliable load balancing - Proven backup startegies - Not having to learn everything I've spent 7 years at a "unicorn" and the first five years we used Digital Ocean VPSs for everything. My main takeaways are that the smaller providers don't have "real" load balancing and that you absolutely should not manage your own databases or logging/metrics, it's a pain in the ass unless you have a team for it. I spent countless hours on learning infrastructure instead of building the product. It worked out in the end, but we would have gotten there much quicker if we'd paid the premium. If I did it again I'd still use VPSs, but with one of the big players and pay for dbs and observability.
• SamirPaul 3y ago
  [flagged]
  • jcrash 3y ago
    Is this chatGPT?
• lakomen 3y ago
  Imo IAM aka OIDC is keeping it simple. Do you want to develop your own auth solution, or focus on features of your service? For the rest, of course. AWS, GCP, Azure, all too expensive. Chances are very high you won't even need more than 3 machines, if you'd like to play it safe, 1 if downtime is acceptable. The challenge is long term. You need to be able to switch after EOL of your current OS version, and easily. Data is something that's expensive. It requires constant backups. Snapshots. If you can afford it, have a dedicated machine or cluster for data. This way you can swap the webapi layer when you receive better offers over the years (cheaper hardware, EOL OS, etc) Do think in blocks or components and layers, but I'm guessing you do that anyway. What cloud does is it adds convenience for ridiculous costs. Personally I have an incubation server where all trials and or developing projects are hosted. Once and if something takes off, it receives dedicated hardware. In the unlikely case that this dedicated setup isn't enough I'll start thinking about cloud. But even in that case, I'm pretty sure, a private cloud with dedicated personnel is cheaper than AWS etc