7 ms·

Three independent, but somewhat related thoughts on this topic: 1). On HOWTO articles about infra (1/2): I'd like to see more articles that lead with requireme

• Aeolun 5y ago
  > Is there any sort of comprehensive infra setup guide out there? This would be the AWS certification at their different levels.
  • pyrale 5y ago
    I would tend to disagree. First, because not everyone is on AWS. Second, because even for people on AWS, some of a company's infra isn't on AWS (e.g. gmail). Third because not every infra setup work has an AWS tool. To take a personal example, we use multiple infra providers. Some of our infra is gmail (currently working on automating it with terraform). Some of it is other infrastructure providers that aren't US-based. etc.
    • sokoloff 5y ago
      What’s the “automating gmail with terraform” angle? G-suite automation? Or GCP?
      • pyrale 5y ago
        g-suite.
    • europa 5y ago
      Can you elaborate on the terraform automation for gmail?
      • pyrale 5y ago
        When we first built our user account system, we had to create a google user and add it to relevant groups in admin console whenever we hired someone, before the automation could kick in. We're tired of that so we're looking into using a terraform provider [1], so that we can declare new users and instantiate their resources with a simple PR. [1]: https://registry.terraform.io/providers/hashicorp/googleworkspace/latest/docs https://registry.terraform.io/providers/hashicorp/googlework...
• nicoburns 5y ago
  The problem with AWS is that you can’t really do anything without understanding 1. IAM, and 2. VPCs/networking. And these are probably the two most complicated parts. For DIY you’re probably best off avoiding AWS
  • flatiron 5y ago
    You can do a lot with EB for simplistic projects. But yes without that you are in for a world of pain using anything else
  • jlkuester7 5y ago
    This 1000%. It was amazing how much more comfortable I was with my suite of VPS's when I moved them from AWS to Digital Ocean! I always describe on-boarding with AWS as "drinking from a firehouse". There is so much to understand and it is hard to tell what you should actually care about. DO, on the other hand, has a much simpler set of services that seem much more applicable to 'non-enterprise' usage. (Plus the documentation for DO is great!)
    • laurent92 5y ago
      I have a recruitment problem. If I tell recruits “We use AWS”, they’ll be happy the be waterboarded when trying to do something, they’ll have the feeling of being in connection with the rest of the world, preparing their resume for the future. If I tell them “I have Digital Ocean, I maintain the load balancer myself, the database myself, ELK myself, and you’ll be expert in Postgres, Ansible, Nginx and Debian”, I sound like an old fart. The future for them in AWS is a click-on-the-button deployment and they’ll master nothing.
    • heurisko 5y ago
      Amazon tried to emulate Digital Ocean with Lightsail. It does feel a bit like a Cinderella service however. The versions of the managed databases you can connect are sometimes several years old.
  • exhaze 5y ago
    Sounds about right based on my experience with AWS for past 8 years. Does anyone (besides AWS/TF/Pulumi, for whom it’s not their core product) attempt to solve the networking and IAM? These two areas, esp AWS networking, despite being fundamental building blocks, just never get coverage. I know I’m being a beggar and beggars can’t be choosers, however, I believe that there are folks here who could solve this problem and make a nice living off of it. Hope someone reads this comment and builds something to address this.
• FpUser 5y ago
  There is a life beyond AWS. I host applications (including ones responsible for 10s of millions in revenue) on dedicated servers rented from Hetzner and OVH. Do not even do containers as my normal deployment is server per particular business (with standby) and the cost of renting a dedicated server comparatively to revenue is microscopic. CI/CD / setup from scratch / backup / restore is handled by a single bash script and has virtually zero administration overhead. For my model I see zero business benefits doing anything on cloud. I did some deployments on Azure due to a particular client requesting it but management overhead was way higher and do not even start on costs vs performance.
  • mihcsab 5y ago
    What do you use for backup?
    • lambdaba 5y ago
      I would guess rsync ;)
    • jimsi 5y ago
      I would recommend to use borgbackup - it is very convenient for security (it provides very flexible encryption options with safe defaults) and efficiency (deduplication)
    • FpUser 5y ago
      scheduled jobs running pg_dump and rsync
  • throwaway984393 5y ago
    The business benefits of the cloud are ability to scale, an API for everything, cheap automation, cheap managed services for everything, and good customer support. Like you say for your model none of those things may be important, but they're killer features for others
    • FpUser 5y ago
• johnchristopher 5y ago
  > It's really nice for a super personal project and I'm sure it helped OP gain a lot of operational experience across several domains, but I just find this type of article to be "DYI porn" for a highly specific subset of DYI. I share (some parts) of the sentiment. Fiddling with a VPS and dockers (and Linux generally) landed me my current job but there's a curve and now I feel the need to up my game with a deeper knowledge and understanding of the different pieces and of the overall picture. Otherwise I am just another admin bash monkey.
• jlkuester7 5y ago
  > I just find this type of article to be "DYI porn" for a highly specific subset of DYI. This seems unnecessarily dismissive. (But maybe that means I am part of the 'specific subset'... :) ). IMO there is a big difference between 'tinkering' and 'self-hosting'. Projects you are tinkering with are typically transitory and can be treated as "pets". But in the 'self-hosted' world, stability and ease of maintenance are huge. In that regard I think the overhead of containers makes total sense, especially when running a bunch of stuff on the same server.
  • throwaway894345 5y ago
    I may also be part of that highly specific subset because I enjoy these kinds of articles as well. It’s interesting to explore different solutions and to und west and their pros and cons even if they aren’t motivated by upfront requirements (you’re probably less likely to come up with an interesting configuration in that case anyway).
• michaelbuckbee 5y ago
  Heroku is "AWS on Rails" taken to an extreme and there have been lots of other in-between services and tools. Cloud 66, Laravel Forge and even AWS's Elastic Beanstalk are all part of that spectrum.
• unclebucknasty 5y ago
  I think there's good value to be had from articles like these, depending on where you're starting and how you approach this kind of work. On the requirements side, one thing I would like to see though is an approach for determining how your applications will scale on cloud infra. For instance on AWS, which RDS and EC2 instances will suffice out of the gate, and at what IOPS/bandwidth? And when will I need to scale them? The metric is simply how many users can I support with acceptable response times on a given configuration? Sure, we know that's highly dependent on the applications, stack, etc. But I've often thought there should be some rubric for approaching this that doesn't require going heavy on performance analyzers, load balance testing, etc, which frequently seem like overkill out of the gate when you're making your initial config selections.
• _xnmw 5y ago
  AWS is complexity-as-a-service which abstracts away so much that you pay a HUGE price when it's time to scale -- and it's a confusing matrix of tradeoffs. I find it easier to login to a server, "sudo apt-get install" and tail some logs, than to try to manage the monstrosity that is AWS. Products that go horizontal-cloud-first are easy to scale, but burn an order of magnitude more money, with far worse realtime performance. At the end of the day, AWS is just an abstraction on top of CPU, RAM and disk -- if your app makes bad use of underlying CPU, RAM and disk, then no amount of cloud magic is going to fix that, it'll just mask the problem and you'll end up paying an arm and a leg. AWS/GCloud are actually incentivized to make it hard to debug and optimize performance, they would want you to waste resources on their platform. See also: https://news.ycombinator.com/item?id=29660117 https://news.ycombinator.com/item?id=29660117
  • mirekrusin 5y ago
    Exactly, at this decade it's still a ripoff = huge margin for them, but market is there, one can't deny that. It'll even out eventually so they're on par with smaller shops and dedicated/colocation, eventually they'll have no choice but to cut margins and leverage scale to kick out smaller competition, they'll delay it, of course, as much as possible with lock-in dark techniques etc – but at the end there is no reason, in theory, why they should not be more attractive price wise with anything else you can come up with. At this decade-tick timeline they're happy taking huge margins and people just throw money without much of first principle etc thinking.
    • laurent92 5y ago
      No: - They’ll leverage GDPR to make it only possible to comply to all privacy standards (SOC, credit card management…) if you use AWS, - Worse, they’ll provide special Linux repos with vulnerability fixes that only they provide. Log4j can make all your distributions obsolete in a single day, and only Amazon Linux users will have that patch, this quickly. - Then insurances will stop covering people who deploy their software manually, because vulnerabilities are too prone to happen.
  • whoevercares 5y ago
• lbriner 5y ago
  2) I think this is a bit like asking, "Why isn't there a comprehensive guide to art" Infra is massively complicated. Not only do you have competing frameworks/modules, you also have these change in different releases of Linux. Some of these are simpler, some are more comprehensive. Some are really hard to use but powerful/well-maintained, others are simple but might not be so good. Some perform well at a cost, others might perform relatively less well but are easier. Sometimes it is worth the effort setting up ufw on a server, sometimes it isn't. BY the time you got to something that was simple enough to create a guide from, and which didn't change each week when some vendor renames a configuration option, it would be very high-level and possibly not very useful.