6 ms·

Time is money, and the more time I spend on infrastructure, the less time I spend on product. And thus is born the incredible demand of infrastructure as a serv

Time is money, and the more time I spend on infrastructure, the less time I spend on product. And thus is born the incredible demand of infrastructure as a service. Thankfully one person's cloud is another person's on prem infrastructure so sysadmin skills will always be in demand. From my perspective in enterprise computing, I now see people taking 2 paths. One where they become super deep sysadmins and work on infra teams supporting large scale deployments (cloud or not) and the other being folks who write code and think of infra as an abstraction upon which they can request services for their code. Both are noble paths and I just hope folks find the path which brings them the most joy.

akira2501 5y ago
That's when it clicked for me.. comparing my hourly salary rate vs. the cost of running these services "in the cloud." Entirely eliminating "system administration" from my duties was absolutely a net win for me and our team.
- rr808 5y ago
  To me System administration is much simpler than cloud administration. Especially for security, I'm never confident cloud applications are fully secure.
  - tpetry 5y ago
    That‘s true! And everyone reading it here is a startup idea: Analyze a cloud infrastructure and report which has access to what, combine that with a simple DSL so i can say that IAM x is only alloweed read-only access to some service. Find ways to break those configured security settings. If on a test run or onboarding this application will find just one security hole (e.g. public s3 bucket) you will have the customer lifetime because he‘s afraid he will make a mistake again.
- marcosdumay 5y ago
  > Entirely eliminating "system administration" from my duties... ... and adding "cloud administration". What is it with people doing completely one-sided analysis even when they experiment the thing by themselves? Is cloud administration less time consuming than system administration? That's not my experience, so I'm quite interested on how it got so.
  - UI_at_80x24 5y ago
    Too many of these people can't/won't/don't see past C-Panel.
  - jhickok 5y ago
    Can be. Paying for SaaS offerings like the gsuite or o365 is a great deal for say, 100 seats, instead of paying someone to administer on prem email. "Cloud administration" can be more work, less work, or about the same work as classic systems administration. That's why carefully running the numbers first should be necessary.
    - marcosdumay 5y ago
      Oh, sure. Offshoring email is much easier than running it yourself. The same isn't true for less standard kinds of service. The more standardized something is the easiest it is to decide what to hire, troubleshoot, and learn to configure your options. The less standardized it is, the harder all of those things become. VMs are very standard, email servers are less so, but not by a huge margin. Web accessible disk space and on-demand interpreters are completely non-standardized and a hell to do anything with. Also, some services do need more upkeep than others. Email is one extreme that requires constant care, file storage and web servers demand much less attention.
  - mark242 5y ago
    > Is cloud administration less time consuming than system administration? Infinitely, and if you look at it from a startup lens it only makes sense. One needs to point only at the recent log4j incident. This is obviously a gigantic black swan event, but even just ongoing security patching at the OS level can be a full-time gig. There is absolutely no substitution for being able to ship code to a platform that just runs it and scales it for you. Andy Jassey had a great slide a few years back at Reinvent, when talking about Lambda -- "in the future, 100% of the code that you write will be business logic". If you really think about that: how many times have you had to write some kind of database sharding logic, or cache invalidation, or maintaining encrypted environment variables, whatever. That idea that you can toss that -- and what that gives to teams, not having to spend massive timesinks and budgets and hiring and all of that on -- effectively -- solved problems, you really start to understand how you can move faster.
- 28304283409234 5y ago
  Counterintuitively, engineers that run their own servers and infra tend to gain a deeper understanding of what it takes to provide an actual running service to end users. And therefor they write software or at least there is better teamwork with "devops" infra folks. This is off course the highly subjective meaning of a greybeard unixadmin.
  - dj_mc_merlin 5y ago
    I'd say that running infrastructure in the cloud still requires the same deep understanding of what's going on under the hood as running your on-prem infra. A lot of annoying things are taken out: some stuff patches automatically, some other things have an easier updating procedure (and obviously the "physical" aspect is taken care of).. but you still only get the basic elements for a proper infrastructure. Your servers need an environment set up, you need to build a network, add load balancing and replication, monitoring etc. etc.. You can provision some of these things from cloud providers, but your infra is going to go to shit unless you actually understand what they're really providing you and how to use it. If the only thing you can do is upload a docker image to a cloud provider and click the "create server" button, then that's not really infra work at all. It's like Wix for sysadmins.
  - oceanplexian 5y ago
    It's also a competitive advantage. Look at Backblaze, their business model simply wouldn't be possible on a cloud provider.
- deleted 5y ago
  [deleted]
- Spooky23 5y ago
  My brother is a CPA running a practice solely focused on auditing and optimizing cloud expense. The amount of money set aflame is astounding. You don’t go cloud to save money. You go cloud to get flexible and reduce capital expense. It’s like leasing a building vs buying. More about tax and accounting.
- nirvdrum 5y ago
  I'm curious how you completely eliminated it. I've heard the claim from others, but it hasn't matched my experience at several companies over the past 13 years (the first place I worked at using AWS was in 2009). While cloud may have a lot of advantages, I don't think it's trivial to run or manage. The AWS dashboard is simply overwhelming. Trying to decide between different, but overlapping services is time consuming. And while you can rebuild somewhat easily, you're also almost certainly going to have to do that as you learn about Amazon's little quirks. Your general RDBMS experience doesn't map to DynamoDB very well and you'll be in for a rough time when you learn you can't just add a new index or whatever. Then you have all these provider-specific APIs. My experience with both Amazon and Google is that their services will return errors that they claim should be impossible, so you get to have fun debugging that in a service that you don't manage. Your application will invariably add a bunch of handling for exceptional cases and accumulate your best guess about how this impossible situation came about. Then you have the constantly shifting devops orchestration tooling space and "best practices". I've lost track of the number of times I've needed to pull my Terraform state file and manually edit this gigantic JSON file because some plugin updated an internal struct in an incompatible way. I'm sure there are people that get an environment up and running in one of IaaS platforms using just the web console. I've never seen it managed that way at any company I've been at. Instead, the devs own the IaaC stuff. It's certainly easier to be in multiple regions that way, but I have a hard time believing any time or money is really saved. Sure, no dedicated ops people, but now your expensive devs have to deal with and probably be on call. Moreover, all that archaic time-sucking Unix admin knowledge acquisition everyone is worried about is just replaced by time-sucking knowledge acquisition of a proprietary service and all its quirks. Maybe we're talking about different levels of "cloud"? I can buy that Heroku is easier than AWS.
- CRConrad 5y ago
  So why are there so many job adverts looking for people with "cloud experience"? If there were no system administration for the user to do, that wouldn't be necessary; they could be looking just for people with experience of running the actual software their job is all about, on top of the "no administration needed" infrastructure.
unknown2374 5y ago
I find it very similar to how understanding of OS and hardware fundamentals can make one a much better software engineer, how infrastructure in the cloud/servers are setup helps make better design decisions. At least in my experience, my hobby of maintaining my own home server helped out immensely in my path in the industry due to knowing what tools are available when working on multi-faceted software designs.
- aledalgrande 5y ago
  It does, but you don't wanna have to deal with it constantly, if you want to be working on a lot of feature work as an application developer.
  - unknown2374 5y ago
    Definitely agree with you on that. Making use of layers of abstraction and delegation is absolutely necessary when working on more and more impactful work.
Ericson2314 5y ago
But the cloud is...full of crap that wastes ton of my time. Simple ass server saves a lot of time. The cloud is also too damn expensive. People are getting ripped off big time and don't want to be embarrassed by the truth, plane and simple.
_xnmw 5y ago
This is a straw-man argument. Of course if the cloud could save me time and energy, I would use it, but it doesn't. In my experience, you spend just as much time in the long run tweaking/configuring the AWS console as you do simply running bash scripts on a baremetal server. That's why "AWS consultant" roles exist as a full-time job. The cloud does NOT save time for me, and it's far worse in many ways: opaque, more expensive, and can lull you into a false sense of security (what do you do, for example, if your cloud providers backups just fail one day?).
- ozim 5y ago
  If you look from business perspective AWS consultant is as opaque as any SysAdmin with bare metal servers. You don't even know if Sys Admin is doing any backups at all. From perspective of a person that does not know anything about administering systems tweaking stuff in AWS is I would say a lot easier than setting up a server properly. So people that know nothing about administering systems pay more because they don't have the knowledge. If you have the knowledge then yes it is cheaper to run your own sever but what is obvious or easy for one person is not really true for someone else.

akira2501 5y ago

That's when it clicked for me.. comparing my hourly salary rate vs. the cost of running these services "in the cloud." Entirely eliminating "system administration" from my duties was absolutely a net win for me and our team.

rr808 5y ago
To me System administration is much simpler than cloud administration. Especially for security, I'm never confident cloud applications are fully secure.
- tpetry 5y ago
  That‘s true! And everyone reading it here is a startup idea: Analyze a cloud infrastructure and report which has access to what, combine that with a simple DSL so i can say that IAM x is only alloweed read-only access to some service. Find ways to break those configured security settings. If on a test run or onboarding this application will find just one security hole (e.g. public s3 bucket) you will have the customer lifetime because he‘s afraid he will make a mistake again.
marcosdumay 5y ago
> Entirely eliminating "system administration" from my duties... ... and adding "cloud administration". What is it with people doing completely one-sided analysis even when they experiment the thing by themselves? Is cloud administration less time consuming than system administration? That's not my experience, so I'm quite interested on how it got so.
- UI_at_80x24 5y ago
  Too many of these people can't/won't/don't see past C-Panel.
- jhickok 5y ago
  Can be. Paying for SaaS offerings like the gsuite or o365 is a great deal for say, 100 seats, instead of paying someone to administer on prem email. "Cloud administration" can be more work, less work, or about the same work as classic systems administration. That's why carefully running the numbers first should be necessary.
  - marcosdumay 5y ago
    Oh, sure. Offshoring email is much easier than running it yourself. The same isn't true for less standard kinds of service. The more standardized something is the easiest it is to decide what to hire, troubleshoot, and learn to configure your options. The less standardized it is, the harder all of those things become. VMs are very standard, email servers are less so, but not by a huge margin. Web accessible disk space and on-demand interpreters are completely non-standardized and a hell to do anything with. Also, some services do need more upkeep than others. Email is one extreme that requires constant care, file storage and web servers demand much less attention.
- mark242 5y ago
  > Is cloud administration less time consuming than system administration? Infinitely, and if you look at it from a startup lens it only makes sense. One needs to point only at the recent log4j incident. This is obviously a gigantic black swan event, but even just ongoing security patching at the OS level can be a full-time gig. There is absolutely no substitution for being able to ship code to a platform that just runs it and scales it for you. Andy Jassey had a great slide a few years back at Reinvent, when talking about Lambda -- "in the future, 100% of the code that you write will be business logic". If you really think about that: how many times have you had to write some kind of database sharding logic, or cache invalidation, or maintaining encrypted environment variables, whatever. That idea that you can toss that -- and what that gives to teams, not having to spend massive timesinks and budgets and hiring and all of that on -- effectively -- solved problems, you really start to understand how you can move faster.
28304283409234 5y ago
Counterintuitively, engineers that run their own servers and infra tend to gain a deeper understanding of what it takes to provide an actual running service to end users. And therefor they write software or at least there is better teamwork with "devops" infra folks. This is off course the highly subjective meaning of a greybeard unixadmin.
- dj_mc_merlin 5y ago
  I'd say that running infrastructure in the cloud still requires the same deep understanding of what's going on under the hood as running your on-prem infra. A lot of annoying things are taken out: some stuff patches automatically, some other things have an easier updating procedure (and obviously the "physical" aspect is taken care of).. but you still only get the basic elements for a proper infrastructure. Your servers need an environment set up, you need to build a network, add load balancing and replication, monitoring etc. etc.. You can provision some of these things from cloud providers, but your infra is going to go to shit unless you actually understand what they're really providing you and how to use it. If the only thing you can do is upload a docker image to a cloud provider and click the "create server" button, then that's not really infra work at all. It's like Wix for sysadmins.
- oceanplexian 5y ago
  It's also a competitive advantage. Look at Backblaze, their business model simply wouldn't be possible on a cloud provider.
deleted 5y ago
[deleted]
Spooky23 5y ago
My brother is a CPA running a practice solely focused on auditing and optimizing cloud expense. The amount of money set aflame is astounding. You don’t go cloud to save money. You go cloud to get flexible and reduce capital expense. It’s like leasing a building vs buying. More about tax and accounting.
nirvdrum 5y ago
I'm curious how you completely eliminated it. I've heard the claim from others, but it hasn't matched my experience at several companies over the past 13 years (the first place I worked at using AWS was in 2009). While cloud may have a lot of advantages, I don't think it's trivial to run or manage. The AWS dashboard is simply overwhelming. Trying to decide between different, but overlapping services is time consuming. And while you can rebuild somewhat easily, you're also almost certainly going to have to do that as you learn about Amazon's little quirks. Your general RDBMS experience doesn't map to DynamoDB very well and you'll be in for a rough time when you learn you can't just add a new index or whatever. Then you have all these provider-specific APIs. My experience with both Amazon and Google is that their services will return errors that they claim should be impossible, so you get to have fun debugging that in a service that you don't manage. Your application will invariably add a bunch of handling for exceptional cases and accumulate your best guess about how this impossible situation came about. Then you have the constantly shifting devops orchestration tooling space and "best practices". I've lost track of the number of times I've needed to pull my Terraform state file and manually edit this gigantic JSON file because some plugin updated an internal struct in an incompatible way. I'm sure there are people that get an environment up and running in one of IaaS platforms using just the web console. I've never seen it managed that way at any company I've been at. Instead, the devs own the IaaC stuff. It's certainly easier to be in multiple regions that way, but I have a hard time believing any time or money is really saved. Sure, no dedicated ops people, but now your expensive devs have to deal with and probably be on call. Moreover, all that archaic time-sucking Unix admin knowledge acquisition everyone is worried about is just replaced by time-sucking knowledge acquisition of a proprietary service and all its quirks. Maybe we're talking about different levels of "cloud"? I can buy that Heroku is easier than AWS.
CRConrad 5y ago
So why are there so many job adverts looking for people with "cloud experience"? If there were no system administration for the user to do, that wouldn't be necessary; they could be looking just for people with experience of running the actual software their job is all about, on top of the "no administration needed" infrastructure.

unknown2374 5y ago

I find it very similar to how understanding of OS and hardware fundamentals can make one a much better software engineer, how infrastructure in the cloud/servers are setup helps make better design decisions. At least in my experience, my hobby of maintaining my own home server helped out immensely in my path in the industry due to knowing what tools are available when working on multi-faceted software designs.

aledalgrande 5y ago
It does, but you don't wanna have to deal with it constantly, if you want to be working on a lot of feature work as an application developer.
- unknown2374 5y ago
  Definitely agree with you on that. Making use of layers of abstraction and delegation is absolutely necessary when working on more and more impactful work.

Ericson2314 5y ago

But the cloud is...full of crap that wastes ton of my time. Simple ass server saves a lot of time. The cloud is also too damn expensive. People are getting ripped off big time and don't want to be embarrassed by the truth, plane and simple.

_xnmw 5y ago

This is a straw-man argument. Of course if the cloud could save me time and energy, I would use it, but it doesn't. In my experience, you spend just as much time in the long run tweaking/configuring the AWS console as you do simply running bash scripts on a baremetal server. That's why "AWS consultant" roles exist as a full-time job. The cloud does NOT save time for me, and it's far worse in many ways: opaque, more expensive, and can lull you into a false sense of security (what do you do, for example, if your cloud providers backups just fail one day?).

ozim 5y ago
If you look from business perspective AWS consultant is as opaque as any SysAdmin with bare metal servers. You don't even know if Sys Admin is doing any backups at all. From perspective of a person that does not know anything about administering systems tweaking stuff in AWS is I would say a lot easier than setting up a server properly. So people that know nothing about administering systems pay more because they don't have the knowledge. If you have the knowledge then yes it is cheaper to run your own sever but what is obvious or easy for one person is not really true for someone else.