4 ms·

That's when it clicked for me.. comparing my hourly salary rate vs. the cost of running these services "in the cloud." Entirely eliminating "system administra

That's when it clicked for me.. comparing my hourly salary rate vs. the cost of running these services "in the cloud." Entirely eliminating "system administration" from my duties was absolutely a net win for me and our team.

rr808 5y ago
To me System administration is much simpler than cloud administration. Especially for security, I'm never confident cloud applications are fully secure.
- tpetry 5y ago
  That‘s true! And everyone reading it here is a startup idea: Analyze a cloud infrastructure and report which has access to what, combine that with a simple DSL so i can say that IAM x is only alloweed read-only access to some service. Find ways to break those configured security settings. If on a test run or onboarding this application will find just one security hole (e.g. public s3 bucket) you will have the customer lifetime because he‘s afraid he will make a mistake again.
marcosdumay 5y ago
> Entirely eliminating "system administration" from my duties... ... and adding "cloud administration". What is it with people doing completely one-sided analysis even when they experiment the thing by themselves? Is cloud administration less time consuming than system administration? That's not my experience, so I'm quite interested on how it got so.
- UI_at_80x24 5y ago
  Too many of these people can't/won't/don't see past C-Panel.
- jhickok 5y ago
  Can be. Paying for SaaS offerings like the gsuite or o365 is a great deal for say, 100 seats, instead of paying someone to administer on prem email. "Cloud administration" can be more work, less work, or about the same work as classic systems administration. That's why carefully running the numbers first should be necessary.
  - marcosdumay 5y ago
    Oh, sure. Offshoring email is much easier than running it yourself. The same isn't true for less standard kinds of service. The more standardized something is the easiest it is to decide what to hire, troubleshoot, and learn to configure your options. The less standardized it is, the harder all of those things become. VMs are very standard, email servers are less so, but not by a huge margin. Web accessible disk space and on-demand interpreters are completely non-standardized and a hell to do anything with. Also, some services do need more upkeep than others. Email is one extreme that requires constant care, file storage and web servers demand much less attention.
- mark242 5y ago
  > Is cloud administration less time consuming than system administration? Infinitely, and if you look at it from a startup lens it only makes sense. One needs to point only at the recent log4j incident. This is obviously a gigantic black swan event, but even just ongoing security patching at the OS level can be a full-time gig. There is absolutely no substitution for being able to ship code to a platform that just runs it and scales it for you. Andy Jassey had a great slide a few years back at Reinvent, when talking about Lambda -- "in the future, 100% of the code that you write will be business logic". If you really think about that: how many times have you had to write some kind of database sharding logic, or cache invalidation, or maintaining encrypted environment variables, whatever. That idea that you can toss that -- and what that gives to teams, not having to spend massive timesinks and budgets and hiring and all of that on -- effectively -- solved problems, you really start to understand how you can move faster.
  - demosito666 5y ago
    > in the future, 100% of the code that you write will be business logic The present reality of Lambda is quite different though. Even though the code of the function itself is more or less "business logic" (although this is a really meaningless term when we're talking about known programming languages and computers), the scaffolding around it with Terraform/CloudFormation/Serverless/etc. is substantial, riddled with quirks and is really time-consuming to figure out and update. I don't think I spend less time on this accidental complexity now when we have most of our logic in Lambda, compared to the times when we were just running Flask apps in a VM. This is not to mention how hard one has to fight to overcome the limitations of the runtime, e.g. adding some "warmers" scripts to reduce cold-start latency (no, provisioned concurrency doesn't help and is ridiculously expensive). And then comes the bill after you accidentally created invocation loop between two functions.
    - mark242 5y ago
      Of course -- you're still writing code to delete a key in Elasticache within your lambda. You're writing yaml code for deployments. Hence the "in the future" portion of this slide. The scale-to-X and scale-to-zero features of Lambda, along with the guaranteed interface to your lambda with predictable input and output requirements, is incredibly empowering for an engineering team. I can absolutely guarantee that we have spent far, far, far less time maintaining our infrastructure than what we would need to be doing if we had a big-buncha-EC2 setup. Imagine that the environment issues get taken care of, because Amazon has teams and teams and teams of engineers who are working on just that. Cloudflare has the zero-cold-start isolates. All these platforms are heavily invested in making your development and deployment experience as easy as it can be. Concentrate on writing your code, and you'll reap the benefits.
28304283409234 5y ago
Counterintuitively, engineers that run their own servers and infra tend to gain a deeper understanding of what it takes to provide an actual running service to end users. And therefor they write software or at least there is better teamwork with "devops" infra folks. This is off course the highly subjective meaning of a greybeard unixadmin.
- dj_mc_merlin 5y ago
  I'd say that running infrastructure in the cloud still requires the same deep understanding of what's going on under the hood as running your on-prem infra. A lot of annoying things are taken out: some stuff patches automatically, some other things have an easier updating procedure (and obviously the "physical" aspect is taken care of).. but you still only get the basic elements for a proper infrastructure. Your servers need an environment set up, you need to build a network, add load balancing and replication, monitoring etc. etc.. You can provision some of these things from cloud providers, but your infra is going to go to shit unless you actually understand what they're really providing you and how to use it. If the only thing you can do is upload a docker image to a cloud provider and click the "create server" button, then that's not really infra work at all. It's like Wix for sysadmins.
- oceanplexian 5y ago
  It's also a competitive advantage. Look at Backblaze, their business model simply wouldn't be possible on a cloud provider.
deleted 5y ago
[deleted]
Spooky23 5y ago
My brother is a CPA running a practice solely focused on auditing and optimizing cloud expense. The amount of money set aflame is astounding. You don’t go cloud to save money. You go cloud to get flexible and reduce capital expense. It’s like leasing a building vs buying. More about tax and accounting.
nirvdrum 5y ago
I'm curious how you completely eliminated it. I've heard the claim from others, but it hasn't matched my experience at several companies over the past 13 years (the first place I worked at using AWS was in 2009). While cloud may have a lot of advantages, I don't think it's trivial to run or manage. The AWS dashboard is simply overwhelming. Trying to decide between different, but overlapping services is time consuming. And while you can rebuild somewhat easily, you're also almost certainly going to have to do that as you learn about Amazon's little quirks. Your general RDBMS experience doesn't map to DynamoDB very well and you'll be in for a rough time when you learn you can't just add a new index or whatever. Then you have all these provider-specific APIs. My experience with both Amazon and Google is that their services will return errors that they claim should be impossible, so you get to have fun debugging that in a service that you don't manage. Your application will invariably add a bunch of handling for exceptional cases and accumulate your best guess about how this impossible situation came about. Then you have the constantly shifting devops orchestration tooling space and "best practices". I've lost track of the number of times I've needed to pull my Terraform state file and manually edit this gigantic JSON file because some plugin updated an internal struct in an incompatible way. I'm sure there are people that get an environment up and running in one of IaaS platforms using just the web console. I've never seen it managed that way at any company I've been at. Instead, the devs own the IaaC stuff. It's certainly easier to be in multiple regions that way, but I have a hard time believing any time or money is really saved. Sure, no dedicated ops people, but now your expensive devs have to deal with and probably be on call. Moreover, all that archaic time-sucking Unix admin knowledge acquisition everyone is worried about is just replaced by time-sucking knowledge acquisition of a proprietary service and all its quirks. Maybe we're talking about different levels of "cloud"? I can buy that Heroku is easier than AWS.
CRConrad 5y ago
So why are there so many job adverts looking for people with "cloud experience"? If there were no system administration for the user to do, that wouldn't be necessary; they could be looking just for people with experience of running the actual software their job is all about, on top of the "no administration needed" infrastructure.

rr808 5y ago

To me System administration is much simpler than cloud administration. Especially for security, I'm never confident cloud applications are fully secure.

tpetry 5y ago
That‘s true! And everyone reading it here is a startup idea: Analyze a cloud infrastructure and report which has access to what, combine that with a simple DSL so i can say that IAM x is only alloweed read-only access to some service. Find ways to break those configured security settings. If on a test run or onboarding this application will find just one security hole (e.g. public s3 bucket) you will have the customer lifetime because he‘s afraid he will make a mistake again.

marcosdumay 5y ago

> Entirely eliminating "system administration" from my duties... ... and adding "cloud administration". What is it with people doing completely one-sided analysis even when they experiment the thing by themselves? Is cloud administration less time consuming than system administration? That's not my experience, so I'm quite interested on how it got so.

UI_at_80x24 5y ago
Too many of these people can't/won't/don't see past C-Panel.
jhickok 5y ago
Can be. Paying for SaaS offerings like the gsuite or o365 is a great deal for say, 100 seats, instead of paying someone to administer on prem email. "Cloud administration" can be more work, less work, or about the same work as classic systems administration. That's why carefully running the numbers first should be necessary.
- marcosdumay 5y ago
  Oh, sure. Offshoring email is much easier than running it yourself. The same isn't true for less standard kinds of service. The more standardized something is the easiest it is to decide what to hire, troubleshoot, and learn to configure your options. The less standardized it is, the harder all of those things become. VMs are very standard, email servers are less so, but not by a huge margin. Web accessible disk space and on-demand interpreters are completely non-standardized and a hell to do anything with. Also, some services do need more upkeep than others. Email is one extreme that requires constant care, file storage and web servers demand much less attention.
mark242 5y ago
> Is cloud administration less time consuming than system administration? Infinitely, and if you look at it from a startup lens it only makes sense. One needs to point only at the recent log4j incident. This is obviously a gigantic black swan event, but even just ongoing security patching at the OS level can be a full-time gig. There is absolutely no substitution for being able to ship code to a platform that just runs it and scales it for you. Andy Jassey had a great slide a few years back at Reinvent, when talking about Lambda -- "in the future, 100% of the code that you write will be business logic". If you really think about that: how many times have you had to write some kind of database sharding logic, or cache invalidation, or maintaining encrypted environment variables, whatever. That idea that you can toss that -- and what that gives to teams, not having to spend massive timesinks and budgets and hiring and all of that on -- effectively -- solved problems, you really start to understand how you can move faster.
- demosito666 5y ago
  > in the future, 100% of the code that you write will be business logic The present reality of Lambda is quite different though. Even though the code of the function itself is more or less "business logic" (although this is a really meaningless term when we're talking about known programming languages and computers), the scaffolding around it with Terraform/CloudFormation/Serverless/etc. is substantial, riddled with quirks and is really time-consuming to figure out and update. I don't think I spend less time on this accidental complexity now when we have most of our logic in Lambda, compared to the times when we were just running Flask apps in a VM. This is not to mention how hard one has to fight to overcome the limitations of the runtime, e.g. adding some "warmers" scripts to reduce cold-start latency (no, provisioned concurrency doesn't help and is ridiculously expensive). And then comes the bill after you accidentally created invocation loop between two functions.
  - mark242 5y ago
    Of course -- you're still writing code to delete a key in Elasticache within your lambda. You're writing yaml code for deployments. Hence the "in the future" portion of this slide. The scale-to-X and scale-to-zero features of Lambda, along with the guaranteed interface to your lambda with predictable input and output requirements, is incredibly empowering for an engineering team. I can absolutely guarantee that we have spent far, far, far less time maintaining our infrastructure than what we would need to be doing if we had a big-buncha-EC2 setup. Imagine that the environment issues get taken care of, because Amazon has teams and teams and teams of engineers who are working on just that. Cloudflare has the zero-cold-start isolates. All these platforms are heavily invested in making your development and deployment experience as easy as it can be. Concentrate on writing your code, and you'll reap the benefits.

28304283409234 5y ago

Counterintuitively, engineers that run their own servers and infra tend to gain a deeper understanding of what it takes to provide an actual running service to end users. And therefor they write software or at least there is better teamwork with "devops" infra folks. This is off course the highly subjective meaning of a greybeard unixadmin.

dj_mc_merlin 5y ago
I'd say that running infrastructure in the cloud still requires the same deep understanding of what's going on under the hood as running your on-prem infra. A lot of annoying things are taken out: some stuff patches automatically, some other things have an easier updating procedure (and obviously the "physical" aspect is taken care of).. but you still only get the basic elements for a proper infrastructure. Your servers need an environment set up, you need to build a network, add load balancing and replication, monitoring etc. etc.. You can provision some of these things from cloud providers, but your infra is going to go to shit unless you actually understand what they're really providing you and how to use it. If the only thing you can do is upload a docker image to a cloud provider and click the "create server" button, then that's not really infra work at all. It's like Wix for sysadmins.
oceanplexian 5y ago
It's also a competitive advantage. Look at Backblaze, their business model simply wouldn't be possible on a cloud provider.

deleted 5y ago

[deleted]

Spooky23 5y ago

My brother is a CPA running a practice solely focused on auditing and optimizing cloud expense. The amount of money set aflame is astounding. You don’t go cloud to save money. You go cloud to get flexible and reduce capital expense. It’s like leasing a building vs buying. More about tax and accounting.

nirvdrum 5y ago

I'm curious how you completely eliminated it. I've heard the claim from others, but it hasn't matched my experience at several companies over the past 13 years (the first place I worked at using AWS was in 2009). While cloud may have a lot of advantages, I don't think it's trivial to run or manage. The AWS dashboard is simply overwhelming. Trying to decide between different, but overlapping services is time consuming. And while you can rebuild somewhat easily, you're also almost certainly going to have to do that as you learn about Amazon's little quirks. Your general RDBMS experience doesn't map to DynamoDB very well and you'll be in for a rough time when you learn you can't just add a new index or whatever. Then you have all these provider-specific APIs. My experience with both Amazon and Google is that their services will return errors that they claim should be impossible, so you get to have fun debugging that in a service that you don't manage. Your application will invariably add a bunch of handling for exceptional cases and accumulate your best guess about how this impossible situation came about. Then you have the constantly shifting devops orchestration tooling space and "best practices". I've lost track of the number of times I've needed to pull my Terraform state file and manually edit this gigantic JSON file because some plugin updated an internal struct in an incompatible way. I'm sure there are people that get an environment up and running in one of IaaS platforms using just the web console. I've never seen it managed that way at any company I've been at. Instead, the devs own the IaaC stuff. It's certainly easier to be in multiple regions that way, but I have a hard time believing any time or money is really saved. Sure, no dedicated ops people, but now your expensive devs have to deal with and probably be on call. Moreover, all that archaic time-sucking Unix admin knowledge acquisition everyone is worried about is just replaced by time-sucking knowledge acquisition of a proprietary service and all its quirks. Maybe we're talking about different levels of "cloud"? I can buy that Heroku is easier than AWS.

CRConrad 5y ago

So why are there so many job adverts looking for people with "cloud experience"? If there were no system administration for the user to do, that wouldn't be necessary; they could be looking just for people with experience of running the actual software their job is all about, on top of the "no administration needed" infrastructure.