5 ms·

Full disclaimer, I'm very much not a sysadmin or devops guy. However, every team I've been on recently has spent a lot of time struggling with gluing their AWS

Full disclaimer, I'm very much not a sysadmin or devops guy. However, every team I've been on recently has spent a lot of time struggling with gluing their AWS stuff together, diagnosing bugs etc. It didn't seem to save a heck of a lot of time at all. I couldn't figure out AWS. But I could figure out how to host sites on a linux VPS. So what's the story here - is serverless something that only makes sense at a certain scale? Because with tools like Caddy the 'old fashioned' way of doing seems really, really easy.

MattGaiser 5y ago
A lot of it is lack of awareness of things like Caddy or any other tools that simplify the process. I did not know about it until I googled it right now. I have spent days/even two weeks figuring out how to set up Nginx and for all I know I did it terribly wrong. I paired it with other tools that I do not even remember. But I would be starting from scratch again if I needed to set another one up. So a lot might come down to that. I was on a team that transitioned from a owned server to cloud as one day one of the test servers went down and after a week of trying, nobody knew how to fix it. We realized at that point that if a server caused a production error, we were utterly screwed as someone who had left set it up and nobody had a clue where to begin fixing it beyond reading endless tutorials and whatever came up in Google searches. The server infrastructure was cobbled together in the first place and for a period was theoretically maintained by people who didn't even know the names of all the parts. At least with cloud, there is an answer of sorts that can be had from the support team.
- peterbabic 5y ago
  Even with Caddy, there are so many rabbit holes ro get down into. My current one is rootless. I feel like in completely different world compared to rootfull. Learned a ton though
bblb 5y ago
Caddyserver is Apache/Nginx killer and we will talk about Caddy in a couple of years as if it's been always the default ("why did we kept fighting with apache/nginx all those years, silly us"). Seriously. It's just a completely different way to think about web servers and automation. I'm just amazed it took all these years to emerge.
- LoveGracePeace 5y ago
  I love Apache, do not love Nginx and won't be looking at Caddy (it's written in Golang). Apache (even Nginx) are easy to set up reverse proxy and can simultaneously serve static content and serve as the https certificate main, as well as a few dozen other things like load balancing and rate limiting, etc.
  - bblb 5y ago
    I'm just a "user" aka sysadmin who sets up stuff, so can't really comment on the Golang. Even just the fully automated LE cert magic in Caddy is a win in average on-premises environment, where TLS certs are still a big pain and massive manual work drain. And the JSON API automation is a big plus also. At least for our use cases Caddy resolves one of the biggest issues.
  - Ku_las33 5y ago
    >> (it's written in Golang) That means no memory related security bugs, which is a huge plus.
    - ihateolives 5y ago
      Not really. JVM is similar in this regard and one of the thing we wrote in our basic Java course was the code that produces memory leak. And when you have memory leak you also may end up with other fun things like applications behaving unexpectedly in low memory conditions.
      - kafkaIncarnate 5y ago
        Java has a garbage collector, you shouldn't have memory leaks unless you are just incorrectly allocating memory, which isn't technically a leak which refers to allocated memory with no reference. Without a reference the garbage collector would pick it up.
      - tpetry 5y ago
        That‘s a really weak argument. You can build memory leaks in every language and at some point your application will have a problem.
lamontcg 5y ago
> is serverless something that only makes sense at a certain scale? Other way around. With enough scale you should be able to make hosting your own datacenter work. The problem is that the people you hire tend to go off buying too much Enterprise-class shit and Empire building and the whole thing winds up costing 10 times as much as it should because they want stuff to play with to resume stuff and to share risk with the vendor and have them to blame. Only thing Amazon did to build out their internal IT ops exceptionally cheaply and eventually sell it as the AWS cloud service was to focus on "frugality" and fire anyone who said expensive words like "SAN". And they were ordered in no uncertain terms to get out of the way of software development and weren't allowed to block changes the way that ITIL and CRBs used to. I didn't realize how difficult that would be to replicate anywhere else and foolishly sold all my AMZN stock options thinking that AWS would quickly get out competed by everyone being able to replicate it by just focusing on cheap horizontal scalability. These days there is some more inherent stickiness to it all since at small scales you can be geographically replicated fairly easily (although lots of people still run in a single region / single AZ -- which indicates that a lot of businesses can tolerate outages so that level of complexity or cost isn't necessary -- but in any head-to-head comparison the "but what if we got our shit together and got geographically distributed?" objection would be raised).
shepherdjerred 5y ago
I've used serverless on small personal projects where paying for a VPS or EC2 instance would be cost prohibitive, e.g. would I really want to pay $5/mo for a small throwaway weekend project? Probably not. But what if the cost is $.0001 per request? It becomes a very convenient way to make all of my personal projects permanently accessible by hosting on S3 + Lambda. Even in large workloads it makes sense. Much of AWS is migrating from instances to AWS Lambda. There are some workloads where persistent instances make sense, but a lot of common use cases are perfect for Lambda or similar serverless technologies.
kokey 5y ago
Serverless was always around and always had a place. It used to be a shell and sftp account on a shared Linux server where you could upload your html and cgi scripts or php files into your own Apache virtual host and get billed per hit on your web site or bandwidth usage. It now just has a lot of tooling around it with a big learning curve and vendor lock in, but at least you get your hits and bandwidth for cheaper than you used to.
pid-1 5y ago
I use serverless for internal systems. - Small API that is used by a couple people everyday? Lambda. - Need to store some data? DynamoDB. - Need to store some files? S3. - Cron? Step Functions. - Need services to communicate with each other? SQS. Those end up being either free or very cheap. If you have high traffic, serverless is actually really expensive. It is only worth it if you have high scale but unpredictable / bursty traffic. > However, every team I've been on recently has spent a lot of time struggling with gluing their AWS stuff together, diagnosing bugs etc. It didn't seem to save a heck of a lot of time at all. I understand AWS so others don't need to and I'm making the money of my life. All 3 big cloud providers have really terrible developer experience and I feel really sorry for folks who just want to get their shit done. I used to think some other player would come in and offer something much easier and simpler to take over this space, but I'm not really seeing any serious contender. Just low code / no code platforms and managed k8s stuff.

MattGaiser 5y ago

A lot of it is lack of awareness of things like Caddy or any other tools that simplify the process. I did not know about it until I googled it right now. I have spent days/even two weeks figuring out how to set up Nginx and for all I know I did it terribly wrong. I paired it with other tools that I do not even remember. But I would be starting from scratch again if I needed to set another one up. So a lot might come down to that. I was on a team that transitioned from a owned server to cloud as one day one of the test servers went down and after a week of trying, nobody knew how to fix it. We realized at that point that if a server caused a production error, we were utterly screwed as someone who had left set it up and nobody had a clue where to begin fixing it beyond reading endless tutorials and whatever came up in Google searches. The server infrastructure was cobbled together in the first place and for a period was theoretically maintained by people who didn't even know the names of all the parts. At least with cloud, there is an answer of sorts that can be had from the support team.

peterbabic 5y ago
Even with Caddy, there are so many rabbit holes ro get down into. My current one is rootless. I feel like in completely different world compared to rootfull. Learned a ton though

bblb 5y ago

Caddyserver is Apache/Nginx killer and we will talk about Caddy in a couple of years as if it's been always the default ("why did we kept fighting with apache/nginx all those years, silly us"). Seriously. It's just a completely different way to think about web servers and automation. I'm just amazed it took all these years to emerge.

LoveGracePeace 5y ago
I love Apache, do not love Nginx and won't be looking at Caddy (it's written in Golang). Apache (even Nginx) are easy to set up reverse proxy and can simultaneously serve static content and serve as the https certificate main, as well as a few dozen other things like load balancing and rate limiting, etc.
- bblb 5y ago
  I'm just a "user" aka sysadmin who sets up stuff, so can't really comment on the Golang. Even just the fully automated LE cert magic in Caddy is a win in average on-premises environment, where TLS certs are still a big pain and massive manual work drain. And the JSON API automation is a big plus also. At least for our use cases Caddy resolves one of the biggest issues.
- Ku_las33 5y ago
  >> (it's written in Golang) That means no memory related security bugs, which is a huge plus.
  - ihateolives 5y ago
    Not really. JVM is similar in this regard and one of the thing we wrote in our basic Java course was the code that produces memory leak. And when you have memory leak you also may end up with other fun things like applications behaving unexpectedly in low memory conditions.
    - kafkaIncarnate 5y ago
      Java has a garbage collector, you shouldn't have memory leaks unless you are just incorrectly allocating memory, which isn't technically a leak which refers to allocated memory with no reference. Without a reference the garbage collector would pick it up.
    - tpetry 5y ago
      That‘s a really weak argument. You can build memory leaks in every language and at some point your application will have a problem.

lamontcg 5y ago

> is serverless something that only makes sense at a certain scale? Other way around. With enough scale you should be able to make hosting your own datacenter work. The problem is that the people you hire tend to go off buying too much Enterprise-class shit and Empire building and the whole thing winds up costing 10 times as much as it should because they want stuff to play with to resume stuff and to share risk with the vendor and have them to blame. Only thing Amazon did to build out their internal IT ops exceptionally cheaply and eventually sell it as the AWS cloud service was to focus on "frugality" and fire anyone who said expensive words like "SAN". And they were ordered in no uncertain terms to get out of the way of software development and weren't allowed to block changes the way that ITIL and CRBs used to. I didn't realize how difficult that would be to replicate anywhere else and foolishly sold all my AMZN stock options thinking that AWS would quickly get out competed by everyone being able to replicate it by just focusing on cheap horizontal scalability. These days there is some more inherent stickiness to it all since at small scales you can be geographically replicated fairly easily (although lots of people still run in a single region / single AZ -- which indicates that a lot of businesses can tolerate outages so that level of complexity or cost isn't necessary -- but in any head-to-head comparison the "but what if we got our shit together and got geographically distributed?" objection would be raised).

shepherdjerred 5y ago

I've used serverless on small personal projects where paying for a VPS or EC2 instance would be cost prohibitive, e.g. would I really want to pay $5/mo for a small throwaway weekend project? Probably not. But what if the cost is $.0001 per request? It becomes a very convenient way to make all of my personal projects permanently accessible by hosting on S3 + Lambda. Even in large workloads it makes sense. Much of AWS is migrating from instances to AWS Lambda. There are some workloads where persistent instances make sense, but a lot of common use cases are perfect for Lambda or similar serverless technologies.

kokey 5y ago

Serverless was always around and always had a place. It used to be a shell and sftp account on a shared Linux server where you could upload your html and cgi scripts or php files into your own Apache virtual host and get billed per hit on your web site or bandwidth usage. It now just has a lot of tooling around it with a big learning curve and vendor lock in, but at least you get your hits and bandwidth for cheaper than you used to.

pid-1 5y ago

I use serverless for internal systems. - Small API that is used by a couple people everyday? Lambda. - Need to store some data? DynamoDB. - Need to store some files? S3. - Cron? Step Functions. - Need services to communicate with each other? SQS. Those end up being either free or very cheap. If you have high traffic, serverless is actually really expensive. It is only worth it if you have high scale but unpredictable / bursty traffic. > However, every team I've been on recently has spent a lot of time struggling with gluing their AWS stuff together, diagnosing bugs etc. It didn't seem to save a heck of a lot of time at all. I understand AWS so others don't need to and I'm making the money of my life. All 3 big cloud providers have really terrible developer experience and I feel really sorry for folks who just want to get their shit done. I used to think some other player would come in and offer something much easier and simpler to take over this space, but I'm not really seeing any serious contender. Just low code / no code platforms and managed k8s stuff.