4 ms·

I've noticed any project that involves AWS requires a huge amount of "programming the machine" versus "writing the application". What I mean by "programming th

I've noticed any project that involves AWS requires a huge amount of "programming the machine" versus "writing the application". What I mean by "programming the machine" is doing technical tasks that relates to making the cloud work in the way you want. I've worked on projects in which tha vast majority of the work seems to be endless "programming the machine".

acdha 2y ago
That’s a choice, like using Kubernetes or being a micro service fundamentalist. A decade ago, you had people building in-house Hadoop clusters for tiny amounts of data instead of doing their jobs analyzing that data, and before that you had the J2EE types building elaborate multi server systems for what could have been simple apps. This repeats constantly because the underlying problem is a broken social environment where people aren’t aligned with what their users really need and that’s rewarded.
- andrewstuart 2y ago
  I think it is because cloud computing demands far more programming the machine than deploying to compute instances does.
  - acdha 2y ago
    Does it, though, or are you just more familiar with traditional servers? There are a lot more things to manage in a traditional environment and things like tools are more primitive so you’ll have to invest a lot more time getting anywhere near equivalent reliability or security. That’s especially important if you have compliance requirements where you’ll be on the hook for things provided or greatly simplified by the cloud platform.
    - andrewstuart 2y ago
      >> There are a lot more things to manage in a traditional environment Can you be specific because that does not sound right to me. I know both pretty well and seems to me there is vastly more things that need to be tuned and managed and configured in cloud.
      - chuckadams 2y ago
        How about you give some specifics of things that need constant fiddling in the cloud but are free on bare metal?
        andrewstuart 2y ago
        Well its a good point, it's my feeling at the moment. I think it would be interesting to do a systematic study of the time required for each.
        chuckadams 2y ago
        In a cloud environment, I turn a few knobs and I can command an absurd amount of compute power, have it autoconfigured with all kinds of service discovery and whatnot. Things that would have taken weeks to months to build in a physical datacenter. Some people take that spare time to train as architecture astronauts I guess, and overcomplicate everything with all kinds of fiddly layers and microservices. I get it, it's fun to play with new toys. Doesn't mean the toys are bad, they're just not necessarily ready for production on day 1.
        andrewstuart 2y ago
        My experience on multiple cloud projects over many years is endless fiddling and problem solving and spaghetti.
        MaKey 2y ago
        That's because it's easy to get started with one of the hundreds of services available in the cloud and before you know it your cloud project is using 50 of them that depend on each other.
      - acdha 2y ago
        Here’s a simple one: if I need to say something accepts inbound 443 from 1.2.3.4, I create an AWS security group rule and will never need to touch that for at least a decade, probably longer. If I’m doing the same thing on premise, the starting cost is learning about something like Cisco hardware which costs a lot more to buy and I am now including in my maintenance work. Over a year, I will spend no time on my AWS rule but I will have to patch the physical switches and deal with primitive configuration tools for a complex networked device. If my organization cares about security, I will have to certify a ton of configuration and practice unrelated to packet filtering whereas I can point auditors at the cloud service’s documentation for everything on their side of the support agreement. What about running code? If I deploy, say, Python Lambdas behind an API Gateway or CloudFront, they’ll run for years without any need to touch them. I don’t need to patch servers, schedule downtime, provision new servers for high load or turn them off when load is low, care that the EL7 generation servers need to be upgraded to EL8, nobody is thinking about load balancer or HTTPS certificate rotation, etc. What you’re paying for, and giving up customization for, is having someone else handle all of that.
- chx 2y ago
  > A decade ago, you had people building in-house Hadoop clusters for tiny amounts of data instead of doing their jobs analyzing that data So said Gary Bernhardt of WAT fame in 2015 https://x.com/garybernhardt/status/600783770925420546 https://x.com/garybernhardt/status/600783770925420546 > Consulting service: you bring your big data problems to me, I say "your data set fits in RAM", you pay me $10,000 for saving you $500,000. Which, in turn, have inspired https://yourdatafitsinram.net/ https://yourdatafitsinram.net/
  - acdha 2y ago
    I remember that, too. There were also just a lot of places where it’s like … your data is already in a SQL database, why not get better at reporting queries until you’re running so many of them that you don’t have an easy path to handle them? It was especially tragicomic in places where they were basically hoping not to need to hire analysts with technical skills, which is possibly the most MBA way ever to very expensively “save” money.
abadpoli 2y ago
For many systems, the application and the machine are so intertwined as to basically be the same thing. A lot of web apps are just boring CRUD APIs. The business logic behind these is nothing special. The differentiating factor for a lot of them is how they scale, or how they integrate with other applications, or their performance, or how fast they are to iterate on, etc. The “machine” offers a lot of possibilities these days that can be taken advantage of, so customizing “the machine” to get what you need out of it is a big focus for many devs.
icedchai 2y ago
Yep. I've also worked on "serverless" projects where similar or more time is spent on IaC (Terraform, etc.) compared to application development. All this effort for something that barely gets any requests.
moduspol 2y ago
One of the core value propositions of AWS is to take advantage of the various services to scale more easily with more linear costs than you would be able to otherwise. Writing and maintaining code that sets up and configures S3, SQS, SNS, DynamoDB, and Lambda is a non-zero cost, but if it then reduces the amount of code I have to write and maintain to "write the application," that's often a good thing. These services are typically solving the hard and/or tedious parts of the application, so it's actually quite valuable to just focus on developing the core differentiating points of the application instead of solving / maintaining highly available object storage or message queues.
tiew9Vii 2y ago
> What I mean by "programming the machine" is doing technical tasks that relates to making the cloud work in the way you want. I take a different perspective. When using a cloud, i need to design my application to work within my selected clouds constraints. A lot of the extra work then comes down to if using a cloud, not just a single VPS on a cloud provider, you are dealing with distributed computing and have to deal with all the issues of distributed computing. Consensus, error handling, etc. I agree though, more often than not it's a lot of extra work but mostly due to dealing with distributed architecture/designing for being cost effective. On personal projects I find designing to be cost effective has me jumping through a lot of hoops and doing novel things. It means I can run multi-region apps with performance better than most webapps for cents/dollars. The equivalent apps I work on in my day job can be costing tens to hundreds of thousands of dollars as they don't jump through the hoops I do. "It's not worth it", "let's look at it later", "we'll just do this, it's good enough for now". As it's someone else's money i just think "meh" and move on as isn't my fight when the majority of people don't want to do it.
crabmusket 2y ago
I feel this strongly. If the project in the OP is meant to run "off the shelf" servers like Express or Next, isn't this a way to reduce the amount of "programming the machine" that needs to be done? That's what I'm hoping. Our approach to lambdas is to delegate all the real work to a "library" and just write a very thin adapter around it dealing with the Lambda details (event format etc.). Hopefully this can take care of as much of the wrapper as possible?
JackSlateur 2y ago
How long would it take to setup an event-driven python function ? I know aws lambda and terraform and, to me, it takes a couple of minutes at most. Indeed, I had to learn it ! Just as I had to learn how to code in Python (which makes me far more effective now), just as I had to learn LVM2 or iproute2 or C or whatever. Once the learning curve is processed, AWS allows you to increase your productivity. Of course, and this is very important, you shall still use the right tool for the right job !

acdha 2y ago

That’s a choice, like using Kubernetes or being a micro service fundamentalist. A decade ago, you had people building in-house Hadoop clusters for tiny amounts of data instead of doing their jobs analyzing that data, and before that you had the J2EE types building elaborate multi server systems for what could have been simple apps. This repeats constantly because the underlying problem is a broken social environment where people aren’t aligned with what their users really need and that’s rewarded.

andrewstuart 2y ago
I think it is because cloud computing demands far more programming the machine than deploying to compute instances does.
- acdha 2y ago
  Does it, though, or are you just more familiar with traditional servers? There are a lot more things to manage in a traditional environment and things like tools are more primitive so you’ll have to invest a lot more time getting anywhere near equivalent reliability or security. That’s especially important if you have compliance requirements where you’ll be on the hook for things provided or greatly simplified by the cloud platform.
  - andrewstuart 2y ago
    >> There are a lot more things to manage in a traditional environment Can you be specific because that does not sound right to me. I know both pretty well and seems to me there is vastly more things that need to be tuned and managed and configured in cloud.
    - chuckadams 2y ago
      How about you give some specifics of things that need constant fiddling in the cloud but are free on bare metal?
      - andrewstuart 2y ago
        Well its a good point, it's my feeling at the moment. I think it would be interesting to do a systematic study of the time required for each.
        chuckadams 2y ago
        In a cloud environment, I turn a few knobs and I can command an absurd amount of compute power, have it autoconfigured with all kinds of service discovery and whatnot. Things that would have taken weeks to months to build in a physical datacenter. Some people take that spare time to train as architecture astronauts I guess, and overcomplicate everything with all kinds of fiddly layers and microservices. I get it, it's fun to play with new toys. Doesn't mean the toys are bad, they're just not necessarily ready for production on day 1.
        andrewstuart 2y ago
        My experience on multiple cloud projects over many years is endless fiddling and problem solving and spaghetti.
        MaKey 2y ago
        That's because it's easy to get started with one of the hundreds of services available in the cloud and before you know it your cloud project is using 50 of them that depend on each other.
    - acdha 2y ago
      Here’s a simple one: if I need to say something accepts inbound 443 from 1.2.3.4, I create an AWS security group rule and will never need to touch that for at least a decade, probably longer. If I’m doing the same thing on premise, the starting cost is learning about something like Cisco hardware which costs a lot more to buy and I am now including in my maintenance work. Over a year, I will spend no time on my AWS rule but I will have to patch the physical switches and deal with primitive configuration tools for a complex networked device. If my organization cares about security, I will have to certify a ton of configuration and practice unrelated to packet filtering whereas I can point auditors at the cloud service’s documentation for everything on their side of the support agreement. What about running code? If I deploy, say, Python Lambdas behind an API Gateway or CloudFront, they’ll run for years without any need to touch them. I don’t need to patch servers, schedule downtime, provision new servers for high load or turn them off when load is low, care that the EL7 generation servers need to be upgraded to EL8, nobody is thinking about load balancer or HTTPS certificate rotation, etc. What you’re paying for, and giving up customization for, is having someone else handle all of that.
chx 2y ago
> A decade ago, you had people building in-house Hadoop clusters for tiny amounts of data instead of doing their jobs analyzing that data So said Gary Bernhardt of WAT fame in 2015 https://x.com/garybernhardt/status/600783770925420546 https://x.com/garybernhardt/status/600783770925420546 > Consulting service: you bring your big data problems to me, I say "your data set fits in RAM", you pay me $10,000 for saving you $500,000. Which, in turn, have inspired https://yourdatafitsinram.net/ https://yourdatafitsinram.net/
- acdha 2y ago
  I remember that, too. There were also just a lot of places where it’s like … your data is already in a SQL database, why not get better at reporting queries until you’re running so many of them that you don’t have an easy path to handle them? It was especially tragicomic in places where they were basically hoping not to need to hire analysts with technical skills, which is possibly the most MBA way ever to very expensively “save” money.

abadpoli 2y ago

For many systems, the application and the machine are so intertwined as to basically be the same thing. A lot of web apps are just boring CRUD APIs. The business logic behind these is nothing special. The differentiating factor for a lot of them is how they scale, or how they integrate with other applications, or their performance, or how fast they are to iterate on, etc. The “machine” offers a lot of possibilities these days that can be taken advantage of, so customizing “the machine” to get what you need out of it is a big focus for many devs.

icedchai 2y ago

Yep. I've also worked on "serverless" projects where similar or more time is spent on IaC (Terraform, etc.) compared to application development. All this effort for something that barely gets any requests.

moduspol 2y ago

One of the core value propositions of AWS is to take advantage of the various services to scale more easily with more linear costs than you would be able to otherwise. Writing and maintaining code that sets up and configures S3, SQS, SNS, DynamoDB, and Lambda is a non-zero cost, but if it then reduces the amount of code I have to write and maintain to "write the application," that's often a good thing. These services are typically solving the hard and/or tedious parts of the application, so it's actually quite valuable to just focus on developing the core differentiating points of the application instead of solving / maintaining highly available object storage or message queues.

tiew9Vii 2y ago

> What I mean by "programming the machine" is doing technical tasks that relates to making the cloud work in the way you want. I take a different perspective. When using a cloud, i need to design my application to work within my selected clouds constraints. A lot of the extra work then comes down to if using a cloud, not just a single VPS on a cloud provider, you are dealing with distributed computing and have to deal with all the issues of distributed computing. Consensus, error handling, etc. I agree though, more often than not it's a lot of extra work but mostly due to dealing with distributed architecture/designing for being cost effective. On personal projects I find designing to be cost effective has me jumping through a lot of hoops and doing novel things. It means I can run multi-region apps with performance better than most webapps for cents/dollars. The equivalent apps I work on in my day job can be costing tens to hundreds of thousands of dollars as they don't jump through the hoops I do. "It's not worth it", "let's look at it later", "we'll just do this, it's good enough for now". As it's someone else's money i just think "meh" and move on as isn't my fight when the majority of people don't want to do it.

crabmusket 2y ago

I feel this strongly. If the project in the OP is meant to run "off the shelf" servers like Express or Next, isn't this a way to reduce the amount of "programming the machine" that needs to be done? That's what I'm hoping. Our approach to lambdas is to delegate all the real work to a "library" and just write a very thin adapter around it dealing with the Lambda details (event format etc.). Hopefully this can take care of as much of the wrapper as possible?

JackSlateur 2y ago

How long would it take to setup an event-driven python function ? I know aws lambda and terraform and, to me, it takes a couple of minutes at most. Indeed, I had to learn it ! Just as I had to learn how to code in Python (which makes me far more effective now), just as I had to learn LVM2 or iproute2 or C or whatever. Once the learning curve is processed, AWS allows you to increase your productivity. Of course, and this is very important, you shall still use the right tool for the right job !