6 ms·

Maybe I’m just turning into one of those old programmers who doesn’t keep with technology, but the incredible inefficiency described and the workarounds for it

Maybe I’m just turning into one of those old programmers who doesn’t keep with technology, but the incredible inefficiency described and the workarounds for it are just mind-boggling to me. Then again, maybe it is cost-effective to burn seconds of CPU time and tens to hundreds of megabytes per request handled. I’m sure some C programmer from the CGI days would be floored by how much resources multithreaded ruby and python app servers use.

edgyquant 5y ago
At this time it is more efficient to use the rich ecosystems provided by modern languages and just scale up than it is to microoptimize against any perceived inefficiencies of using containers and these ecosystems.
GiorgioG 5y ago
Agreed - the cloud providers are laughing all the way to the bank.
erikerikson 5y ago
On the other hand, I've seen organizations run an average of 4% utilization for their VMs.
jeroenhd 5y ago
Despite the absolutely terrible inefficiencies, the scaling benefit these """serverless""" technologies provide is hard to replicate with traditional software. Throwing a Javascript file into some *aaS platform and having that run on a datacenter near the user, automatically upload the output to storage near the user, and scale up dynamically if there's a sudden influx of thousands of users is very hard to accomplish manually without reserving tons of VMs or using some kind of data center API with auto provisioning. Most businesses have no need for such a system, of course, and the costs per function execution are astronomical in comparison to a native program, but for some use cases, the benefits outweigh the costs. As for Ruby and Python, the cost is measured in dev time rather than server costs. Servers are cheap compared to devs, and if a dev can pump out a feature three times as fast because they can make use of a simpler framework, you can afford the overhead of slower programming languages. For this reason I'm watching the Rust space with interest, because the speed is much closer to C++ than to the traditional Java/C# servers most big applications are written in, but the frameworks are slowly shaping up to be similarly complete. Of course, there's a huge difference between Rust and most other languages that cannot be overlooked, but services like Vaultwarden show an interesting opportunity in the server space. It'll still be a while before the tools and frameworks reach a similar level of usability, though.
- reilly3000 5y ago
  > having that run on a datacenter near the user This is the biggest deal IMHO. FaaS in CDN edge DCs are remarkably fast and affordable. If you can respond to an API call faster than a request can get through to a load balancer, you're providing quite a UX improvement regardless of cost. For most people running stateful apps this requires some large tradeoffs in the form of the data layer, but they are getting more reasonable over time. In the case of CloudFlare which may have the strongest edge function hosting now, it used to only support its Worker KV data store (fast but KV only and globally eventually consistent ~10 secs). More recently they have added support for fast connections to their R2 (S3-like) service, and now they have a new service that can proxy to MySQL and Postgres databases. https://blog.cloudflare.com/relational-database-connectors/ https://blog.cloudflare.com/relational-database-connectors/ Its an exciting space, because its not just about CDNs: as 5G towers are rolling out, they are building in edge compute clusters along with them. So now you have a 5G iPhone with faster single-threaded performance than your desktop able to get 1Gbps down and faster response times because of its path to edge compute. ISPs are also getting in the game: https://www.coxedge.com/serverless https://www.coxedge.com/serverless https://www.limelight.com/resources/data-sheet/edge-compute/ https://www.limelight.com/resources/data-sheet/edge-compute/
  - invalidname 5y ago
    I think that's sort of against the pareto principle. Assuming you have the physical data near the user. Great. But most of these functions then go on to fetch stuff from elsewhere with its own delays and problems. Even if S3 or cloudflares equivalent is replicated everywhere there would still be a networking delay and context switch waiting for a response. If performance is THAT important you would be better served with a HUGE machine with a huge cache of the database. That's not something you can do with a FAAS. Other than saying "hello world" fast. I can't think of a proper use case where the performance of FAAS would be realistically better than a dumb monolith. The performance and complexity overheads of FAAS make the case even harder to make.
    - reilly3000 5y ago
vasco 5y ago
When you have a business where salaries are a much larger percentage than hosting on your overall costs, you're going to add to the second to squeeze more out of the first. When you have done the optimisation above, and your bosses keep asking for more features coming out per quarter, and you need to hire more, you're going to opt for the new things everyone is learning, so that you can hire enough without making your life all about hiring. These are trade-offs like any other, and you probably just haven't been exposed to similar constraints or all of this would make at least some sense. There may be better ways, of course.

edgyquant 5y ago

At this time it is more efficient to use the rich ecosystems provided by modern languages and just scale up than it is to microoptimize against any perceived inefficiencies of using containers and these ecosystems.

GiorgioG 5y ago

Agreed - the cloud providers are laughing all the way to the bank.

erikerikson 5y ago

On the other hand, I've seen organizations run an average of 4% utilization for their VMs.

jeroenhd 5y ago

Despite the absolutely terrible inefficiencies, the scaling benefit these """serverless""" technologies provide is hard to replicate with traditional software. Throwing a Javascript file into some *aaS platform and having that run on a datacenter near the user, automatically upload the output to storage near the user, and scale up dynamically if there's a sudden influx of thousands of users is very hard to accomplish manually without reserving tons of VMs or using some kind of data center API with auto provisioning. Most businesses have no need for such a system, of course, and the costs per function execution are astronomical in comparison to a native program, but for some use cases, the benefits outweigh the costs. As for Ruby and Python, the cost is measured in dev time rather than server costs. Servers are cheap compared to devs, and if a dev can pump out a feature three times as fast because they can make use of a simpler framework, you can afford the overhead of slower programming languages. For this reason I'm watching the Rust space with interest, because the speed is much closer to C++ than to the traditional Java/C# servers most big applications are written in, but the frameworks are slowly shaping up to be similarly complete. Of course, there's a huge difference between Rust and most other languages that cannot be overlooked, but services like Vaultwarden show an interesting opportunity in the server space. It'll still be a while before the tools and frameworks reach a similar level of usability, though.

reilly3000 5y ago
> having that run on a datacenter near the user This is the biggest deal IMHO. FaaS in CDN edge DCs are remarkably fast and affordable. If you can respond to an API call faster than a request can get through to a load balancer, you're providing quite a UX improvement regardless of cost. For most people running stateful apps this requires some large tradeoffs in the form of the data layer, but they are getting more reasonable over time. In the case of CloudFlare which may have the strongest edge function hosting now, it used to only support its Worker KV data store (fast but KV only and globally eventually consistent ~10 secs). More recently they have added support for fast connections to their R2 (S3-like) service, and now they have a new service that can proxy to MySQL and Postgres databases. https://blog.cloudflare.com/relational-database-connectors/ https://blog.cloudflare.com/relational-database-connectors/ Its an exciting space, because its not just about CDNs: as 5G towers are rolling out, they are building in edge compute clusters along with them. So now you have a 5G iPhone with faster single-threaded performance than your desktop able to get 1Gbps down and faster response times because of its path to edge compute. ISPs are also getting in the game: https://www.coxedge.com/serverless https://www.coxedge.com/serverless https://www.limelight.com/resources/data-sheet/edge-compute/ https://www.limelight.com/resources/data-sheet/edge-compute/
- invalidname 5y ago
  I think that's sort of against the pareto principle. Assuming you have the physical data near the user. Great. But most of these functions then go on to fetch stuff from elsewhere with its own delays and problems. Even if S3 or cloudflares equivalent is replicated everywhere there would still be a networking delay and context switch waiting for a response. If performance is THAT important you would be better served with a HUGE machine with a huge cache of the database. That's not something you can do with a FAAS. Other than saying "hello world" fast. I can't think of a proper use case where the performance of FAAS would be realistically better than a dumb monolith. The performance and complexity overheads of FAAS make the case even harder to make.
  - reilly3000 5y ago

vasco 5y ago

When you have a business where salaries are a much larger percentage than hosting on your overall costs, you're going to add to the second to squeeze more out of the first. When you have done the optimisation above, and your bosses keep asking for more features coming out per quarter, and you need to hire more, you're going to opt for the new things everyone is learning, so that you can hire enough without making your life all about hiring. These are trade-offs like any other, and you probably just haven't been exposed to similar constraints or all of this would make at least some sense. There may be better ways, of course.