4 ms·
The largest benchmark of Serverless providers
- Dunedan 8y agoLast week I attend a Meetup where the author of these benchmarks gave a talk about his experiments. While the goal is noble, the benchmark suffers from some serious problems: - it's just a snapshot of the performance at one point in time - the number of benchmarks is way too low to be statistically significant - the benchmarks were invoked from a local PC, so the overhead tests did depend on the location of the author and the network he was connected to - the benchmarks don't consider factors like AWS Lambda functions being powered by different types of EC2 instances Because of these problems I wouldn't count on the results, but as the author himself points out, it might be a good starting point for constantly collecting benchmarks of Serverless providers: > To improve quality and validity of this data, I want to create a service that continuously collects data by requesting serverless functions, every hour and every day and provide the recent data on a dedicated website. This will cause quiet some costs for the serverless requests, so if you or your company (talk to your boss!) wants to support me in collecting this data, I plan on setting up a Patreon or OpenCollective account for that. For now if you’re interested, just leave your mail here and I will contact you as soon as the project starts.
- arpinum 8y agoThe most popular use cases for serverless are gluing different services together, where critical metrics are network throughput, network latency, and CPU scheduler. Fibonacci is not an interesting use case on its own, nor is time-insensitive computational work in general. Sustained 100% CPU usage is not affordable on these platforms. When used in conjunction with a database, KV store, or blob storage, total latency for the end user is dominated by the CPU scheduler for small instances, and network quality for larger instances. Except Azure of course, they are still trying to figure out how to bolt serverless onto App Services without re-engineering the stack.