3 ms·

> having that run on a datacenter near the user This is the biggest deal IMHO. FaaS in CDN edge DCs are remarkably fast and affordable. If you can respond to

> having that run on a datacenter near the user This is the biggest deal IMHO. FaaS in CDN edge DCs are remarkably fast and affordable. If you can respond to an API call faster than a request can get through to a load balancer, you're providing quite a UX improvement regardless of cost. For most people running stateful apps this requires some large tradeoffs in the form of the data layer, but they are getting more reasonable over time. In the case of CloudFlare which may have the strongest edge function hosting now, it used to only support its Worker KV data store (fast but KV only and globally eventually consistent ~10 secs). More recently they have added support for fast connections to their R2 (S3-like) service, and now they have a new service that can proxy to MySQL and Postgres databases. https://blog.cloudflare.com/relational-database-connectors/ https://blog.cloudflare.com/relational-database-connectors/ Its an exciting space, because its not just about CDNs: as 5G towers are rolling out, they are building in edge compute clusters along with them. So now you have a 5G iPhone with faster single-threaded performance than your desktop able to get 1Gbps down and faster response times because of its path to edge compute. ISPs are also getting in the game: https://www.coxedge.com/serverless https://www.coxedge.com/serverless https://www.limelight.com/resources/data-sheet/edge-compute/ https://www.limelight.com/resources/data-sheet/edge-compute/

invalidname 5y ago
I think that's sort of against the pareto principle. Assuming you have the physical data near the user. Great. But most of these functions then go on to fetch stuff from elsewhere with its own delays and problems. Even if S3 or cloudflares equivalent is replicated everywhere there would still be a networking delay and context switch waiting for a response. If performance is THAT important you would be better served with a HUGE machine with a huge cache of the database. That's not something you can do with a FAAS. Other than saying "hello world" fast. I can't think of a proper use case where the performance of FAAS would be realistically better than a dumb monolith. The performance and complexity overheads of FAAS make the case even harder to make.
- reilly3000 5y ago
  CF’s Durable Objects product which I neglected to mention above attempts to be that huge database and cache. The CAP theorem isn’t violated, it just embraces consistency while relying on the partition tolerance of their POPs and backbone. Their KV store is more about availability. https://blog.cloudflare.com/durable-objects-ga/ https://blog.cloudflare.com/durable-objects-ga/ I don’t disagree that a huge database can handle an astonishing amount of load but at often unreasonable expense. There as many cases where FaaS is dramatically more affordable to run, both in compute cost and ops time. There are others where its costs spiral out of control under load and with dev effort to fit its constrained programming models. Its hard to say its entirely a trendy frivolity when there are many use cases where its the best tool for the job.
Zababa 5y ago
> This is the biggest deal IMHO. FaaS in CDN edge DCs are remarkably fast and affordable. If you can respond to an API call faster than a request can get through to a load balancer, you're providing quite a UX improvement regardless of cost. How much faster will this be? I think even for servers on the other side of the planet, the max ping will be something like 500ms. I think most pings in the same continent will be something like 100-200ms. On the other hand, a Node hello world was 164ms. That means that compared to just being in the same continent, you don't have much of an advantage. Python is ~30ms for a hello world, That's better. On the other hand, the "competition" is marked as 250ms for a cold start. That's a lot, and I doubt you get the benefits of being close to the user at this point. There may be some things I missed, I don't know much about that space.
- reilly3000 5y ago
  I can really only speak to CloudFlare workers in latency sensitive, user-facing settings, but from my experience their cold start times are insanely low, like 5-10ms. They use V8 Isolates (uberjar for js?) running WASM bytecode, so there is very little runtime overhead. So in that case, there is an actual benefit that being a short hop away, but otherwise I think you're right. I've also used some AWS Lambdas and GCP Cloud Functions in background data processing settings where there are under 1M invocations/month, so its cheaper than running a tiny instance 24/7. In that setting latency doesn't really matter as much, but I was seeing cold starts ~2 seconds and warm start invocations on my node and python code responding within 30-60ms in most cases. Once the container is up and running its reused for follow-on invocations, but eventually(and opaquely) times out and the next invocation will be cold. People who are using serverless runtimes have historically done hacky things to ensure they always have a running container like invoking the function with a cron job. At that point it seems a bit silly not to just run a small vm / container.
  - Zababa 5y ago
    Thanks for the info. 5-10ms seems great, and a real plus for user experience. As you said, cold starts seem to be a killer, especially if you're trying to optimize the slowest requests.
jeroenhd 5y ago
Re: the last part: I dislike the fact that carriers are getting involved in fog computing. It gives them a perverse incentive to do things like slow down VPNs or even disable privacy enhancing technologies (like how some ISPs have been caught disabling Apple's semi-Tor built into the OS. Also, the vast majority of the world doesn't have an iPhone, so assuming everyone is spending over a thousand dollars on the latest and greatest cellphone will lead to even worse websites.
- reilly3000 5y ago
  Yikes, that's more than a bit scary. IMHO Verizon's supercookie and 'Customer Experience Plus' shenanigans deserve massive retribution and restitution such that nobody would dare do that again. If there is one thing that needs to be broken up in big tech its the combo of ISP/carriers and advertising sellers.

invalidname 5y ago

I think that's sort of against the pareto principle. Assuming you have the physical data near the user. Great. But most of these functions then go on to fetch stuff from elsewhere with its own delays and problems. Even if S3 or cloudflares equivalent is replicated everywhere there would still be a networking delay and context switch waiting for a response. If performance is THAT important you would be better served with a HUGE machine with a huge cache of the database. That's not something you can do with a FAAS. Other than saying "hello world" fast. I can't think of a proper use case where the performance of FAAS would be realistically better than a dumb monolith. The performance and complexity overheads of FAAS make the case even harder to make.

reilly3000 5y ago
CF’s Durable Objects product which I neglected to mention above attempts to be that huge database and cache. The CAP theorem isn’t violated, it just embraces consistency while relying on the partition tolerance of their POPs and backbone. Their KV store is more about availability. https://blog.cloudflare.com/durable-objects-ga/ https://blog.cloudflare.com/durable-objects-ga/ I don’t disagree that a huge database can handle an astonishing amount of load but at often unreasonable expense. There as many cases where FaaS is dramatically more affordable to run, both in compute cost and ops time. There are others where its costs spiral out of control under load and with dev effort to fit its constrained programming models. Its hard to say its entirely a trendy frivolity when there are many use cases where its the best tool for the job.

Zababa 5y ago

> This is the biggest deal IMHO. FaaS in CDN edge DCs are remarkably fast and affordable. If you can respond to an API call faster than a request can get through to a load balancer, you're providing quite a UX improvement regardless of cost. How much faster will this be? I think even for servers on the other side of the planet, the max ping will be something like 500ms. I think most pings in the same continent will be something like 100-200ms. On the other hand, a Node hello world was 164ms. That means that compared to just being in the same continent, you don't have much of an advantage. Python is ~30ms for a hello world, That's better. On the other hand, the "competition" is marked as 250ms for a cold start. That's a lot, and I doubt you get the benefits of being close to the user at this point. There may be some things I missed, I don't know much about that space.

reilly3000 5y ago
I can really only speak to CloudFlare workers in latency sensitive, user-facing settings, but from my experience their cold start times are insanely low, like 5-10ms. They use V8 Isolates (uberjar for js?) running WASM bytecode, so there is very little runtime overhead. So in that case, there is an actual benefit that being a short hop away, but otherwise I think you're right. I've also used some AWS Lambdas and GCP Cloud Functions in background data processing settings where there are under 1M invocations/month, so its cheaper than running a tiny instance 24/7. In that setting latency doesn't really matter as much, but I was seeing cold starts ~2 seconds and warm start invocations on my node and python code responding within 30-60ms in most cases. Once the container is up and running its reused for follow-on invocations, but eventually(and opaquely) times out and the next invocation will be cold. People who are using serverless runtimes have historically done hacky things to ensure they always have a running container like invoking the function with a cron job. At that point it seems a bit silly not to just run a small vm / container.
- Zababa 5y ago
  Thanks for the info. 5-10ms seems great, and a real plus for user experience. As you said, cold starts seem to be a killer, especially if you're trying to optimize the slowest requests.

jeroenhd 5y ago

Re: the last part: I dislike the fact that carriers are getting involved in fog computing. It gives them a perverse incentive to do things like slow down VPNs or even disable privacy enhancing technologies (like how some ISPs have been caught disabling Apple's semi-Tor built into the OS. Also, the vast majority of the world doesn't have an iPhone, so assuming everyone is spending over a thousand dollars on the latest and greatest cellphone will lead to even worse websites.

reilly3000 5y ago
Yikes, that's more than a bit scary. IMHO Verizon's supercookie and 'Customer Experience Plus' shenanigans deserve massive retribution and restitution such that nobody would dare do that again. If there is one thing that needs to be broken up in big tech its the combo of ISP/carriers and advertising sellers.