4 ms·

Not really understanding the benefit of running this at the edge to be honest? The additional latency for the request is absolutely negligible compared to the l

Not really understanding the benefit of running this at the edge to be honest? The additional latency for the request is absolutely negligible compared to the latency of LLMs/SD.

tmikaeld 3y ago
Maybe now, but in the future?
mschuster91 3y ago
> Not really understanding the benefit of running this at the edge to be honest? It's primarily a benefit for Cloudflare. Instead of huge and expensive mega-datacenters that AWS/GCP/Azure operates, they can rent cheaper colo space and better distribute workloads. The latter is, I think, the key... AWS basically incentivises you to stay in a single region as long as possible (mostly because the UX of both the web UI and the CLI just sucks when dealing with multiple regions), which means that a lot of users tend to stick in the AWS region most close to themselves, the services aren't really interconnected between regions because that's a headache to set up, while Cloudflare runs "at the edge" from the beginning and people don't even think about introducing silent dependencies on any specific region. And if a Cloudflare DC/region has a massive outage, chances are high no one will notice it because the workloads will just silently shift to somewhere else.
weird-eye-issue 3y ago
It's for companies like us that already run almost everything directly on Cloudflare Workers We integrate with Replicate for SDXL but if this was production ready it would have been likely we went with this instead
fragmede 3y ago
It's a bit of a "if all you have is a hammer, everything looks like a nail" situation. It's not about the latency from you to the edge node, it's about already being in the Cloudflare worker ecosystem as a developer. For voice recognition, latency absolutely matters.
- martinald 3y ago
  ah very fair point on voice - didn't think about that.

4 ms·

Not really understanding the benefit of running this at the edge to be honest? The additional latency for the request is absolutely negligible compared to the l

by martinald 3y ago

Not really understanding the benefit of running this at the edge to be honest? The additional latency for the request is absolutely negligible compared to the latency of LLMs/SD.

tmikaeld 3y ago
Maybe now, but in the future?
mschuster91 3y ago
> Not really understanding the benefit of running this at the edge to be honest? It's primarily a benefit for Cloudflare. Instead of huge and expensive mega-datacenters that AWS/GCP/Azure operates, they can rent cheaper colo space and better distribute workloads. The latter is, I think, the key... AWS basically incentivises you to stay in a single region as long as possible (mostly because the UX of both the web UI and the CLI just sucks when dealing with multiple regions), which means that a lot of users tend to stick in the AWS region most close to themselves, the services aren't really interconnected between regions because that's a headache to set up, while Cloudflare runs "at the edge" from the beginning and people don't even think about introducing silent dependencies on any specific region. And if a Cloudflare DC/region has a massive outage, chances are high no one will notice it because the workloads will just silently shift to somewhere else.
weird-eye-issue 3y ago
It's for companies like us that already run almost everything directly on Cloudflare Workers We integrate with Replicate for SDXL but if this was production ready it would have been likely we went with this instead
fragmede 3y ago
It's a bit of a "if all you have is a hammer, everything looks like a nail" situation. It's not about the latency from you to the edge node, it's about already being in the Cloudflare worker ecosystem as a developer. For voice recognition, latency absolutely matters.
- martinald 3y ago
  ah very fair point on voice - didn't think about that.