4 ms·

That AWS centric idea with Lambda seems pretty reasonable cost-wise too. One issue with Cloudflare Workers is that it is limited in only being able to PUT 100MB

That AWS centric idea with Lambda seems pretty reasonable cost-wise too. One issue with Cloudflare Workers is that it is limited in only being able to PUT 100MB at a time to any host, Cloudflare or not, in an undocumented limit presumably for anti-abuse. I don't believe a similar limitation is present in AWS Lambda. GTR was organically grown from seeing Azure's Blob Storage download API. I got disappointed at the bugs seen and then just patched over it with Cloudflare Workers, but didn't change up to just AWS/S3. I could see myself also going this route too in a clean slate. With the 100MB limit gone, and having to pay for CPU anyway, maybe we could also get proper GPG public key encryption in it too.

kentonv 4y ago
> One issue with Cloudflare Workers is that it is limited in only being able to PUT 100MB at a time to any host, Cloudflare or not, in an undocumented limit presumably for anti-abuse. It's documented here: https://developers.cloudflare.com/workers/platform/limits/#request-limits https://developers.cloudflare.com/workers/platform/limits/#r... This limit actually isn't specific to Workers. It's a general Cloudflare thing, because Cloudflare buffers uploads to disk in order to defend against slow loris attacks. That is, this is actually meant to protect your origin. We've occasionally discussed letting people turn this off if they don't need it but it hasn't come up very often... (I'm the lead engineer for Workers.)
- crazysim 4y ago
  Ah! I guess I missed that. It confused the CF Discord a bit but I guess it is documented.
- crazysim 4y ago
  Actually, the surprise was that it applied to non-Cloudflare hosts such as Azure Storage's endpoint. Request limits against Cloudflare itself? Famous. Against another non-Cloudflare host from a Cloudflare Worker's environment? Much less famous or well known.
  - kentonv 4y ago
    Yeah, that's more or less a bug. The reason for it is that outgoing requests from Workers pass through a lot of the same proxy logic as incoming requests. This made Workers a lot easier to build initially (six months from inception to beta!) but there are a lot of quirks as a result. Outgoing requests from Workers destined for external servers (i.e. not other sites on Cloudflare) really have no reason to be buffered and thus no reason to impose these limits. We should fix that. I also suspect most Workers-based apps don't really need incoming buffering either.

kentonv 4y ago

> One issue with Cloudflare Workers is that it is limited in only being able to PUT 100MB at a time to any host, Cloudflare or not, in an undocumented limit presumably for anti-abuse. It's documented here: https://developers.cloudflare.com/workers/platform/limits/#request-limits https://developers.cloudflare.com/workers/platform/limits/#r... This limit actually isn't specific to Workers. It's a general Cloudflare thing, because Cloudflare buffers uploads to disk in order to defend against slow loris attacks. That is, this is actually meant to protect your origin. We've occasionally discussed letting people turn this off if they don't need it but it hasn't come up very often... (I'm the lead engineer for Workers.)

crazysim 4y ago
Ah! I guess I missed that. It confused the CF Discord a bit but I guess it is documented.
crazysim 4y ago
Actually, the surprise was that it applied to non-Cloudflare hosts such as Azure Storage's endpoint. Request limits against Cloudflare itself? Famous. Against another non-Cloudflare host from a Cloudflare Worker's environment? Much less famous or well known.
- kentonv 4y ago
  Yeah, that's more or less a bug. The reason for it is that outgoing requests from Workers pass through a lot of the same proxy logic as incoming requests. This made Workers a lot easier to build initially (six months from inception to beta!) but there are a lot of quirks as a result. Outgoing requests from Workers destined for external servers (i.e. not other sites on Cloudflare) really have no reason to be buffered and thus no reason to impose these limits. We should fix that. I also suspect most Workers-based apps don't really need incoming buffering either.