5 ms·

> JavaScript is inherently single-threaded, so an isolate can only be executing code on behalf of one request at a time. That said, it is already the case that

> JavaScript is inherently single-threaded, so an isolate can only be executing code on behalf of one request at a time. That said, it is already the case that multiple concurrent requests may be handled by the same isolate (one request may be executing while another is e.g. waiting for a response from a remote server). But today you can't really take advantage of that, because you have no control over exactly which isolate receives any particular request, no any way to communicate between neighboring isolates. This is definitely something we're working on but nothing to announce at this time. I've seen this used in a few contexts, for instance batching requests to a origin (logflare: https://github.com/Logflare/cloudflare-app/blob/master/workers/worker.js https://github.com/Logflare/cloudflare-app/blob/master/worke...). Is this generally seen as ok? I'm planning on building similar things for sending page metrics in parallel to serving traffic. > > Inability to rate limit Workers in a cost effective way gives me sleepless nights Agreed, there is something very challenging with the current design of how requests flow through cloudflare. I have played around with dynamic rules in firewall by manually handling "firewall rules" in a worker as part of a request flow and using KV to share this state. I would then update the cloudflare firewall with api. I had to create a separate service to poll both API and KV to maintain them though.

prdonahue 6y ago
Can you drop me an email with some more details about what you're trying to do with Rate Limiting? I want to understand your use case/challenge a bit better: pat at cloudflare dot com.
ignoramous 6y ago
> Is this generally seen as ok? We use a variation of this technique, and it works. > I have played around with dynamic rules in firewall by manually handling "firewall rules" in a worker as part of a request flow and using KV to share this state. You and I have pretty much the same problem then. Beware though of race conditions. Also, a still cheaper way to keep a counter would be to use CountMinSketch and related probabilistic datastructures, but as before, data-race needs to be handled.
kentonv 6y ago
> Is this generally seen as ok? I'm planning on building similar things for sending page metrics in parallel to serving traffic. It's OK, but you won't see a ton of benefit unless you have a very large amount of traffic -- enough that you're commonly seeing single isolates handling concurrent requests. We're working on something that should be a lot better for this.
- ignoramous 6y ago
  > It's OK, but you won't see a ton of benefit unless you have a very large amount of traffic Our traffic patterns are such that per active-user we would add 100K requests per month when we go live. We send metrics out batched every 10s or so, but stumbled upon this post that warns of internal limit on total number of requests originating from Workers per second set at 2000, account-wide [0]. Is there a way to discuss our usecase and get whitelisted for limits as our traffic isn't malicious, hopefully without having to subscribe to the enterprise plan? I raised a support-ticket but a bot replied that since we are "free customers" it hasn't be been looked at. [0] https://community.cloudflare.com/t/workers-and-sub-requests/151829 https://community.cloudflare.com/t/workers-and-sub-requests/...
  - kentonv 6y ago
    > limit on total number of requests originating from Workers per second set at 2000, account-wide [0]. Oh, there is no such limit. The post you link to is quoting another poster who is quoting something they heard from someone else. It sounds like something that may have originated with someone hearing about a particular implementation detail of our anti-abuse systems but the game of telephone has transformed it into something that's just not true at all. As described in my comment on that thread, we do have certain anti-abuse heuristics which might result in a 1015 error. It's very unusual for people to hit these in legitimate use. Have you seen such an error? 100k requests per user per month doesn't sound like it would come anywhere near being an issue (even with 10M users). FWIW the heuristics are designed such that simply adding more users (who behave the same as existing users) shouldn't ever cause a problem.
    - 6y ago

prdonahue 6y ago

Can you drop me an email with some more details about what you're trying to do with Rate Limiting? I want to understand your use case/challenge a bit better: pat at cloudflare dot com.

ignoramous 6y ago

> Is this generally seen as ok? We use a variation of this technique, and it works. > I have played around with dynamic rules in firewall by manually handling "firewall rules" in a worker as part of a request flow and using KV to share this state. You and I have pretty much the same problem then. Beware though of race conditions. Also, a still cheaper way to keep a counter would be to use CountMinSketch and related probabilistic datastructures, but as before, data-race needs to be handled.

kentonv 6y ago

> Is this generally seen as ok? I'm planning on building similar things for sending page metrics in parallel to serving traffic. It's OK, but you won't see a ton of benefit unless you have a very large amount of traffic -- enough that you're commonly seeing single isolates handling concurrent requests. We're working on something that should be a lot better for this.

ignoramous 6y ago
> It's OK, but you won't see a ton of benefit unless you have a very large amount of traffic Our traffic patterns are such that per active-user we would add 100K requests per month when we go live. We send metrics out batched every 10s or so, but stumbled upon this post that warns of internal limit on total number of requests originating from Workers per second set at 2000, account-wide [0]. Is there a way to discuss our usecase and get whitelisted for limits as our traffic isn't malicious, hopefully without having to subscribe to the enterprise plan? I raised a support-ticket but a bot replied that since we are "free customers" it hasn't be been looked at. [0] https://community.cloudflare.com/t/workers-and-sub-requests/151829 https://community.cloudflare.com/t/workers-and-sub-requests/...
- kentonv 6y ago
  > limit on total number of requests originating from Workers per second set at 2000, account-wide [0]. Oh, there is no such limit. The post you link to is quoting another poster who is quoting something they heard from someone else. It sounds like something that may have originated with someone hearing about a particular implementation detail of our anti-abuse systems but the game of telephone has transformed it into something that's just not true at all. As described in my comment on that thread, we do have certain anti-abuse heuristics which might result in a 1015 error. It's very unusual for people to hit these in legitimate use. Have you seen such an error? 100k requests per user per month doesn't sound like it would come anywhere near being an issue (even with 10M users). FWIW the heuristics are designed such that simply adding more users (who behave the same as existing users) shouldn't ever cause a problem.
  - 6y ago