4 ms·

Workers are pretty neat. I've been using them as a mechanism for serving private Wasabi/Backblaze B2 buckets publicly, ideally so I can host project assets with

by aseipp 8y ago

Workers are pretty neat. I've been using them as a mechanism for serving private Wasabi/Backblaze B2 buckets publicly, ideally so I can host project assets with more predictable costs. This is a very practical and useful case for workers, IMO -- it stops unintended outbound egress from origins, and gives better routing -- both B2 and Wasabi peer with CloudFlare to waive any applicable egress fees. So I basically pay a flat $5/month + B2/Wasabi storage costs (approx $5/mo/TB) and zero egress, more or less. Overall it's worked pretty well; it's also useful for doing things like setting up fully 'virtual' domains when using an object store, e.g. set a CNAME for 'foo.bar.com' to 'example.com' and then write a worker that just intercepts every `foo.bar.com` request and routes appropriately. This has worked surprisingly nicely. For example, I host one of my recent projects under https://riscv.ls0f.pw/ https://riscv.ls0f.pw/ but this subdomain doesn't really "exist" beyond being a CNAME for example.com; it's just routed directly to a B2 bucket subpath, in the background. At the moment I'm working on a mechanism of automating my worker deployments using GitHub Actions and seeing how writing them in OCaml/BuckleScript goes (a much more tedious process), but overall it's pretty neat and useful for writing quick routing glue and whatnot. If anyone from the CloudFlare team is reading this, probably the most important feature I'm looking forward to is resource bindings for cryptographic keys, to be used with the WebCrypto API, so plaintext keys are not visible directly in the worker source code. Is there any ETA on when this might be GA? In my case it's the most important feature currently missing, and it's really crucial for a lot of practical use cases. EDIT: also, another really useful feature (for me), but admittedly maybe niche, would be "atomic-compare-and-update worker script", i.e. I want to submit a JavaScript file to the API and only update the worker iff the new source code is not equal to the existing source code. The reasoning here is that I currently download the worker script and 'diff' it against the new one as a test, and only upload if they're actually different. I could simply re-upload every time but I'm not sure if this invalidates any 'hot' worker instances that are already running. Perhaps the upload API can simply be extended with new return codes to indicate this...

aboutruby 8y ago
> is resource bindings for cryptographic keys, ... , so plaintext keys are not visible directly in the worker source code Maybe using ENV like Heroku does would work
- aseipp 8y ago
  Unfortunately there is no way to propagate an environment variable thru a Worker. The closest equivalent is in fact this "resource binding" mechanism I mention -- but it's a bit different -- where as an environment variable binds `FOO=BAR` in an associative dictionary (where you lookup 'FOO'), setting a resource `FOO=BAR` with Workers actually injects a value named `FOO` into your script, which contains a string. So it's just an ordinary JavaScript variable. So it's not injecting environment variables, it's inserting lexical bindings into the script and then running it. But the end results are more or less the same.
ubercow13 8y ago
In your first example what does the worker do apart from routing egress through Cloudflare? Does it cap bandwidth or impose a transfer quota?
- aseipp 8y ago
  At the moment, no, but in theory if I had access to Workers KV (I do not), I could probably build something like that relatively easily. But, I haven't needed it at all -- maybe later when I plan on uploading some bigger public datasets in the ~100GB range. As I said, it's mostly just an insurance method to shield the buckets from non-CloudFlare access, making sure screw ups/meanies, etc don't get to hit my wallet with egress bandwidth charges, which would make me sad. Peering, not the caching/CDN layer, is the real benefit. Also, the worker includes a mini-router I wrote, making it relatively easy to redirect subdomains or subpaths of a domain to different buckets or subdirectories of a bucket, etc. It's really only like 30 lines of code, tops, but it makes it relatively easy to route multiple projects in a single bucket onto shorter, more stable names. So it's not really anything special at all. It's just convenient and now I don't have to run anything else to serve things "safely" from my object storage. Also, if you use Wasabi, you get an S3 API endpoint, so you don't have to run proxies or write adapter code for your upload-path, which is a huge boon. I actually saved money on my upload-path by being able to get rid of my minio proxy and supporting code by using Workers in this way.
kentonv 8y ago
> the most important feature I'm looking forward to is resource bindings for cryptographic keys, Given the specificity of your request, I'm guessing you're aware that this feature already unofficially exists, with unlisted-but-googlable documentation. :) As long as you always deploy workers via the API -- never the UI -- then it's safe to use the feature now. If you accidentally edit your script in the UI, the secret bindings might be dropped, since the UI doesn't understand them yet. > I could simply re-upload every time but I'm not sure if this invalidates any 'hot' worker instances that are already running. Good news: If you upload exactly the same script, your live workers won't be restarted. This is a side effect of the fact that we use a content-addressable store for scripts. (Similarly, if you go to cloudflareworkers.com and enter the exact same script twice, you'll notice the resulting URL is the same.) Are you actually observing non-negligible cold start times, though? Usually the cold start time is so low that you can't really tell when it happens.
- aseipp 8y ago
  Yes, I did find the google-able documentation, but I wasn't sure if the API worked or not, or it was just a temporary vision of the future, etc -- I guess I simply didn't try. :) Perhaps once my GH Actions Workflow is figured out I can migrate to this, and open source it (though it's tied up with my OCaml code at the moment...) > Are you actually observing non-negligible cold start times, though? Usually the cold start time is so low that you can't really tell when it happens. Honestly, no, it was just something I realized when writing my workflows -- that I don't know from the API how to tell if a worker is "up to date or not" other than downloading and comparing. All of my tests have been exceedingly simple (for now) so it was more just a cautious pre-optimization to make sure I didn't tank something later on a script unrelated to the one I was working on (my workflows upload all scripts across domains during CI, iff they changed). Oh! Here's one more feature request, though: atomic route/filter updates for scripts would be really nice. Right now I kind of iterate through them and match/delete/update them one by one, because overlapping routes cause errors. This means actually doing an insert or update or create is a bit complicated -- you need to actually grab the list of existing routes up front and match IDs with the corresponding route paths, before you can determine what to do (if it already exists, update, if it no longer exists, delete, otherwise, insert, etc) But it would be a huge simplification if I could say "Atomically replace the current routes with this set of routes all at once". I'm not sure how easy this is on your end, though. Now that I think of it, I guess I can just immediately wipe and insert a whole new set, though, since it's not like the current solution is atomic anyway!
judge2020 8y ago
> e.g. set a CNAME for 'foo.bar.com' to 'example.com' Note: try to use an A record instead and set to 192.0.2.1 (or any IP in one of the testnets) based on RFC 5737: https://tools.ietf.org/html/rfc5737 https://tools.ietf.org/html/rfc5737

by aseipp 8y ago

aboutruby 8y ago

> is resource bindings for cryptographic keys, ... , so plaintext keys are not visible directly in the worker source code Maybe using ENV like Heroku does would work

aseipp 8y ago
Unfortunately there is no way to propagate an environment variable thru a Worker. The closest equivalent is in fact this "resource binding" mechanism I mention -- but it's a bit different -- where as an environment variable binds `FOO=BAR` in an associative dictionary (where you lookup 'FOO'), setting a resource `FOO=BAR` with Workers actually injects a value named `FOO` into your script, which contains a string. So it's just an ordinary JavaScript variable. So it's not injecting environment variables, it's inserting lexical bindings into the script and then running it. But the end results are more or less the same.

ubercow13 8y ago

In your first example what does the worker do apart from routing egress through Cloudflare? Does it cap bandwidth or impose a transfer quota?

aseipp 8y ago
At the moment, no, but in theory if I had access to Workers KV (I do not), I could probably build something like that relatively easily. But, I haven't needed it at all -- maybe later when I plan on uploading some bigger public datasets in the ~100GB range. As I said, it's mostly just an insurance method to shield the buckets from non-CloudFlare access, making sure screw ups/meanies, etc don't get to hit my wallet with egress bandwidth charges, which would make me sad. Peering, not the caching/CDN layer, is the real benefit. Also, the worker includes a mini-router I wrote, making it relatively easy to redirect subdomains or subpaths of a domain to different buckets or subdirectories of a bucket, etc. It's really only like 30 lines of code, tops, but it makes it relatively easy to route multiple projects in a single bucket onto shorter, more stable names. So it's not really anything special at all. It's just convenient and now I don't have to run anything else to serve things "safely" from my object storage. Also, if you use Wasabi, you get an S3 API endpoint, so you don't have to run proxies or write adapter code for your upload-path, which is a huge boon. I actually saved money on my upload-path by being able to get rid of my minio proxy and supporting code by using Workers in this way.

kentonv 8y ago

> the most important feature I'm looking forward to is resource bindings for cryptographic keys, Given the specificity of your request, I'm guessing you're aware that this feature already unofficially exists, with unlisted-but-googlable documentation. :) As long as you always deploy workers via the API -- never the UI -- then it's safe to use the feature now. If you accidentally edit your script in the UI, the secret bindings might be dropped, since the UI doesn't understand them yet. > I could simply re-upload every time but I'm not sure if this invalidates any 'hot' worker instances that are already running. Good news: If you upload exactly the same script, your live workers won't be restarted. This is a side effect of the fact that we use a content-addressable store for scripts. (Similarly, if you go to cloudflareworkers.com and enter the exact same script twice, you'll notice the resulting URL is the same.) Are you actually observing non-negligible cold start times, though? Usually the cold start time is so low that you can't really tell when it happens.

aseipp 8y ago
Yes, I did find the google-able documentation, but I wasn't sure if the API worked or not, or it was just a temporary vision of the future, etc -- I guess I simply didn't try. :) Perhaps once my GH Actions Workflow is figured out I can migrate to this, and open source it (though it's tied up with my OCaml code at the moment...) > Are you actually observing non-negligible cold start times, though? Usually the cold start time is so low that you can't really tell when it happens. Honestly, no, it was just something I realized when writing my workflows -- that I don't know from the API how to tell if a worker is "up to date or not" other than downloading and comparing. All of my tests have been exceedingly simple (for now) so it was more just a cautious pre-optimization to make sure I didn't tank something later on a script unrelated to the one I was working on (my workflows upload all scripts across domains during CI, iff they changed). Oh! Here's one more feature request, though: atomic route/filter updates for scripts would be really nice. Right now I kind of iterate through them and match/delete/update them one by one, because overlapping routes cause errors. This means actually doing an insert or update or create is a bit complicated -- you need to actually grab the list of existing routes up front and match IDs with the corresponding route paths, before you can determine what to do (if it already exists, update, if it no longer exists, delete, otherwise, insert, etc) But it would be a huge simplification if I could say "Atomically replace the current routes with this set of routes all at once". I'm not sure how easy this is on your end, though. Now that I think of it, I guess I can just immediately wipe and insert a whole new set, though, since it's not like the current solution is atomic anyway!

judge2020 8y ago

> e.g. set a CNAME for 'foo.bar.com' to 'example.com' Note: try to use an A record instead and set to 192.0.2.1 (or any IP in one of the testnets) based on RFC 5737: https://tools.ietf.org/html/rfc5737 https://tools.ietf.org/html/rfc5737