3 ms·

I tend to think of these services as fulfilling different use cases. For Fargate, the ideal scenario in my workloads is async background processing. Add tasks

I tend to think of these services as fulfilling different use cases. For Fargate, the ideal scenario in my workloads is async background processing. Add tasks to SQS, Fargate pulls tasks off and does the job. Elastically scales up or down and lots of flexibility on machine specs. Ok with some failure rate. For AWS Lambda, recently I like the combo of adding a Cloudflare worker in front and using it as an API gateway. More flexibility on routing, faster performance, and reverse proxy for free which can be good for SEO. And you get all the goodies of Cloudflare like DDOS protection and CDN.

deleted 8y ago
[deleted]
gorbypark 8y ago
Do you have any details about using Cloudflare Workers as an API gateway?
- physcab 8y ago
  Check out the boilerplate here for workers: https://github.com/detroitenglish/cloudflare-worker-webpack-boilerplate/blob/master/README.MD https://github.com/detroitenglish/cloudflare-worker-webpack-... Then invoke lambda with this: https://github.com/mhart/aws4fetch https://github.com/mhart/aws4fetch Important to note that workers have a 15s timeout, so this is really only good for routing. You probably don’t want this to manage tasks that could potentially take longer.
  - kentonv 8y ago
    > Important to note that workers have a 15s timeout Not true -- the timeout on outgoing HTTP requests is (I think) 100 seconds (or unlimited as long as data is streaming). The 15-second limit you may be thinking of is that Workers used to not let you start new outgoing HTTP requests 15 seconds into the event, but already-started requests could continue. This limit was recently removed -- instead, Workers now cancels outgoing requests if the client disconnects, but as long as the client is connected, you can keep making new requests. This was changed to support streaming video use cases where a stream is being assembled out of smaller chunks. (I'm the tech lead for Workers.)
    - physcab 8y ago
      Didn’t know that! Thanks for chiming in.
hartator 8y ago
What do you mean by reverse proxy?
- physcab 8y ago
  If you have a blog running at blog.mycompany.com, that is not going to be as effective for SEO as if the blog were at mycompany.com/blog. You want your primary domain getting the credit. But it’s also not ideal to have a separate subfolder if you have two different services (blog and main product). So the best solution is to reverse proxy so that internet traffic hits /blog, but the worker is actually forwarding the traffic to your internal service.
  - hartator 8y ago
    Got it. So you do that through a Cloudflare worker? Aren’t you loosing the caching perks from cloudflafe? It would nice to be able to do that directy through a page rule.
    - physcab 8y ago
      There are probably multiple ways to reverse proxy, it’s just that CF workers are versatile to handle many use cases. You don’t have to lose the caching benefits. You can still pull your assets through Cloudflare CDN since workers operate at the request level (afaik). In fact it can be even more performant. You can use Workers KV to cache as well. So a request comes in, check KV store, return if found. If not, pull from asset CDN.
      - kentonv 8y ago
        Don't use Workers KV for caching -- use the Cache API: https://developers.cloudflare.com/workers/reference/cache-api/ https://developers.cloudflare.com/workers/reference/cache-ap... KV is a global persistent data store, so reads and writes may have to cross the internet. In comparison, the Cache API reads and writes from the local datacenter's cache. Also, Cache API doesn't cost extra (KV does). However, better than either of these is to formulate your outgoing fetch() calls such that they naturally get the caching properties you want. fetch() goes through Cloudflare's usual caching logic. When that does what you want, it works better because this is the path that has been most optimized over many years.
abd12 8y ago
I think you're right that Fargate and Lambda often serve different use cases. However, I think containers and Lambda can and do serve this particular use case -- handle an API request and forward it to a different system. And Fargate is a good stand-in for containers generally. I wouldn't expect much different performance by using ECS or EKS or EC2 -- it's still going to be a load balancer forwarding to a container instance. Definitely not perfect, but I think it works as a general approximation. For this particular common use case, you have three options. Was curious to see the perf differences between them. * Original author here.

deleted 8y ago

[deleted]

gorbypark 8y ago

Do you have any details about using Cloudflare Workers as an API gateway?

physcab 8y ago
Check out the boilerplate here for workers: https://github.com/detroitenglish/cloudflare-worker-webpack-boilerplate/blob/master/README.MD https://github.com/detroitenglish/cloudflare-worker-webpack-... Then invoke lambda with this: https://github.com/mhart/aws4fetch https://github.com/mhart/aws4fetch Important to note that workers have a 15s timeout, so this is really only good for routing. You probably don’t want this to manage tasks that could potentially take longer.
- kentonv 8y ago
  > Important to note that workers have a 15s timeout Not true -- the timeout on outgoing HTTP requests is (I think) 100 seconds (or unlimited as long as data is streaming). The 15-second limit you may be thinking of is that Workers used to not let you start new outgoing HTTP requests 15 seconds into the event, but already-started requests could continue. This limit was recently removed -- instead, Workers now cancels outgoing requests if the client disconnects, but as long as the client is connected, you can keep making new requests. This was changed to support streaming video use cases where a stream is being assembled out of smaller chunks. (I'm the tech lead for Workers.)
  - physcab 8y ago
    Didn’t know that! Thanks for chiming in.

hartator 8y ago

What do you mean by reverse proxy?

physcab 8y ago
If you have a blog running at blog.mycompany.com, that is not going to be as effective for SEO as if the blog were at mycompany.com/blog. You want your primary domain getting the credit. But it’s also not ideal to have a separate subfolder if you have two different services (blog and main product). So the best solution is to reverse proxy so that internet traffic hits /blog, but the worker is actually forwarding the traffic to your internal service.
- hartator 8y ago
  Got it. So you do that through a Cloudflare worker? Aren’t you loosing the caching perks from cloudflafe? It would nice to be able to do that directy through a page rule.
  - physcab 8y ago
    There are probably multiple ways to reverse proxy, it’s just that CF workers are versatile to handle many use cases. You don’t have to lose the caching benefits. You can still pull your assets through Cloudflare CDN since workers operate at the request level (afaik). In fact it can be even more performant. You can use Workers KV to cache as well. So a request comes in, check KV store, return if found. If not, pull from asset CDN.
    - kentonv 8y ago
      Don't use Workers KV for caching -- use the Cache API: https://developers.cloudflare.com/workers/reference/cache-api/ https://developers.cloudflare.com/workers/reference/cache-ap... KV is a global persistent data store, so reads and writes may have to cross the internet. In comparison, the Cache API reads and writes from the local datacenter's cache. Also, Cache API doesn't cost extra (KV does). However, better than either of these is to formulate your outgoing fetch() calls such that they naturally get the caching properties you want. fetch() goes through Cloudflare's usual caching logic. When that does what you want, it works better because this is the path that has been most optimized over many years.

abd12 8y ago

I think you're right that Fargate and Lambda often serve different use cases. However, I think containers and Lambda can and do serve this particular use case -- handle an API request and forward it to a different system. And Fargate is a good stand-in for containers generally. I wouldn't expect much different performance by using ECS or EKS or EC2 -- it's still going to be a load balancer forwarding to a container instance. Definitely not perfect, but I think it works as a general approximation. For this particular common use case, you have three options. Was curious to see the perf differences between them. * Original author here.