4 ms·

So I'm actually in the process of doing Lambda versus persistent costing right now for a new project, where it's heavily load-based and very spiky, but the work

So I'm actually in the process of doing Lambda versus persistent costing right now for a new project, where it's heavily load-based and very spiky, but the work on each packet of information is actually very lightweight. The tricky part here in AWS is not Lambda, which is pretty reasonable in general--the pitfalls I'm seeing are around data storage. DynamoDB is stealthily very expensive, either provisioned or on-demand. Right now I'm converging on lambdas for compute and RDS/S3 for storage, but what really makes AWS shine for this use case more than anything else is SQS. SQS is so good and that there isn't a great, easily deployed (sorry, RabbitMQ) option for exactly what it does is a real push towards AWS. (GCP Cloud PubSub is close enough, but I know AWS way better than I do GCP.) If I didn't need this, and I felt confident managing RabbitMQ as a queueing solution, I don't think AWS would be a compelling solution because $60 in Hetzner nodes, in a HA configuration, could do a lot of work. (If this thing works I'll need an on-prem solution anyway, so I'll probably have to build that too--but that's "good problems to have".)

Mo3 4y ago
DynamoDB is not required to use cloud functions. You can either use regular RDS like you said, Aurora or even just your own EC2-based cluster (all of these you can attach to the VPC the functions are attached to as well) and there's a lot of nice developments going on like Cloudflare D1. Totally agree on SQS, knowing PubSub as well I'd say they're pretty much on the same level. All the interconnectedness is where cloud platforms shine
- eropple 4y ago
  To clarify, I mentioned DynamoDB because it has a unit-costs model rather than an amortized-over-time cost model, and when you can harness that (and then focus on a simplified COGS model) you can do very well.
gunapologist99 4y ago
NATS and NSQ are better than SQS if you can deploy them yourself. ZeroMQ is a great option as well, but it probably will require some major brain changes, since it doesn't fit most people's mental model of what a queuing server looks like.
- eropple 4y ago
  ZeroMQ is too weird for me, yeah. NATS JetStream and NSQ are both promising but their durability options are unclear and while I have a background in devops/system architecture, I'm looking at building something whose entire staff is literally only me and so while I'm sure I can deploy it, I'm not sure I can effectively run it. SQS will do what I expect it to do, and that's pretty powerful.
  - codegangsta 4y ago
    disclosure: I work at Synadia We do have a hosted version of NATS called NGS that is multi-cloud, multi-geo and is really easy to set up https://synadia.com/ngs https://synadia.com/ngs
    - gunapologist99 4y ago
      Your pricing is really very fair, especially since it's based on resource consumption and not per-request. One suggestion: put your pricing right on the home page. I'm starting to wonder why I am running it myself! And, you probably know a little bit about NATS, too, since, you know, you wrote it! :) https://nats.io/support/ https://nats.io/support/
  - deleted 4y ago
    [deleted]
codegangsta 4y ago
I think NATS is an excellent, and very easy to run and deploy. Pretty much has a "just works" attribute I like to assign to boring technology

Mo3 4y ago

DynamoDB is not required to use cloud functions. You can either use regular RDS like you said, Aurora or even just your own EC2-based cluster (all of these you can attach to the VPC the functions are attached to as well) and there's a lot of nice developments going on like Cloudflare D1. Totally agree on SQS, knowing PubSub as well I'd say they're pretty much on the same level. All the interconnectedness is where cloud platforms shine

eropple 4y ago
To clarify, I mentioned DynamoDB because it has a unit-costs model rather than an amortized-over-time cost model, and when you can harness that (and then focus on a simplified COGS model) you can do very well.

gunapologist99 4y ago

NATS and NSQ are better than SQS if you can deploy them yourself. ZeroMQ is a great option as well, but it probably will require some major brain changes, since it doesn't fit most people's mental model of what a queuing server looks like.

eropple 4y ago
ZeroMQ is too weird for me, yeah. NATS JetStream and NSQ are both promising but their durability options are unclear and while I have a background in devops/system architecture, I'm looking at building something whose entire staff is literally only me and so while I'm sure I can deploy it, I'm not sure I can effectively run it. SQS will do what I expect it to do, and that's pretty powerful.
- codegangsta 4y ago
  disclosure: I work at Synadia We do have a hosted version of NATS called NGS that is multi-cloud, multi-geo and is really easy to set up https://synadia.com/ngs https://synadia.com/ngs
  - gunapologist99 4y ago
    Your pricing is really very fair, especially since it's based on resource consumption and not per-request. One suggestion: put your pricing right on the home page. I'm starting to wonder why I am running it myself! And, you probably know a little bit about NATS, too, since, you know, you wrote it! :) https://nats.io/support/ https://nats.io/support/
- deleted 4y ago
  [deleted]

codegangsta 4y ago

I think NATS is an excellent, and very easy to run and deploy. Pretty much has a "just works" attribute I like to assign to boring technology