3 ms·

> Before going any farther, I wanted to add a link to sign up and support this project! Running this on AWS means that it scales beautifully and is much more re

> Before going any farther, I wanted to add a link to sign up and support this project! Running this on AWS means that it scales beautifully and is much more reliable than if I was trying to host this, but also costs real money. https://github.com/alexander0042/pirateweather/raw/main/docs/images/Arch_Diagram_2023.png https://github.com/alexander0042/pirateweather/raw/main/docs... That looks like a pretty complex architecture. I wonder if instead of financial support, the author could use help figuring out a more cost-effective solution?

bahmboo 4y ago
What specifically is too complex?
- aftbit 4y ago
  Probably this whole thing could run on a single $10/mo DigitalOcean instance in a single monolith, with an appropriate caching CDN in front of it.
  - bahmboo 4y ago
    That's fantastic. Is there any one piece specfically that might run on that? The author discusses using AWS to get free in network transfers since the source material is published there by the guvment. In your analysis how would you offset that incresed cost by moving off network? It would also be interesting to try this all on a $10/month commodity hosting and then open it up for user facing apps such as a darksky type service.
Bellyache5 4y ago
It's not clear that this isn't the more cost-effective solution. Without diving into the pricing of each component shown, I suspect there may be some cost optimizations to make (for one example, I had a hard time discerning from the write-up what the EC2 instance was doing...something related to more weather data collection I think?) but this is approximately "cloud-native" in approach, and the way to minimize costs in the cloud is to design your application to be cloud-native and use managed services and provider-specific capabilities where they make sense (and also tend to offer very low/free per-use pricing). Does that increase vendor lock-in? Definitely. It's a trade-off that needs to be considered for each application. Using the cloud to run a bunch of static VMs is an anti-pattern and a big factor in driving costs through the roof. (If you need a single server, a VPS is a very good approach.) The architecture is complex from the standpoint of having a lot of icons and arrows, but if it was running on a single server would it be _that_ much less complex? At this level of architectural diagram you probably condense most of these service icons down to one or two icons. But drill down further and you still have storage, cron jobs, an API endpoint with throttling/quota logic, CDN, a shared filesystem, the need to access the S3 buckets with weather data that AWS is providing from its Open Data Registry, plus (ideally) fine-grained permissions to lock everything down. Could you run all of this on a single server in a VPS? Yes! Could you run it on a vendor-agnostic K8s cluster? Yes! Would those options automatically be less expensive? Not necessarily.

bahmboo 4y ago

What specifically is too complex?

aftbit 4y ago
Probably this whole thing could run on a single $10/mo DigitalOcean instance in a single monolith, with an appropriate caching CDN in front of it.
- bahmboo 4y ago
  That's fantastic. Is there any one piece specfically that might run on that? The author discusses using AWS to get free in network transfers since the source material is published there by the guvment. In your analysis how would you offset that incresed cost by moving off network? It would also be interesting to try this all on a $10/month commodity hosting and then open it up for user facing apps such as a darksky type service.

Bellyache5 4y ago

It's not clear that this isn't the more cost-effective solution. Without diving into the pricing of each component shown, I suspect there may be some cost optimizations to make (for one example, I had a hard time discerning from the write-up what the EC2 instance was doing...something related to more weather data collection I think?) but this is approximately "cloud-native" in approach, and the way to minimize costs in the cloud is to design your application to be cloud-native and use managed services and provider-specific capabilities where they make sense (and also tend to offer very low/free per-use pricing). Does that increase vendor lock-in? Definitely. It's a trade-off that needs to be considered for each application. Using the cloud to run a bunch of static VMs is an anti-pattern and a big factor in driving costs through the roof. (If you need a single server, a VPS is a very good approach.) The architecture is complex from the standpoint of having a lot of icons and arrows, but if it was running on a single server would it be _that_ much less complex? At this level of architectural diagram you probably condense most of these service icons down to one or two icons. But drill down further and you still have storage, cron jobs, an API endpoint with throttling/quota logic, CDN, a shared filesystem, the need to access the S3 buckets with weather data that AWS is providing from its Open Data Registry, plus (ideally) fine-grained permissions to lock everything down. Could you run all of this on a single server in a VPS? Yes! Could you run it on a vendor-agnostic K8s cluster? Yes! Would those options automatically be less expensive? Not necessarily.