4 ms·

To me it's a bit worrying that someone who is still is a student, builds such a complicated and hard to maintain architecture (https://github.com/alexander0042/

To me it's a bit worrying that someone who is still is a student, builds such a complicated and hard to maintain architecture (https://github.com/alexander0042/pirateweather/raw/main/docs/images/Arch_Diagram_2023.png https://github.com/alexander0042/pirateweather/raw/main/docs...). Maybe it's complicated on purpose, to educate themselves on how to create infrastructure like that, but for a production environment, you want to keep things as simple as possible. I'd have opted for a setup where basically every single "service" in that architecture diagram is one function in a monolith program. Easier to maintain, more performant, easier to deploy and basically better in every single way except you can't individually scale components. But at the scale that the project currently is, I don't think that's needed for a long time, nor is it hard to scale one machine if needed. But again, if it's just a educational exercise in "web scale architecture", then it of course is OK. But I wouldn't want other students to see something like that and think that it's necessarily to create such a monster for a relatively simple "moving data around and wrangle it a bit" program.

anaganisk 4y ago
I don't know if a person pursuing a PhD is a student
- ale42 4y ago
  They're "graduate students", usually. Still have a student status at most universities.
- capableweb 4y ago
  I'm guessing they are studying? That would make them a student, as far as I know. But English is not my first language, maybe there is multiple meanings of student and not all of them fit someone who is "pursuing a PhD".
mynameisvlad 4y ago
This whole comment reads so weird. Why do you or anyone else care how complicated someone else’s architecture is? Why is it only “OK” if it’s an educational exercise? Are you going to be maintaining it? Are you going to even be using it? Why does it matter to you what someone else did in their spare time for their own amusement? Or even to make a commercial product out of it? Either way, it’s not your architecture to backseat architect, let people do what they want.
- capableweb 4y ago
  I think I already elaborated on why? A bunch of people take information they come across on the internet and think that's the only way of doing things, so if someone comes across the post, looks at the architecture they might think building it with so many components is a required thing, not just optional for really big projects. I'm personally dealing with infrastructure at loads of different places, and the amount of engineers who over-complicate their infrastructure pains me, they all go for a distributed architecture before the products they are building even have any users, because they've read bunch of blog posts about building it that way. It sucks to spend so much time and effort on maintaining and improving something when it's not really needed in the first place. Like using a Ferrari to deliver food in a traffic-heavy city, instead of just having a moped. So basically, I want to save myself and others time.
  - mynameisvlad 4y ago
    Gotta love gate keeping architecture in the interest of “saving others time”. If someone takes one look at one project and decides to copy it’s architecture word for word, then it’s on them. Not on the person who built an open source project, for whatever reason, and put it out there for the world. It’s not your architecture to architect. If you want to build your own service to show best practices, then go for it. You weren’t there to make decisions, you don’t know of the reasoning behind them, and it seems ludicrous to backseat architect with context, let alone without any amount of it.
    - capableweb 4y ago
      I don't know what your expertise is exactly, but if someone new in your subject area intentionally made things more complicated than necessary for the scope of the project and publish it as a public resource for others to take inspiration from and learn from, wouldn't you at least have some sort reaction against it? See it as wanting to advance my own trade by providing opinion on the current "everything should be a serverless function" trends in the hope of hopefully be able to move the trade into a different direction.
tony_cannistra 4y ago
Not sure what you think about this, but I'd wager that this fellow posted his detailed architecture diagram for transparency and maybe even to solicit some constructive criticism or feedback. It sounds like you'd be well positioned to offer some feedback, but I'd recommend against that without your practicing some empathy for the stage he's in with his education.
tstrimple 4y ago
This isn't a complicated architecture. It looks like a pretty small and simple cloud deployment. The EC2 looks a bit out of place, but I'm assuming there were technical reasons it couldn't be a lambda instead. Apart from EC2, all of the compute is consumption based so you're just paying for actual utilization. This is a far more efficient approach to building in the cloud than just throwing a monolith on a VM and calling it a day.
- oceanplexian 4y ago
  I've built lots of things like this working at AWS. I probably would have used ECS instead of Fargate, S3 and S3 events instead of EFS, and CW Events instead of EventBridge for simplicity sake, but the "gist" of what he is trying to do is pretty sane and this is what a real-world combination of building blocks would look like. If it were me I'd also try to find a way to modify WGRIB2 to run entirely in Lambda and get the costs down.
- andiareso 4y ago
  Agreed. Guy above doesn't know how to save money or use resources efficiently in the cloud...
- ianburrell 4y ago
  Most architecture look complicated when include everything. He includes the load balancer. It would look simpler as VM but then all the complexity is the things running on VM. Also, much of the complexity is from pulling from data sources. Lots of apps are just the web site and database, but weather app needs the complexity. I bet EventBridge has some important purpose.
andiareso 4y ago
This reads as someone who has never designed a cloud architecture. If he wanted to use a VM he might as well use a VPS. The whole point seems to utilize the resources AWS provides... not some basic VM monolith. That's child's play. [EDIT]: He's a Phd for chrissakes!
- capableweb 4y ago
  Yes, use a VPS or dedicated instance, one machine is usually enough. This is exactly what I'm saying :) And yes, if the whole point is to utilize AWS resources, then of course a architecture like that is fine, as it's a learning exercise (to learn how to utilize resources). My comment includes that as well. As far as I can tell, the author is studying at a university, that makes them a student in my mind. Maybe they do other things than just studying, I didn't look into it very far, hence the "maybes" and other caveats in my initial comment.
  - deleted 4y ago
    [deleted]
  - ianburrell 4y ago
    Someone quoted note where he says using AWS for scalability and reliability. Would single VPS have survived being on Hacker News? Would he need to make new VPS to handle the load? I bet AWS scaled up without problems. Also important, it will scale down when spike is done and will only cost what was used. AWS is more reliable, and from this architecture, probably doesn't have to do anything. If the VPS dies, then the site is down until can build new VM. Or have to run two of them at twice the cost. More since he has to setup database or shared storage. That might be overkill for this site, but for some upfront work, he gets something that needs less maintenance. If going to learn, then should do it the right way. He may be scaling for the future, but it is tradeoff to design now instead of later.
- deleted 4y ago
  [deleted]
evo_9 4y ago
Yeah, 13 years ago I wrote a small signup app for tracking pick-up hockey skates I run regularly. It's a pretty cool system, you have your roster of players and setup skates, and it automatically emails everyone x number of days before the skate. You can see who signs up, pay online, etc and if a goalie who's signed up cancels, the 'Skate Operator' is emailed immediately to take care of that (it's a big deal to not have two goalies). It also has an automatic waitlist, and if someone cancels before the skate, and you're the first person on the waitlist, it moves you to the end of the list and emails you to let them know you're in for the skate. It also sends out a reminder X hours before the skate, day of, etc. After I started using it for my own skates, a few rink managers contacted me to discuss using it for their rink. I met with 2 different groups / municipalities (because most rinks are owned by the city they're in), and it never went any farther than those meetings. They didn't want to use a system by some guy that couldn't maintain it, wouldn't keep it going for more than a year or 3. 13 years later and I'm writing a new version and have run something like 11k skates in that time. There is one small, privately owned rink near my house, has used it the entire time without issue. I've got a few other friends/individuals that use it, too. It is possible for a single person to create and maintain something purely because they need it.
- nonethewiser 4y ago
  How is it hosted?
  - evo_9 4y ago
    AWS virtual machine (I have an old school windows service processing the emails; the rewrite will use a new approach).
dragonwriter 4y ago
> To me it’s a bit worrying that someone who is still is a student, builds such a complicated and hard to maintain architecture To me, it is kind of expected. “Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away,” sure, but perfection – or even distantly approaching it – is the product of expertise and practice.
deleted 4y ago
[deleted]

anaganisk 4y ago

I don't know if a person pursuing a PhD is a student

ale42 4y ago
They're "graduate students", usually. Still have a student status at most universities.
capableweb 4y ago
I'm guessing they are studying? That would make them a student, as far as I know. But English is not my first language, maybe there is multiple meanings of student and not all of them fit someone who is "pursuing a PhD".

mynameisvlad 4y ago

This whole comment reads so weird. Why do you or anyone else care how complicated someone else’s architecture is? Why is it only “OK” if it’s an educational exercise? Are you going to be maintaining it? Are you going to even be using it? Why does it matter to you what someone else did in their spare time for their own amusement? Or even to make a commercial product out of it? Either way, it’s not your architecture to backseat architect, let people do what they want.

capableweb 4y ago
I think I already elaborated on why? A bunch of people take information they come across on the internet and think that's the only way of doing things, so if someone comes across the post, looks at the architecture they might think building it with so many components is a required thing, not just optional for really big projects. I'm personally dealing with infrastructure at loads of different places, and the amount of engineers who over-complicate their infrastructure pains me, they all go for a distributed architecture before the products they are building even have any users, because they've read bunch of blog posts about building it that way. It sucks to spend so much time and effort on maintaining and improving something when it's not really needed in the first place. Like using a Ferrari to deliver food in a traffic-heavy city, instead of just having a moped. So basically, I want to save myself and others time.
- mynameisvlad 4y ago
  Gotta love gate keeping architecture in the interest of “saving others time”. If someone takes one look at one project and decides to copy it’s architecture word for word, then it’s on them. Not on the person who built an open source project, for whatever reason, and put it out there for the world. It’s not your architecture to architect. If you want to build your own service to show best practices, then go for it. You weren’t there to make decisions, you don’t know of the reasoning behind them, and it seems ludicrous to backseat architect with context, let alone without any amount of it.
  - capableweb 4y ago
    I don't know what your expertise is exactly, but if someone new in your subject area intentionally made things more complicated than necessary for the scope of the project and publish it as a public resource for others to take inspiration from and learn from, wouldn't you at least have some sort reaction against it? See it as wanting to advance my own trade by providing opinion on the current "everything should be a serverless function" trends in the hope of hopefully be able to move the trade into a different direction.

tony_cannistra 4y ago

Not sure what you think about this, but I'd wager that this fellow posted his detailed architecture diagram for transparency and maybe even to solicit some constructive criticism or feedback. It sounds like you'd be well positioned to offer some feedback, but I'd recommend against that without your practicing some empathy for the stage he's in with his education.

tstrimple 4y ago

This isn't a complicated architecture. It looks like a pretty small and simple cloud deployment. The EC2 looks a bit out of place, but I'm assuming there were technical reasons it couldn't be a lambda instead. Apart from EC2, all of the compute is consumption based so you're just paying for actual utilization. This is a far more efficient approach to building in the cloud than just throwing a monolith on a VM and calling it a day.

oceanplexian 4y ago
I've built lots of things like this working at AWS. I probably would have used ECS instead of Fargate, S3 and S3 events instead of EFS, and CW Events instead of EventBridge for simplicity sake, but the "gist" of what he is trying to do is pretty sane and this is what a real-world combination of building blocks would look like. If it were me I'd also try to find a way to modify WGRIB2 to run entirely in Lambda and get the costs down.
andiareso 4y ago
Agreed. Guy above doesn't know how to save money or use resources efficiently in the cloud...
ianburrell 4y ago
Most architecture look complicated when include everything. He includes the load balancer. It would look simpler as VM but then all the complexity is the things running on VM. Also, much of the complexity is from pulling from data sources. Lots of apps are just the web site and database, but weather app needs the complexity. I bet EventBridge has some important purpose.

andiareso 4y ago

This reads as someone who has never designed a cloud architecture. If he wanted to use a VM he might as well use a VPS. The whole point seems to utilize the resources AWS provides... not some basic VM monolith. That's child's play. [EDIT]: He's a Phd for chrissakes!

capableweb 4y ago
Yes, use a VPS or dedicated instance, one machine is usually enough. This is exactly what I'm saying :) And yes, if the whole point is to utilize AWS resources, then of course a architecture like that is fine, as it's a learning exercise (to learn how to utilize resources). My comment includes that as well. As far as I can tell, the author is studying at a university, that makes them a student in my mind. Maybe they do other things than just studying, I didn't look into it very far, hence the "maybes" and other caveats in my initial comment.
- deleted 4y ago
  [deleted]
- ianburrell 4y ago
  Someone quoted note where he says using AWS for scalability and reliability. Would single VPS have survived being on Hacker News? Would he need to make new VPS to handle the load? I bet AWS scaled up without problems. Also important, it will scale down when spike is done and will only cost what was used. AWS is more reliable, and from this architecture, probably doesn't have to do anything. If the VPS dies, then the site is down until can build new VM. Or have to run two of them at twice the cost. More since he has to setup database or shared storage. That might be overkill for this site, but for some upfront work, he gets something that needs less maintenance. If going to learn, then should do it the right way. He may be scaling for the future, but it is tradeoff to design now instead of later.
deleted 4y ago
[deleted]

evo_9 4y ago

Yeah, 13 years ago I wrote a small signup app for tracking pick-up hockey skates I run regularly. It's a pretty cool system, you have your roster of players and setup skates, and it automatically emails everyone x number of days before the skate. You can see who signs up, pay online, etc and if a goalie who's signed up cancels, the 'Skate Operator' is emailed immediately to take care of that (it's a big deal to not have two goalies). It also has an automatic waitlist, and if someone cancels before the skate, and you're the first person on the waitlist, it moves you to the end of the list and emails you to let them know you're in for the skate. It also sends out a reminder X hours before the skate, day of, etc. After I started using it for my own skates, a few rink managers contacted me to discuss using it for their rink. I met with 2 different groups / municipalities (because most rinks are owned by the city they're in), and it never went any farther than those meetings. They didn't want to use a system by some guy that couldn't maintain it, wouldn't keep it going for more than a year or 3. 13 years later and I'm writing a new version and have run something like 11k skates in that time. There is one small, privately owned rink near my house, has used it the entire time without issue. I've got a few other friends/individuals that use it, too. It is possible for a single person to create and maintain something purely because they need it.

nonethewiser 4y ago
How is it hosted?
- evo_9 4y ago
  AWS virtual machine (I have an old school windows service processing the emails; the rewrite will use a new approach).

dragonwriter 4y ago

> To me it’s a bit worrying that someone who is still is a student, builds such a complicated and hard to maintain architecture To me, it is kind of expected. “Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away,” sure, but perfection – or even distantly approaching it – is the product of expertise and practice.

deleted 4y ago

[deleted]