3 ms·
Yeah, that's a terrible idea, so don't do something like that. You don't need to break up your monolith to run cronjobs. My setup: I have async worker running
by luhn 6y ago
Yeah, that's a terrible idea, so don't do something like that. You don't need to break up your monolith to run cronjobs.
My setup: I have async worker running alongside my application. Same codebase, same environment, same monolith, just instead of accepting HTTP requests it pops tasks off a queue. The framework handles retrying/reprocessing/failure.
To run cron tasks, I have CloudWatch Events -> SNS -> HTTP endpoint in the monolith which pushes the cron task into the queue.
And this isn't some unusual setup. This is something that people have been dealing with for a very long time, long before Lambda came into existence. Most web frameworks have async tasks built-in or an integration with a framework that can do async tasks.
- morelisp 6y agoNow you need to upgrade. Your HTTP request handling takes like maybe 2s at most bring up a new instance and drain the old one. But you have one daily cron job that takes eight hours, and another that only takes 1s but needs to run hourly. Now you need to upgrade to a new version, which requires an incompatible DB migration affecting the request handling and the hourly job. The long job started at 2PM. How do you do the upgrade, without someone needing to stay late? (If it's e.g. PHP this gets even worse due to lazy package loading, you don't need a DB migration just a changed import path.) I'm not a fan of microservices, especially the ridiculous factoring I see in lambda-style approaches today. But one of our biggest PITAs was a "horizontally scalable monolith" that after 7 years has accrued so so many responsibilities it's either impossible to find a time slot for an upgrade with zero/low downtime, or we have to plan & develop dealing with shared storage where half may upgrade one day and half the next - all the operational overhead of SOA without the benefits.