3 ms·

Thanks for posting, Loved the graphics and layout. They made for enjoyable reading. However maybe you could add or share the motivation for creating the project

Thanks for posting, Loved the graphics and layout. They made for enjoyable reading. However maybe you could add or share the motivation for creating the project? Were their shortcomings of existing job scheduler solutions that you evaluated? Is there an edge case that you have that existing solutions didn't address etc?

uttpal 7y ago
Thanks and I will definitely add that to article, I will just point to major shortcomings here of quartz (the go to job scheduler) scaling beyond one node is difficult because * as it uses db locks to adding more nodes does not scale linearly. * we can add custom partitioning (consistent hashing) on top of multi node multi db setup, but maintaining such setup would be very difficult eg adding and removing nodes, handling node failure (will have to move data around from its db to let other node execute those schedules).
drathi5 7y ago
One of the popular existing job scheduler is Quartz, it is solid and does work pretty well up to a limit. The problem arises where we need to scale-out from a single master DB, there are some workarounds for that, but each has its limitations. (https://hazelcast.com/blog/distributed-task-coordination-with-hazelcast/ https://hazelcast.com/blog/distributed-task-coordination-wit...) While there are some existing solutions like BigBen from Walmart, it might be possible that they were overkill for the task required.