6 ms·
Kala: A Modern Job Scheduler (written in Go)
- endymi0n 11y agoExactly what I was looking for since months now! Loved the architecture of Chronos, but with JVM, Zookeper and Mesos as requirements, it had "heavyweight" written all over the place. This is what the successor of cron should be IMHO. Distributed by design, API driven and dynamic. EDIT: Apparently it's NOT distributed yet. Had hoped for Raft or etcd integration.
- samdolan805 11y agoFeel free to throw issues into the GH tracker so we can get them on the roadmap. My initial use case for this is for complicated time-based triggers that would be an absolute PITA/impossible to model in cron. We both felt the pain of setting up celery, etc.; @ajvb took that pain and put this together.
- thwarted 11y agohttps://github.com/erikh/jobd https://github.com/erikh/jobd uses etcd for distribution of jobs.
- ajvb 11y agoThe initial use case was for a single node setup, Chronos for an early stage startup. I/we will be adding support for it to be distributed here soon, but not until the single-node, single binary with no external requirements setup is production ready. If you have any feature requests, please create an issue in github. The more people request something the sooner it will get implemented. Thanks for the feedback though.
- aukhan 11y agois there way to migrate existing jobs from tools like jenkins to kala ?
- atmosx 11y agoI'm sure it's possible, but it's marked as 'Alpha' in bold letters, "not production ready".
- samdolan805 11y agoCan you start an issue so we can start discussing what this would look like? What kind of APIs would you need for this?
- dmix 11y agoThis looks good, nice work. I need something like this at the moment, but I'm curious what the target company size is for a project like this. For example, we're a 2-3 person team and not quite big enough to have the capacity to run our own deployment of something as critical as Kala/job schedulers. Nor do I want to run another DB like Bolt/Redis when I'm already running another critical DB. I'll probably go with a hosted solution ala http://www.iron.io/worker http://www.iron.io/worker until we can run our own.
- shazow 11y agoBolt is an in-process database, like sqlite or leveldb. And it's native-Go, so there are no dependencies. You don't need to run additional service infrastructure to use it.
- carimura 11y agoJust for clarification, the IronWorker platform by Iron.io supports large enterprise deployments in cloud, on-premises, and hybrid environments, so it scales from small hosted projects to very large deployments processing millions of jobs per hour. http://www.iron.io/enterprise/ http://www.iron.io/enterprise/
- ajvb 11y agodmix, a 2-3 person team is actually exactly what this project was built for. The use case came from a start-up of the past that I was an engineer at that really really needed something like this.
- dmix 11y agoFair enough, different teams have more resources for sys admin work or are at a later stage where they can invest in it. I'm just wary of adding any external pieces of software after being burned a few times staying up late on weekends fixing a broken service.
- dlecorfec 11y agoI see a possible market for a distributed job scheduler in Go, with features from Chronos and Rundeck (and possibly based on an equivalent of the Quartz scheduling library?)
- guruparan18 11y agoSide note: "Kala" means time, in Sanskrit. It also means someone who controls "death". Feminine form of it is "kali". I think it is only fitting to name a scheduler after time (or something that is controlled by it). ;) [1] https://en.wikipedia.org/wiki/K%C4%81la_(time) https://en.wikipedia.org/wiki/K%C4%81la_(time)
- deanclatworthy 11y agoSadly in Finnish it means "fish".
- atmosx 11y agoLuckily in Greek it means 'good'.
- izyda 11y agoLooks good. Very much looking forward to this being production ready. I am currently looking for a scheduler that provides a better UI/UX than cron alone. Currently, what do you all use for this sort of thing? Nextdoor has a not-yet-open-sourced solution: https://engblog.nextdoor.com/2015/06/10/we-do-not-run-cron-jobs-at-nextdoor/ https://engblog.nextdoor.com/2015/06/10/we-do-not-run-cron-j... that looks great. Does anyone have any experience with Yelp's tron? https://github.com/Yelp/Tron https://github.com/Yelp/Tron
- samdolan805 11y agoI work with @ajvb. We'll be working on getting this into production over the next 2 months. To do so, we'll be working on adding redis as a backend and making it distributable. The goal is to make it really easy to get started, but be able to scale out to bigger production systems easily. @ajvb has plans for an awesome dashboard as well to complement this :) Stay tuned... good things ahead.
- izyda 11y agoIs this associated with http://www.222labs.com http://www.222labs.com ? If not, is there an email or somehow I can PM you or @ajvb at?
- samdolan805 11y agoWe work together there. We actually both started working on a job scheduler independently, but joined forces and he ran with it. It's @ajvb's personal project, but we'll be using it at 222 labs for sure. You can shoot me an email at my username without the numbers @ gmail.com and I'll hook in @ajvb to the thread.
- no1youknowz 11y agoPlease add postgres support.
- samdolan805 11y ago