16 ms·
Introducing Chronos: A Replacement for Cron
- philp 14y agoI know a couple guys that will be rather unhappy about the choice of moniker. http://www.getchronos.com/ http://www.getchronos.com/
- rjurney 14y agoI've seen it in action, and it can schedule down to the milisecond across machines.
- solidsnack9000 14y agoTools like this can make a lot of ops housekeeping stuff easier, like log roll ups and deleting old backups and so forth. One doesn't want to depend on particular machines being up; but on the other hand, there's not a 99.999999% uptime requirement.
- nemesisj 14y agoAnd....that blog is going offline in a few weeks. Thanks Posterous/Twitter!
- benatkin 14y agopls...they used a custom domain and they have plenty of time to migrate to a new blogging service and point their DNS to it
- verelo 14y agoThis is brilliant! I've had to do a lot of work creating queue based systems in recent years, and i really like off the shelf solutions to problems like this. Cron is a great tool, but its a mess once you have a bunch of machines and it really just does not scale.
- bdunbar 14y agoBy itself it does not. If you have a bunch of machines, you also have a config system (right?) and they (puppet, chef and so on) have the means to manage cron tasks as a class. You still get cron, and the means to manage them across one's servers.
- darose 14y agoSeems like Chronos covers a lot of the same functionality as Rundeck.
- tptacek 14y agoI'm surprised there aren't more things like this, because Cron replacements are really valuable. We wrote a small C program that serves as a scheduling daemon with Redis; we have a keyspace in Redis that can be used to schedule millisecond-granular periodic or one-shot tasks with a flexible "Chronic"-like specification syntax. It is hugely more convenient and usable than cron. Once you have it, you immediately spot lots of opportunities to factor systems into scheduled jobs that you might have avoided doing if it meant you had to deal with cron.
- codewright 14y agoDon't suppose you could be convinced to release that scheduling daemon as open source, could you?
- tptacek 14y agoSeems like it's all downside and not much upside; it's an unambitious little C project that would mostly give people an opportunity to write blog posts about my C programming style. :) I'm not saying I'll never publish code, just that I've become choosy about it in my advancing years.
- sillysaurus 14y agoWe wrote a small C program that serves as a scheduling daemon with Redis; we have a keyspace in Redis that can be used to schedule millisecond-granular periodic or one-shot tasks with a flexible "Chronic"-like specification syntax. Please open source this. Projects like these are some of the most valuable ones. It's a generalized solution to a very common problem. In fact, what you've described is one of the most elegant designs for this type of problem. It enables any program to schedule other programs just by using a standard Redis interface, for example. Almost every programming language already has a Redis library, meaning it's effectively zero additional work to use your system. Something like your project is sorely needed. If you want help with documentation, I'm happy to offer it. Seems like it's all downside and not much upside; it's an unambitious little C project that would mostly give people an opportunity to write blog posts about my C programming style. :) So was Unix, originally. Those who would deride elegant design due to the choice of language are both short-sighted and mean, and aren't worth worrying about. I know how hard that can be to deal with -- people's negativity tends to bother me a lot, too -- but the world needs more production-grade solutions to real problems. It sounds like your solution has been in use for some time now, and has proven itself effective in the field.
- DannoHung 14y agoAwesome! If anyone's ever had to use Ctrl-M or AutoSys, you know well the supreme horridness of other solutions in this space.
- newman314 14y agoBrr, AutoSys gives me nightmares.
- 3amOpsGuy 14y agoAlways happy to see more tools like this - the bigger the toolbox, the easier our lives are. Of the various approaches I've had to depend on in recent memory, from perl scripts querying a central Db right through to eye wateringly expensive Control-M or Autosys in larger envs, It's plain old cron, fronted by config management (cfengine, puppet, chef, salt - it doesn't matter which) that has proven most dependable, easiest to train others on and simplest to debug.
- stfp 14y agoLooks great, but calling it a replacement for cron is like calling word a replacement for cat.
- hendzen 14y agoInteresting to see that this was built on top of Apache Mesos, a dynamic cluster partitioning framework started by some researchers at Berkeley [1]. So far I've heard about Twitter (and now Airbnb) using Mesos in production. Is anyone else evaluating or using it currently? [1] - http://incubator.apache.org/mesos/papers/nsdi_mesos.pdf http://incubator.apache.org/mesos/papers/nsdi_mesos.pdf
- _harry 14y agoWired had a great article on Mesos that mentions Conviva also uses Mesos: http://www.wired.com/wiredenterprise/2013/03/google-borg-twitter-mesos/all/ http://www.wired.com/wiredenterprise/2013/03/google-borg-twi...
- zobzu 14y agoso... cron's a 200kb self contained C program, super simple, does one thing, and does it well. Chronos is built on top of frameworks, needs a few services to run. Certainly it has value, but marketing it as a cron replacement is wrong imo. Chronos is one of these "web-server-service-tool" thingie, but no, it's not cron.
- bitwize 14y agoEvery time I come across one of these heavyweight, dependency-encrusted "replacements" for basic Unix tools, I'm reminded of L33tStart[0], a fictional, satirical init(1) replacement I wrote about on Reddit -- a sort of parody of systemd. Every day, in a bizarre manifestation of Poe's Law, L33tStart seems less and less a parody. [0] http://www.reddit.com/r/programming/comments/14ay0r/hacker_koan_on_the_proliferation_of_unix_text/c7bntjn http://www.reddit.com/r/programming/comments/14ay0r/hacker_k...
- adambenayoun 14y agoAny chance you could open source L33tStart. Also what type of ponies were you using for your startups? Black or white ponies?
- andrewflnr 14y agoMostly pink and purple, is my impression.
- zobzu 14y agoheh, you made my morning :)
- rsync 14y agoDing! Complex, interdependent systems without slack. Now add production pressures. You have all the ingredients of a "Normal Accident"[1]. You should do the opposite of this. [1] http://en.wikipedia.org/wiki/Normal_Accidents http://en.wikipedia.org/wiki/Normal_Accidents
- 14y ago
- eksith 14y agoThis is almost like saying "A Replacement for Gravity". The benefit having something so simple and singular in function with no dependencies is that there's so little to break. While I appreciate the need for having something with more capability, calling it a "replacement" is a bit facetious. I wish them success and call it an airplane. I'll stick to my hang glider.
- dice 14y agoWhat the ops team isn't telling them (DevOps only goes so far) is that there's a cron job on each node which periodically bounces the Chronos processes (and its 50 dependencies) in order to mitigate memory leaks.
- revscat 14y agoAnyone know why launchd hasn't taken off in this regard? I've used it on my personal systems with success, but rarely (if ever) see it mentioned as a cron replacement. Instead, you frequently see these solutions done from scratch.
- andrewflnr 14y agoLink for the ignorant and lazy, as I once was: http://en.wikipedia.org/wiki/Launchd http://en.wikipedia.org/wiki/Launchd Seems like a decent idea. According to Wikipedia, Ubuntu considered it in 2006 when they were looking for a new unit system, but didn't like that it was under Apple's own license, which was shortly thereafter changed to the Apache license. I'll remember it if I want to do something unconventional on a system.
- SeoxyS 14y agoBecause writing property lists is even more annoying than writing crontab syntax. Seriously, plists suck.
- gnuvince 14y agoAs far as Cron replacements go, I prefer whenjobs [1]. [1] http://people.redhat.com/~rjones/whenjobs/ http://people.redhat.com/~rjones/whenjobs/
- sandGorgon 14y agoIm not sure what mesos brings to the table (possibly distributed dependency management), but for a powerful replacement for cron with dependency management, logging, etc. one could use systemd timers [1] 1. http://jason.the-graham.com/2013/03/06/how-to-use-systemd-timers/ http://jason.the-graham.com/2013/03/06/how-to-use-systemd-ti...
- Zenst 14y agoMany people do much of the fancy stuff with system managment products (Tivoli or BMC patrol spring to mind from past experiences). With that they can do centralised monitoring of the server and services and software and automate responses to defined conditions allowing automatic responses with the right thought and alerting callout when exceptions to the rules occur and when things need looking at. Now all that said, I'm not uptodate on that side of things and even less uptodate on open source alternative, though my quick look at this does indicate that it is a start in the right directon and can only get better. So quick look and glance over gave me a good gut feeling, which is always nice to have.
- erikbern 14y agoWe've built Luigi at Spotify to solve a lot of similar problems: https://github.com/spotify/luigi https://github.com/spotify/luigi Might be worth checking out if you are building large data flows. We probably run 10k Luigi "tasks" every day, of which the majority is Hadoop jobs. They are all organized in a large dependency graph expressed within Python, and you also get visualization, exception handling, atomic file operation, etc.
- philipcristiano 14y agoI was planning to do something similar after I learned about Mesos although I haven't found the time. At the moment I'm using a few processes with MySQL publishing with RabbitMQ to handle distribution and node failures. It doesn't have nearly as nice an interface but RabbitMQ isn't terribly difficult to manage. Thanks for open sourcing!
- tszming 14y agoI am using a similar tool for job automation, history & alert, and it is called Jenkins :)
- dschiptsov 14y agoin Java? Something is deeply wrong with this world.)
- stiff 14y agoI really hope someone will succeed in rolling out a good and eventually popular replacement for Cron, because Cron completely sucks. The crontab file has a weird syntax, that in some respects seems to look like a shell script, but isn't really one and some shell constructs work, other not. There are lots of ways you can make a mistake in writing the commands to be executed in a way that the command you intended won't run but will fail silently and you won't get any trace of an error happening. It is hard to even extract some common parts of commands and put them into a variable. I wasted hours and hours debugging weird cron errors. One case was where the crontab of one of the users was moved over to be the system crontab and strangely didn't work. Well, turns out the system wide crontab has one more field, but cron will not signal an error even in an obvious case like this, it will just fail silently (and this is the thing cron is really good at in my experience).