3 ms·

DevOps tools are rapidly evolving. Now that they've basically finished with machine provisioning, dependency management and orchestration, all major DevOps auto

by 23david 12y ago

DevOps tools are rapidly evolving. Now that they've basically finished with machine provisioning, dependency management and orchestration, all major DevOps automation frameworks are going into managing reactive infrastructure. Enforcing a task schedule is just another form of state enforcement. These are all pretty similar: - configuration management: ensure package oracle-java-8 is installed on machines A,B,C with this specific configuration. - orchestration: ensure my-awesome-java-app on machines A,B,C is running to databases on machine D,E,F - deployment with constraints: ensure that four instances of my-awesome-java-app are running on at least 2 physical machines with over 4TB free disk space. - job runner: ensure that script X runs on a cluster every __ minutes. when script X runs, send the output to script Y I think that you'll see task placement and job scheduling primitives being integrated into DevOps tooling in the next 6-12 months. Saltstack already has many of the primitives in place for building out reactive infrastructure, http://docs.saltstack.com/en/latest/ref/runners/all/salt.runners.queue.html#module-salt.runners.queue http://docs.saltstack.com/en/latest/ref/runners/all/salt.run... Would be great to hear about tools for Puppet, Chef, Ansible, other…

KaiserPro 12y ago
Salt is terribly immature at the moment (I know because I use it professionally) I really wouldn't trust it for running tasks as well. Out of the box it starts to get horribly slow after around 500 nodes. (you need to spool up 600 tasks on 600 machines? that'll take 10 minutes guys.) For distributed cron, we use jenkins. Which has the advantage of keeping "build history" For task placement we use alfred (https://renderman.pixar.com/resources/RPS_13.5/rps_manuals/alfserver/index.html https://renderman.pixar.com/resources/RPS_13.5/rps_manuals/a...) yup its old. However it works like a champ, and its fast. (as in it'll dispatch thousands of tasks a second.) You do hit a limit when you go over 6000 "slots" (each slot accepts one task, and the main dispatcher is single threaded). Dispatching is simple and task building has simple syntax that easily grows to thousands of tasks in one job. monitoring is also simple, as each task ships logs and exit status back to the dispatcher. It also has mechanisms to cope with bad/slow/unhappy machines.

by 23david 12y ago

KaiserPro 12y ago

Salt is terribly immature at the moment (I know because I use it professionally) I really wouldn't trust it for running tasks as well. Out of the box it starts to get horribly slow after around 500 nodes. (you need to spool up 600 tasks on 600 machines? that'll take 10 minutes guys.) For distributed cron, we use jenkins. Which has the advantage of keeping "build history" For task placement we use alfred (https://renderman.pixar.com/resources/RPS_13.5/rps_manuals/alfserver/index.html https://renderman.pixar.com/resources/RPS_13.5/rps_manuals/a...) yup its old. However it works like a champ, and its fast. (as in it'll dispatch thousands of tasks a second.) You do hit a limit when you go over 6000 "slots" (each slot accepts one task, and the main dispatcher is single threaded). Dispatching is simple and task building has simple syntax that easily grows to thousands of tasks in one job. monitoring is also simple, as each task ships logs and exit status back to the dispatcher. It also has mechanisms to cope with bad/slow/unhappy machines.