5 ms·

Similarly, my fundamental question is how you would implement master election for applications running within a docker container using etcd. Or is master electi

by secure 13y ago

Similarly, my fundamental question is how you would implement master election for applications running within a docker container using etcd. Or is master election only for etcd itself and not a capability it provides? Also, all the examples just store simple key/value pairs. What are your best practices for actual config files? Do you store e.g. key=/etc/nginx/nginx.conf value=<contents of that file>? Does that imply all machines will have the same config? Or do you have the machine name in the key space? Or is etcd not actually for this kind of configs? If not, for which kind then?

philips 13y ago
> how you would implement master election for applications running within a docker container using etcd. You build master election on top of the test and set functionality of etcd which gives you a cluster wide locking mechanism. The plan is to provide a simplified master election REST api as a module, just like the dashboard in this release is a module. But, nothing is stopping a regular etcd client from building master election on top of the current API. Something I would like to add to etcdctl is a config file generator which reads from the keys of etcd and fills in fields. You could add the entire file but it is more reasonable to simply add the metadata that is dynamic into etcd and ship a template around. Another example of this is this chef integration: https://github.com/coderanger/etcd-chef https://github.com/coderanger/etcd-chef For some applications, like load balancers, you may want to have the process itself interacting with an etcd cluster. e.g. this HTTP load balancer prototype: https://github.com/calavera/active-proxy https://github.com/calavera/active-proxy
- ash 13y ago
  > You build master election on top of the test and set functionality of etcd which gives you a cluster wide locking mechanism... But, nothing is stopping a regular etcd client from building master election on top of the current API. Could you give some pointers on how to implement master election on top of test-and-set? What is the algorithm? I've tried searching for it, but I've likely missed correct keywords.
  - ash 13y ago
    I've got reply here: https://github.com/coreos/etcd/issues/225 https://github.com/coreos/etcd/issues/225
SEJeff 13y ago
Might I suggest you read up on other key value stores such as redis[1] (Although I don't think etcd supports the various data types redis does). Configuration data != configuration file. You could quite easily write something to generate a nginx config from etcd keys (or maybe do what you suggested if there isn't a size limit), but that's not really the normal idea of a K/V store. Ontop of that, etcd gives you a bulletproof storage system thanks to the raft distributed census system (conceptually similar to zookeeper's zab consensus algorithm[3]). [1] http://redis.io http://redis.io [2] http://highscalability.com/blog/2013/8/7/raft-in-search-of-an-understandable-consensus-algorithm.html http://highscalability.com/blog/2013/8/7/raft-in-search-of-a... [3] http://labs.yahoo.com/files/ladis08.pdf http://labs.yahoo.com/files/ladis08.pdf

by secure 13y ago

philips 13y ago

> how you would implement master election for applications running within a docker container using etcd. You build master election on top of the test and set functionality of etcd which gives you a cluster wide locking mechanism. The plan is to provide a simplified master election REST api as a module, just like the dashboard in this release is a module. But, nothing is stopping a regular etcd client from building master election on top of the current API. Something I would like to add to etcdctl is a config file generator which reads from the keys of etcd and fills in fields. You could add the entire file but it is more reasonable to simply add the metadata that is dynamic into etcd and ship a template around. Another example of this is this chef integration: https://github.com/coderanger/etcd-chef https://github.com/coderanger/etcd-chef For some applications, like load balancers, you may want to have the process itself interacting with an etcd cluster. e.g. this HTTP load balancer prototype: https://github.com/calavera/active-proxy https://github.com/calavera/active-proxy

ash 13y ago
> You build master election on top of the test and set functionality of etcd which gives you a cluster wide locking mechanism... But, nothing is stopping a regular etcd client from building master election on top of the current API. Could you give some pointers on how to implement master election on top of test-and-set? What is the algorithm? I've tried searching for it, but I've likely missed correct keywords.
- ash 13y ago
  I've got reply here: https://github.com/coreos/etcd/issues/225 https://github.com/coreos/etcd/issues/225

SEJeff 13y ago

Might I suggest you read up on other key value stores such as redis[1] (Although I don't think etcd supports the various data types redis does). Configuration data != configuration file. You could quite easily write something to generate a nginx config from etcd keys (or maybe do what you suggested if there isn't a size limit), but that's not really the normal idea of a K/V store. Ontop of that, etcd gives you a bulletproof storage system thanks to the raft distributed census system (conceptually similar to zookeeper's zab consensus algorithm[3]). [1] http://redis.io http://redis.io [2] http://highscalability.com/blog/2013/8/7/raft-in-search-of-an-understandable-consensus-algorithm.html http://highscalability.com/blog/2013/8/7/raft-in-search-of-a... [3] http://labs.yahoo.com/files/ladis08.pdf http://labs.yahoo.com/files/ladis08.pdf