5 ms·

Service Discovery with Mesos-DNS

65 points by presspot 12y ago

m_eiman 12y ago
Doesn't this sound a lot like DNS-SD over mDNS?
- vertex-four 12y ago
  I think it's centralised, so you don't need to either have all your machines on one subnet or configure your routers (if you can) to forward DNS-SD packets between all your subnets, probably creating a small broadcast storm at scale.
  - tknaup 12y ago
    That's right, it's really just a plain ol DNS server that happens to read its records from Mesos.
  - m_eiman 12y ago
    DNS-SD doesn't require mDNS, you can use it with generic DNS servers. There's an example in the RFC, see https://tools.ietf.org/html/rfc6763#section-13.1 https://tools.ietf.org/html/rfc6763#section-13.1 But it's usually used with mDNS on a local network.
preillyme 12y ago
Awesome, Mesos-DNS is simple and stateless. Unlike Consul and SkyDNS, it does not require consensus mechanisms, persistent storage, or a replicated log. This is possible because Mesos-DNS does not implement heartbeats, health monitoring, or lifetime management for applications. This functionality is already available by the Mesos master, slaves, and frameworks.
- vidarh 12y ago
  The same is the case for SkyDNS2. It runs just fine against our Etcd backends. In both cases the combination (SkyDNS + Etcd or Consul) looks far less complicated than Mesos, so I don't really see what you think you gain. If you already have other reasons to want to run Mesos, sure.
  - tknaup 12y ago
    They are simpler but keep it mind all the other things you get from running on Mesos, like provisioning your services, having your services restarted when they crash, workload migration when machines fail, bin-packing so you don't waste resources, etc.
- carlivar 12y ago
  This is great if you're "all in" on Mesos. However the advantages to a truly independent service discovery system (such as Consul etc.) are that it: * Doesn't tie you to Mesos, as it can move to another "datacenter OS" along with your software. * Allows you to mix software-on-Mesos with software that's not.
presspot 12y ago
Here is the Github repo https://github.com/mesosphere/mesos-dns https://github.com/mesosphere/mesos-dns
gtaylor 12y ago
I haven't yet had a chance to play with or research Mesosphere, but I'm hearing it mentioned more and more. Are its goals similar to that of CoreOS and/or Docker Swarm, in that we're slinging around services/containers rather than managing machines or VMs?
- kozyraki 12y ago
  A quick summary of what Mesosphere is working (DCOS - the datacenter opearting system) on is at https://mesosphere.com/ https://mesosphere.com/ . Apache Mesos, the core of Mesosphere's infrastructure, has been in use by companies like Twitter for years to manage clusters with 10s of thousands of servers. http://mesos.apache.org/ http://mesos.apache.org/
kozyraki 12y ago
Thanks for the comments everybody. I am one of the people behind Mesos-DNS and will gladly answer any questions about it. DNS is an obvious way to support many service discovery needs. There were two motivations for implementing Mesos-DNS instead of using another DNS system. First, we needed a DNS system that closely integrates with Mesos. Instead of having every user or framework describe tasks twice (once to Mesos for execution and once to a DNS system), it easier and cleaner to automatically pass task information from Mesos to DNS. Second, we wanted a simple solution. Mesos and its frameworks already implement fault tolerance and life cycle management. We did not want to force Mesos users to deploy another set of consensus mechanisms, persistent storage, or a replicated logs just for DNS. One more interesting detail. SkyDNS, Consul, and Mesos-DNS use the great go-DNS library by Miek Gieben (github.com/miekg/dns). So, their core DNS capabilities are the same. They differ in what they offer for fault-tolerance, life cycle management, load balancing, etc. Mesos-DNS defers all this functionality to Mesos and its framework (KISS).

65 points by presspot 12y ago

m_eiman 12y ago

Doesn't this sound a lot like DNS-SD over mDNS?

vertex-four 12y ago
I think it's centralised, so you don't need to either have all your machines on one subnet or configure your routers (if you can) to forward DNS-SD packets between all your subnets, probably creating a small broadcast storm at scale.
- tknaup 12y ago
  That's right, it's really just a plain ol DNS server that happens to read its records from Mesos.
- m_eiman 12y ago
  DNS-SD doesn't require mDNS, you can use it with generic DNS servers. There's an example in the RFC, see https://tools.ietf.org/html/rfc6763#section-13.1 https://tools.ietf.org/html/rfc6763#section-13.1 But it's usually used with mDNS on a local network.

preillyme 12y ago

Awesome, Mesos-DNS is simple and stateless. Unlike Consul and SkyDNS, it does not require consensus mechanisms, persistent storage, or a replicated log. This is possible because Mesos-DNS does not implement heartbeats, health monitoring, or lifetime management for applications. This functionality is already available by the Mesos master, slaves, and frameworks.

vidarh 12y ago
The same is the case for SkyDNS2. It runs just fine against our Etcd backends. In both cases the combination (SkyDNS + Etcd or Consul) looks far less complicated than Mesos, so I don't really see what you think you gain. If you already have other reasons to want to run Mesos, sure.
- tknaup 12y ago
  They are simpler but keep it mind all the other things you get from running on Mesos, like provisioning your services, having your services restarted when they crash, workload migration when machines fail, bin-packing so you don't waste resources, etc.
carlivar 12y ago
This is great if you're "all in" on Mesos. However the advantages to a truly independent service discovery system (such as Consul etc.) are that it: * Doesn't tie you to Mesos, as it can move to another "datacenter OS" along with your software. * Allows you to mix software-on-Mesos with software that's not.

presspot 12y ago

Here is the Github repo https://github.com/mesosphere/mesos-dns https://github.com/mesosphere/mesos-dns

gtaylor 12y ago

I haven't yet had a chance to play with or research Mesosphere, but I'm hearing it mentioned more and more. Are its goals similar to that of CoreOS and/or Docker Swarm, in that we're slinging around services/containers rather than managing machines or VMs?

kozyraki 12y ago
A quick summary of what Mesosphere is working (DCOS - the datacenter opearting system) on is at https://mesosphere.com/ https://mesosphere.com/ . Apache Mesos, the core of Mesosphere's infrastructure, has been in use by companies like Twitter for years to manage clusters with 10s of thousands of servers. http://mesos.apache.org/ http://mesos.apache.org/

kozyraki 12y ago

Thanks for the comments everybody. I am one of the people behind Mesos-DNS and will gladly answer any questions about it. DNS is an obvious way to support many service discovery needs. There were two motivations for implementing Mesos-DNS instead of using another DNS system. First, we needed a DNS system that closely integrates with Mesos. Instead of having every user or framework describe tasks twice (once to Mesos for execution and once to a DNS system), it easier and cleaner to automatically pass task information from Mesos to DNS. Second, we wanted a simple solution. Mesos and its frameworks already implement fault tolerance and life cycle management. We did not want to force Mesos users to deploy another set of consensus mechanisms, persistent storage, or a replicated logs just for DNS. One more interesting detail. SkyDNS, Consul, and Mesos-DNS use the great go-DNS library by Miek Gieben (github.com/miekg/dns). So, their core DNS capabilities are the same. They differ in what they offer for fault-tolerance, life cycle management, load balancing, etc. Mesos-DNS defers all this functionality to Mesos and its framework (KISS).