19 ms·

Marathon and Kubernetes are in the same level, why do you call it lower level. Mesos, yes, and Kubernetes can run on Mesos too.

KirinDave 11y ago
Maraton schedules long running tasks, but doesn't offer any explicit facilities for binding a homogenous group into a relible responder set. At least, last time I checked. I can see components in the Marathon API to help you build such a thing. But Kubernetes deals with a fundamentally more abstract set of primitives: groups of containers (pods), replicated pods, and automatically managed dispatch points to said groups called services. The Marathon documentation literally recommends you hand-roll a periodically updated haproxy service to accomplish that last bit. While K8S introduces a fair amount of overhead to accomplish this feat reliably, that overhead pays off by having a very responsive and available system. So yeah. K8s is working one level up the stack, as I see it. I think you can make something reasonable with only Marathon, and that thought is backed up by the reality of people doing it. But my own experience suggests K8S does more for you here. Hence my excitement about K8S-mesos.
- ersoft 11y ago
  I managed to achieve load balancing for a fleet of microservices using vulcand [0] and marathon callbacks [1]. It listens for marathon callbacks (for app create/update/scale), or when a task is killed/lost and updates the backends in etcd, propagating to all vulcand load balancers. The tool is available on github [2] [0] https://vulcand.io/proxy.html#backends-and-servers https://vulcand.io/proxy.html#backends-and-servers [1] https://mesosphere.github.io/marathon/docs/event-bus.html https://mesosphere.github.io/marathon/docs/event-bus.html [2] https://github.com/kuende/kameni https://github.com/kuende/kameni
  - KirinDave 11y ago
    This is a good thing. Perhaps you could get the Maraton folks to stop recommending such an awkward solution in preference to your work?
    - ersoft 11y ago
      Well, it depends. My solution is opinionated, using only one vulcand cluster for load balancing, multiple vulcand servers listening on one etcd namespace. For a small number of nodes (< 100) and requests (< 8000 req/s on each vulcand server) this vulcand approach is ok. For large scale, HAProxy/Nginx supports a lot more req/s than vulcand, and I think it can also be configured using confd [0] using a similar aproach: listen for marathon events, update etcd keys, then confd will listen for changes and reload HAProxy/Nginx [0] http://ox86.tumblr.com/post/90554410668/easy-scaling-with-docker-haproxy-and-confd http://ox86.tumblr.com/post/90554410668/easy-scaling-with-do...
KirinDave 11y ago
Just to double down on that previous point, check out the code for haproxy-marathon-bridge. https://github.com/mesosphere/marathon/blob/master/bin/haproxy-marathon-bridge https://github.com/mesosphere/marathon/blob/master/bin/hapro...

KirinDave 11y ago

Maraton schedules long running tasks, but doesn't offer any explicit facilities for binding a homogenous group into a relible responder set. At least, last time I checked. I can see components in the Marathon API to help you build such a thing. But Kubernetes deals with a fundamentally more abstract set of primitives: groups of containers (pods), replicated pods, and automatically managed dispatch points to said groups called services. The Marathon documentation literally recommends you hand-roll a periodically updated haproxy service to accomplish that last bit. While K8S introduces a fair amount of overhead to accomplish this feat reliably, that overhead pays off by having a very responsive and available system. So yeah. K8s is working one level up the stack, as I see it. I think you can make something reasonable with only Marathon, and that thought is backed up by the reality of people doing it. But my own experience suggests K8S does more for you here. Hence my excitement about K8S-mesos.

ersoft 11y ago
I managed to achieve load balancing for a fleet of microservices using vulcand [0] and marathon callbacks [1]. It listens for marathon callbacks (for app create/update/scale), or when a task is killed/lost and updates the backends in etcd, propagating to all vulcand load balancers. The tool is available on github [2] [0] https://vulcand.io/proxy.html#backends-and-servers https://vulcand.io/proxy.html#backends-and-servers [1] https://mesosphere.github.io/marathon/docs/event-bus.html https://mesosphere.github.io/marathon/docs/event-bus.html [2] https://github.com/kuende/kameni https://github.com/kuende/kameni
- KirinDave 11y ago
  This is a good thing. Perhaps you could get the Maraton folks to stop recommending such an awkward solution in preference to your work?
  - ersoft 11y ago
    Well, it depends. My solution is opinionated, using only one vulcand cluster for load balancing, multiple vulcand servers listening on one etcd namespace. For a small number of nodes (< 100) and requests (< 8000 req/s on each vulcand server) this vulcand approach is ok. For large scale, HAProxy/Nginx supports a lot more req/s than vulcand, and I think it can also be configured using confd [0] using a similar aproach: listen for marathon events, update etcd keys, then confd will listen for changes and reload HAProxy/Nginx [0] http://ox86.tumblr.com/post/90554410668/easy-scaling-with-docker-haproxy-and-confd http://ox86.tumblr.com/post/90554410668/easy-scaling-with-do...

KirinDave 11y ago

Just to double down on that previous point, check out the code for haproxy-marathon-bridge. https://github.com/mesosphere/marathon/blob/master/bin/haproxy-marathon-bridge https://github.com/mesosphere/marathon/blob/master/bin/hapro...