4 ms·

I am looking forward to the day when Mesos(or any other PaaS software) will be able to use Ceph to save application(app/container/vm) state. In such a setup app

by lifty 11y ago

I am looking forward to the day when Mesos(or any other PaaS software) will be able to use Ceph to save application(app/container/vm) state. In such a setup applications could move around the compute cluster without loosing state.

jfindley 11y ago
Mesos does now have early-stage support for persistent disks. See https://github.com/apache/mesos/blob/master/docs/persistent-volume.md https://github.com/apache/mesos/blob/master/docs/persistent-... There's also technologies like flocker that will allow you do this with docker volumes (which can then be run inside mesos). Both of these are in a pretty early stage of life, so there's some rough edges, but if you really need it, it's there.
- lifty 11y ago
  Indeed you can cobble together such a solution, but baked in integration at the framework level would be more interesting. If state is exposed at the scheduler level, than its much easier for all scheduler frameworks to take advantage of it.
ownagefool 11y ago
We're already doing this with kubernetes + fleet. Ceph itself seems really stable, though we did have an issue with kubernetes not aquiring a lock on ceph and thus having 2 nodes write in the event that one went unresponsive. This was patched in a later version, but make sure it's merged into the release you go with.
fh973 11y ago
Containers and cluster storage bring you really close to what Google's infrastructure looks like. Especially when the systems are fully fault-tolerant like Mesos and Ceph. One of the drawbacks of Ceph is that its filesystem is not ready for production yet, and you have to resort to block devices that can only accessed from one host at at time. At Quobyte (http://www.quobyte.com http://www.quobyte.com, disclaimer: I am one of the founders), we have a built a fully fault-tolerant distributed file system. This allows concurrent scalable shared access to file systems from any number of hosts. Think of a /data that is accessible from any host and can be mapped in any container. What I found pretty neat was that we could easily do a mysql HA setup on Mesos: put mysql in a container, use a directory on /quobyte for its data, and enable Quobyte's mandatory file locking. When you kill the container, or switch off/unplug its host, the container gets rescheduled and recovers from the shared file system.
TheIronYuppie 11y ago
Full disclosure, I work at Google on Kubernetes. Kubernetes offers this as a plug-in today. https://github.com/kubernetes/kubernetes/tree/master/examples/cephfs https://github.com/kubernetes/kubernetes/tree/master/example...

by lifty 11y ago

jfindley 11y ago

Mesos does now have early-stage support for persistent disks. See https://github.com/apache/mesos/blob/master/docs/persistent-volume.md https://github.com/apache/mesos/blob/master/docs/persistent-... There's also technologies like flocker that will allow you do this with docker volumes (which can then be run inside mesos). Both of these are in a pretty early stage of life, so there's some rough edges, but if you really need it, it's there.

lifty 11y ago
Indeed you can cobble together such a solution, but baked in integration at the framework level would be more interesting. If state is exposed at the scheduler level, than its much easier for all scheduler frameworks to take advantage of it.

ownagefool 11y ago

We're already doing this with kubernetes + fleet. Ceph itself seems really stable, though we did have an issue with kubernetes not aquiring a lock on ceph and thus having 2 nodes write in the event that one went unresponsive. This was patched in a later version, but make sure it's merged into the release you go with.

fh973 11y ago

Containers and cluster storage bring you really close to what Google's infrastructure looks like. Especially when the systems are fully fault-tolerant like Mesos and Ceph. One of the drawbacks of Ceph is that its filesystem is not ready for production yet, and you have to resort to block devices that can only accessed from one host at at time. At Quobyte (http://www.quobyte.com http://www.quobyte.com, disclaimer: I am one of the founders), we have a built a fully fault-tolerant distributed file system. This allows concurrent scalable shared access to file systems from any number of hosts. Think of a /data that is accessible from any host and can be mapped in any container. What I found pretty neat was that we could easily do a mysql HA setup on Mesos: put mysql in a container, use a directory on /quobyte for its data, and enable Quobyte's mandatory file locking. When you kill the container, or switch off/unplug its host, the container gets rescheduled and recovers from the shared file system.

TheIronYuppie 11y ago

Full disclosure, I work at Google on Kubernetes. Kubernetes offers this as a plug-in today. https://github.com/kubernetes/kubernetes/tree/master/examples/cephfs https://github.com/kubernetes/kubernetes/tree/master/example...