3 ms·

This is inferior to the Manatee design: https://github.com/joyent/manatee https://github.com/joyent/manatee Funnily enough I ran into this yesterday when Mist

by jpgvm 11y ago

This is inferior to the Manatee design: https://github.com/joyent/manatee https://github.com/joyent/manatee Funnily enough I ran into this yesterday when Mist was posted and I looked at what other projects were under the umbrella.

deleted 11y ago
[deleted]
dan353hehe 11y ago
True it is, however to use manatee you have to build on ZFS, which unfortunately is not as widely adopted as it should be.
- jpgvm 11y ago
  Aye but ZFS on Linux is rock solid and definitely the way to go. Technically you don't need ZFS to use the Manatee state machine. You could swap out the ZFS snapshot transfer service to use pg_basebackup instead. For anyone interested I have made the appropriate alterations to Manatee to get it to run on Ubuntu with the standard ubuntu-zfs packages. The fork is here: https://github.com/josephglanville/manatee https://github.com/josephglanville/manatee Chef cookbook to help you get started along with the other dependencies: https://github.com/josephglanville/cookbook-manatee https://github.com/josephglanville/cookbook-manatee
tylerflint 11y ago
Much of the architecture of yoke was inspired by manatee. In fact, we've spent the last 2 years working with Joyent on smartos and illumos, so we were heavily inspired by the great engineering behind manatee. Manatee didn't work for our use cases, which should not be interpreted as "manatee is bad". Here are a few of the reasons we forged a different solution: - While node.js is a great language for many things, we didn't want the overhead of running the node.js/v8 runtime alongside postgres. 50M might seem negligible, but with thousands of postgres clusters for our clients every MB adds up. - We didn't want the administrative overhead of managing a zookeeper cluster, and instead built the cluster management semantics directly into the yoke project. - This may be different now, but at the time manatee was very heavily integrated into illumos/smartos. We really love smartos, but recognize that not all clients are able to use this tech.
- lobster_johnson 11y ago
  You guys seem to be creating a bunch of great little tools, but I wish you would spend more time on documentation. For example, in this case it's not at all clear how the cluster management works, or even what the semantics are.
  - tylerflint 11y ago
    I couldn't agree more. We literally just introduced the nanopack cloud-initiative on friday (https://blog.nanobox.io/nanopack-a-new-vision-for-automated-infrastructure/ https://blog.nanobox.io/nanopack-a-new-vision-for-automated-...). Bear with us, we'll get there!

by jpgvm 11y ago

deleted 11y ago

[deleted]

dan353hehe 11y ago

True it is, however to use manatee you have to build on ZFS, which unfortunately is not as widely adopted as it should be.

jpgvm 11y ago
Aye but ZFS on Linux is rock solid and definitely the way to go. Technically you don't need ZFS to use the Manatee state machine. You could swap out the ZFS snapshot transfer service to use pg_basebackup instead. For anyone interested I have made the appropriate alterations to Manatee to get it to run on Ubuntu with the standard ubuntu-zfs packages. The fork is here: https://github.com/josephglanville/manatee https://github.com/josephglanville/manatee Chef cookbook to help you get started along with the other dependencies: https://github.com/josephglanville/cookbook-manatee https://github.com/josephglanville/cookbook-manatee

tylerflint 11y ago

Much of the architecture of yoke was inspired by manatee. In fact, we've spent the last 2 years working with Joyent on smartos and illumos, so we were heavily inspired by the great engineering behind manatee. Manatee didn't work for our use cases, which should not be interpreted as "manatee is bad". Here are a few of the reasons we forged a different solution: - While node.js is a great language for many things, we didn't want the overhead of running the node.js/v8 runtime alongside postgres. 50M might seem negligible, but with thousands of postgres clusters for our clients every MB adds up. - We didn't want the administrative overhead of managing a zookeeper cluster, and instead built the cluster management semantics directly into the yoke project. - This may be different now, but at the time manatee was very heavily integrated into illumos/smartos. We really love smartos, but recognize that not all clients are able to use this tech.

lobster_johnson 11y ago
You guys seem to be creating a bunch of great little tools, but I wish you would spend more time on documentation. For example, in this case it's not at all clear how the cluster management works, or even what the semantics are.
- tylerflint 11y ago
  I couldn't agree more. We literally just introduced the nanopack cloud-initiative on friday (https://blog.nanobox.io/nanopack-a-new-vision-for-automated-infrastructure/ https://blog.nanobox.io/nanopack-a-new-vision-for-automated-...). Bear with us, we'll get there!