6 ms·

Mysos – MySQL on Mesos

59 points by _jomo 11y ago

dijit 11y ago
we're still using mysql? it's still relevant? I never suggest to people who are starting a project to use mysql. (even percona, although I know they do good work). if there is a case for using mysql can someone explain it to me? it feels like a toy in comparison to postgresql, oracle or mssql.. and is too "warty" for small systems either (where sqlite dominates). from what I can tell the only real advocates of mysql seem to be those unwilling to adopt new technology.. but this is all my experience and I'm genuinely curious if anyone sees it differently.
- cwyers 11y ago
  We're still asking this ridiculous question? MySQL supports UPSERT, it supports a lot of partitioning types that Postgres doesn't (and it's first-class partitioning, rather than you having to hack it out of table inheritance). There really are features in MySQL that Postgres lacks, and I haven't even gotten into the world of replication yet. They're different databases with different features and tooling and communities and third-party software support. They should be evaluated on their merits.
- girvo 11y ago
  I'm guessing you've not looked at MariaDB much. I used to feel the same way as you, but having recently looked at MariaDB 10, they've taken things to the next level. I highly recommend taking a look at it before writing MySQL off the way you are, it's 2015 and things have changed.
- threeseed 11y ago
  Alibaba, Facebook, Google, LinkedIn and Twitter are contributors to WebScaleSQL. So yes it is absolutely still relevant.
- SEJeff 11y ago
  Twitter (https://github.com/twitter/mysos https://github.com/twitter/mysos), Facebook (https://github.com/facebook/mysql-5.6 https://github.com/facebook/mysql-5.6), Youtube (https://github.com/youtube/vitess https://github.com/youtube/vitess), just to name a few large users. For everyone saying MySQL isn't "web scale", there are some absolute web titans that seem to be using it quite happily.
  - xu 11y ago
    Yup, Twitter totally use MySQL extensively. This the recent blog post has more details: https://blog.twitter.com/2015/another-look-at-mysql-at-twitter-and-incubating-mysos https://blog.twitter.com/2015/another-look-at-mysql-at-twitt...
  - deleted 11y ago
    [deleted]
justinsb 11y ago
Looks well designed. I wish someone would do this for MongoDB!
fi788 11y ago
This is really interesting, specially if it really reduces the burden of running Mysql with failover or multiple slave databases. This is similar to what is currently available on AWS - RDS, but requires separate database instances and is very tricky to implement on development environments or local infrastructure. I'd be interested in knowing if any of you have tried it or inspected the code in detail.
- stephenr 11y ago
  I'd imagine percona cluster would work for your use case?
chrissnell 11y ago
How does Mysos handle the spin-up of new slaves? One of the pain points of our current Chef-driven environment is the provisioning of new slaves. We use Chef to install and configure MySQL but to get a slave up and running, we use xtrabackup to dump the master's DBs to disk, then rsync them to the slave, then use xtrabackup to restore the dump. Once that's done, we point the new slave at the master and start it slaving. It's totally manual and takes around 15 minutes each time. How does Mysos automate this?
- gpapilion 11y ago
  Looking at to code, it pulls from a backup and restores the state. It manages the cluster slave and master states via zookeeper. When the master fails a slave is promoted. So, in order to run this you need zookeeper, hdfs, and mesos. Its a great solution, but not for everyone.
  - jpgvm 11y ago
    To be fair, if you have Mesos it's highly likely you already have HDFS, ZK is a dependency of Mesos. Though it's not strictly necessary to run HDFS a lot of Mesos native applications and frameworks already expect it. The thing is you can often run HDFS on your Mesos slaves so it ends up stacking up quite nicely, atleast in small deployments.
    - merb 11y ago
      How is HDFS quite nicely in small environments? I mean a minimal HA deployment of HDFS contains at least 5 nodes. 2x Namenode, 3x Data Nodes. Sorry but for "starting out small" this is definitely too much.
      - jpgvm 11y ago
        I don't see how Mesos is going to be useful to you with less than 5 servers.
- xu 11y ago

59 points by _jomo 11y ago

dijit 11y ago

we're still using mysql? it's still relevant? I never suggest to people who are starting a project to use mysql. (even percona, although I know they do good work). if there is a case for using mysql can someone explain it to me? it feels like a toy in comparison to postgresql, oracle or mssql.. and is too "warty" for small systems either (where sqlite dominates). from what I can tell the only real advocates of mysql seem to be those unwilling to adopt new technology.. but this is all my experience and I'm genuinely curious if anyone sees it differently.

cwyers 11y ago
We're still asking this ridiculous question? MySQL supports UPSERT, it supports a lot of partitioning types that Postgres doesn't (and it's first-class partitioning, rather than you having to hack it out of table inheritance). There really are features in MySQL that Postgres lacks, and I haven't even gotten into the world of replication yet. They're different databases with different features and tooling and communities and third-party software support. They should be evaluated on their merits.
girvo 11y ago
I'm guessing you've not looked at MariaDB much. I used to feel the same way as you, but having recently looked at MariaDB 10, they've taken things to the next level. I highly recommend taking a look at it before writing MySQL off the way you are, it's 2015 and things have changed.
threeseed 11y ago
Alibaba, Facebook, Google, LinkedIn and Twitter are contributors to WebScaleSQL. So yes it is absolutely still relevant.
SEJeff 11y ago
Twitter (https://github.com/twitter/mysos https://github.com/twitter/mysos), Facebook (https://github.com/facebook/mysql-5.6 https://github.com/facebook/mysql-5.6), Youtube (https://github.com/youtube/vitess https://github.com/youtube/vitess), just to name a few large users. For everyone saying MySQL isn't "web scale", there are some absolute web titans that seem to be using it quite happily.
- xu 11y ago
  Yup, Twitter totally use MySQL extensively. This the recent blog post has more details: https://blog.twitter.com/2015/another-look-at-mysql-at-twitter-and-incubating-mysos https://blog.twitter.com/2015/another-look-at-mysql-at-twitt...
- deleted 11y ago
  [deleted]

justinsb 11y ago

Looks well designed. I wish someone would do this for MongoDB!

fi788 11y ago

This is really interesting, specially if it really reduces the burden of running Mysql with failover or multiple slave databases. This is similar to what is currently available on AWS - RDS, but requires separate database instances and is very tricky to implement on development environments or local infrastructure. I'd be interested in knowing if any of you have tried it or inspected the code in detail.

stephenr 11y ago
I'd imagine percona cluster would work for your use case?

chrissnell 11y ago

How does Mysos handle the spin-up of new slaves? One of the pain points of our current Chef-driven environment is the provisioning of new slaves. We use Chef to install and configure MySQL but to get a slave up and running, we use xtrabackup to dump the master's DBs to disk, then rsync them to the slave, then use xtrabackup to restore the dump. Once that's done, we point the new slave at the master and start it slaving. It's totally manual and takes around 15 minutes each time. How does Mysos automate this?

gpapilion 11y ago
Looking at to code, it pulls from a backup and restores the state. It manages the cluster slave and master states via zookeeper. When the master fails a slave is promoted. So, in order to run this you need zookeeper, hdfs, and mesos. Its a great solution, but not for everyone.
- jpgvm 11y ago
  To be fair, if you have Mesos it's highly likely you already have HDFS, ZK is a dependency of Mesos. Though it's not strictly necessary to run HDFS a lot of Mesos native applications and frameworks already expect it. The thing is you can often run HDFS on your Mesos slaves so it ends up stacking up quite nicely, atleast in small deployments.
  - merb 11y ago
    How is HDFS quite nicely in small environments? I mean a minimal HA deployment of HDFS contains at least 5 nodes. 2x Namenode, 3x Data Nodes. Sorry but for "starting out small" this is definitely too much.
    - jpgvm 11y ago
      I don't see how Mesos is going to be useful to you with less than 5 servers.
xu 11y ago