4 ms·

> But getting started with PostgreSQL is easier and it scales better, so it's hard for me to agree with using it in 2017. Facebook and Youtube use MySQL, YT vi

by evook 9y ago

> But getting started with PostgreSQL is easier and it scales better, so it's hard for me to agree with using it in 2017. Facebook and Youtube use MySQL, YT via Vitess, in 2017. I am just curious if you could exactly point out what part of PSQL scales better.

dijit 9y ago
The memory locking which I mentioned. Arguing "x uses y therefore it's good" is a poor metric, I personally spent months with a lot of very expensive hardware finding out if we could go forward with mysql (there was a strong push for mysql), but I found PGSQL not having any of the same single server scaling issues (IE; all 40 cores could be uses, all memory could be used, all disk channels could be used) with no hacking of the kernel or memory allocators.
evgen 9y ago
To say that Facebook uses MySQL is a bit of a misnomer. No one at Facebook outside of a hundred or so people on the db team uses MySQL directly, it is just an on-disk storage system with a basic SQL front-end that can be used as an intermediate translation layer and even then it was more of a legacy issue than anything inherent to MySQL as a technology. Please don't use it as an example of why someone should consider MySQL, as there is almost no way anyone else is going to be using it in a similar fashion. Facebook has spent literally thousands of man-years working to make this system scale and perform in the way they need it to, but there is nothing about this effort that required MySQL and in fact a lot of effort was expended working around glaring deficiencies in MySQL. If they had it to do over again I think that no one at Facebook would select MySQL at this time.
- diziet 9y ago
  Plus there's the whole MyRocks thing too...
- evanelias 9y ago
  Just about everything you've said here is factually incorrect. There are far, far fewer than 100 people working on MySQL at Facebook. I say this as a former member of the team, and I am still in close contact with several team members. Additionally, every engineer at FB has the ability to issue ad hoc SQL queries, and many teams do use special-case MySQL DBs that they interact with directly. I was the lead on FB's DBaaS so trust me that I know what I'm talking about here. Over 1000 engineers had used the DBaaS at the time I left the company. There is no legacy issue involved. FB is changing storage engines in MySQL right now; they have the resources to completely move away from MySQL if there was a compelling reason. Your statement of "If they had it to do over again I think that no one at Facebook would select MySQL at this time" is completely incorrect. Even if you ignore FB, a much larger percentage of the largest sites on the internet use MySQL than Postgres.
  - evgen 9y ago
    As a former manager on the ops side who worked closely with the managers of the db teams, and the hadoop/hbase teams, and the presto folks, wormhole, scuba, a few other teams that may not be public knowledge, etc. I can say with absolute certainty that the only people in the company who thought it was not a sad joke that we used MySQL down in the bottom layers of the stack were the people on the db team. And why wouldn't that team like MySQL? Knowledge of its intricacies and bugs and how to work around its problems is what got them this high-paying gig. I will simply state that having also spent a long time inside FB I disagree with your assessment regarding the high regard in which you think MySQL is held within the company.
    - evanelias 9y ago
      > I can say with absolute certainty that the only people in the company who thought it was not a sad joke You can say with absolute certainty that you know the opinion of thousands of engineers at Facebook? This is clearly hyperbole. It does not appear you are interested in having a meaningful discussion on this topic. > I disagree with your assessment regarding the high regard in which you think MySQL is held Where did I say it's held in high regard? I simply said there is no compelling reason to move away from it, and also said your numbers were way off base in terms of how many people are working on MySQL at FB, as well as how many people directly interact with MySQL at FB. I also greatly question how closely you were interacting with the managers of the db teams if you think there were 100 people on these teams! [EDIT: It's also possible that we just have very different definitions of people "on the db team". For people "working on MySQL" I would only include MySQL Eng, DB Client, MySQL Infra PE, and the majority of Data Performance PE. Arguably you could include some RocksDB people at this point, though their work is used in many non-MySQL ways too, and even then that brings the total to still way under 100 people. Overall I would not include teams like Wormhole or TAO as "working on MySQL", though even if you add them in, still significantly under 100.]
  - evanelias 9y ago
    In retrospect my tone should have been softer and less combative here. My apologies. But to give some perspective on why this hits a nerve: By greatly overstating the number of engineers who worked on something, you're directly undermining the effort and ingenuity of the engineers who worked on it. This stings coming from a former coworker. In your second reply (sibling to this one) you seem to state that Facebook's db engineers are motivated by self-preservation (re: staying with MySQL), and you imply they are over-paid. If self-preservation was a motivation at all, FB wouldn't be migrating major data sets from InnoDB to MyRocks, which has a completely different set of performance characteristics and administrative/operational/automation concerns. As for pay, yes the db teams are very well-compensated because they are literally among the top experts in the field, working on an absolutely massive billion-daily-user OLTP system and all of the insane challenges that entails. Put yourself in my shoes. You worked in security, yes? If I posted some factually incorrect comments about your team, and the engineering motivations and talents of your team, on a public forum like this, how would you react?
lima 9y ago
This doesn't mean anything. At some point, it's easier to throw resources at your old stack to make it scale instead of migrating to something else. Example: Facebook and PHP. They decided it was easier to write an entirely new runtime instead of migrating to something else. Often, this results in significant improvements (MySQL and PHP are great examples).

by evook 9y ago

dijit 9y ago

The memory locking which I mentioned. Arguing "x uses y therefore it's good" is a poor metric, I personally spent months with a lot of very expensive hardware finding out if we could go forward with mysql (there was a strong push for mysql), but I found PGSQL not having any of the same single server scaling issues (IE; all 40 cores could be uses, all memory could be used, all disk channels could be used) with no hacking of the kernel or memory allocators.

evgen 9y ago

To say that Facebook uses MySQL is a bit of a misnomer. No one at Facebook outside of a hundred or so people on the db team uses MySQL directly, it is just an on-disk storage system with a basic SQL front-end that can be used as an intermediate translation layer and even then it was more of a legacy issue than anything inherent to MySQL as a technology. Please don't use it as an example of why someone should consider MySQL, as there is almost no way anyone else is going to be using it in a similar fashion. Facebook has spent literally thousands of man-years working to make this system scale and perform in the way they need it to, but there is nothing about this effort that required MySQL and in fact a lot of effort was expended working around glaring deficiencies in MySQL. If they had it to do over again I think that no one at Facebook would select MySQL at this time.

diziet 9y ago
Plus there's the whole MyRocks thing too...
evanelias 9y ago
Just about everything you've said here is factually incorrect. There are far, far fewer than 100 people working on MySQL at Facebook. I say this as a former member of the team, and I am still in close contact with several team members. Additionally, every engineer at FB has the ability to issue ad hoc SQL queries, and many teams do use special-case MySQL DBs that they interact with directly. I was the lead on FB's DBaaS so trust me that I know what I'm talking about here. Over 1000 engineers had used the DBaaS at the time I left the company. There is no legacy issue involved. FB is changing storage engines in MySQL right now; they have the resources to completely move away from MySQL if there was a compelling reason. Your statement of "If they had it to do over again I think that no one at Facebook would select MySQL at this time" is completely incorrect. Even if you ignore FB, a much larger percentage of the largest sites on the internet use MySQL than Postgres.
- evgen 9y ago
  As a former manager on the ops side who worked closely with the managers of the db teams, and the hadoop/hbase teams, and the presto folks, wormhole, scuba, a few other teams that may not be public knowledge, etc. I can say with absolute certainty that the only people in the company who thought it was not a sad joke that we used MySQL down in the bottom layers of the stack were the people on the db team. And why wouldn't that team like MySQL? Knowledge of its intricacies and bugs and how to work around its problems is what got them this high-paying gig. I will simply state that having also spent a long time inside FB I disagree with your assessment regarding the high regard in which you think MySQL is held within the company.
  - evanelias 9y ago
    > I can say with absolute certainty that the only people in the company who thought it was not a sad joke You can say with absolute certainty that you know the opinion of thousands of engineers at Facebook? This is clearly hyperbole. It does not appear you are interested in having a meaningful discussion on this topic. > I disagree with your assessment regarding the high regard in which you think MySQL is held Where did I say it's held in high regard? I simply said there is no compelling reason to move away from it, and also said your numbers were way off base in terms of how many people are working on MySQL at FB, as well as how many people directly interact with MySQL at FB. I also greatly question how closely you were interacting with the managers of the db teams if you think there were 100 people on these teams! [EDIT: It's also possible that we just have very different definitions of people "on the db team". For people "working on MySQL" I would only include MySQL Eng, DB Client, MySQL Infra PE, and the majority of Data Performance PE. Arguably you could include some RocksDB people at this point, though their work is used in many non-MySQL ways too, and even then that brings the total to still way under 100 people. Overall I would not include teams like Wormhole or TAO as "working on MySQL", though even if you add them in, still significantly under 100.]
- evanelias 9y ago
  In retrospect my tone should have been softer and less combative here. My apologies. But to give some perspective on why this hits a nerve: By greatly overstating the number of engineers who worked on something, you're directly undermining the effort and ingenuity of the engineers who worked on it. This stings coming from a former coworker. In your second reply (sibling to this one) you seem to state that Facebook's db engineers are motivated by self-preservation (re: staying with MySQL), and you imply they are over-paid. If self-preservation was a motivation at all, FB wouldn't be migrating major data sets from InnoDB to MyRocks, which has a completely different set of performance characteristics and administrative/operational/automation concerns. As for pay, yes the db teams are very well-compensated because they are literally among the top experts in the field, working on an absolutely massive billion-daily-user OLTP system and all of the insane challenges that entails. Put yourself in my shoes. You worked in security, yes? If I posted some factually incorrect comments about your team, and the engineering motivations and talents of your team, on a public forum like this, how would you react?

lima 9y ago

This doesn't mean anything. At some point, it's easier to throw resources at your old stack to make it scale instead of migrating to something else. Example: Facebook and PHP. They decided it was easier to write an entirely new runtime instead of migrating to something else. Often, this results in significant improvements (MySQL and PHP are great examples).