5 ms·
Background on me; I abuse databases. I currently use pgsql as a k/v store with locking metadata because it's the only database technology that supports being fo
by dijit 10y ago
Background on me; I abuse databases. I currently use pgsql as a k/v store with locking metadata because it's the only database technology that supports being forced onto disk (not just writing into VFS). It's also pushing huge row sizes (1MB)
My team spent many months trying to mangle MySQL into doing a similar thing (which it will do) but eventually we hit a performance stop with MySQL.
We did many things to improve the performance; switched to using fastsockets in the kernel, changed memory allocators (tried two different ones) but at 40cores and 8 pci-e ssds it was spin locking some function in memory.
Not to mention it's history with throwing data away.
Most people use MySQL because they know it already- and I would say that's fair comment. But getting started with PostgreSQL is easier and it scales better, so it's hard for me to agree with using it in 2017.
To get started with a random test db you can do:
mkdir database
pg_ctl -D database initdb
pg_ctl -D database start
psql -h database
- evook 10y ago> But getting started with PostgreSQL is easier and it scales better, so it's hard for me to agree with using it in 2017. Facebook and Youtube use MySQL, YT via Vitess, in 2017. I am just curious if you could exactly point out what part of PSQL scales better.
- dijit 10y agoThe memory locking which I mentioned. Arguing "x uses y therefore it's good" is a poor metric, I personally spent months with a lot of very expensive hardware finding out if we could go forward with mysql (there was a strong push for mysql), but I found PGSQL not having any of the same single server scaling issues (IE; all 40 cores could be uses, all memory could be used, all disk channels could be used) with no hacking of the kernel or memory allocators.
- evgen 10y agoTo say that Facebook uses MySQL is a bit of a misnomer. No one at Facebook outside of a hundred or so people on the db team uses MySQL directly, it is just an on-disk storage system with a basic SQL front-end that can be used as an intermediate translation layer and even then it was more of a legacy issue than anything inherent to MySQL as a technology. Please don't use it as an example of why someone should consider MySQL, as there is almost no way anyone else is going to be using it in a similar fashion. Facebook has spent literally thousands of man-years working to make this system scale and perform in the way they need it to, but there is nothing about this effort that required MySQL and in fact a lot of effort was expended working around glaring deficiencies in MySQL. If they had it to do over again I think that no one at Facebook would select MySQL at this time.
- diziet 10y agoPlus there's the whole MyRocks thing too...
- evanelias 10y agoJust about everything you've said here is factually incorrect. There are far, far fewer than 100 people working on MySQL at Facebook. I say this as a former member of the team, and I am still in close contact with several team members. Additionally, every engineer at FB has the ability to issue ad hoc SQL queries, and many teams do use special-case MySQL DBs that they interact with directly. I was the lead on FB's DBaaS so trust me that I know what I'm talking about here. Over 1000 engineers had used the DBaaS at the time I left the company. There is no legacy issue involved. FB is changing storage engines in MySQL right now; they have the resources to completely move away from MySQL if there was a compelling reason. Your statement of "If they had it to do over again I think that no one at Facebook would select MySQL at this time" is completely incorrect. Even if you ignore FB, a much larger percentage of the largest sites on the internet use MySQL than Postgres.
- evgen 10y agoAs a former manager on the ops side who worked closely with the managers of the db teams, and the hadoop/hbase teams, and the presto folks, wormhole, scuba, a few other teams that may not be public knowledge, etc. I can say with absolute certainty that the only people in the company who thought it was not a sad joke that we used MySQL down in the bottom layers of the stack were the people on the db team. And why wouldn't that team like MySQL? Knowledge of its intricacies and bugs and how to work around its problems is what got them this high-paying gig. I will simply state that having also spent a long time inside FB I disagree with your assessment regarding the high regard in which you think MySQL is held within the company.
- evanelias 10y ago> I can say with absolute certainty that the only people in the company who thought it was not a sad joke You can say with absolute certainty that you know the opinion of thousands of engineers at Facebook? This is clearly hyperbole. It does not appear you are interested in having a meaningful discussion on this topic. > I disagree with your assessment regarding the high regard in which you think MySQL is held Where did I say it's held in high regard? I simply said there is no compelling reason to move away from it, and also said your numbers were way off base in terms of how many people are working on MySQL at FB, as well as how many people directly interact with MySQL at FB. I also greatly question how closely you were interacting with the managers of the db teams if you think there were 100 people on these teams! [EDIT: It's also possible that we just have very different definitions of people "on the db team". For people "working on MySQL" I would only include MySQL Eng, DB Client, MySQL Infra PE, and the majority of Data Performance PE. Arguably you could include some RocksDB people at this point, though their work is used in many non-MySQL ways too, and even then that brings the total to still way under 100 people. Overall I would not include teams like Wormhole or TAO as "working on MySQL", though even if you add them in, still significantly under 100.]
- lima 10y agoThis doesn't mean anything. At some point, it's easier to throw resources at your old stack to make it scale instead of migrating to something else. Example: Facebook and PHP. They decided it was easier to write an entirely new runtime instead of migrating to something else. Often, this results in significant improvements (MySQL and PHP are great examples).
- cel1ne 10y agoCreating a test-db can be even simpler: $ createdb mytest $ psql mytest
- dijit 10y agoThis assumes that you already have a data directory and a running pgsql instance. Mine doesn't. :) Simply having the binaries is enough- this is something that gives pgsql another edge on mysql.
- i336_ 10y agoI'm reminded of the (long winded) article that popped up on here some time ago about a person who was suggested to modify redis' source code to fix a problem they had: https://medium.com/@paydro/hacking-redis-to-save-money-6472332697e4 https://medium.com/@paydro/hacking-redis-to-save-money-64723... (slightly scathing discussion: https://news.ycombinator.com/item?id=10835950 https://news.ycombinator.com/item?id=10835950). I wonder if redis can be convinced to flush data to disk. AFAIK it does it periodically? If only SQLite was a tiiiny bit more scalable. It's the only DB where you have to explicitly disable force-flushing of transactions to disk.