5 ms·

Google and Facebook Team Up to Modernize Old-School Databases

48 points by salemh 13y ago

mikkelewis 13y ago
Currently under what circumstances will we see performance increases, and by how much?
jpalomaki 13y ago
Previous discussion in HN at least here: https://news.ycombinator.com/item?id=7480843 https://news.ycombinator.com/item?id=7480843 Project page: http://webscalesql.org/ http://webscalesql.org/
rakoo 13y ago
I don't think "Webscale" is the right word; they should have used "Worldscale". The web part is irrelevant here.
- peferron 13y ago
  It's a joke referring to the "MongoDB is Web Scale" meme. The FAQ is quite suggestive: Q: Why is it called WebScaleSQL? A: While there are a variety of origin stories for the name depending on who you ask, ... They just don't want to mention it directly. :)
- RyanMcGreal 13y ago
  Every time I see the word "Webscale" I think of this animation: http://www.youtube.com/watch?v=b2F-DItXtZs http://www.youtube.com/watch?v=b2F-DItXtZs
  - coldtea 13y ago
    The name was SPECIFICALLY picked to refer to that animation, it's an inside joke kind of thing.
mixmastamyk 13y ago
What is the rationale of putting this work into Mysql? With its sloppy execution and the dark clouds of Oracle hanging over it? Why not Postgres, or even MariaDB instead?
- davidw 13y ago
  Because they are locked in to Mysql and can't easily switch to something better.
  - MrBuddyCasino 13y ago
    I'd really like to see the evidence for that. Postres > MySQL, and MySQL doesn't have that many lock-in features last time I looked. Any Google/FB/Twitter employees here than can elaborate?
    - HarrisonFisk 13y ago
      (I am a manager of the Data-Perf team at FB which deals with lots of different DB technologies) MySQL (specifically InnoDB) is extremely efficient as a storage backend compared to PostgreSQL. There are a few features that make InnoDB better in many cases: 1. Change buffering for IO bound workloads: If you are IO bound, then InnoDB change buffering is a huge, huge win. It basically is able to reduce IO required for secondary index maintenance by a huge amount. http://dev.mysql.com/doc/innodb/1.1/en/innodb-performance-change_buffering.html http://dev.mysql.com/doc/innodb/1.1/en/innodb-performance-ch... 2. InnoDB compression: When you are space constrained (say using flash storage), then being able to compress your data is a big win. In our case, it reduces space by around 40% which translates directly to 40% less servers required. While you could do something like run PG on ZFS with compression, for an OLTP workload, you want the compression in the DB so that it can do a lot of work to minimize the compressing and decompressing of data. https://dev.mysql.com/doc/refman/5.6/en/innodb-compression-internals.html https://dev.mysql.com/doc/refman/5.6/en/innodb-compression-i... 3. Clustered index: The InnoDB PK is a clustered index. This makes a lot of query patterns (such as range scans of the PK) very cheap. Combined with covering indexes (which PG now has too!), you can really minimize the IO required by properly tuned queries. There are a variety of smaller things as well, such as InnoDB doing logical writes to the redo log vs. PG doing full page writes, so on very high write systems, the REDO log bytes written will be dramatically less. Also MySQL replication has traditionally been more flexible than PG, but PG has made some great strides recently, so I don't know if I would maintain that position still.
collyw 13y ago
So relational databases are old school, and column stores are all the hip trendy thing, despite throwing away a ton of features?
- jlouis 13y ago
  Well, there are three types of databases: Relational, Columnar, and finally massively distributed. They throw away different feature sets in order to handle a certain subset better. The primary reason Relational databases are so friggin' strong is that they have a solid foundation and maturity. Most new "NoSQL" stuff is immature crap which should never be used in production for any kind of persistent data. But they are being used as such, and I do note that matures the products over time.
  - collyw 13y ago
    Tell me about it. I have been hauled into a project, where we are reimplementing a relational database using NoSQL. No idea why, other than the fact that they want to use NoSQL.
- ams6110 13y ago
  We are now clearly in a world where many technology journalists are entirely unaware of the history of computing before the year 2,000 or so.
  - coldtea 13y ago
    Worse, we are now clearly in a world where many developers are entirely unaware of the history of computing before the year 2,000 or so. I mean, we had "NoSQL" before we had SQL. Tons of schema-less, hierarchical etc DBs, talking directly to your program and such. Anybody that uses them without massive scaling needs doesn't know his history.
- avz 13y ago
  Relational databases are not the right choice for large-scale data stores serving large number of users because: - they provide unnecessary features at high cost (e.g. database-wide transactions), - they don't provide features that are essential (e.g. scalability, distribution, graceful degradation). By contrast NoSQL data stores like Bigtable are highly scalable and can easily be stacked over to provide more features if needed (see e.g. Megastore, http://pdos.csail.mit.edu/6.824-2011/papers/jbaker-megastore.pdf http://pdos.csail.mit.edu/6.824-2011/papers/jbaker-megastore...)
arrc 13y ago
Well not just Google / Facebook but Linkedin and Twitter are also in the team.

48 points by salemh 13y ago

Currently under what circumstances will we see performance increases, and by how much?

Previous discussion in HN at least here: https://news.ycombinator.com/item?id=7480843 https://news.ycombinator.com/item?id=7480843 Project page: http://webscalesql.org/ http://webscalesql.org/

rakoo 13y ago

I don't think "Webscale" is the right word; they should have used "Worldscale". The web part is irrelevant here.

peferron 13y ago
It's a joke referring to the "MongoDB is Web Scale" meme. The FAQ is quite suggestive: Q: Why is it called WebScaleSQL? A: While there are a variety of origin stories for the name depending on who you ask, ... They just don't want to mention it directly. :)
RyanMcGreal 13y ago
Every time I see the word "Webscale" I think of this animation: http://www.youtube.com/watch?v=b2F-DItXtZs http://www.youtube.com/watch?v=b2F-DItXtZs
- coldtea 13y ago
  The name was SPECIFICALLY picked to refer to that animation, it's an inside joke kind of thing.

mixmastamyk 13y ago

What is the rationale of putting this work into Mysql? With its sloppy execution and the dark clouds of Oracle hanging over it? Why not Postgres, or even MariaDB instead?

davidw 13y ago
Because they are locked in to Mysql and can't easily switch to something better.
- MrBuddyCasino 13y ago
  I'd really like to see the evidence for that. Postres > MySQL, and MySQL doesn't have that many lock-in features last time I looked. Any Google/FB/Twitter employees here than can elaborate?
  - HarrisonFisk 13y ago
    (I am a manager of the Data-Perf team at FB which deals with lots of different DB technologies) MySQL (specifically InnoDB) is extremely efficient as a storage backend compared to PostgreSQL. There are a few features that make InnoDB better in many cases: 1. Change buffering for IO bound workloads: If you are IO bound, then InnoDB change buffering is a huge, huge win. It basically is able to reduce IO required for secondary index maintenance by a huge amount. http://dev.mysql.com/doc/innodb/1.1/en/innodb-performance-change_buffering.html http://dev.mysql.com/doc/innodb/1.1/en/innodb-performance-ch... 2. InnoDB compression: When you are space constrained (say using flash storage), then being able to compress your data is a big win. In our case, it reduces space by around 40% which translates directly to 40% less servers required. While you could do something like run PG on ZFS with compression, for an OLTP workload, you want the compression in the DB so that it can do a lot of work to minimize the compressing and decompressing of data. https://dev.mysql.com/doc/refman/5.6/en/innodb-compression-internals.html https://dev.mysql.com/doc/refman/5.6/en/innodb-compression-i... 3. Clustered index: The InnoDB PK is a clustered index. This makes a lot of query patterns (such as range scans of the PK) very cheap. Combined with covering indexes (which PG now has too!), you can really minimize the IO required by properly tuned queries. There are a variety of smaller things as well, such as InnoDB doing logical writes to the redo log vs. PG doing full page writes, so on very high write systems, the REDO log bytes written will be dramatically less. Also MySQL replication has traditionally been more flexible than PG, but PG has made some great strides recently, so I don't know if I would maintain that position still.

collyw 13y ago

So relational databases are old school, and column stores are all the hip trendy thing, despite throwing away a ton of features?

jlouis 13y ago
Well, there are three types of databases: Relational, Columnar, and finally massively distributed. They throw away different feature sets in order to handle a certain subset better. The primary reason Relational databases are so friggin' strong is that they have a solid foundation and maturity. Most new "NoSQL" stuff is immature crap which should never be used in production for any kind of persistent data. But they are being used as such, and I do note that matures the products over time.
- collyw 13y ago
  Tell me about it. I have been hauled into a project, where we are reimplementing a relational database using NoSQL. No idea why, other than the fact that they want to use NoSQL.
ams6110 13y ago
We are now clearly in a world where many technology journalists are entirely unaware of the history of computing before the year 2,000 or so.
- coldtea 13y ago
  Worse, we are now clearly in a world where many developers are entirely unaware of the history of computing before the year 2,000 or so. I mean, we had "NoSQL" before we had SQL. Tons of schema-less, hierarchical etc DBs, talking directly to your program and such. Anybody that uses them without massive scaling needs doesn't know his history.
avz 13y ago
Relational databases are not the right choice for large-scale data stores serving large number of users because: - they provide unnecessary features at high cost (e.g. database-wide transactions), - they don't provide features that are essential (e.g. scalability, distribution, graceful degradation). By contrast NoSQL data stores like Bigtable are highly scalable and can easily be stacked over to provide more features if needed (see e.g. Megastore, http://pdos.csail.mit.edu/6.824-2011/papers/jbaker-megastore.pdf http://pdos.csail.mit.edu/6.824-2011/papers/jbaker-megastore...)

arrc 13y ago

Well not just Google / Facebook but Linkedin and Twitter are also in the team.