4 ms·

I still think its amazing that Facebook is running off MySQL. Such a tremendous amount of user data, placed in a system that many deem inferior. I'm not saying

I still think its amazing that Facebook is running off MySQL. Such a tremendous amount of user data, placed in a system that many deem inferior. I'm not saying they made a bad choice, I'm just impressed at how they've managed to make MySQL fit their needs.

GuiA 13y ago
I'm guessing the way they use MySQL is similar to the way they use PHP? Impressive indeed.
RyanZAG 13y ago
I was under the impression they use it more as a sharded "key-values" store? ie. no joins - the part of the database that makes it relational.
- gbog 13y ago
  I concur. In our shop we also use MySQL without the joins. That's like an efficient key-value store, but there must be a better solution somewhere: all the joins are done in client code, and caching too. That's a great amount of error-prone code that probably shouldn't be necessary.
  - bello 13y ago
    "all the joins are done in client code" What's the point of joining in the application level? If you're going to join, why not do it in the database? That should be both faster and more convenient (unless your schemas aren't relational at all, in which case I don't see why you'd use a relational database)
    - RyanZAG 13y ago
      Data for User A is on server 9, and data for User B is on server 43. Joins across servers is not really supported in mysql. Another issue is connections between DB servers. Generally you want your DB server to be as fast as possible, so having it handle the connections to other servers slows everything down. If you offload the DB server connections to your web server, you can easily scale by just adding more web servers and having each DB server handle only its own data. The best solution would be some type of 'mysql proxy' that could run on every web server that would transparently handle the joins between all the different mysql data servers. I think I saw a project attempting to do this awhile back, but didn't really keep track.
    - sicxu 13y ago
      This is because join negatively impact the scalability of a DB: - When huge amount of traffic hits your DB, out of the most commonly run-out resources - CPU, Memory, and IO - CPU is usually the one that runs out first. And Join uses quite a bit of CPU. - When you have a lot of data and have to partition it, you will feel so lucky if you don't have to deal with distributed joins.
rimantas 13y ago
> that many deem inferior. I guess most of those "many" never actually run anything close to the scale of the Facebook (or Wikipedia, which also runs MySQL). If you bother to pay attention what FB DB engineers say you will know that they have their arms shoulder deep into the innards of the database itself, I/O stack, all that jazz.
- davidw 13y ago
  > If you bother to pay attention what FB DB engineers say you will know that they have their arms shoulder deep into the innards of the database itself, I/O stack, all that jazz. Most of us can't afford that, and want something that's pretty good out of the box. I've used Mysql a number of times, but it always seems to have more gotchas for what I need to do with it than Mysql.
  - tim333 13y ago
    Mysql has more gotchas than Mysql??
    - davidw 13y ago
      That Postgres, sorry.
eknkc 13y ago
I wonder if it fits their needs or they are stuck with it? I mean, at some point when you hit petabyte scales, is it even plausible to move to other data stores / rebuild indexes etc? I would not dare to do that.

GuiA 13y ago

I'm guessing the way they use MySQL is similar to the way they use PHP? Impressive indeed.

RyanZAG 13y ago

I was under the impression they use it more as a sharded "key-values" store? ie. no joins - the part of the database that makes it relational.

gbog 13y ago
I concur. In our shop we also use MySQL without the joins. That's like an efficient key-value store, but there must be a better solution somewhere: all the joins are done in client code, and caching too. That's a great amount of error-prone code that probably shouldn't be necessary.
- bello 13y ago
  "all the joins are done in client code" What's the point of joining in the application level? If you're going to join, why not do it in the database? That should be both faster and more convenient (unless your schemas aren't relational at all, in which case I don't see why you'd use a relational database)
  - RyanZAG 13y ago
    Data for User A is on server 9, and data for User B is on server 43. Joins across servers is not really supported in mysql. Another issue is connections between DB servers. Generally you want your DB server to be as fast as possible, so having it handle the connections to other servers slows everything down. If you offload the DB server connections to your web server, you can easily scale by just adding more web servers and having each DB server handle only its own data. The best solution would be some type of 'mysql proxy' that could run on every web server that would transparently handle the joins between all the different mysql data servers. I think I saw a project attempting to do this awhile back, but didn't really keep track.
  - sicxu 13y ago
    This is because join negatively impact the scalability of a DB: - When huge amount of traffic hits your DB, out of the most commonly run-out resources - CPU, Memory, and IO - CPU is usually the one that runs out first. And Join uses quite a bit of CPU. - When you have a lot of data and have to partition it, you will feel so lucky if you don't have to deal with distributed joins.

rimantas 13y ago

> that many deem inferior. I guess most of those "many" never actually run anything close to the scale of the Facebook (or Wikipedia, which also runs MySQL). If you bother to pay attention what FB DB engineers say you will know that they have their arms shoulder deep into the innards of the database itself, I/O stack, all that jazz.

davidw 13y ago
> If you bother to pay attention what FB DB engineers say you will know that they have their arms shoulder deep into the innards of the database itself, I/O stack, all that jazz. Most of us can't afford that, and want something that's pretty good out of the box. I've used Mysql a number of times, but it always seems to have more gotchas for what I need to do with it than Mysql.
- tim333 13y ago
  Mysql has more gotchas than Mysql??
  - davidw 13y ago
    That Postgres, sorry.

eknkc 13y ago

I wonder if it fits their needs or they are stuck with it? I mean, at some point when you hit petabyte scales, is it even plausible to move to other data stores / rebuild indexes etc? I would not dare to do that.