3 ms·

Those slides are incredibly dense and it's a little difficult to tell exactly what they mean. The OP's twitter stream has some good info though: "facebook is c

Those slides are incredibly dense and it's a little difficult to tell exactly what they mean. The OP's twitter stream has some good info though: "facebook is carving out entire chunks from InnoDB to scale up concurrency: no deadlock detection, no read-ahead.." http://twitter.com/igrigorik/status/12117662626 http://twitter.com/igrigorik/status/12117662626 "myisam has single mutex, will fall down after 3+ cores try to access it concurrently. stick with innnodb" http://twitter.com/igrigorik/status/12116958816 http://twitter.com/igrigorik/status/12116958816

igrigorik 16y ago
Yep, just to provide some context / quick review of the talk: It seems like the primary concern for Domas (presenter) is concurrency within InnoDB. Just to get it out of the way, he dismissed MyISAM, and went right into the low level optimizations & tweaks they're working on at Facebook. High level points are: - they're using XFS, and he stressed understanding the block math and how it relates to your record sizes, etc. - using "correct" scheduler has a big impact - swappiness = 0, etc. For InnoDB itself, it seems like they're carving out entire chunks of functionality that he claims "never worked correctly" anyway. For example, deadlock detection is something they disabled outright because it gave them a large perf boost. Likewise, for read-ahead functionality within InnoDB. Those were the two he went into a lot of details. Concurrency + threads: there is no definitive formula, but he suggested setting thread concurrency to be the min concurrency of your lightweight requests (claimed: threads are cheap). For ops / day-to-day, they focus on slow servers and dive right in: custom tools + gdb + freeze the server. They are playing with the idea of providing stored procedures interface to app developers since it'll avoid the round trip time between different queries (TBD). Also, they sometimes go very aggressive (timeouts of 3s, or even 1s), to force internal app devs to respect the DB layer. Also hinted that they're using "smart caches" which are capable of buffering counters / updates and doing one update instead. All in all, their biggest problems seems to be thousands of (app) servers that are always connected, hence the need for high concurrency... (I'm sure I'm forgetting a whole lot, but that's off the top of my head)
- spudlyo 16y ago
  Thanks a lot for the summary. Wish I was there.

igrigorik 16y ago

Yep, just to provide some context / quick review of the talk: It seems like the primary concern for Domas (presenter) is concurrency within InnoDB. Just to get it out of the way, he dismissed MyISAM, and went right into the low level optimizations & tweaks they're working on at Facebook. High level points are: - they're using XFS, and he stressed understanding the block math and how it relates to your record sizes, etc. - using "correct" scheduler has a big impact - swappiness = 0, etc. For InnoDB itself, it seems like they're carving out entire chunks of functionality that he claims "never worked correctly" anyway. For example, deadlock detection is something they disabled outright because it gave them a large perf boost. Likewise, for read-ahead functionality within InnoDB. Those were the two he went into a lot of details. Concurrency + threads: there is no definitive formula, but he suggested setting thread concurrency to be the min concurrency of your lightweight requests (claimed: threads are cheap). For ops / day-to-day, they focus on slow servers and dive right in: custom tools + gdb + freeze the server. They are playing with the idea of providing stored procedures interface to app developers since it'll avoid the round trip time between different queries (TBD). Also, they sometimes go very aggressive (timeouts of 3s, or even 1s), to force internal app devs to respect the DB layer. Also hinted that they're using "smart caches" which are capable of buffering counters / updates and doing one update instead. All in all, their biggest problems seems to be thousands of (app) servers that are always connected, hence the need for high concurrency... (I'm sure I'm forgetting a whole lot, but that's off the top of my head)

spudlyo 16y ago
Thanks a lot for the summary. Wish I was there.