4 ms·
Twitter open sources its MySQL secret sauce
- kuviaq 15y agoBetter link: http://engineering.twitter.com/2012/04/mysql-at-twitter.html http://engineering.twitter.com/2012/04/mysql-at-twitter.html
- revelation 15y agoDid I read 'fork'? If you want the open-source cred, make an attempt to get your patches upstream. Google has done it for MySQL (pre-Oracle).
- amateurguru 15y agoGoogle has their own branch: https://code.google.com/p/google-mysql/ https://code.google.com/p/google-mysql/ Facebook too: https://code.launchpad.net/mysqlatfacebook https://code.launchpad.net/mysqlatfacebook
- polyfractal 15y agoI don't know if it applies in this situation, but certain cases make sense to fork instead of patch. If they are including features that are orthogonal to general MySQL feature set, it shouldn't be patched. Considering their size and unique requirements induced by that size, I imagine a lot of these changes do not apply to small MySQL installations running WordPress (and in fact may be highly detrimental).
- jeremycole 15y agoI think all of the changes pretty much do apply generally, but for us it's easier to go ahead and publish publically to be able to pass these changes upstream, rather than pass them off privately and wait potentially a year or more to get them back in a public release. We want to be able to collaborate with Percona, Facebook, Google, MariaDB, etc., on some of the actual changes, to make things better for everyone. This is the first step towards that goal.
- furyofantares 15y agoIf trying to get their patches upstream doesn't make sense to them internally, it probably won't make sense to others.
- jeremycole 15y agoThis is actually not really about whether it makes sense, but about timelines. We expect some of this to eventually make it to upstream MySQL, but likely in a version newer than we need it in (5.6 instead of 5.5) and not for something like a year.
- hartror 15y agoIt is a fork in that it is a parallel line of development but from reading Twitter's blog the desire is to get some or all of their work upstream. Releasing the fork like this gets them some early OSS & engineering cred while being able to start the conversation on their modifications. We look forward sharing our work with upstream and other downstream MySQL vendors, with a goal to improve the MySQL community. http://engineering.twitter.com/2012/04/mysql-at-twitter.html http://engineering.twitter.com/2012/04/mysql-at-twitter.html
- jeremycole 15y agoIt's a fork in the sense that we're developing without an immediate requirement to get MySQL/Oracle to accept our changes. We hope they will accept the changes, of course, but it isn't and can't be a requirement for our own progress.
- cdibona 15y agoThis might be ideal, but it can be very difficult to work with an upstream company like Oracle (who are the copyright holders to MySQL).
- TazeTSchnitzel 15y agoMaybe we'll see another LibreOffice?
- drivebyacct2 15y agoIs this something that lives in the same space as http://code.google.com/p/vitess/ http://code.google.com/p/vitess/ ?
- simonw 15y agoNot quite - vitess is a proxy that sits in front of MySQL and adds additional features, whereas Twitter are releasing their own fork of MySQL which contains optimisations they have developed for their environment.
- jeremycole 15y agoNo, Vitess is more along the lines of our Gizzard system which has been out there for a while: https://github.com/twitter/gizzard https://github.com/twitter/gizzard
- deleted 15y ago[deleted]
- neilparikh 15y agoLink to github repo: https://github.com/twitter/mysql https://github.com/twitter/mysql
- morganpyne 15y agoIs there any overlap between the work done in Gizzard and the other popular MySQL forks (Percona, MariaDB, Drizzle) or are each of these solving different (incompatible) problems?
- busted 15y agoGizzard is sharding framework that sits on top of, as a separate layer from, MySQL (or any other storage backend, like Redis). It's open sourced right now at https://github.com/twitter/gizzard https://github.com/twitter/gizzard. Twitter's MySQL fork for now is just a way to get in bug fixes and extra features that are needed but not available upstream.
- jeremycole 15y agoYup! And our changes are quite complementary to Gizzard, since it's one of the larger use cases at Twitter for MySQL. More to come in the future to better support big Gizzard systems!
- morganpyne 15y agoThanks for the clarifications. And the work :-)