4 ms·
How big is the team over at pinterest managing the MySQL infrastructure? I think this is the big difference here. I have a particular project. The database h
by no1youknowz 11y ago
How big is the team over at pinterest managing the MySQL infrastructure? I think this is the big difference here.
I have a particular project. The database has well over 1 billion rows in multiple tables which need joins to generate complex reporting.
As the only tech here, there is only so much I can be working on. Before... time spent on trying to get MySQL to work, was time not well spent. I spent countless hours on IRC talking to devs on #mysql/#mariadb and was told many times, you seem to have the indexes in the right places and the query was well written, you are lucky you are getting it that fast.
Sure, if I had a huge team like Instgram (who I think sharded mysql) and Pinterest. I could assign dbas to get my MySQL infrastructure to scale and possibly fork my application to work different.
I even called Percona and Mariadb and was given pricing from $50 to $70k to work the issue. Note, that they didn't even promise to resolve it. They needed between $10 to $20k up front to check my schema and the data and then the other sum to do the work. Really, almost $100k and no concrete answer?
So I don't have a team and I didn't employ Percona/Mariadb to help solve my issue. Instead, I did something completely different. Some may even say drastic.
I need something simple, that will work with the 5x9's and is very robust. So instead, I went to Postgres. I ran to Postgres. I ran and didn't look back.
With Postgres, 80% of my issues went away. PG 9.5 has some features that ALL of my issues will go away and I may be able to have quarterly queries of 10b rows with multiple tables in under 2 seconds. Cloudflare is currently doing it and I'm trying to replicate it.
Bottom line is this. MySQL for me, was a dead end. Postgres seems to every year go from strength to strength.
Postgres isn't shiny. It's dependable, actually cares what you are putting in the database. Makes your schema simple because you can go from var(x)/int(x) to text/int and it's still fast.
I've converted about 10 other dbas who were used to MySQL and thought PG was hard, to switch over. Now, like me. They won't look back.
If you are still using MySQL. Take a look at Postgres. MySQL has lost it's shine. It's now dull.
Fingers crossed PG 9.6 comes with MPP capability. That would really blow the doors off!
- clessg 11y agoI agree. MySQL has been one problem after another. Postgres isn't perfect either, but it is constantly improving and normally does the right thing. The idea of Postgres being a "shiny new toy" is... an interesting viewpoint.
- nadams 11y ago> MySQL has been one problem after another. Please elaborate. MySQL/MaraiaDB can be pretty relaxed when it comes to SQL standards (which you can disable/tweak), but I've yet to have data loss or data corruption. And speaking from experience - the system would be at it's knees and MySQL would happily do it's best to answer queries. I'm not saying Postgres isn't stable (haven't really used it) - I'm just rather suspicious of the comments here. I find it hard to believe that it's as broken as people here claim when big businesses are using it [1]. [1] http://stackoverflow.com/questions/1113381/what-databases-do-the-world-wide-webs-biggest-sites-run-on http://stackoverflow.com/questions/1113381/what-databases-do...
- ltorresv 11y agoSame here. I've been a MySQL DBA for over 10 years and I've never had data loss or corruption even with power supplies dying and servers shutting down completely. I've also used PG in quite a few projects and I can't say I find it better or faster at least for my usage. If anything it's more difficult to manage, there are way less decent tools to monitor, profiling is a PIA and those backslash commands just drive me nuts. I hate stuff like "\dt".
- devonkim 11y agoA lot of the original article is about a retrospective of technologies pinterest seemed to have picked before they were able to afford a fairly large team. If you do have a really huge engineering budget and can afford a commercial support contract, you are now in the business of trying to spend money moreso to maintain your existing business than to try to scale it faster and faster.
- tracker1 11y agoDepending on your reporting needs, or data shaping, you may be better off doing rolling exports, or additional reporting to a Hadoop or ElasticSearch setup for your reporting... Especially since reports are pretty much snapshots in time, not real time... no need to slow down the server for other users.
- chadcatlett 11y agoInstagram used/uses PostgreSQL http://instagram-engineering.tumblr.com/post/10853187575/sharding-ids-at-instagram http://instagram-engineering.tumblr.com/post/10853187575/sha...