3 ms·
Wow, never realized Pinterest had 6 petabytes of data. I wonder if they're including images in that. Even billions of rows is usually around 1-2tb, so makes you
by winrid 2y ago
Wow, never realized Pinterest had 6 petabytes of data. I wonder if they're including images in that. Even billions of rows is usually around 1-2tb, so makes you wonder what they're storing many billions of.
- ddorian43 2y agoIt makes no sense at their scale to store images in Hbase. It's probably just user tracking/analytics.
- badpun 2y agoIn NoSQL world, you often store multiple duplicates of the same data, each of which is optimized for a different use case.
- dehrmann 2y agoThe SQL world calls this an "index."
- etc-hosts 2y agoI am attempting to reach enlightenment by contemplating your wisdom.
- winrid 2y agoNo. Most NoSQL dbs support indexes and secondary indexes too. In SQL world we call this "denormalization". It's called the same thing in NoSQL, too.
- badpun 2y agoIndices are no good if you constantly do large amounts of inserts or updates (say, entire clickstream of a popular global site) - keeping them up to date will massively slow down the inserts/updates. Wheresas, in NoSQL, you can do as inserts as quickly as your disk can write data, and still query the dataset as if it had an index.
- jeffbee 2y agoIf a company has HBase and doesn't have anything else, then the developers will store everything in HBase due to lack of alternatives. They'll store logs in HBase, metrics in HBase, images in HBase.
- evanelias 2y agoThey almost certainly have trillions of rows of data. The popular social media / user-generated content sites are just huge. You can easily get to that size just from the core product OLTP data/metadata, no need to include media. For one comparison, Tumblr hit 100 billion unique rows of relational data in MySQL (on masters, not including replicas) back in October 2012. So they're easily in the trillions of rows today, and Tumblr is smaller than Pinterest!
- winrid 2y agoJust surprising since I've never seen the social side or anyone using it. Guess I'm just not in that social bubble.