3 ms·
I think that’s a large oversimplification of Facebook. While it’s true a lot of FB storage is MySQL backed they also created many complex systems such as: - Ca
by mapme 4y ago
I think that’s a large oversimplification of Facebook. While it’s true a lot of FB storage is MySQL backed they also created many complex systems such as:
- Cassandra (based on dynamo/big table)
- wrote a custom KV store named RocksDb that is open source/now a company
- wrote a custom photos storage system that replaced an NFS based design
- wrote another custom binary object store
- wrote a custom geo distributed graph db (Tao)
- wrote an in house distributed FS replacement for HDFS
https://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf https://www.cs.cornell.edu/projects/ladis2009/papers/lakshma...
https://www.usenix.org/legacy/event/osdi10/tech/full_papers/Beaver.pdf https://www.usenix.org/legacy/event/osdi10/tech/full_papers/...
https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-muralidhar.pdf https://www.usenix.org/system/files/conference/osdi14/osdi14...
https://www.usenix.org/system/files/conference/atc13/atc13-bronson.pdf https://www.usenix.org/system/files/conference/atc13/atc13-b...
https://www.cs.princeton.edu/~wlloyd/papers/tectonic-fast21.pdf https://www.cs.princeton.edu/~wlloyd/papers/tectonic-fast21....
https://m.facebook.com/nt/screen/?params=%7B%22note_id%22%3A10158791582997200%7D&path=%2Fnotes%2Fnote%2F&refsrc=deprecated&_rdr https://m.facebook.com/nt/screen/?params=%7B%22note_id%22%3A...
- kragen 4y agoI think all of this was after their first billion users?
- dismantlethesun 4y agoCassandra was released in 2008. Facebook hit 1 billion users ~ 2012, and 2 billion ~ 2017. Back when Cassandra was released, they had a 'mere' 100 million users. Facebook stats: https://www.statista.com/statistics/264810/number-of-monthly-active-facebook-users-worldwide/ https://www.statista.com/statistics/264810/number-of-monthly...
- kragen 4y agoThanks for the correction! So the more correct statement would be that Facebook scaled to 100 million users with PHP, MySQL, and Memcache, and then to a billion users with Cassandra and Haystack, and then built all the other stuff after their first billion?
- mapme 4y agoYeah, there’s also a lag sometimes ie. They write the paper 1-2 years after building the system. They’ve published some really interesting papers over time. If you filter by year you can see them all! https://research.facebook.com/publications/ https://research.facebook.com/publications/