4 ms·
Twitter, LinkedIn and Facebook all make some bits and pieces of their system available as open-source. You might particularly want to look at Twitter's FlockDB
by mindcrime 16y ago
Twitter, LinkedIn and Facebook all make some bits and pieces of their system available as open-source. You might particularly want to look at Twitter's FlockDB[1], LinkedIn's Kafka[2], and/or Facebook's Scribe[3].
[1]: https://github.com/twitter/flockdb https://github.com/twitter/flockdb
[2]: http://sna-projects.com/kafka/ http://sna-projects.com/kafka/
[3]: https://github.com/facebook/scribe https://github.com/facebook/scribe
FlockDB is designed to deal with the "social graph" connection stuff (who follows who, etc.) and is optimized for that sort of thing. Kafka and Scribe are more about delivering message widely. Poke around, look at their code, chase down and associated papers, and you should be able to sort out an approach... even if you don't adopt any of those codebases directly. FWIW, I'm looking at integrating FlockDB for some stuff I'm working on, but I haven't gotten very far down that path yet, so I don't have a lot more to offer than this.
- rhasson 16y agoThank you this is a really great starting point.