6 ms·
No-one seems to have yet mentioned Facebook's TAO paper, which describes scaling basically this approach to 1 billion reads per second. https://cs.uwaterloo.ca/
by ladon86 11y ago
No-one seems to have yet mentioned Facebook's TAO paper, which describes scaling basically this approach to 1 billion reads per second. https://cs.uwaterloo.ca/~brecht/courses/854-Emerging-2014/readings/data-store/tao-facebook-distributed-datastore-atc-2013.pdf https://cs.uwaterloo.ca/~brecht/courses/854-Emerging-2014/re...
The nice thing about your approach is that you can shard your objects by their id and your edges by their "from" id, and have all lookups go to one box when you shard. Throw in some caching and it scales to multiple boxes really well.
- jordanchan 11y agoYou said it! Actually, Facebook's TAO was an early inspiration for me as well, but it is really customized and complex too, so probably not something for a tiny team. But the concepts from TAO are applicable to many similar scenarios.