4 ms·
I think you might be asking the wrong questions. They key questions are Queries per second and the median response size of the query. For example at reddit (1
by jedberg 2y ago
I think you might be asking the wrong questions. They key questions are Queries per second and the median response size of the query.
For example at reddit (15 years ago) we had 10x more vote traffic than comment traffic, but we only needed two databases to handle votes (technically only one the other was just for redundancy).
But we needed nine comments databases. Mainly because the median query response was so much bigger for comments.
- deleted 2y ago[deleted]
- jascination 2y agoUnrelated: I love HN; a random database question and fkn jedberg is one of the first responders
- chistev 2y agoWho is he?
- RandomBacon 2y agoApparently the first paid employee of reddit https://www.jedberg.net https://www.jedberg.net
- mayli 2y agojedberg.net https://www.jedberg.net https://www.jedberg.net I was the first (paid) employee of reddit, and currently run Site Reliability at Netflix.
- ratg13 2y agoYou can click on HN usernames to read their profile.
- moralestapia 2y agojascinating!
- cogman10 2y agoI'm sure latency matters a lot as well. The users of our apps are pretty tolerant of 5 to 10 minute request times for some of our pages, which means we've been able to get away with just a few servers for several TBs of data stored and served. (100+mb responses are not unusual for us). If we had to rethink and redesign the system to cut down those times, we'd need a lot more databases and a much cleverer storage strategy than we currently have. While I'm sure response time for Reddit is really important, I could imagine that an IOT serving system needs almost nothing in to hit something like a 10 to 20 second response time for 5k devices.
- ajhool 2y agoThank you very much! Did you start with separate databases or wait until you needed to scale workloads separately before breaking things up?
- chistev 2y agoI imagine they didn't optimize prematurely.
- dpkirchner 2y agoNot to denigrate jedberg but a lot of folks optimized prematurely before realizing they shouldn't. (I've certainly been guilty of it.)