5 ms·
" Sites that are read-only or store blob data in something like S3 can often avoid sharding" Still too broad. Sorry, this is a pet peeve of mine, where tech pe
by jshen 14y ago
" Sites that are read-only or store blob data in something like S3 can often avoid sharding"
Still too broad. Sorry, this is a pet peeve of mine, where tech people assume that everyone else has the same issues as them. For example, I worked on an ecommerce site that made over a mil a year. They had less than 10k products, and will never need sharding. They are not read-only, they have people updating their products on a daily basis through the site.
- nsanch 14y agoSorry, those were meant to be examples, not an exhaustive list.
- jshen 14y agoLet me flip it around. You said, "I wouldn't build a serious site on top of anything that didn't have some sort of built-in sharding story". I think the opposite. Most serious sites will never need sharding.
- taligent 14y agoYou're joking, right ? Every large site on the internet has talked about strategies around sharding. At some point you are surely going to hit the physical limitations of one database on one server.
- jshen 14y agoHe said "serious site". He did not say "large site". For example, techcrunch is most certainly a serious site. It is NOT large scale and in need of sharding.
- nsanch 14y agoMy definition of seriousness includes some relatively large scale. Your definition of seriousness appears to mean any site that is important to the person or business running it. Is that a fair assessment? I think your definition is better, and I should've said "a potentially-large site" or something like that.
- jshen 14y agoYes, this. Thanks for listening. It's a pet peeve of mine because most people work on serious sites, and a minority of them have large scale data, but people often imply that everyone needs to solve massive scaling problems even if they will never have them.
- leothekim 14y agoWhat do you mean by "serious"? That's broadly stated as well.
- jshen 14y agotechcrunch is a serious site. They don't need sharding. They have thousands of posts, not millions, and certainly not billions or trillions.