3 ms·
Sharding is basically a data storage concern, so it's not something that the language can really control. There are however, frameworks that enforce some level
by hn_throwaway_99 8y ago
Sharding is basically a data storage concern, so it's not something that the language can really control.
There are however, frameworks that enforce some level of "shardability". The Datastore (now Firestore) in AppEngine/Firebase comes to mind, in that your data is shared all over the place (the details are hidden from you), and you can't write un-indexed queries that don't scale well. The Datastore has some limitations that definitely seem awkward if you're coming from a relational DB world (e.g. no joins, limits on the kinds of inequality queries you can do, etc.), but these limitations are there specifically so Datastore can guarantee performant distributed queries.
- sb8244 8y agoSharding is not just about data storage. You can shard parallel units of work across multiple CPU cores in order to prevent single CPU bottlenecks. Erlang encourages shardable solutions from OTP concepts being so strongly integrated into Erlang.
- StreamBright 8y agoAbsolutely not. While working for Amazon we had to think about a lot how to "shard" our fleets to have the least amount of items in every cache and maintain high hit ratios. If you let all traffic hit a single cluster your cache hit ratios will be terrible and most customers are going to experience worst case latency which is absolutely do not work for popular websites.