4 ms·

> they'd written a monolith instead of building a system where the message delivery was designed to be sharded; n Do some languages handle this differently? I

> they'd written a monolith instead of building a system where the message delivery was designed to be sharded; n Do some languages handle this differently? Isnt this based on how the code was written rather than a language natively sharding?

hn_throwaway_99 7y ago
Sharding is basically a data storage concern, so it's not something that the language can really control. There are however, frameworks that enforce some level of "shardability". The Datastore (now Firestore) in AppEngine/Firebase comes to mind, in that your data is shared all over the place (the details are hidden from you), and you can't write un-indexed queries that don't scale well. The Datastore has some limitations that definitely seem awkward if you're coming from a relational DB world (e.g. no joins, limits on the kinds of inequality queries you can do, etc.), but these limitations are there specifically so Datastore can guarantee performant distributed queries.
- sb8244 7y ago
  Sharding is not just about data storage. You can shard parallel units of work across multiple CPU cores in order to prevent single CPU bottlenecks. Erlang encourages shardable solutions from OTP concepts being so strongly integrated into Erlang.
- StreamBright 7y ago
  Absolutely not. While working for Amazon we had to think about a lot how to "shard" our fleets to have the least amount of items in every cache and maintain high hit ratios. If you let all traffic hit a single cluster your cache hit ratios will be terrible and most customers are going to experience worst case latency which is absolutely do not work for popular websites.
njharman 7y ago
Yes erlang does. Unless you try hard and work against the language erlang systems are written differently than ruby/rails program. I used "system" and "program" there specifically. One of the key things erlang "forces". Languages do matter.
vidarh 7y ago
Yes, that was largely my point. While Rails at the time probably did sort-of encourage a monolith, there was nothing requiring it to be that way; it was their design choices, not Ruby or Rails that caused their scaling problems at the time they moved away from Rails. When they were rewriting anyways, it's very much possibly that moving off Rails was the right choice anyway, but that was incidental to the far bigger problem of their broken architecture.

hn_throwaway_99 7y ago

Sharding is basically a data storage concern, so it's not something that the language can really control. There are however, frameworks that enforce some level of "shardability". The Datastore (now Firestore) in AppEngine/Firebase comes to mind, in that your data is shared all over the place (the details are hidden from you), and you can't write un-indexed queries that don't scale well. The Datastore has some limitations that definitely seem awkward if you're coming from a relational DB world (e.g. no joins, limits on the kinds of inequality queries you can do, etc.), but these limitations are there specifically so Datastore can guarantee performant distributed queries.

sb8244 7y ago
Sharding is not just about data storage. You can shard parallel units of work across multiple CPU cores in order to prevent single CPU bottlenecks. Erlang encourages shardable solutions from OTP concepts being so strongly integrated into Erlang.
StreamBright 7y ago
Absolutely not. While working for Amazon we had to think about a lot how to "shard" our fleets to have the least amount of items in every cache and maintain high hit ratios. If you let all traffic hit a single cluster your cache hit ratios will be terrible and most customers are going to experience worst case latency which is absolutely do not work for popular websites.

njharman 7y ago

Yes erlang does. Unless you try hard and work against the language erlang systems are written differently than ruby/rails program. I used "system" and "program" there specifically. One of the key things erlang "forces". Languages do matter.

vidarh 7y ago

Yes, that was largely my point. While Rails at the time probably did sort-of encourage a monolith, there was nothing requiring it to be that way; it was their design choices, not Ruby or Rails that caused their scaling problems at the time they moved away from Rails. When they were rewriting anyways, it's very much possibly that moving off Rails was the right choice anyway, but that was incidental to the far bigger problem of their broken architecture.