3 ms·

I worked at a company who re-implemented the entire Dynamo paper and API, and it was exactly the same story. Completely eliminated all my illusions about the su

I worked at a company who re-implemented the entire Dynamo paper and API, and it was exactly the same story. Completely eliminated all my illusions about the supposed superiority of distributed systems. It was a mound of tires held together with duct tape, with a tiki torch in each tire.

yftsui 5y ago
Dynamo paper and DynamoDB are two very different things…
- 0xbadcafebee 5y ago
  Err you're right, they re-used something that implemented the Dynamo paper (Riak) and implemented on top of it the DynamoDB API.
AtlasBarfed 5y ago
Did they have a spare 100 million hanging around to burn? That seems pretty ridiculous. Why did they not just run cassandra?
- 0xbadcafebee 5y ago
  They did have 100 million to burn, but my mostly-wild-guess is it was closer to $1.5M/yr. But that gives you an in-house SaaS DB used across a hundred other teams/products/services, so it actually saved money (and nothing else matched its performance/CAP/functionality). Cassandra is too opinionated and its CAP behavior wasn't great for a service like this, so they built on top of Riak. (This also eliminated any thoughts I had about Erlang being some uber-language for distributed systems, as there were (are?) tons of bugs and missing edge cases in Riak)
  - staticassertion 5y ago
    Erlang gives you great primitives for building reliable protocols, but they're just primitives, and there are tons of footguns since building protocols is hard.
  - AtlasBarfed 5y ago
    Because Riak uses vector clocks instead of cell timestamps? Cassandra's ONE/QUORUM/ALL consistency levels otherwise allow tuning for tolerance of CP vs AP, don't they?
    - 0xbadcafebee 5y ago
      To be honest I don't know, I wasn't there for the initial decision, but I know it wasn't just about CAP. It could have been as simple as Riak was easier to use (which I don't know either)
- spookthesunset 5y ago
  > Why did they not just run cassandra? Not Invented Here can run very deep in some branches of an organization. Depending on how engineering performance evaluations work, writing a homebrew database could totally be something that aligns with the company incentives. It might not make a single bit of sense from a business standpoint but hey, if the company rewards such behavior don't be surprised when engineers flush millions down the tube "innovating" a brand new wheel.

yftsui 5y ago

Dynamo paper and DynamoDB are two very different things…

0xbadcafebee 5y ago
Err you're right, they re-used something that implemented the Dynamo paper (Riak) and implemented on top of it the DynamoDB API.

AtlasBarfed 5y ago

Did they have a spare 100 million hanging around to burn? That seems pretty ridiculous. Why did they not just run cassandra?

0xbadcafebee 5y ago
They did have 100 million to burn, but my mostly-wild-guess is it was closer to $1.5M/yr. But that gives you an in-house SaaS DB used across a hundred other teams/products/services, so it actually saved money (and nothing else matched its performance/CAP/functionality). Cassandra is too opinionated and its CAP behavior wasn't great for a service like this, so they built on top of Riak. (This also eliminated any thoughts I had about Erlang being some uber-language for distributed systems, as there were (are?) tons of bugs and missing edge cases in Riak)
- staticassertion 5y ago
  Erlang gives you great primitives for building reliable protocols, but they're just primitives, and there are tons of footguns since building protocols is hard.
- AtlasBarfed 5y ago
  Because Riak uses vector clocks instead of cell timestamps? Cassandra's ONE/QUORUM/ALL consistency levels otherwise allow tuning for tolerance of CP vs AP, don't they?
  - 0xbadcafebee 5y ago
    To be honest I don't know, I wasn't there for the initial decision, but I know it wasn't just about CAP. It could have been as simple as Riak was easier to use (which I don't know either)
spookthesunset 5y ago
> Why did they not just run cassandra? Not Invented Here can run very deep in some branches of an organization. Depending on how engineering performance evaluations work, writing a homebrew database could totally be something that aligns with the company incentives. It might not make a single bit of sense from a business standpoint but hey, if the company rewards such behavior don't be surprised when engineers flush millions down the tube "innovating" a brand new wheel.