3 ms·

What about DynamoDB? I'm pretty sure they could open source a lot but they just don't want. Google had the same pathetic excuse when asked why they don't open s

What about DynamoDB? I'm pretty sure they could open source a lot but they just don't want. Google had the same pathetic excuse when asked why they don't open source appengine or datastore.

talawahdotnet 8y ago
I mean it would be cool if they did, but the secret sauce behind DynamoDb is really the scale, without it it probably wouldn't have the performance characteristics end users are looking for. Nor do I think AWS ever wants to be in the business of supporting a non-managed piece of software.
mbell 8y ago
> What about DynamoDB? From what I recall DynamoDB is a somewhat hacky API layer over a bunch of MySQL clusters. I'm not sure anyone would, or should, attempt to use a non-managed version of it.
- chrisseaton 8y ago
  Where do you recall that from? Your description doesn’t match their published paper in any way.
  - iampims 8y ago
    My understanding is that DynamoDB != Dynamo Paper
  - manigandham 8y ago
    DynamoDB is different from their published paper, which is mostly about designing a highly available key/value system that can be run on top of any other datastore. The paper does mention MySQL as a storage option. The recent MySQL info comes from this 2016 thread on DynamoDB storing empty strings: https://news.ycombinator.com/item?id=13170746 https://news.ycombinator.com/item?id=13170746 Confirmed by this comment specifically: https://news.ycombinator.com/item?id=13173927 https://news.ycombinator.com/item?id=13173927 The latest 2018 ReInvent deep dive on DynamoDB doesn't reveal anything but still fits if mysql is powering the storage nodes: https://www.youtube.com/watch?v=yvBR71D0nAQ https://www.youtube.com/watch?v=yvBR71D0nAQ
manigandham 8y ago
Nobody is entitled to proprietary development by these companies. Amazon wrote the Dynamo paper which gave us Cassandra/Scylla and Google produced BigTable which was replicated by HBase. Releasing stuff isn't simple, the hosted services only work because of the rest of their infrastructure. Writing about the tech is usually the best way to spur clean open-source versions that we can all use.
yjftsjthsd-h 8y ago
I don't think that excuse is particularly pathetic; I would be shocked if appengine can run without Borg or some equivalent, at which point open sourcing it is a lovely thought and completely worthless to everyone. It's the equivalent of open sourcing a program in a language for which all compilers are proprietary and generally not available (K comes to mind). Not literally 100% without value, but close enough.
oh-kumudo 8y ago
Amazon isn't good at open sourcing stuff, I am 100% with you. But there is more to this as well. AWS is a huge ecosystem by itself, a lot their services intertwined with each other, simply open source DynamoDB isn't enough without open sourcing the networking/storage functionality that is probably provided by other services.

What about DynamoDB? I'm pretty sure they could open source a lot but they just don't want. Google had the same pathetic excuse when asked why they don't open source appengine or datastore.

talawahdotnet 8y ago

I mean it would be cool if they did, but the secret sauce behind DynamoDb is really the scale, without it it probably wouldn't have the performance characteristics end users are looking for. Nor do I think AWS ever wants to be in the business of supporting a non-managed piece of software.

mbell 8y ago

> What about DynamoDB? From what I recall DynamoDB is a somewhat hacky API layer over a bunch of MySQL clusters. I'm not sure anyone would, or should, attempt to use a non-managed version of it.

chrisseaton 8y ago
Where do you recall that from? Your description doesn’t match their published paper in any way.
- iampims 8y ago
  My understanding is that DynamoDB != Dynamo Paper
- manigandham 8y ago
  DynamoDB is different from their published paper, which is mostly about designing a highly available key/value system that can be run on top of any other datastore. The paper does mention MySQL as a storage option. The recent MySQL info comes from this 2016 thread on DynamoDB storing empty strings: https://news.ycombinator.com/item?id=13170746 https://news.ycombinator.com/item?id=13170746 Confirmed by this comment specifically: https://news.ycombinator.com/item?id=13173927 https://news.ycombinator.com/item?id=13173927 The latest 2018 ReInvent deep dive on DynamoDB doesn't reveal anything but still fits if mysql is powering the storage nodes: https://www.youtube.com/watch?v=yvBR71D0nAQ https://www.youtube.com/watch?v=yvBR71D0nAQ

manigandham 8y ago

Nobody is entitled to proprietary development by these companies. Amazon wrote the Dynamo paper which gave us Cassandra/Scylla and Google produced BigTable which was replicated by HBase. Releasing stuff isn't simple, the hosted services only work because of the rest of their infrastructure. Writing about the tech is usually the best way to spur clean open-source versions that we can all use.

yjftsjthsd-h 8y ago

I don't think that excuse is particularly pathetic; I would be shocked if appengine can run without Borg or some equivalent, at which point open sourcing it is a lovely thought and completely worthless to everyone. It's the equivalent of open sourcing a program in a language for which all compilers are proprietary and generally not available (K comes to mind). Not literally 100% without value, but close enough.

oh-kumudo 8y ago

Amazon isn't good at open sourcing stuff, I am 100% with you. But there is more to this as well. AWS is a huge ecosystem by itself, a lot their services intertwined with each other, simply open source DynamoDB isn't enough without open sourcing the networking/storage functionality that is probably provided by other services.