4 ms·

I fully agree that AWS should do a better job of contributing back to OSS communities both in terms of code, money and man hours, however I don't think most ppl

I fully agree that AWS should do a better job of contributing back to OSS communities both in terms of code, money and man hours, however I don't think most ppl appreciate that a lot of AWS' secret sauce just doesn't make sense to open source even if they wanted to. They build their own custom nitro ASICs to integrate with KVM, for networking, storage, security, etc. That code doesn't make sense anywhere but AWS. They build things in ways that are tightly integrated with their internal systems and at a scale that few have experience with. Take their new Envoy based service mesh for example, I really do hope that they make meaningful contributions back to the main project, but I also suspect they will doing lots of custom integration with their own hardware and internal network that will never make sense to support in an OSS project. Who knows, their Annapurna Labs divsion might even be building a custom card that includes Envoy functionality. As to them being known for destroying OSS businesses I am drawing a blank there, can you provide some examples?

thefounder 8y ago
What about DynamoDB? I'm pretty sure they could open source a lot but they just don't want. Google had the same pathetic excuse when asked why they don't open source appengine or datastore.
- talawahdotnet 8y ago
  I mean it would be cool if they did, but the secret sauce behind DynamoDb is really the scale, without it it probably wouldn't have the performance characteristics end users are looking for. Nor do I think AWS ever wants to be in the business of supporting a non-managed piece of software.
- mbell 8y ago
  > What about DynamoDB? From what I recall DynamoDB is a somewhat hacky API layer over a bunch of MySQL clusters. I'm not sure anyone would, or should, attempt to use a non-managed version of it.
  - chrisseaton 8y ago
    Where do you recall that from? Your description doesn’t match their published paper in any way.
    - iampims 8y ago
      My understanding is that DynamoDB != Dynamo Paper
    - manigandham 8y ago
      DynamoDB is different from their published paper, which is mostly about designing a highly available key/value system that can be run on top of any other datastore. The paper does mention MySQL as a storage option. The recent MySQL info comes from this 2016 thread on DynamoDB storing empty strings: https://news.ycombinator.com/item?id=13170746 https://news.ycombinator.com/item?id=13170746 Confirmed by this comment specifically: https://news.ycombinator.com/item?id=13173927 https://news.ycombinator.com/item?id=13173927 The latest 2018 ReInvent deep dive on DynamoDB doesn't reveal anything but still fits if mysql is powering the storage nodes: https://www.youtube.com/watch?v=yvBR71D0nAQ https://www.youtube.com/watch?v=yvBR71D0nAQ
- manigandham 8y ago
  Nobody is entitled to proprietary development by these companies. Amazon wrote the Dynamo paper which gave us Cassandra/Scylla and Google produced BigTable which was replicated by HBase. Releasing stuff isn't simple, the hosted services only work because of the rest of their infrastructure. Writing about the tech is usually the best way to spur clean open-source versions that we can all use.
- yjftsjthsd-h 8y ago
  I don't think that excuse is particularly pathetic; I would be shocked if appengine can run without Borg or some equivalent, at which point open sourcing it is a lovely thought and completely worthless to everyone. It's the equivalent of open sourcing a program in a language for which all compilers are proprietary and generally not available (K comes to mind). Not literally 100% without value, but close enough.
- oh-kumudo 8y ago
  Amazon isn't good at open sourcing stuff, I am 100% with you. But there is more to this as well. AWS is a huge ecosystem by itself, a lot their services intertwined with each other, simply open source DynamoDB isn't enough without open sourcing the networking/storage functionality that is probably provided by other services.
wmf 8y ago
That's why MongoDB's SSPL license is kind of disingenuous: they don't really want the source code to AWS; they want to get paid. The Commons Clause is more honest about getting paid.
hueving 8y ago
This is a lame excuse. Google has open sourced all kinds of Linux kernel work and they were doing custom hardware with huge proprietary internal systems way before AWS was a thing. Open source takes work and the foresight to separate your custom shit from things genuinely helpful to the community. Amazon doesn't encourage or incentivize that kind of thinking.
- talawahdotnet 8y ago
  > Open source takes work and the foresight to separate your custom shit from things genuinely helpful to the community. That is exactly what AWS has done in releasing Firecracker[1]. Better late than never. 1. https://firecracker-microvm.github.io https://firecracker-microvm.github.io

thefounder 8y ago

What about DynamoDB? I'm pretty sure they could open source a lot but they just don't want. Google had the same pathetic excuse when asked why they don't open source appengine or datastore.

talawahdotnet 8y ago
I mean it would be cool if they did, but the secret sauce behind DynamoDb is really the scale, without it it probably wouldn't have the performance characteristics end users are looking for. Nor do I think AWS ever wants to be in the business of supporting a non-managed piece of software.
mbell 8y ago
> What about DynamoDB? From what I recall DynamoDB is a somewhat hacky API layer over a bunch of MySQL clusters. I'm not sure anyone would, or should, attempt to use a non-managed version of it.
- chrisseaton 8y ago
  Where do you recall that from? Your description doesn’t match their published paper in any way.
  - iampims 8y ago
    My understanding is that DynamoDB != Dynamo Paper
  - manigandham 8y ago
    DynamoDB is different from their published paper, which is mostly about designing a highly available key/value system that can be run on top of any other datastore. The paper does mention MySQL as a storage option. The recent MySQL info comes from this 2016 thread on DynamoDB storing empty strings: https://news.ycombinator.com/item?id=13170746 https://news.ycombinator.com/item?id=13170746 Confirmed by this comment specifically: https://news.ycombinator.com/item?id=13173927 https://news.ycombinator.com/item?id=13173927 The latest 2018 ReInvent deep dive on DynamoDB doesn't reveal anything but still fits if mysql is powering the storage nodes: https://www.youtube.com/watch?v=yvBR71D0nAQ https://www.youtube.com/watch?v=yvBR71D0nAQ
manigandham 8y ago
Nobody is entitled to proprietary development by these companies. Amazon wrote the Dynamo paper which gave us Cassandra/Scylla and Google produced BigTable which was replicated by HBase. Releasing stuff isn't simple, the hosted services only work because of the rest of their infrastructure. Writing about the tech is usually the best way to spur clean open-source versions that we can all use.
yjftsjthsd-h 8y ago
I don't think that excuse is particularly pathetic; I would be shocked if appengine can run without Borg or some equivalent, at which point open sourcing it is a lovely thought and completely worthless to everyone. It's the equivalent of open sourcing a program in a language for which all compilers are proprietary and generally not available (K comes to mind). Not literally 100% without value, but close enough.
oh-kumudo 8y ago
Amazon isn't good at open sourcing stuff, I am 100% with you. But there is more to this as well. AWS is a huge ecosystem by itself, a lot their services intertwined with each other, simply open source DynamoDB isn't enough without open sourcing the networking/storage functionality that is probably provided by other services.

wmf 8y ago

That's why MongoDB's SSPL license is kind of disingenuous: they don't really want the source code to AWS; they want to get paid. The Commons Clause is more honest about getting paid.

hueving 8y ago

This is a lame excuse. Google has open sourced all kinds of Linux kernel work and they were doing custom hardware with huge proprietary internal systems way before AWS was a thing. Open source takes work and the foresight to separate your custom shit from things genuinely helpful to the community. Amazon doesn't encourage or incentivize that kind of thinking.

talawahdotnet 8y ago
> Open source takes work and the foresight to separate your custom shit from things genuinely helpful to the community. That is exactly what AWS has done in releasing Firecracker[1]. Better late than never. 1. https://firecracker-microvm.github.io https://firecracker-microvm.github.io