4 ms·

I don't see why AWS is alone in "Tier 1" on a list that claims to prioritize standard technology and outsourcing undifferentiated heavy lifting. The post gives

I don't see why AWS is alone in "Tier 1" on a list that claims to prioritize standard technology and outsourcing undifferentiated heavy lifting. The post gives an example of switching from self-hosted MongoDB onto AWS DynamoDB. But AWS DynamoDB is proprietary technology, not standard. Amazon doesn't even claim that DynamoDB is standard; they argue that DynamoDB is the best in the world, worth the premium in price. It's not at all "undifferentiated heavy lifting." It sounds like they made the right decision with DynamoDB, but they did it by violating the pillars of their own philosophy, adopting a differentiated proprietary solution for their scalability problems.

curun1r 9y ago
It's the "self-hosting" part that's the "heavy lifting" for MongoDB. They could've also chosen a hosted Mongo service, and that would've worked too. The idea is to spend time focusing on your own product and do as little as possible outside of that core competency. Setting up, optimizing, monitoring, upgrading and everything else that goes into running your own database, however standard, falls outside of that core. The more you can do to avoid doing work that isn't directly building features and experience for your customers, the more you'll be successful. By undifferentiated, they mean outsource anything that doesn't differentiate their own product offering. Build things that give you a competitive advantage over other products in your space. Outsource everything else because wasting work on it won't help you get customers. Of course DynamoDB is differentiated for Amazon, but it's undifferentiated for Intercom...it's just a persistence strategy that customers should never know about.
scarface74 9y ago
DynamoDB is not even a good replacement for Mongo. DynamoDB only lets you index two (?) fields and everything else is a full table scan. At least go with a hosted Mongo solution.
- mooreds 9y ago
  Actually out can have multiple indices, but they are eventually consistent: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GSI.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg...
- ciaranlee 9y ago
  It’s certainly nowhere near as full featured. But in our case it is entirely appropriate. Over time we simplified our use of MongoDB to the stage where it’s effectively a k/v store. Swapping that simple pattern over to DynamoDB was easy, saves us a ton of money, and means we get to shed a ton of operational responsibility. Also, you are incorrect about only being able to index two fields in DynamoDB. Though we luckily don’t need anything other than the primary key.
rich_archbold 9y ago
Thanks for taking the time to read our blog post and for sharing your thoughts! For us, the concept of “standard” technology is not about whether it’s proprietary or open source, more that we have a “standard way of doing things. So in this case it was moving toward saying, “hey, if you’re an engineer at Intercom, and need a scalable key-value store for the thing that you’re building, we think you should lean toward using DynamoDB, rather than any of the other technology options, because that’s the one that we’re committed to making a first developer experience for internally for the long term”. The reason for having AWS alone as a Tier 1 option is similar. We’re already very heavily invested in making developing on top of AWS a great developer experience at Intercom, and we’re committed to doing so for the long term. Therefore we think that it makes sense for us to advocate for always exploring relevant offerings that put out. I recognize that I could have elaborated on the philosophy a lot more, there’s always a tension between writing too much and too little. And the philosophy is just that, not dogma and not right or wrong, just trying to give insight into how we think. Thanks again for the feedback! Rich.
- dajonker 9y ago
  Somehow the recommendations feel like you are making them to the reader, and in that regards it doesn't feel right to just recommend AWS. Other public clouds provide equivalent or sometimes even better services.
  - rich_archbold 9y ago
    Hi, Rich, blog author here again. Your feedback is super fair. That is how it comes across in the blog. In the talk that came before the blog post, I go into a lot more detail here. I have some slides where I explain that these are "our" standard technologies, and are not intended to be "the" standard technologies that everyone should use. I gave some other examples of perfectly good sets of "standard" technologies which other companies could choose for lots of good reasons. The main point that I/we were advocating for was that, in our opinion (just an opinion, not dogma, not definitely right or wrong), there are some strong benefits to making deliberate decisions about what technologies you consider standard, safe, easy and fast to use, versus those that are not. Some of those benefits we see include: 1. speed of technical decision making 2. incentivizing engineers to develop deep technical expertise in specific technologies 3. incentivizing managers to fund training in specific technologies 4. creating fungibility within your engineering team to make it easier for engineers to transfer between teams and come up to speed quickly 5. minimizing ongoing operational overhead as there's fewer technologies that need to be battle-hardened with operational tooling

curun1r 9y ago

It's the "self-hosting" part that's the "heavy lifting" for MongoDB. They could've also chosen a hosted Mongo service, and that would've worked too. The idea is to spend time focusing on your own product and do as little as possible outside of that core competency. Setting up, optimizing, monitoring, upgrading and everything else that goes into running your own database, however standard, falls outside of that core. The more you can do to avoid doing work that isn't directly building features and experience for your customers, the more you'll be successful. By undifferentiated, they mean outsource anything that doesn't differentiate their own product offering. Build things that give you a competitive advantage over other products in your space. Outsource everything else because wasting work on it won't help you get customers. Of course DynamoDB is differentiated for Amazon, but it's undifferentiated for Intercom...it's just a persistence strategy that customers should never know about.

scarface74 9y ago

DynamoDB is not even a good replacement for Mongo. DynamoDB only lets you index two (?) fields and everything else is a full table scan. At least go with a hosted Mongo solution.

mooreds 9y ago
Actually out can have multiple indices, but they are eventually consistent: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GSI.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg...
ciaranlee 9y ago
It’s certainly nowhere near as full featured. But in our case it is entirely appropriate. Over time we simplified our use of MongoDB to the stage where it’s effectively a k/v store. Swapping that simple pattern over to DynamoDB was easy, saves us a ton of money, and means we get to shed a ton of operational responsibility. Also, you are incorrect about only being able to index two fields in DynamoDB. Though we luckily don’t need anything other than the primary key.

rich_archbold 9y ago

Thanks for taking the time to read our blog post and for sharing your thoughts! For us, the concept of “standard” technology is not about whether it’s proprietary or open source, more that we have a “standard way of doing things. So in this case it was moving toward saying, “hey, if you’re an engineer at Intercom, and need a scalable key-value store for the thing that you’re building, we think you should lean toward using DynamoDB, rather than any of the other technology options, because that’s the one that we’re committed to making a first developer experience for internally for the long term”. The reason for having AWS alone as a Tier 1 option is similar. We’re already very heavily invested in making developing on top of AWS a great developer experience at Intercom, and we’re committed to doing so for the long term. Therefore we think that it makes sense for us to advocate for always exploring relevant offerings that put out. I recognize that I could have elaborated on the philosophy a lot more, there’s always a tension between writing too much and too little. And the philosophy is just that, not dogma and not right or wrong, just trying to give insight into how we think. Thanks again for the feedback! Rich.

dajonker 9y ago
Somehow the recommendations feel like you are making them to the reader, and in that regards it doesn't feel right to just recommend AWS. Other public clouds provide equivalent or sometimes even better services.
- rich_archbold 9y ago
  Hi, Rich, blog author here again. Your feedback is super fair. That is how it comes across in the blog. In the talk that came before the blog post, I go into a lot more detail here. I have some slides where I explain that these are "our" standard technologies, and are not intended to be "the" standard technologies that everyone should use. I gave some other examples of perfectly good sets of "standard" technologies which other companies could choose for lots of good reasons. The main point that I/we were advocating for was that, in our opinion (just an opinion, not dogma, not definitely right or wrong), there are some strong benefits to making deliberate decisions about what technologies you consider standard, safe, easy and fast to use, versus those that are not. Some of those benefits we see include: 1. speed of technical decision making 2. incentivizing engineers to develop deep technical expertise in specific technologies 3. incentivizing managers to fund training in specific technologies 4. creating fungibility within your engineering team to make it easier for engineers to transfer between teams and come up to speed quickly 5. minimizing ongoing operational overhead as there's fewer technologies that need to be battle-hardened with operational tooling