3 ms·

I don't have a dog in this race; I'm not partnered with any cloud provider. I will say that based upon what the article discusses, they save a bunch of money on

• sidlls 6y ago
  That depends on the scale and other particulars that are only shared/known inside the company. In my experience, beyond a certain scale it simply doesn't make any sense to use managed services anymore. There is a high initial upfront cost in development hours and hardware that is amortized over a very long time after, and this upfront cost is partially paid for by the reduced cloud bill.
  • glogla 6y ago
    That also depends on the markup. EKS on EC2 costs EC2 costs + flat money for control plane, so that might make more sense than running your own Kubernetes on EC2. (although I have no experience to say how much time this actually saves you) Managed Kafka costing 2x the cost of EC2 infrastructure? Probably not worth it.
• kfk 6y ago
  Most ETL processes should be declarative and configured in text files like you configure a CloudFormation or Terraform. Once you have that, the execution piece is relatively straightforward. I think a lot of issues and costs with ETL come from poor architecture decisions. For instance, we all made the mistake of running Extract processes with too many transformations. An extract should be an extract, transformation should come later.