3 ms·

Thanks for thorough answer! Seems like you've platformized(!) yourself to an extent, have you considered going full on with k8s on top of metal (their machines)

Thanks for thorough answer! Seems like you've platformized(!) yourself to an extent, have you considered going full on with k8s on top of metal (their machines) to offset some of the concerns about hardware?

sksjvsla 1y ago
Thanks for the compliment. We used AWS EKS in the old days and we never liked the extreme complexity of it. With two Spring Boot apps, a database and Redis running across Ubuntu servers, we found simpler tools to distribute and scale workloads. Since compute is dirt cheap, we over-provision and sleep well. We have live alerts and quarterly reviews (just looking at a dashboard!) to assess if we balance things well. K8s on EKS was not pleasant, I wanna make sure I never learn how much worse it can get across European VPS providers.
- NewJazz 1y ago
  Hmm, what was so unpleasant about EKS if you don't mind my asking?
  - mdaniel 1y ago
    I'm guessing the answer is going to center around the word "complexity" cited in their original comment. That is: I would guess it's YAGNI more than EKS itself There's an ongoing thread (one of many) exploring the different perspectives on that debate: https://news.ycombinator.com/item?id=44317825 https://news.ycombinator.com/item?id=44317825
  - mystifyingpoi 1y ago
    He said "in the old days" so probably before addons, managed nodegroups or auto mode. This must have been hell.
    - NewJazz 1y ago
      Ah. Yeah I'm setting things up with managed node groups and it doesn't seem so bad so far... Waiting for the other shoe to drop after so much doom saying though haha. Luckily we removed the need for anything stateful, so I can ignore the EBS-CSI shortcomings. Also trying to keep it simple/minimal when it comes to ingress and networking.
      - mystifyingpoi 1y ago
        Cool. I've never had any issue with EBS CSI driver itself, the biggest issue were idiosyncracies of EBS itself, like the mount count limit or availability zone requirements. These need ugly workarounds, like limiting your volumes to a single AZ, so, no HA. On the other side, their VPC CNI plugin and their ingress controller are pretty much set and forget.
        NewJazz 1y ago
        Yeah basically it is that EBS limitation (AZ-specific) combined with autoscaling causing quirky failure modes unless you are careful about how you set things up. https://github.com/kubernetes-sigs/aws-ebs-csi-driver/issues/923 https://github.com/kubernetes-sigs/aws-ebs-csi-driver/issues...

sksjvsla 1y ago

Thanks for the compliment. We used AWS EKS in the old days and we never liked the extreme complexity of it. With two Spring Boot apps, a database and Redis running across Ubuntu servers, we found simpler tools to distribute and scale workloads. Since compute is dirt cheap, we over-provision and sleep well. We have live alerts and quarterly reviews (just looking at a dashboard!) to assess if we balance things well. K8s on EKS was not pleasant, I wanna make sure I never learn how much worse it can get across European VPS providers.

NewJazz 1y ago
Hmm, what was so unpleasant about EKS if you don't mind my asking?
- mdaniel 1y ago
  I'm guessing the answer is going to center around the word "complexity" cited in their original comment. That is: I would guess it's YAGNI more than EKS itself There's an ongoing thread (one of many) exploring the different perspectives on that debate: https://news.ycombinator.com/item?id=44317825 https://news.ycombinator.com/item?id=44317825
- mystifyingpoi 1y ago
  He said "in the old days" so probably before addons, managed nodegroups or auto mode. This must have been hell.
  - NewJazz 1y ago
    Ah. Yeah I'm setting things up with managed node groups and it doesn't seem so bad so far... Waiting for the other shoe to drop after so much doom saying though haha. Luckily we removed the need for anything stateful, so I can ignore the EBS-CSI shortcomings. Also trying to keep it simple/minimal when it comes to ingress and networking.
    - mystifyingpoi 1y ago
      Cool. I've never had any issue with EBS CSI driver itself, the biggest issue were idiosyncracies of EBS itself, like the mount count limit or availability zone requirements. These need ugly workarounds, like limiting your volumes to a single AZ, so, no HA. On the other side, their VPC CNI plugin and their ingress controller are pretty much set and forget.
      - NewJazz 1y ago
        Yeah basically it is that EBS limitation (AZ-specific) combined with autoscaling causing quirky failure modes unless you are careful about how you set things up. https://github.com/kubernetes-sigs/aws-ebs-csi-driver/issues/923 https://github.com/kubernetes-sigs/aws-ebs-csi-driver/issues...