10 ms·

Efficiency is an area where there can be a lot of cost hiding. We recently saved a lot of money by: - Using the Kubernetes Vertical Pod Autoscaler (https://git

Efficiency is an area where there can be a lot of cost hiding. We recently saved a lot of money by: - Using the Kubernetes Vertical Pod Autoscaler (https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler https://github.com/kubernetes/autoscaler/tree/master/vertica...) for CPU and memory scaling, and switching to metrics like requests per second and connection count for horizontal scaling - Collecting metrics on container CPU and memory allocation vs utilization - Writing scripts to auto-submit PRs to SREs with better recommended sizing based on actual usage - Tuning our VM instance sizes and autoscaler configs A few engineers were able to save the company several times their salary with a few months of work, and plan to 10x the savings over the next year

totomz 7y ago
Actually, HakunaCloud works on cloud servers, not containers. With Kubernetes you pay for the worker nodes, not for running containers. HakuaCloud starto&stop cloud servers, so you actually pay only for the time the servers are needed
- dharmab 7y ago
  The number of running nodes is a direct function of the number and size of your containers. The cluster autoscaler stops and and start servers based on how many containers are desired, which is in turn based on the Horizontal Pod Autoscaler inputs (e.g. incoming requests).
justicezyx 7y ago
Very nice! I suspect the economy of Cloud will soon change in such a way that Cloud providers will be focusing more on Enterprise, which have very little capability of doing similar things. And then, a lot of companies like Hakuna Cloud (reported here) will emerge, just like what happened in the last iteration of IT infrastructure.
najarvg 7y ago
Would love to see a more detailed write-up, if one exists, on your experience. We're nowhere near your scale but will probably ramp up in the next year or so.
- colechristensen 7y ago
  During a ramp up the limiting factor is usually engineer hours, new features, and ability to have the load at all and not cost.
- avip 7y ago
  Pre-scale just overprovision and pay $ to Bezos instead of $$ your precious engineers. 96% you won't "ramp up" anyhow.
- dharmab 7y ago
  I'm going to try to convince the lead on this project to present at an upcoming Kubecon :)
cbluth 7y ago
> Tuning our VM instance sizes and autoscaler configs Do you run kubernetes in VMs on baremetal?
- dharmab 7y ago
  Yes, we own some datacenters as well as use multiple cloud providers. You are still responsible for running your own worker VMs with a managed Kubernetes anyway.
qubit23 7y ago
If you ever open-source your rightsizing scripts, I think a lot of people (including myself) would find that useful. While the vertical pod autoscaler attempts to solve this problem, it's incompatible with horizontal pod autoscalers based on CPU and memory, and thus unusable in a lot of cases. Going with a semi-automated solution might be ideal.
- dharmab 7y ago
  You can just run vpa-recommender without vpa-updater to get VPA recommendations without the autoscaling. That should work for most people.
ma2rten 7y ago
I am confused by your use of the word cost hiding. Do you mean cost saving?
- dharmab 7y ago
  as in hidden costs- costs that the organization is unaware of because it's wrapped up in r&d/goods delivered on the accounting sheets. Its='s not until you pull in the utilization metrics that it reveals itself as waste.

totomz 7y ago

Actually, HakunaCloud works on cloud servers, not containers. With Kubernetes you pay for the worker nodes, not for running containers. HakuaCloud starto&stop cloud servers, so you actually pay only for the time the servers are needed

dharmab 7y ago
The number of running nodes is a direct function of the number and size of your containers. The cluster autoscaler stops and and start servers based on how many containers are desired, which is in turn based on the Horizontal Pod Autoscaler inputs (e.g. incoming requests).

justicezyx 7y ago

Very nice! I suspect the economy of Cloud will soon change in such a way that Cloud providers will be focusing more on Enterprise, which have very little capability of doing similar things. And then, a lot of companies like Hakuna Cloud (reported here) will emerge, just like what happened in the last iteration of IT infrastructure.

najarvg 7y ago

Would love to see a more detailed write-up, if one exists, on your experience. We're nowhere near your scale but will probably ramp up in the next year or so.

colechristensen 7y ago
During a ramp up the limiting factor is usually engineer hours, new features, and ability to have the load at all and not cost.
avip 7y ago
Pre-scale just overprovision and pay $ to Bezos instead of $$ your precious engineers. 96% you won't "ramp up" anyhow.
dharmab 7y ago
I'm going to try to convince the lead on this project to present at an upcoming Kubecon :)

cbluth 7y ago

> Tuning our VM instance sizes and autoscaler configs Do you run kubernetes in VMs on baremetal?

dharmab 7y ago
Yes, we own some datacenters as well as use multiple cloud providers. You are still responsible for running your own worker VMs with a managed Kubernetes anyway.

qubit23 7y ago

If you ever open-source your rightsizing scripts, I think a lot of people (including myself) would find that useful. While the vertical pod autoscaler attempts to solve this problem, it's incompatible with horizontal pod autoscalers based on CPU and memory, and thus unusable in a lot of cases. Going with a semi-automated solution might be ideal.

dharmab 7y ago
You can just run vpa-recommender without vpa-updater to get VPA recommendations without the autoscaling. That should work for most people.

ma2rten 7y ago

I am confused by your use of the word cost hiding. Do you mean cost saving?

dharmab 7y ago
as in hidden costs- costs that the organization is unaware of because it's wrapped up in r&d/goods delivered on the accounting sheets. Its='s not until you pull in the utilization metrics that it reveals itself as waste.