3 ms·

I'd say there are other reasons behind using virtualization than simply quick system spin-up, such as easier/better integration with (Dev|Sys)Op tools, higher h

by druiid 14y ago

I'd say there are other reasons behind using virtualization than simply quick system spin-up, such as easier/better integration with (Dev|Sys)Op tools, higher hardware->instance densities than you'll ever see from physical... etc. I think the performance hit for many people is overstated. Unless you're at the long-tail of CPU bound instances I imagine it won't become very apparent. That said, if I'm missing something/incorrect then someone please correct me!

tedunangst 14y ago
It sounds like the reason they have so many machines is because they need a lot of CPU. Compressing those machines onto fewer CPUs would not be a win.
runako 14y ago
I don't work at Etsy or have any specific knowledge, but it sounds like they already have the tool integration they need with their current setup. Increasing densities through virtualization allows you to perform more discrete functions, but obviously reduces the aggregate capacity to do so. Etsy specifically has a goal of minimizing instance types, but virtualization would enable them go in a direction counter to their goals. In my experience, the biggest weakness of virtualization is subpar I/O, and I would imagine that this would be an issue for Etsy as well. Virtualization has a very real cost. If the performance hit (CPU & I/O) is (say) 20%, that's 20% of your capacity you pay to keep operational but don't get to use. The benefits don't outweigh that cost for every situation.
avleenvig 14y ago
So, this is somewhat untrue for us. For example, let's look at some common web stack infrastructure: * Web servers * Database servers * Monitoring servers * Asynchronous task processors * Scheduled task servers (eg, cron jobs) In all of these cases, we want the service on that hardware to be able to use 100% CPU. For example, it doesn't make sense to get N web servers, and then run M virtual instances with apache on them, when each instance will max out its available resources. Just run N dedicated web servers :-) The same is true for pretty much everything else too, sometimes for different reasons. Eg, you don't want your database servers or monitoring system to be impacted by multi-tenancy issues. - Avleen Vig, Staff Operations Engineer, Etsy

by druiid 14y ago

tedunangst 14y ago

It sounds like the reason they have so many machines is because they need a lot of CPU. Compressing those machines onto fewer CPUs would not be a win.

runako 14y ago

I don't work at Etsy or have any specific knowledge, but it sounds like they already have the tool integration they need with their current setup. Increasing densities through virtualization allows you to perform more discrete functions, but obviously reduces the aggregate capacity to do so. Etsy specifically has a goal of minimizing instance types, but virtualization would enable them go in a direction counter to their goals. In my experience, the biggest weakness of virtualization is subpar I/O, and I would imagine that this would be an issue for Etsy as well. Virtualization has a very real cost. If the performance hit (CPU & I/O) is (say) 20%, that's 20% of your capacity you pay to keep operational but don't get to use. The benefits don't outweigh that cost for every situation.

avleenvig 14y ago

So, this is somewhat untrue for us. For example, let's look at some common web stack infrastructure: * Web servers * Database servers * Monitoring servers * Asynchronous task processors * Scheduled task servers (eg, cron jobs) In all of these cases, we want the service on that hardware to be able to use 100% CPU. For example, it doesn't make sense to get N web servers, and then run M virtual instances with apache on them, when each instance will max out its available resources. Just run N dedicated web servers :-) The same is true for pretty much everything else too, sometimes for different reasons. Eg, you don't want your database servers or monitoring system to be impacted by multi-tenancy issues. - Avleen Vig, Staff Operations Engineer, Etsy