3 ms·

Tangent: > Energy Efficiency in the Cloud I recently moved the infrastructure for one of my company's products, from ~10 smallish EC2 VMs, plus durable storag

Tangent: > Energy Efficiency in the Cloud I recently moved the infrastructure for one of my company's products, from ~10 smallish EC2 VMs, plus durable storage in RDS, EFS, and S3, to a single dedicated box at OVH with local storage and backups in S3 Glacier and rsync.net. The application stack is the same in both cases. I wonder which is more energy efficient. The OVH box has a 6-core CPU, 32 GB of RAM, two 512 GB SSDs, and two 6 TB hard drives. I suppose AWS, because of their massive scale and use of virtualization, can be more energy-efficient than OVH's comparatively low-capacity dedicated servers; notice how absurdly high the specs are on all of the EC2 bare-metal instances. On the OVH box, I set the Linux CPU frequency governor to "performance", to make sure all requests get the fastest possible response even though the box is ~98% idle. I wonder how much energy that's actually wasting. The main reason for the move was to save money. And yes, since we moved to one box, we decided we could tolerate some reduction in uptime (this is a consumer app). But I wonder what the relative costs would look like if the externalities of having that relatively low-capacity, mostly idle box to ourselves, as opposed to using several small EC2 VMs (also mostly idle), were factored in. Maybe the only really fair answer is for everyone who doesn't actually own and operate their own servers to go all serverless and be nickled and dimed for every little thing. Edit: And FWIW, no, we're not using energy-efficient programming languages at the application layer. Mostly CPython. I guess I could experiment with PyPy, and/or rewrite some of the smaller services in Rust.

lumost 5y ago
It sounds like you had invested in a high redundancy deployment in AWS, and switched to a single host with a single DB in a colo facility given the background that you could tolerate downtime and the people time to remediate the outage. I'm curious, what made you choose to also move to a colo facility vs. allocating a single ec2 instance to act as a VPS? I'd imagine the costs would have been comparable, and you could still get automated backups/recovery with EBS + snapshots.
- mwcampbell 5y ago
  Fair point. I was just thinking about that myself. One big reason is to avoid being charged for outgoing data transfer; with OVH, that's unmetered, with a gigabit port. Also, the OVH box, with the specs I laid out above, costs ~$150 per month. For roughly that same price on EC2, we could get an m5ad.xlarge on-demand instance, with 2 cores, 16 GB of RAM, and a 150 GB SSD. We'd have to add EBS and/or S3 (non-Glacier) storage on top of that, and as noted above, outgoing data transfer.
zozbot234 5y ago
> On the OVH box, I set the Linux CPU frequency governor to "performance", to make sure all requests get the fastest possible response I'm not sure that'll meaningfully improve latency given how much more relevant other factors can be. Have you measured a real difference? If not, you might be burning CPU-cycles for nothing.
- mwcampbell 5y ago
  Some things felt noticeably faster to me, but that could of course be the placebo effect. More importantly, the CPU usage percentages reported by ps have fallen (those are actually large enough to measure in a few processes, because "100" means a single CPU thread and not the whole CPU). That implies that the CPU is in fact spending less time on some things.
  - zozbot234 5y ago
    > More importantly, the CPU usage percentages reported by ps have fallen That's not a meaningful comparison when the CPU itself is running at a higher frequency. Lower frequency enables the CPU to waste less effort, e.g. while waiting on memory or I/O.
    - vlovich123 5y ago
      I don’t know if the same holds true in servers, but in mobile you want to race to idle. In other words, consistent high CPU usage at a low frequency is worse than spiking the frequency up to get CPU usage down to 0. I would imagine that holds too although it’s more complicated because a server usage model is different (perhaps a constant baseload to begin with changes the calculus).
      - zozbot234 5y ago
        Race to idle is good in a compute heavy workload (up to thermal limits), but I'm not sure it is when the CPU is mostly waiting for memory. Also IIRC, the point of using "performance" in the first place is that it keeps the CPU at its design frequency even when idle - so it will be less able to boost further when needed.

lumost 5y ago

It sounds like you had invested in a high redundancy deployment in AWS, and switched to a single host with a single DB in a colo facility given the background that you could tolerate downtime and the people time to remediate the outage. I'm curious, what made you choose to also move to a colo facility vs. allocating a single ec2 instance to act as a VPS? I'd imagine the costs would have been comparable, and you could still get automated backups/recovery with EBS + snapshots.

mwcampbell 5y ago
Fair point. I was just thinking about that myself. One big reason is to avoid being charged for outgoing data transfer; with OVH, that's unmetered, with a gigabit port. Also, the OVH box, with the specs I laid out above, costs ~$150 per month. For roughly that same price on EC2, we could get an m5ad.xlarge on-demand instance, with 2 cores, 16 GB of RAM, and a 150 GB SSD. We'd have to add EBS and/or S3 (non-Glacier) storage on top of that, and as noted above, outgoing data transfer.

zozbot234 5y ago

> On the OVH box, I set the Linux CPU frequency governor to "performance", to make sure all requests get the fastest possible response I'm not sure that'll meaningfully improve latency given how much more relevant other factors can be. Have you measured a real difference? If not, you might be burning CPU-cycles for nothing.

mwcampbell 5y ago
Some things felt noticeably faster to me, but that could of course be the placebo effect. More importantly, the CPU usage percentages reported by ps have fallen (those are actually large enough to measure in a few processes, because "100" means a single CPU thread and not the whole CPU). That implies that the CPU is in fact spending less time on some things.
- zozbot234 5y ago
  > More importantly, the CPU usage percentages reported by ps have fallen That's not a meaningful comparison when the CPU itself is running at a higher frequency. Lower frequency enables the CPU to waste less effort, e.g. while waiting on memory or I/O.
  - vlovich123 5y ago
    I don’t know if the same holds true in servers, but in mobile you want to race to idle. In other words, consistent high CPU usage at a low frequency is worse than spiking the frequency up to get CPU usage down to 0. I would imagine that holds too although it’s more complicated because a server usage model is different (perhaps a constant baseload to begin with changes the calculus).
    - zozbot234 5y ago
      Race to idle is good in a compute heavy workload (up to thermal limits), but I'm not sure it is when the CPU is mostly waiting for memory. Also IIRC, the point of using "performance" in the first place is that it keeps the CPU at its design frequency even when idle - so it will be less able to boost further when needed.