8 ms·

Google said 40% of their data server operating cost comes from paying for energy, it makes business sense to switch to this type of DRAM. I couldn't find the

Google said 40% of their data server operating cost comes from paying for energy, it makes business sense to switch to this type of DRAM. I couldn't find the actual youtube video where a google employee shows the 40% energy cost, if you do, please link it. Meanwhile here is another graph that is somewhat similar: https://perspectives.mvdirona.com/2008/11/cost-of-power-in-large-scale-data-centers/ https://perspectives.mvdirona.com/2008/11/cost-of-power-in-l...

adrr 8y ago
I am not an expert on microchips but looking at Apple's A12 vs Intel chips in terms of power and energy usage. Wouldn't the industry really benefit moving away from X86 architecture to something else in terms of reducing power consumption? These savings could be passed to users of cloud providers.
- matthewmacleod 8y ago
  There's not much that intrinsically makes an aarch chip lower-power or more energy-efficient than an x86 or x64 one – energy use will be more about the design goals of the microarchitecture than anything else. There's a paper from a few years ago the looked into this in some detail – ftp://doc.nit.ac.ir/cee/computer/alinejhad.saeedeh/old/95-96-1/MS/Advanced%20Architecture/project/paper/6-ISA%20Wars%20Understanding%20the%20Relevance%20of%20ISA%20being%20RISC%20or%20CISC.pdf – there's no immediate reason to think the conclusions should have changed.
- jjeaff 8y ago
  We have about 40 years of software development on x86. I'm not even sure we could reuse any of that when moving to a new architecture.
  - noir_lord 8y ago
    Does that matter for some cases.? Linux supports arm and we have massive companies where economies of scale and a Linux/OSS plus their code could all be cross compiled, I mean if your the average biggish none tech company then yeah you might be stuck on x86 because of windows but that isn't everyone.
    - zitterbewegung 8y ago
      In any organization if you have an internal tool or software it’s not like you can press a button and make sure it is cross compiled to ARM from x86. That is a huge and expensive undertaking.
      - phkahler 8y ago
        >> In any organization if you have an internal tool or software it’s not like you can press a button and make sure it is cross compiled to ARM from x86. That is a huge and expensive undertaking. That depends on the software. Many linux distributions can be compiled for x86, ARM, PPC, RISC-V, and even some others. Cross compiling is not difficult if your software is written with even a little of that in mind. Windows itself used to be available for 3 or 4 instruction sets - and an OS requires low level hardware support. Applications should be largely target independent.
        kevin_thibedeau 8y ago
        Lots of x86 code plays fast and loose with alignment which will blow up on ARM.
        monocasa 8y ago
        Modern big ARMs let you play fast and loose with alignment as well. Hell, a CortexM3 lets you have unaligned loads and stores too.
        jschwartzi 8y ago
        Even if they don't you can just turn unaligned access emulation on in your OS kernel. This was a thing in QNX and is also a thing in Linux.
        monocasa 8y ago
        I think it's fair to not want to turn that on in the kernel on chips that don't support it in servers. It's really expensive on those kinds of cores to trap into the kernel (a KPTI makes it dramatically more expensive). That being said, server cores with their long me or pipelines are the most likely to not have an issue with unaligned access, so it's probably a moot point.
  - simias 8y ago
    Unless your software is written in assembly you can probably reuse a lot of that when moving to a new architecture. And given that interpreted/JITed languages are pretty mainstream these days I'm sure many developers wouldn't even notice the difference if you swapped their CPUs during the night.
- JetSetWilly 8y ago
  Even if one arch is intrinsically lower power than the other, I am not sure that x86 would be the one to come off worse. For example, could one not argue that an x86 binary / code size is on average smaller and more dense than the ARM equivalent, therefore on x86 cache misses will be less common, less ram needs to be used etc etc. Perhaps a denser binary is better for power consumption? Although this effect would be very marginal even if it is present at all.
  - adrr 8y ago
    Interesting on the code size. How does pipeline length effect this? ARM is a lot shorter so a miss doesn't matter as much. Intel once tried to push long pipelines with their netburst architecture but real-world scenarios didn't really show any improvements. Are the compilers better at branch prediction now?
- rbanffy 8y ago
  In the end it all comes down to the energy cost of doing a computation. With good architectures and compilers, the code is causing as few unnecessary bit flips as possible to get to the results and, unless you have a significant difference in how much energy it takes do that, all CPUs will end up close. But CPUs are just part of the equation - you can make significant gains in other parts of the computer and memory is a good target. When we got to the point PSUs live as independent things plugged into the same rack as your servers, it means we are really concerned about not feeding one amp more than we need to.
  - janekm 8y ago
    Mainstream CPUs are heavily optimised for latency over energy consumption (using speculative execution to effectively waste a multiple of the energy required for the computation itself). It'd be an interesting exercise to try to design an architecture optimises energy consumption over latency (perhaps rather than deep cache hierarchies, using cores that can power gate while they are waiting for memory accesses...)
    - rbanffy 8y ago
      That's a very good point. Even when one generates optimal code, the CPU will run all sorts of computations that do not contribute to reach the right result, but only to get there more quickly. A lot of the low-power ARM processors do that - just avoiding speculative execution like you pointed out is a huge win if power is your main concern. Another interesting take are barrel processors that can avoid scheduling stalled contexts, but they are only effective when we can feed them enough threads the time it spends waiting is less than a "normal" processor would. Early Xeon Phi's (and Cell's PPUs) behaved like barrel processors - each core could dispatch 4 (2 on the PPY) threads, but it did so round-robin. Scheduling only one thread per core didn't make them run any faster.
- jandrewrogers 8y ago
  No, the power efficiency of (micro)architectures is highly dependent on how well the workload fits design assumptions. CPU cores spend their time in one of several states with different power profiles. You can increase the power efficiency of one state by reducing the power efficiency of another, so assumptions about what a CPU core will spend most of its time doing matter quite a bit in terms of optimizing throughput per watt. If a CPU core spends most of its time idle, it makes sense to optimize power consumption of idle time even if it makes actual computing or waiting for memory more expensive. Intel microarchitectures tend to be heavily optimized around a workload that assumes a CPU core is doing heavy scalar computation all the time. If your workload and code actually looks like that -- and many kinds of server workloads do -- you can't beat them for performance per watt. But this comes at the cost of things like power consumption when idle. Most personal computing, on the other hand, spends most of its time idle and may benefit significantly from architectures that optimize more for power consumption at low utilization levels. There are many ways to attack these problems at the architectural level. This becomes complicated to optimize in intermittent compute-intensive environments, since over-optimizing for the idle time effectively increases the amount of time you spend in a less efficient compute-intensive state (see also: Apple's big/little core hybrid ARM architecture). There was a lot of empirical work done on this in the supercomputing world, which is sensitive to power costs and had built ARM clusters. The consensus seemed to be in that environment, for example, that throughput per watt under max load was the driver of power costs, which favored Intel. Modeling this is difficult because it is a complex interaction between specific pieces of code and specific architecture implementations. Even just comparing Intel and AMD, they have different tradeoffs for real code because the architectures are optimized differently. For example, while some codes run very efficiently on AMD's Epyc, I have a couple old codes that have terrible performance on Epyc that can be traced to design decisions made by AMD. You can't design a CPU that is the best at everything or even most things.
  - adrr 8y ago
    Thank you for the explanation. First time I've heard it in such understandable concepts. Do you think the industry will move to specialized processors dependant on workload type? For example, for serverless, cloud providers could move workloads to the most efficient microarchitecture.
- spamizbad 8y ago
  In addition to what others have said there’s one area to consider: the “uncore” performance: memory controllers, PCIE busses, core crosslinks, cache coherency etc. Both Intel and AMD have demonstrated they can scale these effectively. The A12X isn’t even playing in the same league when it comes to uncore functionality. I’m sure if you created a version of the A12X with the same cache, memory, PCIE, core count, and interconnections the power picture looks a lot different. This is part of the reason why “datacenter arm” CPUs never get much of a foothold: as soon as you start ramping up your uncore transistor count, your power advantage erodes significantly, to the point where Intel and AMD begin to yield better performance-per-watt in typical server workloads due to a tightly optimized microarch and an “uncore” to feed it.
hinkley 8y ago
It’s not unreasonable though. For every x watts of server you need y watts of cooling. Years ago Google complained that they couldn’t fill up their data centers because they had hit the limits of electrical code. Servers plus A/C took them into no man’s land as far as electrical code was concerned. That also means wasted real estate.
- stefan_ 8y ago
  That seems unlikely. An aluminum smelter uses far more energy than a data center ever will.
  - hinkley 8y ago
    Yeah that’s a good question. Smelters don’t generate their own power generally, right? But I believe they also use a lot of air cooling so most of that draw is going to be electrolysis. Wikipedia has a list of aluminum smelters and I notice none of the biggest are in the US. Is that just because, or due to code limitations?
    - oasisbob 8y ago
      The Pacific Northwest used to have a lot more Aluminum smelting than it does now - the power was provided by large hydro power contracts. Not sure if it was changes in these contracts or other market forces that caused the decline.
      - oasisbob 8y ago
        Decided to answer my own question - here's a great history of the NW aluminum industry: https://www.nwcouncil.org/reports/columbia-river-history/aluminum https://www.nwcouncil.org/reports/columbia-river-history/alu... The last active NW aluminum smelters closed in 2016. Most of the rest were victims of Enron and the 2000 energy crisis.
    - mikeyouse 8y ago
      Aluminum smelting is dirty and uses a ton of power -- we generally don't subsidize power here (in stark contrast to China / Argentina / etc.) and have increasingly strict environmental controls so we're offshoring most production and pollution associated with smelting from ore. We do have a lot of 'secondary' production which is much cleaner since it uses aluminum scrap as the input.
wyldfire 8y ago
Are ECC LPDDRx memory sticks available? If not, perhaps they should consider it. But Google probably couldn't switch until/unless they're produced.
- wtallis 8y ago
  LPDDR doesn't come in modular form. It's always soldered onto the same PCB as the processor. The number of packages used is determined by the data bus width of the processor. If Google has access to a processor with a DRAM controller that supports LPDDR4 and ECC, they can design a suitable motherboard.
  - ChuckMcM 8y ago
    That is typical usage but it isn't the required usage. It is straight forward to generate a controller ASIC that would provide the appropriate bus signals and drive levels for these to operate on a module.
    - kingosticks 8y ago
      The signal integrity at 0.6V at whatever frequency these are at, across a connector must be horrible. Are people doing this?
      - ChuckMcM 8y ago
        I have seen LPDDR modules where the ASIC provides the connector signals. So they run 0.6V between the LPDDR chips and the ASIC, then the ASIC runs 1.8V on the connector.
        kingosticks 8y ago
        I don't think I follow your topology. The asic is driving 1.8V to a dimm with 3 LPDDR chips on it? At ddr4 speeds?
        8y ago
jaxtellerSoA 8y ago
Then it also stands to reason that it makes business sense to switch from Intel Xeon to AMD Epyc.
- derefr 8y ago
  Yes, it might; though, unlike memory, a processor is more than just a frequency/feature size and a TDP. A processor also has specialty instructions which workloads may have been optimized for. I would expect that right now, Google is 1. seeing to the re-engineering of certain math libraries they use internally for Google Search et al, and 2. like AWS, intending to create separate cloud-VM instance classes for Intel vs. AMD processors, where each workload type can “pay its own bills.” If AMD continues to dominate in server TDP, then I would expect this to lead to the AMD instance classes getting cheaper while the Intel classes remain the same; and so most customers switching to AMD instances, save for the customers who themselves have workloads optimized for Xeon-specific SIMD instructions.
- chx 8y ago
  Care to link to processor efficiency benchmarks underlining your statement?
  - cde-v 8y ago
    https://www.anandtech.com/show/11544/intel-skylake-ep-vs-amd-epyc-7000-cpu-battle-of-the-decade/22 https://www.anandtech.com/show/11544/intel-skylake-ep-vs-amd...
    - toast0 8y ago
      That page is most assuredly not comparing apples to apples. The isn't noted at all, it's more of a comparison on peak power usage, which you could maybe use to plan cooling, and a comment on idle power usage. But you would need to compare total watts for enough systems to get the throughput you need, which I suspect is not the same and is likely workload dependent. For home use, where you are likely to only have a single system, you can directly compare idle usage, but for Google how many idle systems depends on your thrroughput needs --- and maybe Google can orchestrate shutdown / reuse of idle servers off-peak, so it might be moot.
masklinn 8y ago
IIRC DDR4 already has pretty much the same energy consumption as LPDDR4, LPDDR4 gains when not in active use (sleep / hibernation). LPDDR4X does improve on LPDDR4 by a bit, but I don't know how sensible ~20% less energy consumption on RAM is in a server context. Especially when you're trading density for it.
- kingosticks 8y ago
  Do you not think the IO voltage reduction alone gets you that? Or are you saying its just not worth it?
dgacmu 8y ago
This was becoming true before 2010, but no longer holds - server and chip efficiency has improved quite a bit (correlating with the stop in chip frequency increases), and datacenter PUEs have improved sufficiently to take more of the edge off. Power is still a big cost, but it's not 40%.
- dilyevsky 8y ago
  Significant portion of that electricity spend is to power cooling systems tho
  - dgacmu 8y ago
    No, that's PUE - a measure of the overall efficiency of computing. Today's leading datacenters have a PUE of about 1.12, meaning 12% of their total power is spent on non-compute things (losses in electrical conversion and cooling, for the most part). A decade ago, PUE numbers were much more commonly closer to 2, (Google's were in the 1.2-1.3 range even in 2008, but things had improved quite a bit since 2005 already, and MSFT and YHOO were working on it), but modern datacenter design has improved that substantially. see, e.g., https://www.google.com/about/datacenters/efficiency/internal/ https://www.google.com/about/datacenters/efficiency/internal...
    - BurnGpuBurn 8y ago
      This is a strange figure. If 12% of power is used on non-compute things, what is the other 88% of power converted into? The computed information? In my understanding, 100% of the power consumed is converted to heat. Measuring efficiency should have a unit like Gigaflops per Megawatt or something, not a fictional percentage. That way you can also account for hardware getting more energy efficient over time, which is now just barely represented in the numbers. It's hard to get a right unit though, because for instance archive.org has a big data center but focuses on storage instead of processing power.
      - na85 8y ago
        I'm not sure how much, but some of the energy is transformed into the electromotive force that literally pushes metal molecules around in the circuitry of your pc, so it can't be 100% heat.

adrr 8y ago

I am not an expert on microchips but looking at Apple's A12 vs Intel chips in terms of power and energy usage. Wouldn't the industry really benefit moving away from X86 architecture to something else in terms of reducing power consumption? These savings could be passed to users of cloud providers.

matthewmacleod 8y ago
There's not much that intrinsically makes an aarch chip lower-power or more energy-efficient than an x86 or x64 one – energy use will be more about the design goals of the microarchitecture than anything else. There's a paper from a few years ago the looked into this in some detail – ftp://doc.nit.ac.ir/cee/computer/alinejhad.saeedeh/old/95-96-1/MS/Advanced%20Architecture/project/paper/6-ISA%20Wars%20Understanding%20the%20Relevance%20of%20ISA%20being%20RISC%20or%20CISC.pdf – there's no immediate reason to think the conclusions should have changed.
jjeaff 8y ago
We have about 40 years of software development on x86. I'm not even sure we could reuse any of that when moving to a new architecture.
- noir_lord 8y ago
  Does that matter for some cases.? Linux supports arm and we have massive companies where economies of scale and a Linux/OSS plus their code could all be cross compiled, I mean if your the average biggish none tech company then yeah you might be stuck on x86 because of windows but that isn't everyone.
  - zitterbewegung 8y ago
    In any organization if you have an internal tool or software it’s not like you can press a button and make sure it is cross compiled to ARM from x86. That is a huge and expensive undertaking.
    - phkahler 8y ago
      >> In any organization if you have an internal tool or software it’s not like you can press a button and make sure it is cross compiled to ARM from x86. That is a huge and expensive undertaking. That depends on the software. Many linux distributions can be compiled for x86, ARM, PPC, RISC-V, and even some others. Cross compiling is not difficult if your software is written with even a little of that in mind. Windows itself used to be available for 3 or 4 instruction sets - and an OS requires low level hardware support. Applications should be largely target independent.
      - kevin_thibedeau 8y ago
        Lots of x86 code plays fast and loose with alignment which will blow up on ARM.
        monocasa 8y ago
        Modern big ARMs let you play fast and loose with alignment as well. Hell, a CortexM3 lets you have unaligned loads and stores too.
        jschwartzi 8y ago
        Even if they don't you can just turn unaligned access emulation on in your OS kernel. This was a thing in QNX and is also a thing in Linux.
        monocasa 8y ago
        I think it's fair to not want to turn that on in the kernel on chips that don't support it in servers. It's really expensive on those kinds of cores to trap into the kernel (a KPTI makes it dramatically more expensive). That being said, server cores with their long me or pipelines are the most likely to not have an issue with unaligned access, so it's probably a moot point.
- simias 8y ago
  Unless your software is written in assembly you can probably reuse a lot of that when moving to a new architecture. And given that interpreted/JITed languages are pretty mainstream these days I'm sure many developers wouldn't even notice the difference if you swapped their CPUs during the night.
JetSetWilly 8y ago
Even if one arch is intrinsically lower power than the other, I am not sure that x86 would be the one to come off worse. For example, could one not argue that an x86 binary / code size is on average smaller and more dense than the ARM equivalent, therefore on x86 cache misses will be less common, less ram needs to be used etc etc. Perhaps a denser binary is better for power consumption? Although this effect would be very marginal even if it is present at all.
- adrr 8y ago
  Interesting on the code size. How does pipeline length effect this? ARM is a lot shorter so a miss doesn't matter as much. Intel once tried to push long pipelines with their netburst architecture but real-world scenarios didn't really show any improvements. Are the compilers better at branch prediction now?
rbanffy 8y ago
In the end it all comes down to the energy cost of doing a computation. With good architectures and compilers, the code is causing as few unnecessary bit flips as possible to get to the results and, unless you have a significant difference in how much energy it takes do that, all CPUs will end up close. But CPUs are just part of the equation - you can make significant gains in other parts of the computer and memory is a good target. When we got to the point PSUs live as independent things plugged into the same rack as your servers, it means we are really concerned about not feeding one amp more than we need to.
- janekm 8y ago
  Mainstream CPUs are heavily optimised for latency over energy consumption (using speculative execution to effectively waste a multiple of the energy required for the computation itself). It'd be an interesting exercise to try to design an architecture optimises energy consumption over latency (perhaps rather than deep cache hierarchies, using cores that can power gate while they are waiting for memory accesses...)
  - rbanffy 8y ago
    That's a very good point. Even when one generates optimal code, the CPU will run all sorts of computations that do not contribute to reach the right result, but only to get there more quickly. A lot of the low-power ARM processors do that - just avoiding speculative execution like you pointed out is a huge win if power is your main concern. Another interesting take are barrel processors that can avoid scheduling stalled contexts, but they are only effective when we can feed them enough threads the time it spends waiting is less than a "normal" processor would. Early Xeon Phi's (and Cell's PPUs) behaved like barrel processors - each core could dispatch 4 (2 on the PPY) threads, but it did so round-robin. Scheduling only one thread per core didn't make them run any faster.
jandrewrogers 8y ago
No, the power efficiency of (micro)architectures is highly dependent on how well the workload fits design assumptions. CPU cores spend their time in one of several states with different power profiles. You can increase the power efficiency of one state by reducing the power efficiency of another, so assumptions about what a CPU core will spend most of its time doing matter quite a bit in terms of optimizing throughput per watt. If a CPU core spends most of its time idle, it makes sense to optimize power consumption of idle time even if it makes actual computing or waiting for memory more expensive. Intel microarchitectures tend to be heavily optimized around a workload that assumes a CPU core is doing heavy scalar computation all the time. If your workload and code actually looks like that -- and many kinds of server workloads do -- you can't beat them for performance per watt. But this comes at the cost of things like power consumption when idle. Most personal computing, on the other hand, spends most of its time idle and may benefit significantly from architectures that optimize more for power consumption at low utilization levels. There are many ways to attack these problems at the architectural level. This becomes complicated to optimize in intermittent compute-intensive environments, since over-optimizing for the idle time effectively increases the amount of time you spend in a less efficient compute-intensive state (see also: Apple's big/little core hybrid ARM architecture). There was a lot of empirical work done on this in the supercomputing world, which is sensitive to power costs and had built ARM clusters. The consensus seemed to be in that environment, for example, that throughput per watt under max load was the driver of power costs, which favored Intel. Modeling this is difficult because it is a complex interaction between specific pieces of code and specific architecture implementations. Even just comparing Intel and AMD, they have different tradeoffs for real code because the architectures are optimized differently. For example, while some codes run very efficiently on AMD's Epyc, I have a couple old codes that have terrible performance on Epyc that can be traced to design decisions made by AMD. You can't design a CPU that is the best at everything or even most things.
- adrr 8y ago
  Thank you for the explanation. First time I've heard it in such understandable concepts. Do you think the industry will move to specialized processors dependant on workload type? For example, for serverless, cloud providers could move workloads to the most efficient microarchitecture.
spamizbad 8y ago
In addition to what others have said there’s one area to consider: the “uncore” performance: memory controllers, PCIE busses, core crosslinks, cache coherency etc. Both Intel and AMD have demonstrated they can scale these effectively. The A12X isn’t even playing in the same league when it comes to uncore functionality. I’m sure if you created a version of the A12X with the same cache, memory, PCIE, core count, and interconnections the power picture looks a lot different. This is part of the reason why “datacenter arm” CPUs never get much of a foothold: as soon as you start ramping up your uncore transistor count, your power advantage erodes significantly, to the point where Intel and AMD begin to yield better performance-per-watt in typical server workloads due to a tightly optimized microarch and an “uncore” to feed it.

hinkley 8y ago

It’s not unreasonable though. For every x watts of server you need y watts of cooling. Years ago Google complained that they couldn’t fill up their data centers because they had hit the limits of electrical code. Servers plus A/C took them into no man’s land as far as electrical code was concerned. That also means wasted real estate.

stefan_ 8y ago
That seems unlikely. An aluminum smelter uses far more energy than a data center ever will.
- hinkley 8y ago
  Yeah that’s a good question. Smelters don’t generate their own power generally, right? But I believe they also use a lot of air cooling so most of that draw is going to be electrolysis. Wikipedia has a list of aluminum smelters and I notice none of the biggest are in the US. Is that just because, or due to code limitations?
  - oasisbob 8y ago
    The Pacific Northwest used to have a lot more Aluminum smelting than it does now - the power was provided by large hydro power contracts. Not sure if it was changes in these contracts or other market forces that caused the decline.
    - oasisbob 8y ago
      Decided to answer my own question - here's a great history of the NW aluminum industry: https://www.nwcouncil.org/reports/columbia-river-history/aluminum https://www.nwcouncil.org/reports/columbia-river-history/alu... The last active NW aluminum smelters closed in 2016. Most of the rest were victims of Enron and the 2000 energy crisis.
  - mikeyouse 8y ago
    Aluminum smelting is dirty and uses a ton of power -- we generally don't subsidize power here (in stark contrast to China / Argentina / etc.) and have increasingly strict environmental controls so we're offshoring most production and pollution associated with smelting from ore. We do have a lot of 'secondary' production which is much cleaner since it uses aluminum scrap as the input.

wyldfire 8y ago

Are ECC LPDDRx memory sticks available? If not, perhaps they should consider it. But Google probably couldn't switch until/unless they're produced.

wtallis 8y ago
LPDDR doesn't come in modular form. It's always soldered onto the same PCB as the processor. The number of packages used is determined by the data bus width of the processor. If Google has access to a processor with a DRAM controller that supports LPDDR4 and ECC, they can design a suitable motherboard.
- ChuckMcM 8y ago
  That is typical usage but it isn't the required usage. It is straight forward to generate a controller ASIC that would provide the appropriate bus signals and drive levels for these to operate on a module.
  - kingosticks 8y ago
    The signal integrity at 0.6V at whatever frequency these are at, across a connector must be horrible. Are people doing this?
    - ChuckMcM 8y ago
      I have seen LPDDR modules where the ASIC provides the connector signals. So they run 0.6V between the LPDDR chips and the ASIC, then the ASIC runs 1.8V on the connector.
      - kingosticks 8y ago
        I don't think I follow your topology. The asic is driving 1.8V to a dimm with 3 LPDDR chips on it? At ddr4 speeds?
        8y ago

jaxtellerSoA 8y ago

Then it also stands to reason that it makes business sense to switch from Intel Xeon to AMD Epyc.

derefr 8y ago
Yes, it might; though, unlike memory, a processor is more than just a frequency/feature size and a TDP. A processor also has specialty instructions which workloads may have been optimized for. I would expect that right now, Google is 1. seeing to the re-engineering of certain math libraries they use internally for Google Search et al, and 2. like AWS, intending to create separate cloud-VM instance classes for Intel vs. AMD processors, where each workload type can “pay its own bills.” If AMD continues to dominate in server TDP, then I would expect this to lead to the AMD instance classes getting cheaper while the Intel classes remain the same; and so most customers switching to AMD instances, save for the customers who themselves have workloads optimized for Xeon-specific SIMD instructions.
chx 8y ago
Care to link to processor efficiency benchmarks underlining your statement?
- cde-v 8y ago
  https://www.anandtech.com/show/11544/intel-skylake-ep-vs-amd-epyc-7000-cpu-battle-of-the-decade/22 https://www.anandtech.com/show/11544/intel-skylake-ep-vs-amd...
  - toast0 8y ago
    That page is most assuredly not comparing apples to apples. The isn't noted at all, it's more of a comparison on peak power usage, which you could maybe use to plan cooling, and a comment on idle power usage. But you would need to compare total watts for enough systems to get the throughput you need, which I suspect is not the same and is likely workload dependent. For home use, where you are likely to only have a single system, you can directly compare idle usage, but for Google how many idle systems depends on your thrroughput needs --- and maybe Google can orchestrate shutdown / reuse of idle servers off-peak, so it might be moot.

masklinn 8y ago

IIRC DDR4 already has pretty much the same energy consumption as LPDDR4, LPDDR4 gains when not in active use (sleep / hibernation). LPDDR4X does improve on LPDDR4 by a bit, but I don't know how sensible ~20% less energy consumption on RAM is in a server context. Especially when you're trading density for it.

kingosticks 8y ago
Do you not think the IO voltage reduction alone gets you that? Or are you saying its just not worth it?

dgacmu 8y ago

This was becoming true before 2010, but no longer holds - server and chip efficiency has improved quite a bit (correlating with the stop in chip frequency increases), and datacenter PUEs have improved sufficiently to take more of the edge off. Power is still a big cost, but it's not 40%.

dilyevsky 8y ago
Significant portion of that electricity spend is to power cooling systems tho
- dgacmu 8y ago
  No, that's PUE - a measure of the overall efficiency of computing. Today's leading datacenters have a PUE of about 1.12, meaning 12% of their total power is spent on non-compute things (losses in electrical conversion and cooling, for the most part). A decade ago, PUE numbers were much more commonly closer to 2, (Google's were in the 1.2-1.3 range even in 2008, but things had improved quite a bit since 2005 already, and MSFT and YHOO were working on it), but modern datacenter design has improved that substantially. see, e.g., https://www.google.com/about/datacenters/efficiency/internal/ https://www.google.com/about/datacenters/efficiency/internal...
  - BurnGpuBurn 8y ago
    This is a strange figure. If 12% of power is used on non-compute things, what is the other 88% of power converted into? The computed information? In my understanding, 100% of the power consumed is converted to heat. Measuring efficiency should have a unit like Gigaflops per Megawatt or something, not a fictional percentage. That way you can also account for hardware getting more energy efficient over time, which is now just barely represented in the numbers. It's hard to get a right unit though, because for instance archive.org has a big data center but focuses on storage instead of processing power.
    - na85 8y ago
      I'm not sure how much, but some of the energy is transformed into the electromotive force that literally pushes metal molecules around in the circuitry of your pc, so it can't be 100% heat.