6 ms·

Macs are a small market, but Apple definitely sets trends. Once they adopt a technology, people "know" it works and it's safe to switch. If servers can run on

Macs are a small market, but Apple definitely sets trends. Once they adopt a technology, people "know" it works and it's safe to switch. If servers can run on cheap ARM hardware that uses less electricity and runs far cooler without much of a performance difference, that's a massive improvement. Electricity savings would more than justify a switch. Apple has shown that it's possible at the desktop/laptop level. Now they or someone else can prove it works at the server level as well.

easygenes 5y ago
For most purposes, electricity is almost a rounding error in datacenter asset management concerns. Server compute is expensive. It is all about performance. ARM is just starting to get competitive in that ballpark. So we will see. All of these ARM server chips are going to be very low on the pecking order for TSMC fab time, so they won't have the edge that Apple had by buying up the first slot on the latest manufacturing node. Even being a process node ahead, M1 only just squeaks by with comparable single threaded performance to Zen 3 cores, so ARM still has some catching up to do in the performance realm.
- hencoappel 5y ago
  As far as I'm aware, efficiency is key for datacenters. It can lower direct electricity costs, but also the cost of cooling. I've always been under the impression that the cost of running is much more than the cost of the hardware which is why they are willing to pay so much for the best/most efficient hardware?
- tdrdt 5y ago
  "For most purposes, electricity is almost a rounding error in datacenter asset management concerns." I don't think this is true. It is not only power consumption but also power backup. If you can use smaller diesel engines and smaller battery packs this will lower the cost. And why do you think datacenters have been raising temperatures? It saves a ton of money: https://www.datacenterknowledge.com/archives/2008/10/14/google-raise-your-data-center-temperature https://www.datacenterknowledge.com/archives/2008/10/14/goog... HP estimated they saved 8 million. That is not a rounding error. A server that uses less power will also generate less heat. I have to admit it is years ago I worked for a company owning datacenters but at that time the highest costs were always: power and connectivity.
  - easygenes 5y ago
    That article cites a 100,000 sq. ft. data-center as the source for that $8 million savings estimate, along with roughly a 30% power savings figure. No mention on the period, so presumably it would be a TCO for the servers going into the initial buildout. 100,000 sq. ft. is roughly enough space for about 250,000 1U servers. Say conservatively the servers are in the ballpark of $2,000 each. That's $500 million just in server hardware costs. If the total electric cost is around 3*$8 = ~$24 million over their lifetime, then we're talking about <5% of just the server expenses. Never mind the facility and staff costs (and, as you've said, connectivity). So maybe not a rounding error, but way down the priorities list.
    - deleted 5y ago
      [deleted]
    - klelatti 5y ago
      These are AC costs not total costs and the $8m is probably an annual figure - although it's not clear.
      - easygenes 5y ago
        Yeah, unfortunately it's a bit vague on the period. It is not unreasonable to assume an annual figure, in which case you're looking at something closer to 20% of the server costs for A/C power (assuming an average life of about 4 years). Problem is, that sounds high... the coefficient of performance for chillers is around 4 to 7 [1]. That puts ~15-30% of the total energy demand from chillers, though often datacenters do budget a factor up to about 60% of equipment power demand for cooling power demand. Sum energy related expenses for datacenters tends to be around 10-15% of the total costs [2]. So it would be odd to have such high cooling costs. A TCO over 4 years seems more in line with typical figures. 1: https://www.energy.gov.au/sites/default/files/hvac-factsheet-chiller-efficiency.pdf 2: https://www.missioncriticalmagazine.com/ext/resources/MC/Home/Files/PDFs/(TUI3011B)SimpleModelDetermingTrueTCO.pdf
- oefrha 5y ago
  Are you speaking from experience doing data center asset management or are you just speculating? I don’t know about data centers, but for colocation, electricity cost is far from a rounding error. Also, cooling is a major factor that is directly related to power consumption, as pointed out by siblings.
  - easygenes 5y ago
    See [1], figure 1 there is a fairly typical data center costs breakdown at the highest level for TCO. Total energy is by far the lowest, at around 10-15%. Site infrastructure, IT infrastructure, and staff are the real cost priorities. There's a caveat to that in the planning phase of a data center: A lot of the site infrastructure costs are a function of the total power requirement. So if - when building out a data center - you can get more power efficiency, then that does translate to a significant cost savings. Cooling tends to be somewhere between a 20 and 60% add on to the direct power consumption of a server. 1: https://www.missioncriticalmagazine.com/ext/resources/MC/Home/Files/PDFs/(TUI3011B)SimpleModelDetermingTrueTCO.pdf
- iorrus 5y ago
  In Ireland the national grid is under strain due to electricity usage from data centres, so much so that there is discussion to deny permission for new ones. I very much doubt that the electricity costs are negligible, it’s also one of the reason data centres are in Ireland as due to the relatively mild climate it reduces hearing/cooling costs. https://www.irishtimes.com/news/politics/data-centres-could-use-70-of-ireland-s-electricity-by-2030-committee-to-hear-1.4685589 https://www.irishtimes.com/news/politics/data-centres-could-...
  - Symbiote 5y ago
    Isn't more of the reason political (taxes etc)? The same climate benefits apply to northern England and Scotland, but Great Britain's grid's capacity has around 10 times the capacity. (Ireland's grid's peak demand was 6.8GW, Great Britain's 63GW.) Although the same argument then applies for siting the datacentres in continental Europe, where the grid is around 667GW. Perhaps this is part of Facebook's reasoning for a second datacentre in Denmark. (Most of Denmark, including Facebook's existing datacentre, is connected to the continental European grid. The eastern islands are part of the Scandinavian grid.) [1] https://www.thelocal.dk/20211013/facebook-eyes-second-danish-data-centre-after-buying-large-plot/ https://www.thelocal.dk/20211013/facebook-eyes-second-danish...
  - easygenes 5y ago
    Will be interesting to see where datacenters wind up in Europe this next decade. My money is on Norway. Norway seems poised to build out significantly. They're actively inviting datacenters [1]. They also seem to have lower electric costs for large businesses [2] vs Ireland [3]. FAANG are already populating the Nordics too [4]. 1: https://www.datacenterdynamics.com/en/news/norway-wants-data-centers-to-locate-there-and-be-more-sustainable-too/ 2: https://www.statista.com/statistics/595859/electricity-industry-price-norway/ 3: https://www.statista.com/statistics/595806/electricity-industry-price-ireland/ 4: https://www.zdnet.com/pictures/the-nordic-datacenter-boom/
- Symbiote 5y ago
  The average consumer cost for 1kWh of electricity in the EU is 20¢. A well-used server might consume 500W on average. units '500W * 1 year * 0.20 €/kWh' '€' € 876 I'm sure businesses get a discount, but we haven't paid for cooling yet. This is hardly a rounding error.
  - ksec 5y ago
    >I'm sure businesses get a discount Not as large as some may expect once you factor in power delivery guarantee and redundancy. But basically yes. Every single HyperScaler has been pointing out electricity as one of their largest item on their TCO, 2nd only to hardware cost.
  - anthk 5y ago
    Which units is that? Mine does return stuff in Euros but with a fraction.
    - Symbiote 5y ago
      GNU Units version 2.19 with readline, with utf8, locale en_IE and no customizations. units '500W * 1 year * 0.20/kWh' '1' would also work.
      - anthk 5y ago
        Do you have a custom ~/.units file? user@xirl>units '500W * 1 year * 0.20/kWh' '1' * 876.58128 / 0.0011407955
qwertox 5y ago
It basically says that the "intel inside"-logo on laptops has become meaningless.
tdrdt 5y ago
You are giving too much praise to Apple. They never set trends in the datacenter world. Arm servers have been sold for some years now. AWS, Oracle and others already made the switch. And the fastest super computer (Fugaku, Japan) uses ARM.
- cultofmetatron 5y ago
  I'm building server software for my startup. My development environment is a linux docker container running on a linux vm running on an x86 mac. Its running the same binary as the docker container running in kubernetes on an x86 aws server. The fact that apple is making arm mainstream means if I switch to arm, so are a lot of others working to make the arm port of my stack a first class consideration. Suddenly the gavatron cores aws has are becoming a whole lot more viable. I personally plan to move our api instances over to them once the erlang vm's arm port matures a bit.
smackeyacky 5y ago
In CPU choices, when Apple have lead (PowerPC) nobody followed. Their other CPU choices have hardly been brave. 6502? Not the first. 680x0? Followers. Intel they were way behind and all their laptop and desktop gear was a generation behind everything other companies were using. Phones bit of a mixed bag but their computing gear has always been a bit old hat since the 1980s. The M1 is the first time since PowerPC they took a risk. My guess is that it will turn out the same PowerPC. I really want a linux powered ARM workstation but Apple aren't going to make that.
- zeckalpha 5y ago
  https://en.m.wikipedia.org/wiki/MessagePad https://en.m.wikipedia.org/wiki/MessagePad
  - anthk 5y ago
    Irrelevant, Palm did it better. No one got a Newton, but every mid-boss or manager got a Palm.
    - zeckalpha 5y ago
      But Palm didn’t drive the CPU direction of the industry as GP claimed.
      - anthk 5y ago
        Neither did the Newton.
        zeckalpha 5y ago
        > Their other CPU choices have hardly been brave.
- MisterTea 5y ago
  > In CPU choices, when Apple have lead (PowerPC) nobody followed. PowerPC was developed by IBM. Apple got involved when Motorola could not deliver a faster 68k to beat Intel's 486 which was passing 50MHz. So they joined IBM along with Motorola and formed the AIM alliance (Apple, IBM, Motorola) to build better processors. > The M1 is the first time since PowerPC they took a risk. There's little to no risc (buh-dum-tish) in moving to Arm these days. It's a well supported and understood architecture and found everywhere. > I really want a linux powered ARM workstation but Apple aren't going to make that. Apple has the power to steer its own ecosystem. When AMD was working on its Arm A series server processors I kept thinking "This sounds like a backwards approach destined to fail. Why not start by making a performant Arm SoC with 2-4 cores and GPU with a TDP of 10-20W? Target it at consoles/tv/laptops/desktop computing and jump start the desktop Arm market which will naturally lead to demand for Arm servers." The Idea was an Arm SoC that could fill the gap between low power/performance Arm SoC's for mobile/embedded and the power hungry yet performant x86 chips. Basically an AMD version of the M1. That could have really changed things but the big issue AMD would face is where's the Arm Desktop software ecosystem? That's why Apple can take these "risks", they have full control over the whole stack.
fullstop 5y ago
> Once they adopt a technology, people "know" it works and it's safe to switch. Not always. Firewire, for example.
- LargoLasskhyfv 5y ago
  Afaik it flies in figher jets, experimental military UAVs and also civil satellites. Just not so common in consumer tech anymore. Besides that it's fun to debug with on older systems, or even new ones if they have it. If not it can be plugged in from anything between 30 to 50 universal credit units from 1 to 4 port Firewire800 as PCIe 1x to 4x. It's fun to have in a homelab. You can even tunnel IP over it.

easygenes 5y ago

For most purposes, electricity is almost a rounding error in datacenter asset management concerns. Server compute is expensive. It is all about performance. ARM is just starting to get competitive in that ballpark. So we will see. All of these ARM server chips are going to be very low on the pecking order for TSMC fab time, so they won't have the edge that Apple had by buying up the first slot on the latest manufacturing node. Even being a process node ahead, M1 only just squeaks by with comparable single threaded performance to Zen 3 cores, so ARM still has some catching up to do in the performance realm.

hencoappel 5y ago
As far as I'm aware, efficiency is key for datacenters. It can lower direct electricity costs, but also the cost of cooling. I've always been under the impression that the cost of running is much more than the cost of the hardware which is why they are willing to pay so much for the best/most efficient hardware?
tdrdt 5y ago
"For most purposes, electricity is almost a rounding error in datacenter asset management concerns." I don't think this is true. It is not only power consumption but also power backup. If you can use smaller diesel engines and smaller battery packs this will lower the cost. And why do you think datacenters have been raising temperatures? It saves a ton of money: https://www.datacenterknowledge.com/archives/2008/10/14/google-raise-your-data-center-temperature https://www.datacenterknowledge.com/archives/2008/10/14/goog... HP estimated they saved 8 million. That is not a rounding error. A server that uses less power will also generate less heat. I have to admit it is years ago I worked for a company owning datacenters but at that time the highest costs were always: power and connectivity.
- easygenes 5y ago
  That article cites a 100,000 sq. ft. data-center as the source for that $8 million savings estimate, along with roughly a 30% power savings figure. No mention on the period, so presumably it would be a TCO for the servers going into the initial buildout. 100,000 sq. ft. is roughly enough space for about 250,000 1U servers. Say conservatively the servers are in the ballpark of $2,000 each. That's $500 million just in server hardware costs. If the total electric cost is around 3*$8 = ~$24 million over their lifetime, then we're talking about <5% of just the server expenses. Never mind the facility and staff costs (and, as you've said, connectivity). So maybe not a rounding error, but way down the priorities list.
  - deleted 5y ago
    [deleted]
  - klelatti 5y ago
    These are AC costs not total costs and the $8m is probably an annual figure - although it's not clear.
    - easygenes 5y ago
      Yeah, unfortunately it's a bit vague on the period. It is not unreasonable to assume an annual figure, in which case you're looking at something closer to 20% of the server costs for A/C power (assuming an average life of about 4 years). Problem is, that sounds high... the coefficient of performance for chillers is around 4 to 7 [1]. That puts ~15-30% of the total energy demand from chillers, though often datacenters do budget a factor up to about 60% of equipment power demand for cooling power demand. Sum energy related expenses for datacenters tends to be around 10-15% of the total costs [2]. So it would be odd to have such high cooling costs. A TCO over 4 years seems more in line with typical figures. 1: https://www.energy.gov.au/sites/default/files/hvac-factsheet-chiller-efficiency.pdf 2: https://www.missioncriticalmagazine.com/ext/resources/MC/Home/Files/PDFs/(TUI3011B)SimpleModelDetermingTrueTCO.pdf
oefrha 5y ago
Are you speaking from experience doing data center asset management or are you just speculating? I don’t know about data centers, but for colocation, electricity cost is far from a rounding error. Also, cooling is a major factor that is directly related to power consumption, as pointed out by siblings.
- easygenes 5y ago
  See [1], figure 1 there is a fairly typical data center costs breakdown at the highest level for TCO. Total energy is by far the lowest, at around 10-15%. Site infrastructure, IT infrastructure, and staff are the real cost priorities. There's a caveat to that in the planning phase of a data center: A lot of the site infrastructure costs are a function of the total power requirement. So if - when building out a data center - you can get more power efficiency, then that does translate to a significant cost savings. Cooling tends to be somewhere between a 20 and 60% add on to the direct power consumption of a server. 1: https://www.missioncriticalmagazine.com/ext/resources/MC/Home/Files/PDFs/(TUI3011B)SimpleModelDetermingTrueTCO.pdf
iorrus 5y ago
In Ireland the national grid is under strain due to electricity usage from data centres, so much so that there is discussion to deny permission for new ones. I very much doubt that the electricity costs are negligible, it’s also one of the reason data centres are in Ireland as due to the relatively mild climate it reduces hearing/cooling costs. https://www.irishtimes.com/news/politics/data-centres-could-use-70-of-ireland-s-electricity-by-2030-committee-to-hear-1.4685589 https://www.irishtimes.com/news/politics/data-centres-could-...
- Symbiote 5y ago
  Isn't more of the reason political (taxes etc)? The same climate benefits apply to northern England and Scotland, but Great Britain's grid's capacity has around 10 times the capacity. (Ireland's grid's peak demand was 6.8GW, Great Britain's 63GW.) Although the same argument then applies for siting the datacentres in continental Europe, where the grid is around 667GW. Perhaps this is part of Facebook's reasoning for a second datacentre in Denmark. (Most of Denmark, including Facebook's existing datacentre, is connected to the continental European grid. The eastern islands are part of the Scandinavian grid.) [1] https://www.thelocal.dk/20211013/facebook-eyes-second-danish-data-centre-after-buying-large-plot/ https://www.thelocal.dk/20211013/facebook-eyes-second-danish...
- easygenes 5y ago
  Will be interesting to see where datacenters wind up in Europe this next decade. My money is on Norway. Norway seems poised to build out significantly. They're actively inviting datacenters [1]. They also seem to have lower electric costs for large businesses [2] vs Ireland [3]. FAANG are already populating the Nordics too [4]. 1: https://www.datacenterdynamics.com/en/news/norway-wants-data-centers-to-locate-there-and-be-more-sustainable-too/ 2: https://www.statista.com/statistics/595859/electricity-industry-price-norway/ 3: https://www.statista.com/statistics/595806/electricity-industry-price-ireland/ 4: https://www.zdnet.com/pictures/the-nordic-datacenter-boom/
Symbiote 5y ago
The average consumer cost for 1kWh of electricity in the EU is 20¢. A well-used server might consume 500W on average. units '500W * 1 year * 0.20 €/kWh' '€' € 876 I'm sure businesses get a discount, but we haven't paid for cooling yet. This is hardly a rounding error.
- ksec 5y ago
  >I'm sure businesses get a discount Not as large as some may expect once you factor in power delivery guarantee and redundancy. But basically yes. Every single HyperScaler has been pointing out electricity as one of their largest item on their TCO, 2nd only to hardware cost.
- anthk 5y ago
  Which units is that? Mine does return stuff in Euros but with a fraction.
  - Symbiote 5y ago
    GNU Units version 2.19 with readline, with utf8, locale en_IE and no customizations. units '500W * 1 year * 0.20/kWh' '1' would also work.
    - anthk 5y ago
      Do you have a custom ~/.units file? user@xirl>units '500W * 1 year * 0.20/kWh' '1' * 876.58128 / 0.0011407955

qwertox 5y ago

It basically says that the "intel inside"-logo on laptops has become meaningless.

tdrdt 5y ago

You are giving too much praise to Apple. They never set trends in the datacenter world. Arm servers have been sold for some years now. AWS, Oracle and others already made the switch. And the fastest super computer (Fugaku, Japan) uses ARM.

cultofmetatron 5y ago
I'm building server software for my startup. My development environment is a linux docker container running on a linux vm running on an x86 mac. Its running the same binary as the docker container running in kubernetes on an x86 aws server. The fact that apple is making arm mainstream means if I switch to arm, so are a lot of others working to make the arm port of my stack a first class consideration. Suddenly the gavatron cores aws has are becoming a whole lot more viable. I personally plan to move our api instances over to them once the erlang vm's arm port matures a bit.

smackeyacky 5y ago

In CPU choices, when Apple have lead (PowerPC) nobody followed. Their other CPU choices have hardly been brave. 6502? Not the first. 680x0? Followers. Intel they were way behind and all their laptop and desktop gear was a generation behind everything other companies were using. Phones bit of a mixed bag but their computing gear has always been a bit old hat since the 1980s. The M1 is the first time since PowerPC they took a risk. My guess is that it will turn out the same PowerPC. I really want a linux powered ARM workstation but Apple aren't going to make that.

zeckalpha 5y ago
https://en.m.wikipedia.org/wiki/MessagePad https://en.m.wikipedia.org/wiki/MessagePad
- anthk 5y ago
  Irrelevant, Palm did it better. No one got a Newton, but every mid-boss or manager got a Palm.
  - zeckalpha 5y ago
    But Palm didn’t drive the CPU direction of the industry as GP claimed.
    - anthk 5y ago
      Neither did the Newton.
      - zeckalpha 5y ago
        > Their other CPU choices have hardly been brave.
MisterTea 5y ago
> In CPU choices, when Apple have lead (PowerPC) nobody followed. PowerPC was developed by IBM. Apple got involved when Motorola could not deliver a faster 68k to beat Intel's 486 which was passing 50MHz. So they joined IBM along with Motorola and formed the AIM alliance (Apple, IBM, Motorola) to build better processors. > The M1 is the first time since PowerPC they took a risk. There's little to no risc (buh-dum-tish) in moving to Arm these days. It's a well supported and understood architecture and found everywhere. > I really want a linux powered ARM workstation but Apple aren't going to make that. Apple has the power to steer its own ecosystem. When AMD was working on its Arm A series server processors I kept thinking "This sounds like a backwards approach destined to fail. Why not start by making a performant Arm SoC with 2-4 cores and GPU with a TDP of 10-20W? Target it at consoles/tv/laptops/desktop computing and jump start the desktop Arm market which will naturally lead to demand for Arm servers." The Idea was an Arm SoC that could fill the gap between low power/performance Arm SoC's for mobile/embedded and the power hungry yet performant x86 chips. Basically an AMD version of the M1. That could have really changed things but the big issue AMD would face is where's the Arm Desktop software ecosystem? That's why Apple can take these "risks", they have full control over the whole stack.

fullstop 5y ago

> Once they adopt a technology, people "know" it works and it's safe to switch. Not always. Firewire, for example.

LargoLasskhyfv 5y ago
Afaik it flies in figher jets, experimental military UAVs and also civil satellites. Just not so common in consumer tech anymore. Besides that it's fun to debug with on older systems, or even new ones if they have it. If not it can be plugged in from anything between 30 to 50 universal credit units from 1 to 4 port Firewire800 as PCIe 1x to 4x. It's fun to have in a homelab. You can even tunnel IP over it.