4 ms·

Microsoft shelves its underwater data center

32 points by namanyayg 2y ago

glimshe 2y ago
[flagged]
deleted 2y ago
[deleted]
yelnatz 2y ago
With an improvement of 87.5% to reliability compared to land-based data centers. I wonder what factors led them to discontinue the project.
- withinboredom 2y ago
  And I wonder if they meant 187.5% (this is better) or 87.5% (this is worse).
- Maxatar 2y ago
  That it's significantly cheaper to deal with the failures than it is to prevent them.
- poulpy123 2y ago
  I would expect price and practicability of physical intervention
- wongarsu 2y ago
  Sure, they are less likely to fail in the stable environment, but you can't get to it until you pull up the entire vessel. On land 5.9% of servers required a technician to walk over there and replace components. At sea 0.7% of servers failed irrecoverably. On the other hand 0.7% dead hardware sounds like a marginal cost. What likely killed it was the cost to deploy and recover them, plus all the hassle with sea cables for data and power. Building glorified warehouses with redundant power and AC is likely a lot cheaper than deploying these metal tanks in the ocean.
- cjbgkagh 2y ago
  Less noise and dust down there - I think the air is purged with nitrogen as well so no oxidization. They could do the same thing cheaper on land and recycle the waste heat. Edit: just read the article - they also state temperature stability as a factor. Having the servers more easy to access and repair also makes them more likely to fail. Also spaces that humans can enter require a bunch of regulation that takes up space and adds to the expense. So the math could easily work out in favor of a sealed box of computers that no-one is allowed to access and any broken servers just get switched off.
oldpersonintx 2y ago
such a data center would have been under a lot of pressure to perform
cm2187 2y ago
Wouldn't it be easier to just submerge some big heat exchanger?
- Faaak 2y ago
  At the end of the day, you're still wasting heat whereas you could heat a whole district with it.
  - cm2187 2y ago
    But the point is that it is passive and works all year long, which I suspect wouldn't be the case of a residential heating system.
tboyd47 2y ago
> “I’m not building subsea data centers anywhere in the world.” She later added, “My team worked on it, and it worked. We learned a lot about operations below sea level and vibration and impacts on the server. So, we’ll apply those learning to other cases.” It doesn't sound like they really discontinued the project. It sounds like it got moved to another team.
- wongarsu 2y ago
  The article they got the quote from has some more quotes from the interview [1]. In context it sounds to me like they lost a lot of R&D budget ("have to get more focused") and as a consequence don't pursue subsea data centers any more. But they learned a lot that is applicable to other projects they are still running. They mention liquid immersion, which is a less extreme take on the idea of trading lower failure rates for lower maintainability. 1 https://www.datacenterdynamics.com/en/news/microsoft-confirms-project-natick-underwater-data-center-is-no-more/ https://www.datacenterdynamics.com/en/news/microsoft-confirm...
deleted 2y ago
[deleted]
gorgoiler 2y ago
The article alludes to the technique for dealing with failures inside a sealed underwater container: you never actually replace hardware but instead work around failures by reprovisioning services onto different nodes. Over time more and more nodes will fail until the container becomes exhausted and needs replacing. That therefore begs the question: what are the narrowest points of failure in a system like this? For example it would be naive to have 140 fungible compute nodes if they all went through a single networking switch. Node workloads can be moved but a switch failure makes the entire container useless. This works physically as well. Marine vessels have sealed bulkheads such that a hull breach in one end doesn’t cause flooding in the other end. How do these vessels fan out all the other points of failure and what are the error rates in those components?
- wongarsu 2y ago
  It's worth noting that redundant networking gear and redundant power are the norm even in normal data centers. Having an entire row of racks go offline because of a router failure is already considered unacceptable and well-mitigated. Their biggest point of failure is probably the land connection. Looks like it's one big cable bundle. Those get damaged all the time from sea life, fishing, people throwing anchors where they shouldn't etc. Land-based datacenters have the same issue with excavators damaging power or network lines, but it's easy to run multiple lines far apart from each other when you are on land.
kulor 2y ago
Without knowing the security measures in place, it strikes me as a ripe vulnerability to any determined nefarious actor to destroy or infiltrate this submersible data centre.
ChrisArchitect 2y ago
Some more discussion about the project in 2020 when they presented some of the findings: https://news.ycombinator.com/item?id=24470954 https://news.ycombinator.com/item?id=24470954

32 points by namanyayg 2y ago

glimshe 2y ago

[flagged]

deleted 2y ago

[deleted]

yelnatz 2y ago

With an improvement of 87.5% to reliability compared to land-based data centers. I wonder what factors led them to discontinue the project.

withinboredom 2y ago
And I wonder if they meant 187.5% (this is better) or 87.5% (this is worse).
Maxatar 2y ago
That it's significantly cheaper to deal with the failures than it is to prevent them.
poulpy123 2y ago
I would expect price and practicability of physical intervention
wongarsu 2y ago
Sure, they are less likely to fail in the stable environment, but you can't get to it until you pull up the entire vessel. On land 5.9% of servers required a technician to walk over there and replace components. At sea 0.7% of servers failed irrecoverably. On the other hand 0.7% dead hardware sounds like a marginal cost. What likely killed it was the cost to deploy and recover them, plus all the hassle with sea cables for data and power. Building glorified warehouses with redundant power and AC is likely a lot cheaper than deploying these metal tanks in the ocean.
cjbgkagh 2y ago
Less noise and dust down there - I think the air is purged with nitrogen as well so no oxidization. They could do the same thing cheaper on land and recycle the waste heat. Edit: just read the article - they also state temperature stability as a factor. Having the servers more easy to access and repair also makes them more likely to fail. Also spaces that humans can enter require a bunch of regulation that takes up space and adds to the expense. So the math could easily work out in favor of a sealed box of computers that no-one is allowed to access and any broken servers just get switched off.

oldpersonintx 2y ago

such a data center would have been under a lot of pressure to perform

cm2187 2y ago

Wouldn't it be easier to just submerge some big heat exchanger?

Faaak 2y ago
At the end of the day, you're still wasting heat whereas you could heat a whole district with it.
- cm2187 2y ago
  But the point is that it is passive and works all year long, which I suspect wouldn't be the case of a residential heating system.

tboyd47 2y ago

> “I’m not building subsea data centers anywhere in the world.” She later added, “My team worked on it, and it worked. We learned a lot about operations below sea level and vibration and impacts on the server. So, we’ll apply those learning to other cases.” It doesn't sound like they really discontinued the project. It sounds like it got moved to another team.

wongarsu 2y ago
The article they got the quote from has some more quotes from the interview [1]. In context it sounds to me like they lost a lot of R&D budget ("have to get more focused") and as a consequence don't pursue subsea data centers any more. But they learned a lot that is applicable to other projects they are still running. They mention liquid immersion, which is a less extreme take on the idea of trading lower failure rates for lower maintainability. 1 https://www.datacenterdynamics.com/en/news/microsoft-confirms-project-natick-underwater-data-center-is-no-more/ https://www.datacenterdynamics.com/en/news/microsoft-confirm...

deleted 2y ago

[deleted]

gorgoiler 2y ago

The article alludes to the technique for dealing with failures inside a sealed underwater container: you never actually replace hardware but instead work around failures by reprovisioning services onto different nodes. Over time more and more nodes will fail until the container becomes exhausted and needs replacing. That therefore begs the question: what are the narrowest points of failure in a system like this? For example it would be naive to have 140 fungible compute nodes if they all went through a single networking switch. Node workloads can be moved but a switch failure makes the entire container useless. This works physically as well. Marine vessels have sealed bulkheads such that a hull breach in one end doesn’t cause flooding in the other end. How do these vessels fan out all the other points of failure and what are the error rates in those components?

wongarsu 2y ago
It's worth noting that redundant networking gear and redundant power are the norm even in normal data centers. Having an entire row of racks go offline because of a router failure is already considered unacceptable and well-mitigated. Their biggest point of failure is probably the land connection. Looks like it's one big cable bundle. Those get damaged all the time from sea life, fishing, people throwing anchors where they shouldn't etc. Land-based datacenters have the same issue with excavators damaging power or network lines, but it's easy to run multiple lines far apart from each other when you are on land.

kulor 2y ago

Without knowing the security measures in place, it strikes me as a ripe vulnerability to any determined nefarious actor to destroy or infiltrate this submersible data centre.

ChrisArchitect 2y ago

Some more discussion about the project in 2020 when they presented some of the findings: https://news.ycombinator.com/item?id=24470954 https://news.ycombinator.com/item?id=24470954