4 ms·

Is there a reliable modular datacenter construction technique that allows for easy scaling/expansion? Seems that could have a dramatic impact on bringing these

Is there a reliable modular datacenter construction technique that allows for easy scaling/expansion? Seems that could have a dramatic impact on bringing these within reach.

datadrivenangel 2y ago
Racks.
- unclebucknasty 2y ago
  Yeah, I was thinking something a little more balanced than building an AWS-scale datacenter with full infrastructure—capable of accommodating thousands of racks—then starting with ten 42Us.
kjs3 2y ago
Sun had their 'modular data center' concept[1], which was a self contained DC in a standard shipping container. You could chain them together. It wasn't obscenely expensive relatively speaking as I recall. I'm not sure if Sun was 'first', but these days there are all sorts of companies that do the same thing (I was recently in one from Delta). I don't know if they are applicable to the reality on the ground in Nigeria. [1] https://en.wikipedia.org/wiki/Sun_Modular_Datacenter https://en.wikipedia.org/wiki/Sun_Modular_Datacenter
- unclebucknasty 2y ago
  Very interesting. Yes, this is along the lines of what I was thinking. Seems you could calibrate the size of each unit according to anticipated growth and the max a site could sustain, area-wise (i.e. go larger to balance number of containers). Then, prep the site with easily extensible cabling for power/networking between units. It seems (completely intuiting here) that if the reality on the ground could accommodate full data centers, then this would be possible. And according to the article, the cost was 1% of traditional data centers.
  - kjs3 2y ago
    I don't think there was much choice when it came to the size of the unit. It was a shipping container...the size is standardized. So more "install the first one and start filling it. When you get to some percentage of maxing it out, start budgeting for the next one." the cost was 1% of traditional data centers That smells an awful lot like an apples-to-oranges marketspeak. Perhaps "1% the cost for 1% the capacity". Ignoring the things you get when you build out a full DC that these don't have, I very much doubt if you bought, say, 300k sq ft of these it would be 1%.
    - unclebucknasty 2y ago
      >I don't think there was much choice when it came to the size of the unit Yeah, I got the sense the design was intended for smaller on-demand needs versus strategically building out massive datacenters. So, my thinking is that you would borrow from the base idea, but you'd upsize each unit to cut down on the actual number of units (and attendant overhead). Just considering that if you truly are building out a datacenter, you'd probably do better to have fewer, larger units. On the flip side though, the shipping container format does make it very appealing logistically. >That smells an awful lot like an apples-to-oranges marketspeak Yeah, they are no doubt being selective in what they are comparing. That claim has to do with the cost of "connecting [these units] to existing infrastructure". So, I'm guessing they are assuming infrastructure, then comparing the cost of one of these units to construction of equivalent DC square footage. 1% still seems aggressive, but I could be convinced it's "significantly less". Of course, assuming the infrastructure starts to get murky, since a full DC build is going to be more intertwined with its infra.
      - kjs3 2y ago
        I got the sense the design was intended for smaller on-demand needs versus strategically building out massive datacenters This is absolutely the case. This was not intended to replace small, permanent DCs, much less massive ones. Think "I need compute on space limited offshore oil rig" or "I need some compute here for 6 months". you'd upsize each unit to cut down on the actual number of units I've been out of involvement in building out DCs for a long time, so my info is dated, but... As I said, there are now other companies doing variations on this. The Delta unit I mentioned upthread is built into a (standard?) 18-wheel tractor trailer (significantly larger than a shipping container). But much bigger than that you start getting into "how do I move it to the site" issues, which means costs due to transport are going to go up fast, and you're back to "does it just make sense to build a fixed DC", perhaps with a careful eye towards "I'll want to easily tack on more capacity later". Even if you figure out a way to do off-site pre-fab economically, I don't see how it would make sense to do "DC built out of X many identical prefab units" but instead you'd want to do "X many rack units, and Y many network units, and Z many air handling units, and W power distribution units, and V many CnC units, and...". I know there's a lot of talk about how building houses out of prefab components is going to change the world (one day), but unless you are building a lot of data centers, I don't know how you build out the infrastructure to deliver on that that's going to be cheaper than "haul materials to the site and build it there like we've done forever". My completely unsubstantiated view is the volume isn't there.
- abdullahkhalids 2y ago
  Very cool. Huwawai also made a similar thing [1], called the FusionModule1000A All-in-One Container Data Center. But interestingly (as far as I can tell) without any compute, just power, cooling, safety etc. Presumably costs about a million USD [2]. [1] https://web.archive.org/web/20170709195517/https://e.huawei.com/us/products/network-energy/dc-facilities/ids1000-a https://web.archive.org/web/20170709195517/https://e.huawei.... [2] https://www.finditprice.com/index/it_product/search.html?product_series=FusionModule1000-A https://www.finditprice.com/index/it_product/search.html?pro...
  - kjs3 2y ago
    I don't think Sun's base model came with any compute either. It was racks/power/cooling and then you put servers and networking in. Sun would happily sell those to you...just more SKUs on the invoice.
donavanm 2y ago
As others mentioned “everyone” has done variations of havac-and-racks-in-teu for the past 25 years. Heres Amazons version circa 2010, https://www.datacenterdynamics.com/en/news/amazon-sheds-light-on-its-data-center-modules/ https://www.datacenterdynamics.com/en/news/amazon-sheds-ligh... You find them in old satellite photos of places like us-west-2. You still need site prep to support the weight, you still need power, external CRAC/H and/or water supply, and access & repairability sucks.

Is there a reliable modular datacenter construction technique that allows for easy scaling/expansion? Seems that could have a dramatic impact on bringing these within reach.

datadrivenangel 2y ago

Racks.

unclebucknasty 2y ago
Yeah, I was thinking something a little more balanced than building an AWS-scale datacenter with full infrastructure—capable of accommodating thousands of racks—then starting with ten 42Us.

kjs3 2y ago

Sun had their 'modular data center' concept[1], which was a self contained DC in a standard shipping container. You could chain them together. It wasn't obscenely expensive relatively speaking as I recall. I'm not sure if Sun was 'first', but these days there are all sorts of companies that do the same thing (I was recently in one from Delta). I don't know if they are applicable to the reality on the ground in Nigeria. [1] https://en.wikipedia.org/wiki/Sun_Modular_Datacenter https://en.wikipedia.org/wiki/Sun_Modular_Datacenter

unclebucknasty 2y ago
Very interesting. Yes, this is along the lines of what I was thinking. Seems you could calibrate the size of each unit according to anticipated growth and the max a site could sustain, area-wise (i.e. go larger to balance number of containers). Then, prep the site with easily extensible cabling for power/networking between units. It seems (completely intuiting here) that if the reality on the ground could accommodate full data centers, then this would be possible. And according to the article, the cost was 1% of traditional data centers.
- kjs3 2y ago
  I don't think there was much choice when it came to the size of the unit. It was a shipping container...the size is standardized. So more "install the first one and start filling it. When you get to some percentage of maxing it out, start budgeting for the next one." the cost was 1% of traditional data centers That smells an awful lot like an apples-to-oranges marketspeak. Perhaps "1% the cost for 1% the capacity". Ignoring the things you get when you build out a full DC that these don't have, I very much doubt if you bought, say, 300k sq ft of these it would be 1%.
  - unclebucknasty 2y ago
    >I don't think there was much choice when it came to the size of the unit Yeah, I got the sense the design was intended for smaller on-demand needs versus strategically building out massive datacenters. So, my thinking is that you would borrow from the base idea, but you'd upsize each unit to cut down on the actual number of units (and attendant overhead). Just considering that if you truly are building out a datacenter, you'd probably do better to have fewer, larger units. On the flip side though, the shipping container format does make it very appealing logistically. >That smells an awful lot like an apples-to-oranges marketspeak Yeah, they are no doubt being selective in what they are comparing. That claim has to do with the cost of "connecting [these units] to existing infrastructure". So, I'm guessing they are assuming infrastructure, then comparing the cost of one of these units to construction of equivalent DC square footage. 1% still seems aggressive, but I could be convinced it's "significantly less". Of course, assuming the infrastructure starts to get murky, since a full DC build is going to be more intertwined with its infra.
    - kjs3 2y ago
      I got the sense the design was intended for smaller on-demand needs versus strategically building out massive datacenters This is absolutely the case. This was not intended to replace small, permanent DCs, much less massive ones. Think "I need compute on space limited offshore oil rig" or "I need some compute here for 6 months". you'd upsize each unit to cut down on the actual number of units I've been out of involvement in building out DCs for a long time, so my info is dated, but... As I said, there are now other companies doing variations on this. The Delta unit I mentioned upthread is built into a (standard?) 18-wheel tractor trailer (significantly larger than a shipping container). But much bigger than that you start getting into "how do I move it to the site" issues, which means costs due to transport are going to go up fast, and you're back to "does it just make sense to build a fixed DC", perhaps with a careful eye towards "I'll want to easily tack on more capacity later". Even if you figure out a way to do off-site pre-fab economically, I don't see how it would make sense to do "DC built out of X many identical prefab units" but instead you'd want to do "X many rack units, and Y many network units, and Z many air handling units, and W power distribution units, and V many CnC units, and...". I know there's a lot of talk about how building houses out of prefab components is going to change the world (one day), but unless you are building a lot of data centers, I don't know how you build out the infrastructure to deliver on that that's going to be cheaper than "haul materials to the site and build it there like we've done forever". My completely unsubstantiated view is the volume isn't there.
abdullahkhalids 2y ago
Very cool. Huwawai also made a similar thing [1], called the FusionModule1000A All-in-One Container Data Center. But interestingly (as far as I can tell) without any compute, just power, cooling, safety etc. Presumably costs about a million USD [2]. [1] https://web.archive.org/web/20170709195517/https://e.huawei.com/us/products/network-energy/dc-facilities/ids1000-a https://web.archive.org/web/20170709195517/https://e.huawei.... [2] https://www.finditprice.com/index/it_product/search.html?product_series=FusionModule1000-A https://www.finditprice.com/index/it_product/search.html?pro...
- kjs3 2y ago
  I don't think Sun's base model came with any compute either. It was racks/power/cooling and then you put servers and networking in. Sun would happily sell those to you...just more SKUs on the invoice.

donavanm 2y ago

As others mentioned “everyone” has done variations of havac-and-racks-in-teu for the past 25 years. Heres Amazons version circa 2010, https://www.datacenterdynamics.com/en/news/amazon-sheds-light-on-its-data-center-modules/ https://www.datacenterdynamics.com/en/news/amazon-sheds-ligh... You find them in old satellite photos of places like us-west-2. You still need site prep to support the weight, you still need power, external CRAC/H and/or water supply, and access & repairability sucks.