4 ms·

I have to second this. While it takes mich effort and in-depth knowledge do build up from an “empty” cage it’s still far from dealing with everything from build

I have to second this. While it takes mich effort and in-depth knowledge do build up from an “empty” cage it’s still far from dealing with everything from building permits, to plan and realize a data center to code including redundant power lines, AC and fibre. Still kudos going this path in the cloud-centric time we live in.

j45 2y ago
Having been around and through both, setting up a cage or two is very different than the entire facility.
- HaZeust 2y ago
  I think you and GP are in agreement.
matt-p 2y ago
Yes, the second is much more work, orders of magnitude at least.
- motorest 2y ago
  > Yes, the second is much more work, orders of magnitude at least. I feel it's important to stress that the difficulty level of collocating something, let alone actually building a data center, is exactly what makes cloud computing so enticing and popular. Everyone focuses on trivia items like OpEx vs CapEx and dynamic scaling, but the massive task of actually plugging in the hardware in a secure setting and get it to work reliably is a massive undertaking.
  - matt-p 2y ago
    I just honestly don't agree with that at all. That's the easy bit, the bit I don't enjoy is organising backups and storage in general. But it's not 'hard'.
llm_trw 2y ago
Do I have stories. One of the better was the dead possum in the drain during a thunderstorm. >So do we throw the main switch before we get electroduced? Or do we try to poke enough holes in it that it gets flushed out? And what about the half million in servers that are going to get ruined? Sign up to my patreon to find out how the story ended.
- pinoy420 2y ago
  Give me a link to your patreon
  - Imustaskforhelp 2y ago
    pay to the man's patreon and then tell me the story please!
manquer 2y ago
While it is more complex to actually build out the center , a lot of that is specific to the regional you are doing it. Thy will vary by country, by state or even county , setting up a DC in the Bay Area and say one in Ohio or Utah is a very different endeavor with different design considerations.
- itsoktocry 2y ago
  >Thy will vary by country, by state or even county , setting up a DC in the Bay Area and say one in Ohio or Utah is a very different endeavor with different design considerations. What point are you trying to make? It does not matter where you are in the world, or what local laws exist or permits are required, racking up servers in a cage is much less difficult than physically building a data center (of which racking up servers is a part).
  - manquer 2y ago
    I meant that the learning from doing actual build outs aren't going to translate in other geographies and regulatory climates, not that the work is less difficult or not interesting and important. Also people doing the build outs of a DC aren't likely keen on talking about permits and confidential agreements in the industry quite publicly. Yes the title is click baity, but that is par of the course these days.
    - xiconfjs 2y ago
      Sure, every business has confidential agreements which are usually kept secret but there are even on youtube a few people/companies who gave deep insides in the bits and bytes of building a data center from ground up across multiple hours of documentation. And the confidential business agreements in the data center world are up to a certain level the same as any other businesses.
- quickthrowman 2y ago
  > Thy will vary by country, by state or even county , setting up a DC in the Bay Area and say one in Ohio or Utah is a very different endeavor with different design considerations. Regarding data centers that cost 9 figures and up: For the largest players, there’s not a ton of variation. A combination of evaporative cooling towers and chillers are used to reject heat. This is a consequence of evaporative open loop cooling being 2-3x more efficient than a closed-loop system. There will be multiple medium-voltage electrical services, usually from different utilities or substations, with backup generators and UPSes and paralleling switchgear to handle failover between normal, emergency, and critical power sources. There’s not a lot of variation since the two main needs of a data center are reliable electricity and the ability to remove heat from the space, and those are well-solved problems in mature engineering disciplines (ME and EE). The huge players are plopping these all across the country and repeatability/reliability is more important than tailoring the build to the local climate. FWIW my employer has done billions of dollars of data center construction work for some of the largest tech companies (members of Mag7) and I’ve reviewed construction plans for multiple data centers.

j45 2y ago

Having been around and through both, setting up a cage or two is very different than the entire facility.

HaZeust 2y ago
I think you and GP are in agreement.

matt-p 2y ago

Yes, the second is much more work, orders of magnitude at least.

motorest 2y ago
> Yes, the second is much more work, orders of magnitude at least. I feel it's important to stress that the difficulty level of collocating something, let alone actually building a data center, is exactly what makes cloud computing so enticing and popular. Everyone focuses on trivia items like OpEx vs CapEx and dynamic scaling, but the massive task of actually plugging in the hardware in a secure setting and get it to work reliably is a massive undertaking.
- matt-p 2y ago
  I just honestly don't agree with that at all. That's the easy bit, the bit I don't enjoy is organising backups and storage in general. But it's not 'hard'.

llm_trw 2y ago

Do I have stories. One of the better was the dead possum in the drain during a thunderstorm. >So do we throw the main switch before we get electroduced? Or do we try to poke enough holes in it that it gets flushed out? And what about the half million in servers that are going to get ruined? Sign up to my patreon to find out how the story ended.

pinoy420 2y ago
Give me a link to your patreon
- Imustaskforhelp 2y ago
  pay to the man's patreon and then tell me the story please!

manquer 2y ago

While it is more complex to actually build out the center , a lot of that is specific to the regional you are doing it. Thy will vary by country, by state or even county , setting up a DC in the Bay Area and say one in Ohio or Utah is a very different endeavor with different design considerations.

itsoktocry 2y ago
>Thy will vary by country, by state or even county , setting up a DC in the Bay Area and say one in Ohio or Utah is a very different endeavor with different design considerations. What point are you trying to make? It does not matter where you are in the world, or what local laws exist or permits are required, racking up servers in a cage is much less difficult than physically building a data center (of which racking up servers is a part).
- manquer 2y ago
  I meant that the learning from doing actual build outs aren't going to translate in other geographies and regulatory climates, not that the work is less difficult or not interesting and important. Also people doing the build outs of a DC aren't likely keen on talking about permits and confidential agreements in the industry quite publicly. Yes the title is click baity, but that is par of the course these days.
  - xiconfjs 2y ago
    Sure, every business has confidential agreements which are usually kept secret but there are even on youtube a few people/companies who gave deep insides in the bits and bytes of building a data center from ground up across multiple hours of documentation. And the confidential business agreements in the data center world are up to a certain level the same as any other businesses.
quickthrowman 2y ago
> Thy will vary by country, by state or even county , setting up a DC in the Bay Area and say one in Ohio or Utah is a very different endeavor with different design considerations. Regarding data centers that cost 9 figures and up: For the largest players, there’s not a ton of variation. A combination of evaporative cooling towers and chillers are used to reject heat. This is a consequence of evaporative open loop cooling being 2-3x more efficient than a closed-loop system. There will be multiple medium-voltage electrical services, usually from different utilities or substations, with backup generators and UPSes and paralleling switchgear to handle failover between normal, emergency, and critical power sources. There’s not a lot of variation since the two main needs of a data center are reliable electricity and the ability to remove heat from the space, and those are well-solved problems in mature engineering disciplines (ME and EE). The huge players are plopping these all across the country and repeatability/reliability is more important than tailoring the build to the local climate. FWIW my employer has done billions of dollars of data center construction work for some of the largest tech companies (members of Mag7) and I’ve reviewed construction plans for multiple data centers.