6 ms·

The density they're getting here is significantly higher than AWS Outposts, which is interesting. The top-end (~$600k) AWS Outposts seem to max out at around 1k

The density they're getting here is significantly higher than AWS Outposts, which is interesting. The top-end (~$600k) AWS Outposts seem to max out at around 1k CPUs and 4.5 TB RAM in a rack (e.g. 12x m5.24xlarge = 12x 384 GB), while this rack can house 2k CPUs and 30 TB (!) RAM.

psanford 5y ago
Outposts seem like a solution to the problem, "for regulatory or compliance reasons we are required for data to reside and be processed within a physical space we control." For that problem, an organization that is otherwise on AWS might find Outposts to be appealing. I can imagine an engineering team's response to such a requirement as "Oh yeah? Fine, its but its going to cost you $600k per year per rack!". I believe Oxide is attempting to capture a much broader market than that.
ptomato 5y ago
it's better than that, actually, as an AWS vCPU is a hyperthread, not a full core, and these would have 4096 hyperthreads.
kaliszad 5y ago
Yes, well it isn't that dense either. As I have written, it's 32 CPUs (16x 2 CPUs). 1 TB of RAM per CPU is not that huge a deal, it's perhaps 16x 64 GB (Milan uses 8 channels, 2 DIMMs per channel is reasonable), if you consider that is 16 GB of RAM per core. In HPC, you would probably shrink it to 1/4 of the volume (half width, 1 U dual socket server). Oxide probably focuses on optimal thermal efficiency since their limit isn't the space so much as the power density/ max. power per rack in existing DCs, which they are already pushing hard. (Of course they have lower power options too but they probably will not use 2048 cores.)
- spamizbad 5y ago
  The problem with pushing higher compute density is you're running into the limits of what most DCs can provide in terms of power and cooling for a single rack. Usually it's specialized HPC facilities or hyperscalers pushing the power and cooling to handle stuff like that. Those people aren't likely Oxide's customers - they've already got their own hardware solutions.
- Tuna-Fish 5y ago
  There are 32 half-width compute blades there, so probably single socket servers with 64 cores each.
  - kaliszad 5y ago
    It does look like it. Perhaps it is easier for maintenance, if you don't want to start with a full rack. The upgrade granularity is higher.

psanford 5y ago

Outposts seem like a solution to the problem, "for regulatory or compliance reasons we are required for data to reside and be processed within a physical space we control." For that problem, an organization that is otherwise on AWS might find Outposts to be appealing. I can imagine an engineering team's response to such a requirement as "Oh yeah? Fine, its but its going to cost you $600k per year per rack!". I believe Oxide is attempting to capture a much broader market than that.

ptomato 5y ago

it's better than that, actually, as an AWS vCPU is a hyperthread, not a full core, and these would have 4096 hyperthreads.

kaliszad 5y ago

Yes, well it isn't that dense either. As I have written, it's 32 CPUs (16x 2 CPUs). 1 TB of RAM per CPU is not that huge a deal, it's perhaps 16x 64 GB (Milan uses 8 channels, 2 DIMMs per channel is reasonable), if you consider that is 16 GB of RAM per core. In HPC, you would probably shrink it to 1/4 of the volume (half width, 1 U dual socket server). Oxide probably focuses on optimal thermal efficiency since their limit isn't the space so much as the power density/ max. power per rack in existing DCs, which they are already pushing hard. (Of course they have lower power options too but they probably will not use 2048 cores.)

spamizbad 5y ago
The problem with pushing higher compute density is you're running into the limits of what most DCs can provide in terms of power and cooling for a single rack. Usually it's specialized HPC facilities or hyperscalers pushing the power and cooling to handle stuff like that. Those people aren't likely Oxide's customers - they've already got their own hardware solutions.
Tuna-Fish 5y ago
There are 32 half-width compute blades there, so probably single socket servers with 64 cores each.
- kaliszad 5y ago
  It does look like it. Perhaps it is easier for maintenance, if you don't want to start with a full rack. The upgrade granularity is higher.