4 ms·

What I find odd is that I know of a lot of companies looking a moving back to their own hardware. They are going to need a hell of a lot fewer boxes today than

What I find odd is that I know of a lot of companies looking a moving back to their own hardware. They are going to need a hell of a lot fewer boxes today than in 2009 (15 years ago) This is 2009: https://www.intc.com/news-events/press-releases/detail/1307/intel-introduces-core-i7-xeon-3400-and-first-core https://www.intc.com/news-events/press-releases/detail/1307/... I can get a 128 core system today, if I need the density. And that is the thing, density is through the roof, and pricing is roughly the same as 2009. The same thing has happened to storage performance and (to a degree) scale. What would have taken a few racks 15 years ago barely fills one today. 30% percent of AWS's revenue is profit. Thats what you're leaving on the table. For a lot of companies (at least the ones that can plan ahead and do basic accounting) that does not make sense. Even more so when lending is tight and every dollar counts.

jedberg 3y ago
> 30% percent of AWS's revenue is profit. If you built exactly what AWS has you wouldn't recapture that because most of that comes from operating efficiencies at scale. Yes, you can get some of that back, but it won't be 30%.
- chasd00 3y ago
  I think the idea is a company would only build what they use instead of replicating AWS feature for feature. If they just build what the need then I think 30% is on the low side.
  - jedberg 3y ago
    Power and cooling are the two main things that they amortize over many customers, and everyone needs those.
    - vidarh 3y ago
      They're however low margin commodity services at a colo provider. Building your own data centers is the last of a long list of steps up in difficulty and one most people wont need. Renting by the rack in a colo facility with a support contract is however approachable from the point your cost is above ~$10k / months. Depending on location, once you're above ~10 racks or so per physical location, a dedicated cage can start to make sense, and a bit above that a data centre where you can rent office space for a couple of your own dedicated staff. Even many colo providers haven't owned all of their own setups for years. I think 2001 or so was the first time we had facilities in a colo building where half the building has been leased, ready made w/raised floors, HVAC, generators etc. to another colo provider which "just" had to staff it and handle sales and accounts. You can go to a commercial real estate broker and get a fractional data centre building, or you can go to a data centre operators and get a fractional functional, staffed data centre all the way from large percentages of the space with locked steel cages or walls separatibg your part from the rest down to individual 1U server slots.
      - vbezhenar 3y ago
        How do you deal with hardware management? It seems that at this scale, hardware does not break often enough to warrant full-time system administrator position. However when hardware breaks, you need to fix it ASAP, you can't spend time looking for someone on the market for one-off job.
        vidarh 3y ago
        Firstly, if you need to fix hardware ASAP, you're dangerously underprivisoned. You avoid that either by having excess capacity - it's still cheaper than cloud unless you ridiculously overprovision - or you pick a provider that also offers managed servers or cloud servers, and fall back on using that for elasticity (this, incidentally, makes public cloud even less attractive cost-wise - being able to tie in VMs or managed servers into your infra to handle spikes means you can run your own servers far closer to capacity, and so the cost per unit of compute drops dramatically). But when you do need to, any colo provider offers "remote hands" and 24/7 staffing, so if you want you can usually have your server shipped straight to the data centre and just raise a ticket to have them rack and connect it. I have also been second level support for companies for this - getting people, or companies, on retainer separate from the data centre operator, to handle all your ops needs including going out to the data centre in the middle of the night to fix issues their on-site staff can't, is a service you trivially buy. But any reasonable setup these days will have IPMI and/or network boot, so you can have it slotted in an automatically boot into an installer image providing remote access to complete the setup, as well as to power cycle servers and check console output, so hands-on access is rarely needed beyond the initial racking, pulling and replacing hot-swappable drives, or eventual removal. In practice, last time I had racks where we owned the servers as opposed to managed servers where none of this was our problem (still vastly cheaper than public cloud, though somewhat less flexible if you want it to remain cheap), it took on average a fraction of a day per year per rack time spent dealing with anything lower level than our orchestration layer. I tracked the time, and it remained fairly stable over many years, including in cases where our rack costs were so low that it was justified to keep servers in the racks years past the point where they'd been written off.
- SoftTalker 3y ago
  Yes you have to think of the overhead. It's not just buying a rack and some hardware. You need power, cooling, standby generators, redundant internet connectivity, firewalls, people who know how to run all of those things, and people to manage those people. Amazon and Google and Microsoft spread the cost of all of that out over many customers, you're paying for it yourself if you're doing it yourself. As long as there is no collusion, having at least a few big cloud providers should eventually drive prices down to close to the marginal cost of providing the service. Your job is to avoid lock-in to any specific provider's platform.
  - kaliszad 3y ago
    Yes, but your offering doesn't have to be so general as for instance on your hardware there does not have to be any sharing of resources with non-trusted parties. You also don't need a lot of the accounting and billing stuff internally. And in general, if you are willing to drop a nine or two on availability/ reliability, the costs fall much further down. Obviously, you have to do your own estimates and calculations but beating AWS on price in certain areas isn't really hard even if you just use a different hoster/ cloud. Source: Ran a SaaS business with a fraction of the cost that I normally hear from people.
    - vidarh 3y ago
      We used to price out an AWS transition every year at an old job. As well as managed hosting providers. 6-7 years in Hetzner reached parity with our colo setup and we started using them (our colos were near London, with London real estate prices). AWS however never got within a factor of 2. Which is fine for many businesses where a 2x on hosting does little to your margins, or you e.g. serve customers whose infra is in AWS, but for my employer at the time, moving to AWS would have meant bankruptcy.
  - vidarh 3y ago
    Most people rent rack space, or cages, rather than build data centers. I worked at Yahoo back when they were still 10k employees, and "their" data centers in London were still someone else's - they rented cages (if you haven't worked in coloed data centers before, you can rent by the 19" inch rack, or sometimes fractions, or you can have them wall of a section for just your racks). This is a commodity service - there are several data centers within 20 minutes of me, and dozens within an hour. All of the services you list, including the staff and access to them on a per incident or per hour basis is a commodity to the point that owners of these providers are increasingly real estate companies because they are "boring", relatively low margin plays priced by the square metre and kilowatts of power and HVAC capacity, not tech companies. So is services for these data centre operators. There's a company within walking distance of me whose sole product is software for optimising HVAC costs for data centers. My current employers core offering is digital twin solutions - being able to model and collect data from building sensors. Turns out a data centre needs mostly the same set of sensors as any other modern building. Just different densities and layouts. Yes, you want raised floors for cable ducts. Most offices have dropped ceilings for the same reasons but less volume. It's a basic real estate play, and building your own is not where you save over a public cloud unless you're a unicorn, because you can recapture most of the margin by going to a provider that doesn't hide that from you. It's a basic real estate play - the margins when you rent Colo space are nowhere near AWS level margins.
- withinboredom 3y ago
  You mostly don't need all the things AWS built. You need, maybe, 10-20% of it.
  - jedberg 3y ago
    Right but you need the power and cooling, which is the main thing they amortize over many customers.
    - withinboredom 3y ago
      I pay a colo for power (which also covers cooling) and the costs are still waaaay less expensive than EC2; orders of magnitude less. Like <300USD a month (amortized) for hundreds of cores, hundreds of gb of ram, hundreds of tb of disk space. The same costs for a EC2 instances (reserved 3yr, so it compares nicely) is thousands of USD per month.
      - latchkey 3y ago
        Anecdotal response. You don't scale to every single business trying to run this hardware. If that happened, I guarantee that your bills would go up significantly. I've actually been hearing recently of people getting booted from their data center cause the DC is trying to reconfigure their customer base in order to increase rack density.
        withinboredom 3y ago
        That's simply because there isn't as much demand, so supply is down. If demand goes up, supply goes up eventually, but price goes up until an equilibrium is reached. Eco 101.
        latchkey 3y ago
        A year ago, I had a call with Digital Reality on a Friday. I was looking for 5-10mw that they had available. They called me back on Monday to let me know that it was gone. Mind you, this was also $0.35/kwh, which is like residential rates! CoreWeave is in the process of opening 25 more data centers, concurrently. Eco 101 on steroids.
- deleted 3y ago
  [deleted]
ben-schaaf 3y ago
> I can get a 128 core system today, if I need the density. 256 core now with bergamo.
- latchkey 3y ago
  This is what we have in our MI300x box. 2x9754. Beast of a CPU. Intel has nothing even close.
  - zer00eyz 3y ago
    OHHHH That's like 500 cores in one box? cores/threads total? How much ram? disk? do you know the all in cost and rack U? There are lots of folks whose entire production infrastructure is half that core count or less.
    - latchkey 3y ago
      Whole box is a beast. 3TB ram, 122TB nvme... plus the 8x MI300x (192gb/each). 2x 400G cx7 NICs. Here are the CPU specs: https://www.amd.com/en/products/cpu/amd-epyc-9754 https://www.amd.com/en/products/cpu/amd-epyc-9754 We went with higher cores vs. higher clocks/cache due to wanting multi-tenancy on this box. MI300x is so new that we wanted to be able to give people 1-2 gpus at a time (via VM/pcie pass through), so we figured more cores are better for now. In the future, we will buy a mix of different CPUs (including Intel as well). Yes, of course I know the costs. It ain't cheap. ;-)
      - zer00eyz 3y ago
        >> Yes, of course I know the costs. It ain't cheap. ;-) I hate that we have to be this way about prices (its stupid) Core density is through the roof... Your one box today is the equal of whats likely a 15x multiplier. 15 years ago that much compute would have spanned 2 racks. Or jammed one full of blade servers (and they were expensive). It's in what? 4? 8? u of space?
        3y ago

jedberg 3y ago

> 30% percent of AWS's revenue is profit. If you built exactly what AWS has you wouldn't recapture that because most of that comes from operating efficiencies at scale. Yes, you can get some of that back, but it won't be 30%.

chasd00 3y ago
I think the idea is a company would only build what they use instead of replicating AWS feature for feature. If they just build what the need then I think 30% is on the low side.
- jedberg 3y ago
  Power and cooling are the two main things that they amortize over many customers, and everyone needs those.
  - vidarh 3y ago
    They're however low margin commodity services at a colo provider. Building your own data centers is the last of a long list of steps up in difficulty and one most people wont need. Renting by the rack in a colo facility with a support contract is however approachable from the point your cost is above ~$10k / months. Depending on location, once you're above ~10 racks or so per physical location, a dedicated cage can start to make sense, and a bit above that a data centre where you can rent office space for a couple of your own dedicated staff. Even many colo providers haven't owned all of their own setups for years. I think 2001 or so was the first time we had facilities in a colo building where half the building has been leased, ready made w/raised floors, HVAC, generators etc. to another colo provider which "just" had to staff it and handle sales and accounts. You can go to a commercial real estate broker and get a fractional data centre building, or you can go to a data centre operators and get a fractional functional, staffed data centre all the way from large percentages of the space with locked steel cages or walls separatibg your part from the rest down to individual 1U server slots.
    - vbezhenar 3y ago
      How do you deal with hardware management? It seems that at this scale, hardware does not break often enough to warrant full-time system administrator position. However when hardware breaks, you need to fix it ASAP, you can't spend time looking for someone on the market for one-off job.
      - vidarh 3y ago
        Firstly, if you need to fix hardware ASAP, you're dangerously underprivisoned. You avoid that either by having excess capacity - it's still cheaper than cloud unless you ridiculously overprovision - or you pick a provider that also offers managed servers or cloud servers, and fall back on using that for elasticity (this, incidentally, makes public cloud even less attractive cost-wise - being able to tie in VMs or managed servers into your infra to handle spikes means you can run your own servers far closer to capacity, and so the cost per unit of compute drops dramatically). But when you do need to, any colo provider offers "remote hands" and 24/7 staffing, so if you want you can usually have your server shipped straight to the data centre and just raise a ticket to have them rack and connect it. I have also been second level support for companies for this - getting people, or companies, on retainer separate from the data centre operator, to handle all your ops needs including going out to the data centre in the middle of the night to fix issues their on-site staff can't, is a service you trivially buy. But any reasonable setup these days will have IPMI and/or network boot, so you can have it slotted in an automatically boot into an installer image providing remote access to complete the setup, as well as to power cycle servers and check console output, so hands-on access is rarely needed beyond the initial racking, pulling and replacing hot-swappable drives, or eventual removal. In practice, last time I had racks where we owned the servers as opposed to managed servers where none of this was our problem (still vastly cheaper than public cloud, though somewhat less flexible if you want it to remain cheap), it took on average a fraction of a day per year per rack time spent dealing with anything lower level than our orchestration layer. I tracked the time, and it remained fairly stable over many years, including in cases where our rack costs were so low that it was justified to keep servers in the racks years past the point where they'd been written off.
SoftTalker 3y ago
Yes you have to think of the overhead. It's not just buying a rack and some hardware. You need power, cooling, standby generators, redundant internet connectivity, firewalls, people who know how to run all of those things, and people to manage those people. Amazon and Google and Microsoft spread the cost of all of that out over many customers, you're paying for it yourself if you're doing it yourself. As long as there is no collusion, having at least a few big cloud providers should eventually drive prices down to close to the marginal cost of providing the service. Your job is to avoid lock-in to any specific provider's platform.
- kaliszad 3y ago
  Yes, but your offering doesn't have to be so general as for instance on your hardware there does not have to be any sharing of resources with non-trusted parties. You also don't need a lot of the accounting and billing stuff internally. And in general, if you are willing to drop a nine or two on availability/ reliability, the costs fall much further down. Obviously, you have to do your own estimates and calculations but beating AWS on price in certain areas isn't really hard even if you just use a different hoster/ cloud. Source: Ran a SaaS business with a fraction of the cost that I normally hear from people.
  - vidarh 3y ago
    We used to price out an AWS transition every year at an old job. As well as managed hosting providers. 6-7 years in Hetzner reached parity with our colo setup and we started using them (our colos were near London, with London real estate prices). AWS however never got within a factor of 2. Which is fine for many businesses where a 2x on hosting does little to your margins, or you e.g. serve customers whose infra is in AWS, but for my employer at the time, moving to AWS would have meant bankruptcy.
- vidarh 3y ago
  Most people rent rack space, or cages, rather than build data centers. I worked at Yahoo back when they were still 10k employees, and "their" data centers in London were still someone else's - they rented cages (if you haven't worked in coloed data centers before, you can rent by the 19" inch rack, or sometimes fractions, or you can have them wall of a section for just your racks). This is a commodity service - there are several data centers within 20 minutes of me, and dozens within an hour. All of the services you list, including the staff and access to them on a per incident or per hour basis is a commodity to the point that owners of these providers are increasingly real estate companies because they are "boring", relatively low margin plays priced by the square metre and kilowatts of power and HVAC capacity, not tech companies. So is services for these data centre operators. There's a company within walking distance of me whose sole product is software for optimising HVAC costs for data centers. My current employers core offering is digital twin solutions - being able to model and collect data from building sensors. Turns out a data centre needs mostly the same set of sensors as any other modern building. Just different densities and layouts. Yes, you want raised floors for cable ducts. Most offices have dropped ceilings for the same reasons but less volume. It's a basic real estate play, and building your own is not where you save over a public cloud unless you're a unicorn, because you can recapture most of the margin by going to a provider that doesn't hide that from you. It's a basic real estate play - the margins when you rent Colo space are nowhere near AWS level margins.
withinboredom 3y ago
You mostly don't need all the things AWS built. You need, maybe, 10-20% of it.
- jedberg 3y ago
  Right but you need the power and cooling, which is the main thing they amortize over many customers.
  - withinboredom 3y ago
    I pay a colo for power (which also covers cooling) and the costs are still waaaay less expensive than EC2; orders of magnitude less. Like <300USD a month (amortized) for hundreds of cores, hundreds of gb of ram, hundreds of tb of disk space. The same costs for a EC2 instances (reserved 3yr, so it compares nicely) is thousands of USD per month.
    - latchkey 3y ago
      Anecdotal response. You don't scale to every single business trying to run this hardware. If that happened, I guarantee that your bills would go up significantly. I've actually been hearing recently of people getting booted from their data center cause the DC is trying to reconfigure their customer base in order to increase rack density.
      - withinboredom 3y ago
        That's simply because there isn't as much demand, so supply is down. If demand goes up, supply goes up eventually, but price goes up until an equilibrium is reached. Eco 101.
        latchkey 3y ago
        A year ago, I had a call with Digital Reality on a Friday. I was looking for 5-10mw that they had available. They called me back on Monday to let me know that it was gone. Mind you, this was also $0.35/kwh, which is like residential rates! CoreWeave is in the process of opening 25 more data centers, concurrently. Eco 101 on steroids.
deleted 3y ago
[deleted]

ben-schaaf 3y ago

> I can get a 128 core system today, if I need the density. 256 core now with bergamo.

latchkey 3y ago
This is what we have in our MI300x box. 2x9754. Beast of a CPU. Intel has nothing even close.
- zer00eyz 3y ago
  OHHHH That's like 500 cores in one box? cores/threads total? How much ram? disk? do you know the all in cost and rack U? There are lots of folks whose entire production infrastructure is half that core count or less.
  - latchkey 3y ago
    Whole box is a beast. 3TB ram, 122TB nvme... plus the 8x MI300x (192gb/each). 2x 400G cx7 NICs. Here are the CPU specs: https://www.amd.com/en/products/cpu/amd-epyc-9754 https://www.amd.com/en/products/cpu/amd-epyc-9754 We went with higher cores vs. higher clocks/cache due to wanting multi-tenancy on this box. MI300x is so new that we wanted to be able to give people 1-2 gpus at a time (via VM/pcie pass through), so we figured more cores are better for now. In the future, we will buy a mix of different CPUs (including Intel as well). Yes, of course I know the costs. It ain't cheap. ;-)
    - zer00eyz 3y ago
      >> Yes, of course I know the costs. It ain't cheap. ;-) I hate that we have to be this way about prices (its stupid) Core density is through the roof... Your one box today is the equal of whats likely a 15x multiplier. 15 years ago that much compute would have spanned 2 racks. Or jammed one full of blade servers (and they were expensive). It's in what? 4? 8? u of space?
      - 3y ago