3 ms·

>Would the other providers compete on (non-public) price just to gain (new) market share? It's possible, but I maintain it's an extraordinary proposition, consi

by lsc 8y ago

>Would the other providers compete on (non-public) price just to gain (new) market share? It's possible, but I maintain it's an extraordinary proposition, considering Microsoft's overall software ecosystem integration advantage with Azure (much better lock-in) and Google's various attempts at features, including AI hardware. The better their lock-in, the more sense it makes for them to lose money at first, if that's what it takes to get you into bondage. (assuming they have a way of raising the prices later, or of dropping their costs later) I'm just saying that it would totally make sense to do this if it got you more business. Maybe they only do it for especially juicy customers? I mean, yeah, I'm reaching for some way that it might conceivably be cheaper to rent servers when you have already done the work of setting up your own datacenter, and it's hard. (now, I do understand that most in house datacenters have ridiculous policies that managers would be happy to pay 5x to get around. But if you are at f50 scale, you can setup your own "internal cloud" with amazon like policy affordances.) In the aughts, the standard thing to do when starting a VPS provider was to come in as cheap as you possibly could, get as many customers as you could, and then just leave your prices the same as the hardware (and thus power) costs fell out from under you. I mean, I don't know anyone who lost money on their marginal customer, but plenty started out with margins that wouldn't have been any good long-term. That said, I don't know of any cloud infrastructure providers who seem to be losing money on their marginal customer, and as you point out, there's very good evidence that AWS makes a killing. My impression is that it's an industry with pretty good margins, once you scale enough to amortize your labor costs, which is harder than it looks. I mean, in those days, hardware(and thus power) costs were dropping dramatically, labor costs were not, and even if you have a scalable system where one person can handle a lot of servers, you need a minimum size team, which gets expensive fast. >(Did you mean the parenthetical to apply to this or to the statement about bandwidth?) the recent change is that hardware has gotten... more expensive in the last little bit. not just ddr4 (but mostly ddr4... and that's most of your cost, a lot of the time, as that's the thing you least want to oversubscribe) To me? this is shocking. Honestly, I haven't really been keeping up with bandwidth prices since I sold my thing, so I wouldn't know as much about that side of things. >Is there anywhere that tracks this? I've only ever done that-point-in-time pricing analysis, never historical tracking. pcpartpicker has nice historical graphs for hardware pricing that are publicly accessible (but that don't go back so far) One of my super early projects was a web price scraper for books... I wish I had kept up with that project and expanded it to computer parts, because there's a lot of data that I don't know how to get anymore. https://pcpartpicker.com/trends/price/memory/ https://pcpartpicker.com/trends/price/memory/ >I tend to wonder how much of the difference is due to Amazon's specific choice of hardware for each generation. That is, would the trend look different if you limited the comparison to the cost of hardware with only the same specs as theirs (which are, of course, not necessarily, or even ever, the best bang-for-the-buck)? From what I've seen, the x86 server market is pretty efficient. Yes, yes, the professionals will take you if you let them, but it's pretty easy to get to within 10% of the tray price even on small purchases, and it's hard to get much below tray prices, even on large purchases - (dramxchange is a pretty good resource if you want to know about 'tray price' for ram) I mean, x86 hardware is... x86 hardware. Dell vs. HP vs. supermicro- it mostly even uses the same chipsets, and most of the cost is the cpu/ram, so there's really not a lot of difference in who you buy from, as long as you get quotes from more than one vendor. The times I've seen people get really ripped off, they got sold different types of hardware; like one place I worked was getting sold blade servers using FBDIMMs. Blade servers for "efficency" - of course, there was only enough power for one blade chassis per rack. And FBDIMMs used like 3x the power of low voltage reg. ecc ddr2. I pointed out the existence of the low power xeons that took reg. ecc ddr2 rather than FBDIMMs, and ran enough benchmarks to show that they were about as good as the regular xeons of similar speed... and used a lot less power. I'm not aware of any similar traps in the market right now, but I am a little out of the loop. My personal theory as to the lack of cost concern in "cloud" is that times are hot. When times are hot? you jump. make hay while the sun is shining. If spinning up cloud servers is faster than developing your datacenter infrastructure? who cares if it costs 5x as much. Setting up openstack is a huge pain in the ass, and you're going to have growing pains for the next six months, if your team is really good. My personal theory is that this trend will reverse with the next downturn. Meanwhile, I'm working for other people, which is pretty nice in a lot of ways, even though it does feel sad that all the time I spent in managing x86 hardware is something people just don't care about anymore.

mmt 8y ago
> nice historical graphs for hardware pricing that are publicly accessible Even then, that's only half the comparison, though. Do you know of anyone keeping historical AWS prices? That would seem more likely, actually, than a reliable pc-parts database. > I mean, x86 hardware is... x86 hardware. Dell vs. HP vs. supermicro- it mostly even uses the same chipsets, and most of the cost is the cpu/ram, so there's really not a lot of difference in who you buy from, as long as you get quotes from more than one vendor. Noooo.. no you too! :) I find that this is true, but only if the workload is primarily CPU- and RAM-bound. My sense is that somewhere between the dot-com boom and now (500x increase in speed?) CPUs got fast enough, and software got parallel enough, that the challenge in scaling shifted CPU capacity to I/O. (More RAM always seems to help, but I agree there's not much differentiation there). As such, at any given point in time, each vendor's offering at a particular (otherwise competitive) price point might not make sense. The mobo might not have enough PCIe slots with enough lanes (or too many on too few slots). The onboard 10GE might use a chipset that performs poorly under OpenBSD. The chassis might hold comically few disks (ahem, Dell) or have no SAS expander backplane option. The other differentiator, is at the high end, for >2S systems. It's arguably no longer "commodity" hardware, but the parts are more similar than different, and there's no lock-in if one can wait at least the full release cycle. Still, it's rare that anyone is brave enough to buck the trend and pay the 2x-4x price premium to scale "up" before embarking on the engineering nightmare that is scaling "out" (which I'm not convinced has any less of a hardware price premium, just that it can grow without bound). > Blade servers for "efficency" I've never understood that mass delusion, either, though at least not there do exist high-power-density datacenters that can house them. I assume someone managed to spread the myth that it's space that's expensive at a datacenter, rather than power and cooling. > I'm not aware of any similar traps in the market right now I'm not, either, at this very moment, but every time I've looked, out of curiosity, there's something like that. All the vendors will have at least one product that seems objectively worse in every way to the older products, including being more proprietary, that I wonder if it isn't some custom design that flopped that they're trying to dump. >My personal theory as to the lack of cost concern in "cloud" is that times are hot. Agreed. Easy money, be it from VCs, low interest rates, and/or a booming economy (perhaps fueled by the first two) helps explain it. > Setting up openstack is a huge pain in the ass, Is that really the go-to option? I'd think there are much lighter-weight alternatives. I'd have thought provisioning the base infrastructure were much less a PITA than writing the "infrastructure as code" which one has to do even for public cloud. > growing pains for the next six months, Is it really that long? Or is that just the perception? > if your team is really good. Maybe that's the problem. Good help is hard to find? Still, what about 1-2 years down the line of making hay at those 5x costs? There are plenty of companies like that and, at some point, it has to at least be slightly tempting to go through the effort, including hiring. Maybe the money really is too easy. > My personal theory is that this trend will reverse with the next downturn. I've been thinking the same thing, but I'm wondering what will happen with all that excess capacity that cloud providers are suddenly left holding. > Meanwhile, I'm working for other people, which is pretty nice in a lot of ways, even though it does feel sad that all the time I spent in managing x86 hardware is something people just don't care about anymore. I've always worked for other people (which also makes me worry about too strong a downturn), and the hardware management has only been a part of what I do. I'm not so much sad at the apathy toward one of my skills. It's more that I'm so limited in how much performance I can squeeze out of the infrastructure or how low I can keep the cost. It makes for weird tension.

by lsc 8y ago

mmt 8y ago

> nice historical graphs for hardware pricing that are publicly accessible Even then, that's only half the comparison, though. Do you know of anyone keeping historical AWS prices? That would seem more likely, actually, than a reliable pc-parts database. > I mean, x86 hardware is... x86 hardware. Dell vs. HP vs. supermicro- it mostly even uses the same chipsets, and most of the cost is the cpu/ram, so there's really not a lot of difference in who you buy from, as long as you get quotes from more than one vendor. Noooo.. no you too! :) I find that this is true, but only if the workload is primarily CPU- and RAM-bound. My sense is that somewhere between the dot-com boom and now (500x increase in speed?) CPUs got fast enough, and software got parallel enough, that the challenge in scaling shifted CPU capacity to I/O. (More RAM always seems to help, but I agree there's not much differentiation there). As such, at any given point in time, each vendor's offering at a particular (otherwise competitive) price point might not make sense. The mobo might not have enough PCIe slots with enough lanes (or too many on too few slots). The onboard 10GE might use a chipset that performs poorly under OpenBSD. The chassis might hold comically few disks (ahem, Dell) or have no SAS expander backplane option. The other differentiator, is at the high end, for >2S systems. It's arguably no longer "commodity" hardware, but the parts are more similar than different, and there's no lock-in if one can wait at least the full release cycle. Still, it's rare that anyone is brave enough to buck the trend and pay the 2x-4x price premium to scale "up" before embarking on the engineering nightmare that is scaling "out" (which I'm not convinced has any less of a hardware price premium, just that it can grow without bound). > Blade servers for "efficency" I've never understood that mass delusion, either, though at least not there do exist high-power-density datacenters that can house them. I assume someone managed to spread the myth that it's space that's expensive at a datacenter, rather than power and cooling. > I'm not aware of any similar traps in the market right now I'm not, either, at this very moment, but every time I've looked, out of curiosity, there's something like that. All the vendors will have at least one product that seems objectively worse in every way to the older products, including being more proprietary, that I wonder if it isn't some custom design that flopped that they're trying to dump. >My personal theory as to the lack of cost concern in "cloud" is that times are hot. Agreed. Easy money, be it from VCs, low interest rates, and/or a booming economy (perhaps fueled by the first two) helps explain it. > Setting up openstack is a huge pain in the ass, Is that really the go-to option? I'd think there are much lighter-weight alternatives. I'd have thought provisioning the base infrastructure were much less a PITA than writing the "infrastructure as code" which one has to do even for public cloud. > growing pains for the next six months, Is it really that long? Or is that just the perception? > if your team is really good. Maybe that's the problem. Good help is hard to find? Still, what about 1-2 years down the line of making hay at those 5x costs? There are plenty of companies like that and, at some point, it has to at least be slightly tempting to go through the effort, including hiring. Maybe the money really is too easy. > My personal theory is that this trend will reverse with the next downturn. I've been thinking the same thing, but I'm wondering what will happen with all that excess capacity that cloud providers are suddenly left holding. > Meanwhile, I'm working for other people, which is pretty nice in a lot of ways, even though it does feel sad that all the time I spent in managing x86 hardware is something people just don't care about anymore. I've always worked for other people (which also makes me worry about too strong a downturn), and the hardware management has only been a part of what I do. I'm not so much sad at the apathy toward one of my skills. It's more that I'm so limited in how much performance I can squeeze out of the infrastructure or how low I can keep the cost. It makes for weird tension.