3 ms·
I'm not an economist, but you touched on an observation that I've made that I can't quite articulate using the appropriate technical terminology: When products
by jiggawatts 5y ago
I'm not an economist, but you touched on an observation that I've made that I can't quite articulate using the appropriate technical terminology:
When products are not fully vertically integrated, and there's a large markup dowstream, the upstream manufacturing is highly incentivesed to "penny pinch" in a way that would be perceived to be highly counter-productive in an integrated organisation.
The example that first made me notice this was power management in server systems. If you're running $100K worth DBMS software (licenses) on a $100K box, all of which has a further $10-100M of web apps and systems relying on it, then it would be madness to throttle that CPU to save $50 in electricity annually, right?
Well, guess what: most places do this! It happens because they outsource the low-margin upstream component (colocation/hosting/cloud), and when you're a data centre provider living on thin margins, the cost of power / cooling can be very substantial relative to your own costs. If you rent the rack space for that server out for $500 per annum, that $50 of electricity is 10% more in your pocket! That's a big deal!
Meanwhile, that $50 saved has slowed down $100M worth of stuff, sometimes by 50% or more. But... that's not the hosting provider's problem.
(PS: Azure does this, and they don't let you turn it off.)
- MaxBarraclough 5y ago> that $50 saved has slowed down $100M worth of stuff, sometimes by 50% or more. But... that's not the hosting provider's problem Provided their clients don't figure out what's going on and move to another provider. If the client has millions of dollars riding on that infrastructure, they'd likely be doing enough performance-monitoring to notice this kind of thing, no? > Azure does this, and they don't let you turn it off Azure really don't offer any never-throttled instances? What about their dedicated hosts?
- jiggawatts 5y ago> Provided their clients don't figure out what's going on I'm always pleasantly surprised when I see power management correctly configured at a client precisely because it is so rarely done correctly. Even if cloud vendors do fix this one thing, there's about a dozen more such issues where the suppliers' interests don't line up with the downstream consumers' interests. > Azure really don't offer any never-throttled instances? What about their dedicated hosts? I don't believe so. For example, their highest performing single node is the HBv3, with 2x AMD EPYC 7V13 processors for a total of 120 cores / 240 threads. On this page: https://docs.microsoft.com/en-us/azure/virtual-machines/workloads/hpc/hbv3-series-overview https://docs.microsoft.com/en-us/azure/virtual-machines/work... There is a box that says: Nodes per Socket (NPS) = 2 L3 as NUMA = Disabled NUMA domains within VM OS = 4 C-states = Enabled The technical name for CPU power management is called "C-states". In other words, they've left it on the default (enabled) on a machine that they charge $40K/year for!
- maayank 5y ago> I'm always pleasantly surprised when I see power management correctly configured at a client precisely because it is so rarely done correctly. Let's say I have Linux instances in the cloud. What would be the steps to take for proper power management?
- jiggawatts 5y agoAs far as I know, there are no steps you can take if you're running your instance in a virtual machine. Only the hypervisor host and/or the firmware can control the processor power and sleep states.
- cwizou 5y agoTL;DR: C-States improve performance. I'm not sure calling C-States throttling is completely accurate, at least not in the way that you imply. Modern CPUs have multiple mechanisms that can be called throttling in some way, but C-States are the least offensive and in modern CPUs actually increase performance. There's a good basic explanation here [1] but in short : - C-States is about idle cores, and powering them down. You can actually configure how deep you can power them down. Waking up does have a tiny cost in latency, yes, but it's mostly irrelevant (your core spends most of it's time waiting for data). - P-States are what I would call "ok" throttling, those happen when non-idle, modern cores can scale their frequency up and down based on how much workload they have. Again, in theory purely about power saving and close to 0 impact on performance. This was simple, and used to be just about power saving, disabling was ok. But things changed massively about 10+ years ago with the introductions of various Turbo mechanisms. Turbo is about looking at your CPU as a whole (and not per core), and maximising performance of the loaded cores by pushing their frequency, using the power budget that used to be allocated to the idle cores (that are now powered off). The way its done is by adding a few extra P-states that push the frequency a few hundred MHz. The more "C-State-down" cores you have, the higher the frequency of the working cores is pushed. Extreme example : you have a 28 cores CPUs running a single thread workload on one core only, with 27 idle cores. With C-States disabled, you lose Turbo and are stuck with the single core running at the "non-Turbo" frequency. With C-States enabled, because you idled the non used cores, your working core gets higher clock speeds and higher performances. (Things get more complex with AVX2 because those units have a large power consumption, and this is where you get into the very bad throttling) [1] : https://software.intel.com/content/www/us/en/develop/blogs/c-states-and-p-states-are-very-different.html https://software.intel.com/content/www/us/en/develop/blogs/c...
- hdjjhhvvhga 5y ago> Provided their clients don't figure out what's going on As far as I can tell, very few people ask similar questions. Those who do, come to interesting conclusions: https://jan.rychter.com/enblog/cloud-server-cpu-performance-comparison-2019-12-12 https://jan.rychter.com/enblog/cloud-server-cpu-performance-...
- jiggawatts 5y agoIt's so rare to see cloud benchmarks! I'm surprised this guy hasn't had a scary takedown letter from some lawyers yet... "There. I've said it. These [Xeon] processors are slow. Dog slow, in fact." My gimmick recently has been to upgrade Intel CPU VMs to AMD VMs. One benefit of the cloud is that this is literally just a menu selection. If it was on-prem, it would be a one-month argument to convince people that it's risk free, then a one-month argument with some beancounter, then a month of waiting while the pallets of kit ship in from Asia, then another two months of rack mounting and prepping for production. "They let them pack lots of slow cores onto a single CPU die, put that into a server, and then sell twice that number of cores as "vCPUs" to us." Azure also charges for licenses on a per vCPU basis as if* they were physical CPUs! This means that in effect they've doubled their Windows and SQL revenue stream.
- joe_the_user 5y agoHmm, I agree with your overall description - I believe economist describe problems of this sort as externalities[1]. The example of externalities that comes to mind is shipping companies running absurdly large ships to shave margins and so being concerned that their behemoth might lodge in the Suez Canal and disrupt much of world trade. IE, producers not being concerned about the larger impact of their decisions. But your example leaves me scratching my head. My impression is that cloud providers effectively selling CPU cycles by the dollar. There must be some pricing tradeoff for fast versus slow cycles. If a cloud provider can sell 20 throttled instances cheaper than 17 unthrottled instances, that's a deal that might suit buyer and seller. But maybe there's something I'd missing here. [1] https://en.wikipedia.org/wiki/Externality https://en.wikipedia.org/wiki/Externality