5 ms·

I have a pet theory that the recent focus on CPUs with a high count of low-perf cores (this article is about intel but AMD has an equivalent offering) is being

by paol 2y ago

I have a pet theory that the recent focus on CPUs with a high count of low-perf cores (this article is about intel but AMD has an equivalent offering) is being driven by hyperscalers like Amazon and the perverse incentive caused by their pricing structures. Cloud providers have structured their compute offerings around dollars per core. This creates the incentive to deploy the densest possible core count processors, and zero incentive to make those cores faster. It can't have escaped people's notice how slow cloud VCPUs are. My 6 year old crappy laptop CPU has the same single core perf as a cloud instance that costs hundreds of dollars per month. And the frustration is not just cost, when you actually need good performance for something that doesn't scale so well horizontally, it's hard to get it.

re-thc 2y ago
> being driven by hyperscalers like Amazon and the perverse incentive caused by their pricing structures Not really. They can and do price it otherwise e.g. with different instance types. If we had to blame something then it's the move away from the traditional software licensing model. In the past, self-install software e.g. databases, OSes, etc would charge per core / CPU and that would be a huge chunk of the expenses added up. Now that we're free from this things are a lot different. Having said that, the desktop side is still aiming for high-perf cores for e.g. gaming. It's not lost - just harder and harder to do. > It can't have escaped people's notice how slow cloud VCPUs are It's not that slow. It comes down to choosing the right instances and staying current. EPYC 4th gen single core on e.g. Amazon i.e. c7a is not the same as a 6th year old laptop CPU? It's also not hundreds of dollars per month (reserved / spot instances are the way to go). Also this is nothing new. I remember the days of companies running their own VMWare data centers and those server Xeons were a lot slower. There are just heat and density concerns even if you had to do it yourself.
- paol 2y ago
  Funny enough c7a is just what we chose recently for our CI runners precisely because of the better CPU performance. It's only better relatively to the terribleness of the t* types though. And it definitely costs hundreds of dollars per month. Just ran the calculator and 8 VPCUs (to keep the rough laptop analogy) comes out at $316.
  - re-thc 2y ago
    > It's only better relatively to the terribleness of the t* types though. It's never a value add to use the t* types. That's poor misinformation or at least outdated information that people keep spreading. The advantage of c7a or Graviton 3 though is you get real cores. Most of the t* get you vCPUs, and 1/2 of it are just threads. It can be up to 2x faster in that sense. > Just ran the calculator and 8 VPCUs (to keep the rough laptop analogy) comes out at $316. Does your laptop have 8 real cores not threads that can 24/7 keep its peak speeds? Most overheat and throttle... are we comparing the same thing? ??? USD? c7a.2xlarge (8 vCPU) spot instance pricing is $0.1032 = ~$80/month (us-east-1) Even on just a 3 year compute savings plan and we're on $144.71/month. You're not required to commit to a particular instance type - just the spending, so all good? This is excluding storage. Either way there's no reason to use EC2 On-Demand without a savings plan or spot. At least commit some of it a year and it's a big saving. Otherwise don't use AWS. Side note: EPYC 5th gen has been announced and we'll likely see way more improvements on top. Pricing will definitely go down. Also: this is just a rant on AWS. Hetzner, OVH or even Oracle cloud is way cheaper, like maybe 1/2 to 1/10 of this $316... it's not a cloud issue - just AWS.
    - paol 2y ago
      You're slightly off in your understanding of VCPUs. All AWS instances "get" VCPUs. A VCPU is a single thread in architectures with SMT and a single core in architectures without. So for example c7a instances which are AMD EPYCs have 1 VCPU = 1/2 core. They have is a page that lists this for every single instance type which I can't be bothered to search for right now. As for spot instances, they're a great if you can use them, but you can't hold their price as being representative. Now using savings plan prices as representative is defensible, but it won't bring the cost below the "hundreds per month" order of magnitude.
      - phonon 2y ago
        Hyperthreading is turned off on m7a/c7a/r7a. "One of the major differences between M7a instances and the previous generations of instances, such as M6a instances, is their vCPU to physical processor core mapping. Every vCPU on a M7a instance is a physical CPU core. This means there is no Simultaneous Multi-Threading (SMT). By contrast, every vCPU on prior generations such as M6a instances is a thread of a CPU core." https://aws.amazon.com/ec2/instance-types/m7a/ https://aws.amazon.com/ec2/instance-types/m7a/
authorfly 2y ago
DHH made this point a while ago when talking about this potentially springboarding a walk back to self-hosting/on-device software for companies. I think the main point was that running CI on a MacBook PRO happens about 3-5x faster than CI on the cloud. The benefit of things running faster on the cloud is tearing away.
- re-thc 2y ago
  > The benefit of things running faster on the cloud is tearing away Was that the benefit? I thought it's reliability, scalability, etc. I remember having a FAST shared server for CI. "Someone" was in charge of it. Every time it went down everyone lost 1/2 a day trying to ask "someone" to reboot it.
  - Kon-Peki 2y ago
    It wasn't switching to the cloud that saved your dev team. It was getting rid of policy/thinking that allowed the "ownership" of a critical service to be so opaque/siloed that it took half a day to reboot a computer.
    - re-thc 2y ago
      > It was getting rid of policy/thinking that allowed the "ownership" of a critical service to be so opaque/siloed that it took half a day to reboot a computer. No 1 got rid of any policy. Cloud meant any team could just take out their company card and sign up. Same as how SaaS got popular. The problem is, what is critical? That server was non-production and there were 100s to 1000s of things more critical to deal with. It was low priority. Maybe you know but to many up the chain in many orgs - they don't know. I've seen way worse scenarios - things on someone's laptop with ports opened that were production critical. Cloud isn't the cure, you're right, but it forced a lot of change.
      - Kon-Peki 2y ago
        > Cloud meant any team could just take out their company card and sign up > Cloud isn't the cure, you're right, but it forced a lot of change. That's the thing - there are plenty of organizations that operate solely in the cloud but do not give their teams the authority/ability to spend without prior approval. And there are plenty of organizations that have no cloud presence that allow their teams the discretion to spend money when spending money is the right answer. For a lot of companies, the move to the cloud was a super large benefit. I'm just arguing that the real benefit to those organizations was a side effect and that many organizations probably still don't realize this.
londons_explore 2y ago
Pretty sure "one VCPU" is scaled to the core performance. So if they have a powerful core that has 1.5x the instruction throughput, it gets sold to users as 1.5 VCPU's.
- MaKey 2y ago
  At least with Azure that's not the case. Depending on the VM family one vCPU is one core or one thread.
- paol 2y ago
  Not in AWS. 1 VCPU = 1 thread (so half a core in architectures with 2-way SMT, which is most of them).
  - yencabulator 2y ago
    It is, except when it isn't. T2 etc are burstable and each vCPU seems to be guaranteed only 10% of a CPU thread(/core), even while it's called a "vCPU".
    - rbanffy 2y ago
      So, in AWS (and most others), a vCPU is something between nothing and one single thread, depending on how much you pay for it. I wonder why IBM doesn’t do more with POWER10 in their cloud, because each core can go up to, IIRC, 8 threads. Same for Oracle, BTW, but they are no longer investing on SPARC, just milking until it dries up completely.
kriz9 2y ago
Last I checked it is even worse if you need a fast NVMe. 120000 IOPS will cost like $7,800 on Google Cloud.
klausa 2y ago
I think it's far more likely that it's driven by the fact that most CPUs are sold in laptops; and P/E core split lets them have better battery life. Those design decisions then bubble up to server space, not the other way around.
- paol 2y ago
  But the decision to make server chips entirely of E cores must come from some customers demanding core count density above all else. I'm suggesting where that demand comes from.
  - chipdart 2y ago
    > (...) must come from some customers demanding core count density above all else. I don't think you realize that that's not a niche, whimsical requirement that you can only explain with conspiracy theories. For decades, all hosting providers have been paying a premium to have more computational power with a smaller size and power footprint. This is not a AWS whim. All cloud providers want this, and have been paying a premium for this. If anything, AWS had the budget to roll out their own CPUs. How long ago did AMD launched its Bulldozer line?
- greenicon 2y ago
  Modern CPUs are mostly limited by their energy budget/heat and smaller cores are more efficient. Also, if you have a workload scaling to, say, 8 cores chances are good that it also scales to 20+, allowing the usage of many small cores. Hence, having a few fast and then several smaller, more efficient cores should be also useful for workstations and servers.
chipdart 2y ago
> Cloud providers have structured their compute offerings around dollars per core. This creates the incentive to deploy the densest possible core count processors, and zero incentive to make those cores faster. I think your take is too conspiratorial. There are plenty of good justifications for this sort of approach. One of the selling points that stand out is that the bulk of the threads/processes don't have high computational needs and act more as event handlers, and typically they stay idle while waiting for IO, which represents the bulk of their workloads. Why would you waste your budget on high-performance cores if most of what you do is sitting idle while waiting for data? There's a reason why things like node.js are a success.
- paol 2y ago
  You may be right in general about my take being too cynical, but I have to counter this in particular: > and typically they stay idle while waiting for IO That's just not how you size servers. If most of your cores are idle you will add more work to the server, or you will have bough one with fewer cores to begin with.
  - lionkor 2y ago
    There's plenty of work that will max out your NiC before maxing out all cores, or even one, with reasonably efficient code. At some point youre just actually pushing 10 Gbit/s through a process that is using a few 10-15% per core.
    - paol 2y ago
      You're missing my point. Servers are very expensive and are sized to task. If your NICs (or whatever) bottleneck the server with 32 cores worth of load, then you buy a server with 32 cores, not one with 64 cores.
      - lionkor 2y ago
        Fair
  - 2y ago
pif 2y ago
> when you actually need good performance for something that doesn't scale so well horizontally Truth is, you don't have the right to need fast cores any more. Those times are over, when you could write poor code and buy a more expensive CPU. Now, if you want your software to scale, it's up to you to make it scale horizontally. Either this, or suck it up!
bayindirh 2y ago
The thing is, most cloud systems sit at idle-ish loads, sans a few select components (file inegestion, file conversion, AI inference, high load databases). These idle microservices needs bursts, not long strips of straight road. Hence having a lot of low power cores which can be quickly scheduled and have ample bandwidth to the rest of the system is what makes them feel snappy on the cloud. Another fact is, currently processors are really way more powerful and have more capacity demanded by the tasks. I have a new N100 NUC system. It has 32GB single channel memory, a paltry 500MB/sec SATA and another paltry 800MB/sec NVMe disk, and it's way snappier than my old desktop, which is more than enough for me (It's running Debian & KDE). So having tons of E cores on a bog standard server won't hurt people who are trying to serve CRUD apps, but they may need a couple of P-core filled servers for DB, etc. It's not very different from what Sun did with Niagara processors which moped the floor (and moped again just for the giggles) with Intel in web serving and small file processing tasks (500% difference, c'mon now). The only downside is, we're going deeper on specialization cycle. Now, processors are the new accelerators, which are task specialized, and your server may not be just repurposed for something else on the other rack. P.S.: I'm not a big Oracle fan, but the ARM servers they offer at OCI are way more snappier than the Premium Intel server I have at DigitalOcean. Result may change if I test their "long-straight performance", but ARM one works like local machine response wise.
dzonga 2y ago
or it has escaped people -- how CPU's have gotten blazing fast. we can't notice of coure due to how software is these days. your average desktop CPU might have 8Cores / 16 threads. if you rent a dedicated machine n skip the 'cloud sharing' your average web app -- won't be able to exhaust a single machine with 128GB Ram to 2TB Ram and 64 core CPU. Now we're talking about 288 Cores with the Intel Efficient CPU's.
- marginalia_nu 2y ago
  CPUs have had an enormous performance boost over the last few years and I do indeed think it's happened without many realizing. For a big part of the 2010s CPUs were kind of stagnant since Intel didn't really have much competition and was happy resting on their laurels and dropping slow incremental upgrades (like a 2012 CPU and a 2016 CPU were basically the same), and AMD were floundering and playing catch up, but when they did catch up and really blow past Intel, it really lit a fire under Intel's ass and now there's real competition for x86 CPUs again. I think it's a pretty favorable description of cloud providers that they will sell you 2015 performance for 2024 dollars.
  - rbanffy 2y ago
    It’s quite incredible how the cores themselves got faster - I was betting on that stagnation and core count increases, but, it seems, they have managed to delay that point for a very long time, and with no wall in sight for now. Having said that, making software more parallel is still a good investment.
bravetraveler 2y ago
That's just physics/scaling and pricing to represent it Space is practically finite, if more of the same performance can be crammed in, that's still a win! Your laptop and servers are solving for entirely different workloads and constraints. The egregious pricing has always been there, but the proportional nature makes sense. It's less perverse incentive and more... natural characteristic
jiggawatts 2y ago
There's another aspect to this, which I'm not sure even has a proper name. Misaligned incentives isn't quite it, more like... incentive scale misalignment. It goes like this: Let's say you run a company that has a critical database that cost $10M in software licensing, $100M in software development, and is central to your $1B business. Now go host it on a cloud server that has margins of $10 per core per month. If that cloud vendor can cut your performance by 10% to save $1 without you switching providers, they will do that, because across millions of cores and millions of customers, that's huge money... for them. Sure, you've lost $111 million worth of performance on $1.11B of your stuff, but that's your problem. The cloud vendor's executives are whistling a happy tune while flipping through a yacht prospectus. They don't care about your lost efficiency, they care about their single digit dollar margins. You'll see this any time some huge company uses a platform, input material, or service that is very low cost relative to the value of the service or product that it goes into. As another random example: the quality of the cotton cloth that goes into name-brand T-shirts. You can buy a $100 shirt that has 10c of material in it because some guy worked very hard to make it as thin as physically possible to cut that down from 11c. He doesn't care if your nipples show through. Conversely, the companies with the best products in the world typically have good vertical integration. E.g.: Apple. PS: All major cloud providers use processor "C-states" to save a few dollars per server per month on their power and cooling costs. Turning that off is the magic turbo button on database servers. I do this for kit that I have direct access to. You can't do this in the cloud, and never will be allowed to, no matter how many millions of dollars this costs you. PPS: All cloud SLAs refund the cost of the service you purchased, not the value of the service you were unable to sell due to the outage.
dmitrygr 2y ago
This was tried before: https://en.wikipedia.org/wiki/SeaMicro https://en.wikipedia.org/wiki/SeaMicro

by paol 2y ago

re-thc 2y ago

> being driven by hyperscalers like Amazon and the perverse incentive caused by their pricing structures Not really. They can and do price it otherwise e.g. with different instance types. If we had to blame something then it's the move away from the traditional software licensing model. In the past, self-install software e.g. databases, OSes, etc would charge per core / CPU and that would be a huge chunk of the expenses added up. Now that we're free from this things are a lot different. Having said that, the desktop side is still aiming for high-perf cores for e.g. gaming. It's not lost - just harder and harder to do. > It can't have escaped people's notice how slow cloud VCPUs are It's not that slow. It comes down to choosing the right instances and staying current. EPYC 4th gen single core on e.g. Amazon i.e. c7a is not the same as a 6th year old laptop CPU? It's also not hundreds of dollars per month (reserved / spot instances are the way to go). Also this is nothing new. I remember the days of companies running their own VMWare data centers and those server Xeons were a lot slower. There are just heat and density concerns even if you had to do it yourself.

paol 2y ago
Funny enough c7a is just what we chose recently for our CI runners precisely because of the better CPU performance. It's only better relatively to the terribleness of the t* types though. And it definitely costs hundreds of dollars per month. Just ran the calculator and 8 VPCUs (to keep the rough laptop analogy) comes out at $316.
- re-thc 2y ago
  > It's only better relatively to the terribleness of the t* types though. It's never a value add to use the t* types. That's poor misinformation or at least outdated information that people keep spreading. The advantage of c7a or Graviton 3 though is you get real cores. Most of the t* get you vCPUs, and 1/2 of it are just threads. It can be up to 2x faster in that sense. > Just ran the calculator and 8 VPCUs (to keep the rough laptop analogy) comes out at $316. Does your laptop have 8 real cores not threads that can 24/7 keep its peak speeds? Most overheat and throttle... are we comparing the same thing? ??? USD? c7a.2xlarge (8 vCPU) spot instance pricing is $0.1032 = ~$80/month (us-east-1) Even on just a 3 year compute savings plan and we're on $144.71/month. You're not required to commit to a particular instance type - just the spending, so all good? This is excluding storage. Either way there's no reason to use EC2 On-Demand without a savings plan or spot. At least commit some of it a year and it's a big saving. Otherwise don't use AWS. Side note: EPYC 5th gen has been announced and we'll likely see way more improvements on top. Pricing will definitely go down. Also: this is just a rant on AWS. Hetzner, OVH or even Oracle cloud is way cheaper, like maybe 1/2 to 1/10 of this $316... it's not a cloud issue - just AWS.
  - paol 2y ago
    You're slightly off in your understanding of VCPUs. All AWS instances "get" VCPUs. A VCPU is a single thread in architectures with SMT and a single core in architectures without. So for example c7a instances which are AMD EPYCs have 1 VCPU = 1/2 core. They have is a page that lists this for every single instance type which I can't be bothered to search for right now. As for spot instances, they're a great if you can use them, but you can't hold their price as being representative. Now using savings plan prices as representative is defensible, but it won't bring the cost below the "hundreds per month" order of magnitude.
    - phonon 2y ago
      Hyperthreading is turned off on m7a/c7a/r7a. "One of the major differences between M7a instances and the previous generations of instances, such as M6a instances, is their vCPU to physical processor core mapping. Every vCPU on a M7a instance is a physical CPU core. This means there is no Simultaneous Multi-Threading (SMT). By contrast, every vCPU on prior generations such as M6a instances is a thread of a CPU core." https://aws.amazon.com/ec2/instance-types/m7a/ https://aws.amazon.com/ec2/instance-types/m7a/

authorfly 2y ago

DHH made this point a while ago when talking about this potentially springboarding a walk back to self-hosting/on-device software for companies. I think the main point was that running CI on a MacBook PRO happens about 3-5x faster than CI on the cloud. The benefit of things running faster on the cloud is tearing away.

re-thc 2y ago
> The benefit of things running faster on the cloud is tearing away Was that the benefit? I thought it's reliability, scalability, etc. I remember having a FAST shared server for CI. "Someone" was in charge of it. Every time it went down everyone lost 1/2 a day trying to ask "someone" to reboot it.
- Kon-Peki 2y ago
  It wasn't switching to the cloud that saved your dev team. It was getting rid of policy/thinking that allowed the "ownership" of a critical service to be so opaque/siloed that it took half a day to reboot a computer.
  - re-thc 2y ago
    > It was getting rid of policy/thinking that allowed the "ownership" of a critical service to be so opaque/siloed that it took half a day to reboot a computer. No 1 got rid of any policy. Cloud meant any team could just take out their company card and sign up. Same as how SaaS got popular. The problem is, what is critical? That server was non-production and there were 100s to 1000s of things more critical to deal with. It was low priority. Maybe you know but to many up the chain in many orgs - they don't know. I've seen way worse scenarios - things on someone's laptop with ports opened that were production critical. Cloud isn't the cure, you're right, but it forced a lot of change.
    - Kon-Peki 2y ago
      > Cloud meant any team could just take out their company card and sign up > Cloud isn't the cure, you're right, but it forced a lot of change. That's the thing - there are plenty of organizations that operate solely in the cloud but do not give their teams the authority/ability to spend without prior approval. And there are plenty of organizations that have no cloud presence that allow their teams the discretion to spend money when spending money is the right answer. For a lot of companies, the move to the cloud was a super large benefit. I'm just arguing that the real benefit to those organizations was a side effect and that many organizations probably still don't realize this.

londons_explore 2y ago

Pretty sure "one VCPU" is scaled to the core performance. So if they have a powerful core that has 1.5x the instruction throughput, it gets sold to users as 1.5 VCPU's.

MaKey 2y ago
At least with Azure that's not the case. Depending on the VM family one vCPU is one core or one thread.
paol 2y ago
Not in AWS. 1 VCPU = 1 thread (so half a core in architectures with 2-way SMT, which is most of them).
- yencabulator 2y ago
  It is, except when it isn't. T2 etc are burstable and each vCPU seems to be guaranteed only 10% of a CPU thread(/core), even while it's called a "vCPU".
  - rbanffy 2y ago
    So, in AWS (and most others), a vCPU is something between nothing and one single thread, depending on how much you pay for it. I wonder why IBM doesn’t do more with POWER10 in their cloud, because each core can go up to, IIRC, 8 threads. Same for Oracle, BTW, but they are no longer investing on SPARC, just milking until it dries up completely.

kriz9 2y ago

Last I checked it is even worse if you need a fast NVMe. 120000 IOPS will cost like $7,800 on Google Cloud.

klausa 2y ago

I think it's far more likely that it's driven by the fact that most CPUs are sold in laptops; and P/E core split lets them have better battery life. Those design decisions then bubble up to server space, not the other way around.

paol 2y ago
But the decision to make server chips entirely of E cores must come from some customers demanding core count density above all else. I'm suggesting where that demand comes from.
- chipdart 2y ago
  > (...) must come from some customers demanding core count density above all else. I don't think you realize that that's not a niche, whimsical requirement that you can only explain with conspiracy theories. For decades, all hosting providers have been paying a premium to have more computational power with a smaller size and power footprint. This is not a AWS whim. All cloud providers want this, and have been paying a premium for this. If anything, AWS had the budget to roll out their own CPUs. How long ago did AMD launched its Bulldozer line?
greenicon 2y ago
Modern CPUs are mostly limited by their energy budget/heat and smaller cores are more efficient. Also, if you have a workload scaling to, say, 8 cores chances are good that it also scales to 20+, allowing the usage of many small cores. Hence, having a few fast and then several smaller, more efficient cores should be also useful for workstations and servers.

chipdart 2y ago

> Cloud providers have structured their compute offerings around dollars per core. This creates the incentive to deploy the densest possible core count processors, and zero incentive to make those cores faster. I think your take is too conspiratorial. There are plenty of good justifications for this sort of approach. One of the selling points that stand out is that the bulk of the threads/processes don't have high computational needs and act more as event handlers, and typically they stay idle while waiting for IO, which represents the bulk of their workloads. Why would you waste your budget on high-performance cores if most of what you do is sitting idle while waiting for data? There's a reason why things like node.js are a success.

paol 2y ago
You may be right in general about my take being too cynical, but I have to counter this in particular: > and typically they stay idle while waiting for IO That's just not how you size servers. If most of your cores are idle you will add more work to the server, or you will have bough one with fewer cores to begin with.
- lionkor 2y ago
  There's plenty of work that will max out your NiC before maxing out all cores, or even one, with reasonably efficient code. At some point youre just actually pushing 10 Gbit/s through a process that is using a few 10-15% per core.
  - paol 2y ago
    You're missing my point. Servers are very expensive and are sized to task. If your NICs (or whatever) bottleneck the server with 32 cores worth of load, then you buy a server with 32 cores, not one with 64 cores.
    - lionkor 2y ago
      Fair
- 2y ago

pif 2y ago

> when you actually need good performance for something that doesn't scale so well horizontally Truth is, you don't have the right to need fast cores any more. Those times are over, when you could write poor code and buy a more expensive CPU. Now, if you want your software to scale, it's up to you to make it scale horizontally. Either this, or suck it up!

bayindirh 2y ago

The thing is, most cloud systems sit at idle-ish loads, sans a few select components (file inegestion, file conversion, AI inference, high load databases). These idle microservices needs bursts, not long strips of straight road. Hence having a lot of low power cores which can be quickly scheduled and have ample bandwidth to the rest of the system is what makes them feel snappy on the cloud. Another fact is, currently processors are really way more powerful and have more capacity demanded by the tasks. I have a new N100 NUC system. It has 32GB single channel memory, a paltry 500MB/sec SATA and another paltry 800MB/sec NVMe disk, and it's way snappier than my old desktop, which is more than enough for me (It's running Debian & KDE). So having tons of E cores on a bog standard server won't hurt people who are trying to serve CRUD apps, but they may need a couple of P-core filled servers for DB, etc. It's not very different from what Sun did with Niagara processors which moped the floor (and moped again just for the giggles) with Intel in web serving and small file processing tasks (500% difference, c'mon now). The only downside is, we're going deeper on specialization cycle. Now, processors are the new accelerators, which are task specialized, and your server may not be just repurposed for something else on the other rack. P.S.: I'm not a big Oracle fan, but the ARM servers they offer at OCI are way more snappier than the Premium Intel server I have at DigitalOcean. Result may change if I test their "long-straight performance", but ARM one works like local machine response wise.

dzonga 2y ago

or it has escaped people -- how CPU's have gotten blazing fast. we can't notice of coure due to how software is these days. your average desktop CPU might have 8Cores / 16 threads. if you rent a dedicated machine n skip the 'cloud sharing' your average web app -- won't be able to exhaust a single machine with 128GB Ram to 2TB Ram and 64 core CPU. Now we're talking about 288 Cores with the Intel Efficient CPU's.

marginalia_nu 2y ago
CPUs have had an enormous performance boost over the last few years and I do indeed think it's happened without many realizing. For a big part of the 2010s CPUs were kind of stagnant since Intel didn't really have much competition and was happy resting on their laurels and dropping slow incremental upgrades (like a 2012 CPU and a 2016 CPU were basically the same), and AMD were floundering and playing catch up, but when they did catch up and really blow past Intel, it really lit a fire under Intel's ass and now there's real competition for x86 CPUs again. I think it's a pretty favorable description of cloud providers that they will sell you 2015 performance for 2024 dollars.
- rbanffy 2y ago
  It’s quite incredible how the cores themselves got faster - I was betting on that stagnation and core count increases, but, it seems, they have managed to delay that point for a very long time, and with no wall in sight for now. Having said that, making software more parallel is still a good investment.

bravetraveler 2y ago

That's just physics/scaling and pricing to represent it Space is practically finite, if more of the same performance can be crammed in, that's still a win! Your laptop and servers are solving for entirely different workloads and constraints. The egregious pricing has always been there, but the proportional nature makes sense. It's less perverse incentive and more... natural characteristic

jiggawatts 2y ago

There's another aspect to this, which I'm not sure even has a proper name. Misaligned incentives isn't quite it, more like... incentive scale misalignment. It goes like this: Let's say you run a company that has a critical database that cost $10M in software licensing, $100M in software development, and is central to your $1B business. Now go host it on a cloud server that has margins of $10 per core per month. If that cloud vendor can cut your performance by 10% to save $1 without you switching providers, they will do that, because across millions of cores and millions of customers, that's huge money... for them. Sure, you've lost $111 million worth of performance on $1.11B of your stuff, but that's your problem. The cloud vendor's executives are whistling a happy tune while flipping through a yacht prospectus. They don't care about your lost efficiency, they care about their single digit dollar margins. You'll see this any time some huge company uses a platform, input material, or service that is very low cost relative to the value of the service or product that it goes into. As another random example: the quality of the cotton cloth that goes into name-brand T-shirts. You can buy a $100 shirt that has 10c of material in it because some guy worked very hard to make it as thin as physically possible to cut that down from 11c. He doesn't care if your nipples show through. Conversely, the companies with the best products in the world typically have good vertical integration. E.g.: Apple. PS: All major cloud providers use processor "C-states" to save a few dollars per server per month on their power and cooling costs. Turning that off is the magic turbo button on database servers. I do this for kit that I have direct access to. You can't do this in the cloud, and never will be allowed to, no matter how many millions of dollars this costs you. PPS: All cloud SLAs refund the cost of the service you purchased, not the value of the service you were unable to sell due to the outage.

dmitrygr 2y ago

This was tried before: https://en.wikipedia.org/wiki/SeaMicro https://en.wikipedia.org/wiki/SeaMicro