6 ms·

> Furthermore, these R8g instances provide up to three times more vCPUs and memory than Graviton 3-based R7g instances. So, uh, 75% more bandwidth but 200% mor

> Furthermore, these R8g instances provide up to three times more vCPUs and memory than Graviton 3-based R7g instances. So, uh, 75% more bandwidth but 200% more cores is a net reduction in available bandwidth per core. If you rented 32 cores of a server before, you got a certain amount of bandwidth. If you now rent the same 32 cores, you'll have about half the realized bandwidth?

athrun 3y ago
apparently the 75% improvement is per core: https://news.ycombinator.com/item?id=38451909 https://news.ycombinator.com/item?id=38451909
- tedunangst 3y ago
  Well, somebody is lying. Amazon previously claimed graviton3 had 300GB/sec memory bandwidth. https://www.mgt-commerce.com/blog/aws-announces-general-availability-of-graviton-3-processors/ https://www.mgt-commerce.com/blog/aws-announces-general-avai... 300 x 1.75 = 525, ~536. Per chip. If the new chip has more cores, then where is the extra bandwidth hiding?
  - deleted 3y ago
    [deleted]
  - phonon 3y ago
    50% more cores per chip. 75% more bandwidth per chip. So more bandwidth per core.
- scq 3y ago
  That comment reads more like they're talking about compute performance, not memory bandwidth.
krasin 3y ago
Memory bandwidth is becoming the ultimate spec. In related news, Apple reduced memory bandwidth in their M3 chips by 25% compared to M1/M2: https://www.macrumors.com/2023/10/31/apple-m3-pro-less-memory-bandwidth/ https://www.macrumors.com/2023/10/31/apple-m3-pro-less-memor...
- fswd 3y ago
  I'm so glad we're seeing memory specs next to the core count, awesome.
- Aurornis 3y ago
  > Memory bandwidth is becoming the ultimate spec. Only in applications that are constrained by memory bandwidth. Many applications are not constrained by the memory bandwidth on modern architectures with significant cache. If you have an application constrained by memory bandwidth, carefully selecting the server would make sense. > In related news, Apple reduced memory bandwidth in their M3 chips by 25% compared to M1/M2: Not exactly. M3 Max has the same 400GB/sec memory bandwidth as the top end M1 and M2 chips. It’s only certain lower tiers that have less memory bandwidth than their equivalent tiers in previous gens. But it probably doesn’t matter for most applications.
  - admax88qqq 3y ago
    > Only in applications that are constrained by memory bandwidth. Sure but I think OPs point is that memory bandwidth is the bottleneck in more applications that you think. Cache only helps if you're accessing the same data over and over. My gut instinct is the amount of applications that do a lot of processing but only on a small amount of data are in the minority. The opposite seems a lot more common.
    - jltsiren 3y ago
      I would assume that applications that process data faster than a few gigabytes per CPU-second are rare. It's more common that things are constrained by computation or memory latency.
      - Xorlev 3y ago
        You'd think so, but for datacenter workloads it's absolutely common, especially if you're just scheduling a bunch of containers together. Computation also doesn't happen in a vacuum, unless you're doing some fairly trivial processing you're likely loading quite a bit of memory, perhaps many multiples of what your business logic is actually doing. It's also not as easy as GB/s/core, since cores aren't entirely uniform, and data access may be across core complexes.
CuriousCosmic 3y ago
~~Probably however Graviton3 instances will not be going away any time soon (Grav2 instances are still around). They'll likely fill a different niche where the Grav4 instances are going to be better for larger workloads that can make use of the whole CPU whole Grav3 instances will be available for people with smaller workloads.~~ Edit: Nevermind. A reply confirms the memory uplift is per core.
phkahler 3y ago
>> So, uh, 75% more bandwidth but 200% more cores is a net reduction in available bandwidth per core. My guess is that most cloud customers use fairly little compute and low average bandwidth. But everyone wants "dedicated" cores so this makes a ton of sense and Amazon is raking in billions due to stupidity. Remember the recent story how a couple clicks cut the company cloud bill by a shit-ton. That's probably the norm.
- pants2 3y ago
  Ever since the Spectre/Meltdown attacks came out, it became a big liability not to do dedicated cores.
  - jacquesm 3y ago
    And with the crappy scheduling your typical OS provides and containerization as the virtualization mechanism of choice it makes good sense to assign one or more cores to a container.
  - ithkuil 3y ago
    I wonder if there is a timing attack that can be subjected to if you don't also have dedicated bandwidth to main memory
diffeomorphism 3y ago
> 200% more cores The article says R7g is 64 cores for a full CPU and R8g is 96. Where are you getting the 200% from?
ksec 3y ago
The key here is 3x more vCPU. I wasn't even aware ARM had SMT design, so it is highly likely to be the first dual socket / CPU design for ARM. That is 192 CPU per instances. I assume AWS will continue to work with AMD and provide 256 vCPU. Would love to see them being compared. ( ARM with Physical CPU core and AMD are SMT thread. )

athrun 3y ago

apparently the 75% improvement is per core: https://news.ycombinator.com/item?id=38451909 https://news.ycombinator.com/item?id=38451909

tedunangst 3y ago
Well, somebody is lying. Amazon previously claimed graviton3 had 300GB/sec memory bandwidth. https://www.mgt-commerce.com/blog/aws-announces-general-availability-of-graviton-3-processors/ https://www.mgt-commerce.com/blog/aws-announces-general-avai... 300 x 1.75 = 525, ~536. Per chip. If the new chip has more cores, then where is the extra bandwidth hiding?
- deleted 3y ago
  [deleted]
- phonon 3y ago
  50% more cores per chip. 75% more bandwidth per chip. So more bandwidth per core.
scq 3y ago
That comment reads more like they're talking about compute performance, not memory bandwidth.

krasin 3y ago

Memory bandwidth is becoming the ultimate spec. In related news, Apple reduced memory bandwidth in their M3 chips by 25% compared to M1/M2: https://www.macrumors.com/2023/10/31/apple-m3-pro-less-memory-bandwidth/ https://www.macrumors.com/2023/10/31/apple-m3-pro-less-memor...

fswd 3y ago
I'm so glad we're seeing memory specs next to the core count, awesome.
Aurornis 3y ago
> Memory bandwidth is becoming the ultimate spec. Only in applications that are constrained by memory bandwidth. Many applications are not constrained by the memory bandwidth on modern architectures with significant cache. If you have an application constrained by memory bandwidth, carefully selecting the server would make sense. > In related news, Apple reduced memory bandwidth in their M3 chips by 25% compared to M1/M2: Not exactly. M3 Max has the same 400GB/sec memory bandwidth as the top end M1 and M2 chips. It’s only certain lower tiers that have less memory bandwidth than their equivalent tiers in previous gens. But it probably doesn’t matter for most applications.
- admax88qqq 3y ago
  > Only in applications that are constrained by memory bandwidth. Sure but I think OPs point is that memory bandwidth is the bottleneck in more applications that you think. Cache only helps if you're accessing the same data over and over. My gut instinct is the amount of applications that do a lot of processing but only on a small amount of data are in the minority. The opposite seems a lot more common.
  - jltsiren 3y ago
    I would assume that applications that process data faster than a few gigabytes per CPU-second are rare. It's more common that things are constrained by computation or memory latency.
    - Xorlev 3y ago
      You'd think so, but for datacenter workloads it's absolutely common, especially if you're just scheduling a bunch of containers together. Computation also doesn't happen in a vacuum, unless you're doing some fairly trivial processing you're likely loading quite a bit of memory, perhaps many multiples of what your business logic is actually doing. It's also not as easy as GB/s/core, since cores aren't entirely uniform, and data access may be across core complexes.

CuriousCosmic 3y ago

~~Probably however Graviton3 instances will not be going away any time soon (Grav2 instances are still around). They'll likely fill a different niche where the Grav4 instances are going to be better for larger workloads that can make use of the whole CPU whole Grav3 instances will be available for people with smaller workloads.~~ Edit: Nevermind. A reply confirms the memory uplift is per core.

phkahler 3y ago

>> So, uh, 75% more bandwidth but 200% more cores is a net reduction in available bandwidth per core. My guess is that most cloud customers use fairly little compute and low average bandwidth. But everyone wants "dedicated" cores so this makes a ton of sense and Amazon is raking in billions due to stupidity. Remember the recent story how a couple clicks cut the company cloud bill by a shit-ton. That's probably the norm.

pants2 3y ago
Ever since the Spectre/Meltdown attacks came out, it became a big liability not to do dedicated cores.
- jacquesm 3y ago
  And with the crappy scheduling your typical OS provides and containerization as the virtualization mechanism of choice it makes good sense to assign one or more cores to a container.
- ithkuil 3y ago
  I wonder if there is a timing attack that can be subjected to if you don't also have dedicated bandwidth to main memory

diffeomorphism 3y ago

> 200% more cores The article says R7g is 64 cores for a full CPU and R8g is 96. Where are you getting the 200% from?

ksec 3y ago

The key here is 3x more vCPU. I wasn't even aware ARM had SMT design, so it is highly likely to be the first dual socket / CPU design for ARM. That is 192 CPU per instances. I assume AWS will continue to work with AMD and provide 256 vCPU. Would love to see them being compared. ( ARM with Physical CPU core and AMD are SMT thread. )