6 ms·

This reminds me of Tilera, who had a 64 core mesh connected CPU ten about ten years ago. The problems seemed to be it was harder to optimize due to the mesh con

This reminds me of Tilera, who had a 64 core mesh connected CPU ten about ten years ago. The problems seemed to be it was harder to optimize due to the mesh connectivity (like NUMA but multidimensional), low clock speeds, and lack of improvement after an initially promising launch. Will this be the same? It seems possible. Does it really get more work done per watt than x86? And why does the article say "These Altra CPUs have no turbo mechanism" right below a graphic saying "3.0 Ghz Turbo"?

PaulHoule 6y ago
These chips are practical and can go into servers that are similar in performance to x86 servers. ARM has well-thought out NUMA support, probably a system this size or larger should be divided into logical partitions anyway. (e.g. out of 128 cores maybe you pick 4 to be management processors to begin with).
wtallis 6y ago
> And why does the article say "These Altra CPUs have no turbo mechanism" right below a graphic saying "3.0 Ghz Turbo"? These chips obviously have variable clock speed, but apparently nothing like the complicated boost mechanisms on recent x86 processors. My guess is that Turbo speed here is simply full speed, and doesn't depend significantly on how many cores are active, and doesn't let the chip exceed its nominal TDP for short (or not so short) bursts the way x86 processors do.
- rbanffy 6y ago
  > and doesn't depend significantly on how many cores are active, and doesn't let the chip exceed its nominal TDP for short (or not so short) bursts the way x86 processors do Either that, or 3 GHz always exceeds the envelope and the chip is throttling clocks down all the time to keep itself inside the allowed power envelope.
- zozbot234 6y ago
  > doesn't let the chip exceed its nominal TDP for short (or not so short) bursts the way x86 processors do. That's more of an artifact of how TDP is defined than anything else. I doubt that this could peg even a single core at 3.0 GHz given a reasonable cooling setup, let alone run all cores @ 3.0 GHz.
  - greggyb 6y ago
    Why do you think this won't keep a single core at 3.0GHz? You can get ~3.4GHz average sustained all-core speed on a 3990x (64-core, nominal 280W TDP). This is with an off-the-shelf AIO cooler.[0] Note: top-end air coolers are often competitive with AIOs, and can be had for $80-$100.[1] If you're buying a several thousand dollar CPU, dropping even $500 (much higher than you'd need for closed loop liquid or high-end air cooling) on cooling doesn't seem unreasonable. [0] https://www.anandtech.com/show/15483/amd-threadripper-3990x-review/2 https://www.anandtech.com/show/15483/amd-threadripper-3990x-... [1] https://www.youtube.com/watch?v=7VzXHUTqE7E https://www.youtube.com/watch?v=7VzXHUTqE7E
    - stan_rogers 6y ago
      Neither the AIO nor the large tower (air) cooler are going to fly in a sever rack.
      - greggyb 6y ago
        No. Servers will be utilizing jet-sounding fans, or air conditioned intake, or rack-level water cooling, or any number of other cooling solutions that can handle high the heat output. 250W is not an absurd figure to shove in a server CPU. If you're buying an 80-core CPU, you're not going to be skimping on the cooling solution. Especially given the target market for Ampere is cloud providers, you can expect these to be racked in enclosures that provide sufficient cooling for their operational needs.
rbanffy 6y ago
You need a lot of memory bandwidth and large caches, or else the cores will starve. That's also why IBM mainframes have up to 4.5 GB of L4 cache.
- zozbot234 6y ago
  That's true of all high-frequency/high-core count hardware. Which is why running Java or Python codes on this hardware makes very little sense. Rust is more like it. Golang in a pinch.
  - blackoil 6y ago
    Noob ques. Is there any fundamental limitation in Java or more like JVM will need to evolve to optimally use such architecture ??
    - twic 6y ago
      AIUI the relevant weakness of Java here is that it typically has worse memory density and locality than something like Rust. Consider code which linearly goes through a list of points in 2D space and does some calculation on the coordinates. In Rust, the list is a Vec<(f64, f64)>. The Vec is a small object containing a pointer to a large block of data which contains all the points packed tightly together. Once the program has dereferenced the pointer and loaded the first point, all the others are immediately after it in memory, in order, containing nothing but the coordinates, and so the processor's cacheing and prefetching will make them available very quickly. In Java, the list is an ArrayList<Point2D.Double>. The ArrayList is a small object containing a pointer to an array of pointers to more small objects, one for each point. Each of the small objects has a two-word object header on it. The pointer plus header means that for every two words of coordinate, there are three words of overhead, so the cache is used much less effectively. The small objects aren't necessarily anywhere near one another in memory, or in order, so prefetching won't help. There are a couple of ways the Java situation can be improved. Firstly, today, you can replace the naive ArrayList<Point2D.Double> with a more compact structure which keeps all the coordinates in a single big array. This gives you the same efficiency as Rust, but requires programming effort (unless you can find an existing library which does it!), and may give you an API that is less efficient (if it copies coordinates to objects on retrieval) or convenient (if it gives you some cursor/flyweight API). Secondly, in the future, the JVM could get smarter. In principle, it could do the above rewriting as an optimisation, although i wouldn't want to rely on that. A good garbage collector could bring the small objects together in memory, to improve locality a bit. Thirdly, in the near-ish future, Java will get value types [0] which behave a lot more like Rust's types. That would give you equally good density and locality without having to jump through hoops. [0] http://openjdk.java.net/projects/valhalla/ http://openjdk.java.net/projects/valhalla/
jillesvangurp 6y ago
It depends a bit on how you utilize these CPUs. A lot of server software is optimized for just a few cores. Even products optimized for using more than 1 thread tend to be tested and used mostly with 4/8 core configurations. And then of course there are a few popular server-side languages that are effectively single threaded typically (e.g. python) and use multiple processes to leverage multiple cores. Launching 80 python processes on an 80 core machine may not be the best way to utilize available resources compared to e.g. a Java process with a few hundred threads. With non blocking IO and async processing that can be good enough but to fully utilize dozens/hundreds of CPU cores from a single process, you basically want something that can do both threading and async. But assuming each core performs at a reasonable percentage of e.g. a Xeon core (lets say 40%) and doesn't slow down when all cores are fully loaded, you would expect a CPU with 80 cores to more than keep up with a 16 or even 32 core Xeon. Of course the picture gets murkier if you throw in specialized instructions for vector processing, GPUs, etc.
- ddorian43 6y ago
  Yes most software is limited cores (example: encoding videos). The best (efficient) way to utilize that many cores IS to have 1 pinned process/thread per-core: https://www.scylladb.com/ https://www.scylladb.com/ https://github.com/scylladb/seastar/ https://github.com/scylladb/seastar/ That would be the same in Python too. A problem is that you can't share the kernel pages for the code, and you need to have a shared-cache. Probably 0-mem-copy with no deserialization example: lmdb + Flat Buffers. Nicer is to have 1/2x cores, but each core being 2x faster ;)

PaulHoule 6y ago

These chips are practical and can go into servers that are similar in performance to x86 servers. ARM has well-thought out NUMA support, probably a system this size or larger should be divided into logical partitions anyway. (e.g. out of 128 cores maybe you pick 4 to be management processors to begin with).

wtallis 6y ago

> And why does the article say "These Altra CPUs have no turbo mechanism" right below a graphic saying "3.0 Ghz Turbo"? These chips obviously have variable clock speed, but apparently nothing like the complicated boost mechanisms on recent x86 processors. My guess is that Turbo speed here is simply full speed, and doesn't depend significantly on how many cores are active, and doesn't let the chip exceed its nominal TDP for short (or not so short) bursts the way x86 processors do.

rbanffy 6y ago
> and doesn't depend significantly on how many cores are active, and doesn't let the chip exceed its nominal TDP for short (or not so short) bursts the way x86 processors do Either that, or 3 GHz always exceeds the envelope and the chip is throttling clocks down all the time to keep itself inside the allowed power envelope.
zozbot234 6y ago
> doesn't let the chip exceed its nominal TDP for short (or not so short) bursts the way x86 processors do. That's more of an artifact of how TDP is defined than anything else. I doubt that this could peg even a single core at 3.0 GHz given a reasonable cooling setup, let alone run all cores @ 3.0 GHz.
- greggyb 6y ago
  Why do you think this won't keep a single core at 3.0GHz? You can get ~3.4GHz average sustained all-core speed on a 3990x (64-core, nominal 280W TDP). This is with an off-the-shelf AIO cooler.[0] Note: top-end air coolers are often competitive with AIOs, and can be had for $80-$100.[1] If you're buying a several thousand dollar CPU, dropping even $500 (much higher than you'd need for closed loop liquid or high-end air cooling) on cooling doesn't seem unreasonable. [0] https://www.anandtech.com/show/15483/amd-threadripper-3990x-review/2 https://www.anandtech.com/show/15483/amd-threadripper-3990x-... [1] https://www.youtube.com/watch?v=7VzXHUTqE7E https://www.youtube.com/watch?v=7VzXHUTqE7E
  - stan_rogers 6y ago
    Neither the AIO nor the large tower (air) cooler are going to fly in a sever rack.
    - greggyb 6y ago
      No. Servers will be utilizing jet-sounding fans, or air conditioned intake, or rack-level water cooling, or any number of other cooling solutions that can handle high the heat output. 250W is not an absurd figure to shove in a server CPU. If you're buying an 80-core CPU, you're not going to be skimping on the cooling solution. Especially given the target market for Ampere is cloud providers, you can expect these to be racked in enclosures that provide sufficient cooling for their operational needs.

rbanffy 6y ago

You need a lot of memory bandwidth and large caches, or else the cores will starve. That's also why IBM mainframes have up to 4.5 GB of L4 cache.

zozbot234 6y ago
That's true of all high-frequency/high-core count hardware. Which is why running Java or Python codes on this hardware makes very little sense. Rust is more like it. Golang in a pinch.
- blackoil 6y ago
  Noob ques. Is there any fundamental limitation in Java or more like JVM will need to evolve to optimally use such architecture ??
  - twic 6y ago
    AIUI the relevant weakness of Java here is that it typically has worse memory density and locality than something like Rust. Consider code which linearly goes through a list of points in 2D space and does some calculation on the coordinates. In Rust, the list is a Vec<(f64, f64)>. The Vec is a small object containing a pointer to a large block of data which contains all the points packed tightly together. Once the program has dereferenced the pointer and loaded the first point, all the others are immediately after it in memory, in order, containing nothing but the coordinates, and so the processor's cacheing and prefetching will make them available very quickly. In Java, the list is an ArrayList<Point2D.Double>. The ArrayList is a small object containing a pointer to an array of pointers to more small objects, one for each point. Each of the small objects has a two-word object header on it. The pointer plus header means that for every two words of coordinate, there are three words of overhead, so the cache is used much less effectively. The small objects aren't necessarily anywhere near one another in memory, or in order, so prefetching won't help. There are a couple of ways the Java situation can be improved. Firstly, today, you can replace the naive ArrayList<Point2D.Double> with a more compact structure which keeps all the coordinates in a single big array. This gives you the same efficiency as Rust, but requires programming effort (unless you can find an existing library which does it!), and may give you an API that is less efficient (if it copies coordinates to objects on retrieval) or convenient (if it gives you some cursor/flyweight API). Secondly, in the future, the JVM could get smarter. In principle, it could do the above rewriting as an optimisation, although i wouldn't want to rely on that. A good garbage collector could bring the small objects together in memory, to improve locality a bit. Thirdly, in the near-ish future, Java will get value types [0] which behave a lot more like Rust's types. That would give you equally good density and locality without having to jump through hoops. [0] http://openjdk.java.net/projects/valhalla/ http://openjdk.java.net/projects/valhalla/

jillesvangurp 6y ago

It depends a bit on how you utilize these CPUs. A lot of server software is optimized for just a few cores. Even products optimized for using more than 1 thread tend to be tested and used mostly with 4/8 core configurations. And then of course there are a few popular server-side languages that are effectively single threaded typically (e.g. python) and use multiple processes to leverage multiple cores. Launching 80 python processes on an 80 core machine may not be the best way to utilize available resources compared to e.g. a Java process with a few hundred threads. With non blocking IO and async processing that can be good enough but to fully utilize dozens/hundreds of CPU cores from a single process, you basically want something that can do both threading and async. But assuming each core performs at a reasonable percentage of e.g. a Xeon core (lets say 40%) and doesn't slow down when all cores are fully loaded, you would expect a CPU with 80 cores to more than keep up with a 16 or even 32 core Xeon. Of course the picture gets murkier if you throw in specialized instructions for vector processing, GPUs, etc.

ddorian43 6y ago
Yes most software is limited cores (example: encoding videos). The best (efficient) way to utilize that many cores IS to have 1 pinned process/thread per-core: https://www.scylladb.com/ https://www.scylladb.com/ https://github.com/scylladb/seastar/ https://github.com/scylladb/seastar/ That would be the same in Python too. A problem is that you can't share the kernel pages for the code, and you need to have a shared-cache. Probably 0-mem-copy with no deserialization example: lmdb + Flat Buffers. Nicer is to have 1/2x cores, but each core being 2x faster ;)