4 ms·

I'm a little confused about the value proposition here. This is about 60 CPUs at 1GHz with little memory and low IPC. Price, before promotion, is $2k, and after

by cpks 12y ago

I'm a little confused about the value proposition here. This is about 60 CPUs at 1GHz with little memory and low IPC. Price, before promotion, is $2k, and after, $250. I can get an FX-9590 for $250, which has 8 cores, but running at 4.7GHz, and higher IPC. In terms of raw compute speed, it seems like the FX-9590 will be at least the same speed. But: * I can use standard programming tools. * I only distribute computes 8-ways, not 60-ways. That's easier. * Anything which is not easily made parallel is much faster. My experience is that there tends to be a lot of that. * I have less work to transfer data back and forth. For big data, that actually takes a fair bit of time. When would the Phi be faster? When would I want to use it?

nkurz 12y ago
In terms of raw compute speed, it seems like the FX-9590 will be at least the same speed. They aren't really comparable. An FX-9590 at 4.7GHz is about 300 GFlops, but the the Xeon Phi 31S1P is slightly over 1 TFlop --- more than 3x as many floating point operations per second. The spec that probably shows the most difference is memory bandwidth. The FX-9590 with 1866 memory achieves 30GB/s. The Phi reaches 320GB/s, about 10 times as much. The presumption is that you might put 4 of these in a server, in addition to the standard processors. The price is actually quite a bit better, too, as some vendors are offering the Phi at $125 each in quantities of 10: http://www.colfax-intl.com/nd/xeonphi/31s1p-promo.aspx http://www.colfax-intl.com/nd/xeonphi/31s1p-promo.aspx This article offers some good information about cases where the Phi is a good (or bad) choice: https://software.intel.com/sites/default/files/article/330164/an-overview-of-programming-for-intel-xeon-processors-and-intel-xeon-phi-coprocessors_1.pdf https://software.intel.com/sites/default/files/article/33016... The short answer would be that the Phi might be the best choice in cases where you need to perform billions of energy efficient floating point operations on a working set small enough to fit in the cards RAM, where the degree of branching is such that a "normal" GPU would be inappropriate, and where the problem justifies considerable programmer time for optimization. Financial and scientific models are the usual examples.
- tormeh 12y ago
  Ok, but if the normal price is 2000USD, then that leaves space for a lot of 9590s. You could get almost the same memory bandwidth and 3x as much processing power with 10 9590s. And remember that's floating point performance; the bulldozer architecture the 9590 is based on gives you about effectively 1.5x as many cores when doing integer operations. So, ok, the Phi is useful at the current price, but at the 2000USD price other people are talking about in the thread it seems pretty useless.
  - decwakeboarder 12y ago
    You also need n number of motherboards, RAM, power supplies, etc as you have CPUs. The Phi looks a lot better when you fit 5-6 in a server.
  - awalton 12y ago
    Everyone in this thread keeps railing on its floating point perf, which is about on par with parts like nVidia's Tesla and similarly priced... but its integer performance is why people are really buying these cards up. All GPUs are floating point monsters but integer pansies because that's what their graphics workload heritage turned them into. Xeon Phi is a Computing Monster, with crazy Integer and Floating Point perf, because its heritage was "Let's try to make a graphics card out of CPUs" and not visa versa. It's similar to buying a whole stack of x86 chips just to get to the SSE/AVX units. Only, now you don't need a whole stack of computers, you need a handful of host computers and a stack of add-on cards. And if you think $2000 is expensive for 60 x86 cores (~$35/core) when all you care about is the SSE/AVX unit and how many vector ops you can cram through it, you're definitely not Intel's target.
    - nkurz 12y ago
      Thanks, you are right that I shouldn't have focussed on just floating point. I emphasized it because it better matched the frequently quoted "teraflop in a single card" measurement. As you say, the Phi is equally strong at vector integer operations as floating point, in both cases operating on 512-bit vectors. But I'll quibble with the assertion that all GPU's are "integer pansies", although it's a fantastic phrase. Historically, NVidia GPU's are much stronger at floating point operations than integer because of their graphics origins, and NVidia's Tesla is definitely what Intel views as nearest to the Phi's target market. But AMD/ATI cards have excellent vector integer performance, several times that of NVidia/Tesla, and much better per dollar than Phi until this recent price drop. This is why AMD cards were the best choice for bitcoin mining until the recent arrival of dedicated ASIC's: http://www.extremetech.com/computing/153467-amd-destroys-nvidia-bitcoin-mining http://www.extremetech.com/computing/153467-amd-destroys-nvi...
Coding_Cat 12y ago
It is my understanding that it sits in between a GPU and a CPU in terms of "ideal work". Like a GPU it favours SIMD work, but it is better at dealing with branches and a lot of random memory accesses[1] (it still favours linear work of course). It also uses a more familiar architecture, leading Intel to claim it is easier to master than GPU-programing, but I personally disagree as x64 is already much more complex than GPUs. In terms of FLOPS the FX-9590 is still significantly slower. I don't know the exact numbers, but it will be hundreds of MegaFlops, whereas the Phi runs at ~1 TFlop. In part because the cores are basically low-end (in-order) atoms with modules bolted on for HPC (avx-512 instructions, instructions for exponentiation for example). Data transfer to the card is a lot slower than reading from ram, correct, but on the other hand once it is on the card it is a lot faster (wiki says the fastest Phi has 350 Gb/s, Corsair claims a maximum of 70 Gb/s for DDR4)[2]. So generally, as long as you have a decent ratio of work/memory access and your work is parallel you could use a Phi to speed things a long. You would want to use one in the same situations as were you'd want to use a GPU, and the Phi would be preferable if your computations and memory access patterns are nor perfectly homogeneous (e.g. branching which can't be rewritten), as this kills performance on GPUs. [1] I'm pretty sure Intel claimed this, and it makes sense when you think about it, but I can't seem to find a source: Could someone confirm this? [2] Do note that it 350Gb/s shared between the cores, not 350 Gb/s each.
- nkurz 12y ago
  as x64 is already much more complex than GPUs I'd be interested to hear about your experiences. By "more complex", do you mean the x64 instruction set itself, or the out-of-order superscalar processors that use it?
  - Coding_Cat 12y ago
    Basically the whole package. The instruction set is pretty big, but this is not much of an issue as you generally just need a subset for a given problem (I generally only tweak SSE/AVX code) and it has a lot of if's and but's when it comes to performance, the out-of-order execution also adds a degree of uncertainty, branch prediction and prefetching, a complex(ish) cache... There is a lot of things one can fiddle with in order to improve performance. GPUs are much stricter in their design, which can lead to some headaches when having to completely change the way you formulate a problem, but once you're past that step, the simplicity of the architecture makes it a lot nicer to optimize in my opinion. (And my personal experience has been that the tools are cheaper (free) and more stable as well, but your mileage may vary).
jandrewrogers 12y ago
The Xeon Phi uses a different computing architecture than either a CPU or a GPU so some intuitions about using it will be off. The Phi is essentially a modern barrel processor that uses the AMD64 ISA (it does not understand legacy x86 modes), a really nice vector ALU implementation, and memory bandwidth that is more like a GPU than a CPU. While it will run normal 64-bit software reasonably well with no special considerations, it will not be efficient without tweaking code design. CPU-targeted code attempts to optimize IPC in a single thread; barrel processors are designed to hide latency and can't drive IPC through single thread optimization. The reason barrel processors are interesting is that they can be incredibly efficient with their clock cycles. Unlike either CPUs or GPUs, it is relatively easy to get sustained throughput that approaches the theoretical IPC of the silicon for diverse software. The Xeon Phi mentioned in the article has 114 ALUs; it is possible to ensure all of those ALUs are doing useful work every single clock cycle, unlike the much smaller number of ALUs in your CPU. CPUs and GPUs have higher theoretical throughput in some cases but various parts of their ALUs typically spend a significant part of their time idle. Contrary to marketing, you do not want to program these like an ordinary CPU even though the cores are truly general purpose (unlike a GPU). Thread behavior is unlike CPUs or GPUs. Barrel processors cannot saturate a single core with a single thread! The Xeon Phi has 228 independent threads and you need to use them all the time. The way barrel processors work is if the hardware supports N threads then each clock cycle you can saturate all the ALUs if some subset M of those threads are not stalled. The M-of-N ratio varies by barrel processor design but is typically 20-50% in my experience. Each clock cycle, a core selects an immediately runnable operation from the basket of threads it can see and executes it; as long as something is runnable in that basket, the core will do real work that clock cycle. Xeon Phi has a 50% M-of-N requirement, so you need a minimum of 114 threads that are not stalled every clock cycle to saturate the processor. The way you ensure that you hit the 114 threshold is to schedule all 228 hardware thread slots with useful work. For programming, this changes the way you reason about locality and concurrency. If we assume that some percentage of threads can be safely stalled or blocked with no impact on throughput then it changes the way you design your algorithms and data structures. A little additional latency on a subset of threads won't hurt performance, especially if it increases task concurrency. On a CPU stalled threads are expensive, as it leads to idle cores or context switches. For architectures like Phi, you design your data structures and algorithms around relatively small, semi-independently work units so that there is always a large number of tasks that can be assigned to a thread even though this reduces locality. Below a certain threshold, thread concurrency is approximately free because the cores will schedule around stalls due to contention. I like barrel processors quite a lot. Once you get used to the model, it is an easier architecture with which to achieve efficient massively threaded parallelism and throughput than either CPUs or GPUs. More importantly, they are hard to beat for efficiency for general purpose computing when software is designed for the architecture since so few clock cycles are wasted.

by cpks 12y ago

nkurz 12y ago

In terms of raw compute speed, it seems like the FX-9590 will be at least the same speed. They aren't really comparable. An FX-9590 at 4.7GHz is about 300 GFlops, but the the Xeon Phi 31S1P is slightly over 1 TFlop --- more than 3x as many floating point operations per second. The spec that probably shows the most difference is memory bandwidth. The FX-9590 with 1866 memory achieves 30GB/s. The Phi reaches 320GB/s, about 10 times as much. The presumption is that you might put 4 of these in a server, in addition to the standard processors. The price is actually quite a bit better, too, as some vendors are offering the Phi at $125 each in quantities of 10: http://www.colfax-intl.com/nd/xeonphi/31s1p-promo.aspx http://www.colfax-intl.com/nd/xeonphi/31s1p-promo.aspx This article offers some good information about cases where the Phi is a good (or bad) choice: https://software.intel.com/sites/default/files/article/330164/an-overview-of-programming-for-intel-xeon-processors-and-intel-xeon-phi-coprocessors_1.pdf https://software.intel.com/sites/default/files/article/33016... The short answer would be that the Phi might be the best choice in cases where you need to perform billions of energy efficient floating point operations on a working set small enough to fit in the cards RAM, where the degree of branching is such that a "normal" GPU would be inappropriate, and where the problem justifies considerable programmer time for optimization. Financial and scientific models are the usual examples.

tormeh 12y ago
Ok, but if the normal price is 2000USD, then that leaves space for a lot of 9590s. You could get almost the same memory bandwidth and 3x as much processing power with 10 9590s. And remember that's floating point performance; the bulldozer architecture the 9590 is based on gives you about effectively 1.5x as many cores when doing integer operations. So, ok, the Phi is useful at the current price, but at the 2000USD price other people are talking about in the thread it seems pretty useless.
- decwakeboarder 12y ago
  You also need n number of motherboards, RAM, power supplies, etc as you have CPUs. The Phi looks a lot better when you fit 5-6 in a server.
- awalton 12y ago
  Everyone in this thread keeps railing on its floating point perf, which is about on par with parts like nVidia's Tesla and similarly priced... but its integer performance is why people are really buying these cards up. All GPUs are floating point monsters but integer pansies because that's what their graphics workload heritage turned them into. Xeon Phi is a Computing Monster, with crazy Integer and Floating Point perf, because its heritage was "Let's try to make a graphics card out of CPUs" and not visa versa. It's similar to buying a whole stack of x86 chips just to get to the SSE/AVX units. Only, now you don't need a whole stack of computers, you need a handful of host computers and a stack of add-on cards. And if you think $2000 is expensive for 60 x86 cores (~$35/core) when all you care about is the SSE/AVX unit and how many vector ops you can cram through it, you're definitely not Intel's target.
  - nkurz 12y ago
    Thanks, you are right that I shouldn't have focussed on just floating point. I emphasized it because it better matched the frequently quoted "teraflop in a single card" measurement. As you say, the Phi is equally strong at vector integer operations as floating point, in both cases operating on 512-bit vectors. But I'll quibble with the assertion that all GPU's are "integer pansies", although it's a fantastic phrase. Historically, NVidia GPU's are much stronger at floating point operations than integer because of their graphics origins, and NVidia's Tesla is definitely what Intel views as nearest to the Phi's target market. But AMD/ATI cards have excellent vector integer performance, several times that of NVidia/Tesla, and much better per dollar than Phi until this recent price drop. This is why AMD cards were the best choice for bitcoin mining until the recent arrival of dedicated ASIC's: http://www.extremetech.com/computing/153467-amd-destroys-nvidia-bitcoin-mining http://www.extremetech.com/computing/153467-amd-destroys-nvi...

Coding_Cat 12y ago

It is my understanding that it sits in between a GPU and a CPU in terms of "ideal work". Like a GPU it favours SIMD work, but it is better at dealing with branches and a lot of random memory accesses[1] (it still favours linear work of course). It also uses a more familiar architecture, leading Intel to claim it is easier to master than GPU-programing, but I personally disagree as x64 is already much more complex than GPUs. In terms of FLOPS the FX-9590 is still significantly slower. I don't know the exact numbers, but it will be hundreds of MegaFlops, whereas the Phi runs at ~1 TFlop. In part because the cores are basically low-end (in-order) atoms with modules bolted on for HPC (avx-512 instructions, instructions for exponentiation for example). Data transfer to the card is a lot slower than reading from ram, correct, but on the other hand once it is on the card it is a lot faster (wiki says the fastest Phi has 350 Gb/s, Corsair claims a maximum of 70 Gb/s for DDR4)[2]. So generally, as long as you have a decent ratio of work/memory access and your work is parallel you could use a Phi to speed things a long. You would want to use one in the same situations as were you'd want to use a GPU, and the Phi would be preferable if your computations and memory access patterns are nor perfectly homogeneous (e.g. branching which can't be rewritten), as this kills performance on GPUs. [1] I'm pretty sure Intel claimed this, and it makes sense when you think about it, but I can't seem to find a source: Could someone confirm this? [2] Do note that it 350Gb/s shared between the cores, not 350 Gb/s each.

nkurz 12y ago
as x64 is already much more complex than GPUs I'd be interested to hear about your experiences. By "more complex", do you mean the x64 instruction set itself, or the out-of-order superscalar processors that use it?
- Coding_Cat 12y ago
  Basically the whole package. The instruction set is pretty big, but this is not much of an issue as you generally just need a subset for a given problem (I generally only tweak SSE/AVX code) and it has a lot of if's and but's when it comes to performance, the out-of-order execution also adds a degree of uncertainty, branch prediction and prefetching, a complex(ish) cache... There is a lot of things one can fiddle with in order to improve performance. GPUs are much stricter in their design, which can lead to some headaches when having to completely change the way you formulate a problem, but once you're past that step, the simplicity of the architecture makes it a lot nicer to optimize in my opinion. (And my personal experience has been that the tools are cheaper (free) and more stable as well, but your mileage may vary).

jandrewrogers 12y ago

The Xeon Phi uses a different computing architecture than either a CPU or a GPU so some intuitions about using it will be off. The Phi is essentially a modern barrel processor that uses the AMD64 ISA (it does not understand legacy x86 modes), a really nice vector ALU implementation, and memory bandwidth that is more like a GPU than a CPU. While it will run normal 64-bit software reasonably well with no special considerations, it will not be efficient without tweaking code design. CPU-targeted code attempts to optimize IPC in a single thread; barrel processors are designed to hide latency and can't drive IPC through single thread optimization. The reason barrel processors are interesting is that they can be incredibly efficient with their clock cycles. Unlike either CPUs or GPUs, it is relatively easy to get sustained throughput that approaches the theoretical IPC of the silicon for diverse software. The Xeon Phi mentioned in the article has 114 ALUs; it is possible to ensure all of those ALUs are doing useful work every single clock cycle, unlike the much smaller number of ALUs in your CPU. CPUs and GPUs have higher theoretical throughput in some cases but various parts of their ALUs typically spend a significant part of their time idle. Contrary to marketing, you do not want to program these like an ordinary CPU even though the cores are truly general purpose (unlike a GPU). Thread behavior is unlike CPUs or GPUs. Barrel processors cannot saturate a single core with a single thread! The Xeon Phi has 228 independent threads and you need to use them all the time. The way barrel processors work is if the hardware supports N threads then each clock cycle you can saturate all the ALUs if some subset M of those threads are not stalled. The M-of-N ratio varies by barrel processor design but is typically 20-50% in my experience. Each clock cycle, a core selects an immediately runnable operation from the basket of threads it can see and executes it; as long as something is runnable in that basket, the core will do real work that clock cycle. Xeon Phi has a 50% M-of-N requirement, so you need a minimum of 114 threads that are not stalled every clock cycle to saturate the processor. The way you ensure that you hit the 114 threshold is to schedule all 228 hardware thread slots with useful work. For programming, this changes the way you reason about locality and concurrency. If we assume that some percentage of threads can be safely stalled or blocked with no impact on throughput then it changes the way you design your algorithms and data structures. A little additional latency on a subset of threads won't hurt performance, especially if it increases task concurrency. On a CPU stalled threads are expensive, as it leads to idle cores or context switches. For architectures like Phi, you design your data structures and algorithms around relatively small, semi-independently work units so that there is always a large number of tasks that can be assigned to a thread even though this reduces locality. Below a certain threshold, thread concurrency is approximately free because the cores will schedule around stalls due to contention. I like barrel processors quite a lot. Once you get used to the model, it is an easier architecture with which to achieve efficient massively threaded parallelism and throughput than either CPUs or GPUs. More importantly, they are hard to beat for efficiency for general purpose computing when software is designed for the architecture since so few clock cycles are wasted.