9 ms·

What's New in CPUs Since the 80s and How Does It Affect Programmers?

pjmlp 12y ago
All features that make C look high level nowadays, contrary to what many think. None of them are exposed on ANSI C.
- agumonkey 12y ago
  Are there any 'finished' language giving access to the whole machinery ?
  - pjmlp 12y ago
    As far as I know, no. Hence why even if C looks like a portable high level Assembler, there are many modern CPU features not available. Even most common extensions just cover part of them.
    - selckin 12y ago
      So who uses these features then?
      - agumonkey 12y ago
        Wild guess: libs (intel cilk?), compiler intrisics through reified assembly.
      - bhouston 12y ago
        I've used a lot of SSE2 and prefetch instincs in Visual C++ for math / data heavy high performance computing applications. It can make a fairly big different -- like more than 50% performance improvements for key parts of the code.
        vardump 12y ago
        That depends heavily on with CPU and memory subsystem combination you're running it on. You need to know how far to prefetch and how to order your data in memory.
      - kabdib 12y ago
        Games on consoles, where the hardware architecture is frozen and you can optimize to the metal all you want. Not all titles do this, since most titles are cross-platform and you're not going to do tons of optimization for a specific one unless there is a huge payoff.
        tarpherder 12y ago
        It is done on PC as well, but we can't assume very recent instruction sets. We'll have to do with semi recent instruction sets.
      - TallGuyShort 12y ago
        Specialized HPC workloads
      - lgeek 12y ago
        Many of the capabilities described in the article are just hardware features which don't need any control from application developers, but their effectiveness can be improved if the developers are aware of them (e.g. caches, TLBs). Others properties are exploited automatically by compilers (e.g. instruction scheduling for improving throughput in a superscalar processor) Instruction set extensions (which I believe might be what you were thinking of) are used via intrinsics and assembly programming by pretty much any performance-oriented code. For example look at the dozens of architecture-specific implementations of glibc functions: x86-64[0], arm[1]. [0] https://sourceware.org/git/?p=glibc.git;a=tree;f=sysdeps/x86_64/multiarch https://sourceware.org/git/?p=glibc.git;a=tree;f=sysdeps/x86... [1] https://sourceware.org/git/?p=glibc.git;a=tree;f=sysdeps/arm/armv7/multiarch https://sourceware.org/git/?p=glibc.git;a=tree;f=sysdeps/arm...
    - bhouston 12y ago
      > Even most common extensions just cover part of them. The headers I linked to here: https://news.ycombinator.com/item?id=8873764 https://news.ycombinator.com/item?id=8873764 Cover most useful Intel CPU extensions. And you can use them easily in code, it just sucks when you have to do CPU feature detection and have different code paths for different CPUs.
      - bluetomcat 12y ago
        > it just sucks when you have to do CPU feature detection and have different code paths for different CPUs. In the Linux kernel they went so far as to actually replacing instructions at boot-time at known locations, based on the capabilities of the CPU the code is executing on. This prevents them from maintaining many paths in the compiled code.
      - pjmlp 12y ago
        Yes, but as I answered they are compiler specific and cover only part of the CPU.
- bluetomcat 12y ago
  ANSI C doesn't expose any intrinsics, but once you get used to GCC extensions and attributes, it starts to feel a lot more "low-level". Think of __builtin_expect, __builtin_prefetch, alignment attributes, etc.
  - pjmlp 12y ago
    That is compiler specific without any portability guarantee to another compiler, you can just write Assembly instead, for what is worth.
    - bluetomcat 12y ago
      It's not quite like writing assembly because these intrinsics are meant to translate to the "best" instructions for the particular ISA that the code is targeted for. You are able to compile the same chunk of code for two different architectures, for example. If you were to write inline assembly, you would be forced to have a separate chunk of code for each architecture and surround them with #ifdefs.
      - pjmlp 12y ago
        Assuming the same compiler is used.
    - falcolas 12y ago
      The code can be made portable with a few simple macros (which is why they are implemented they way they are in GCC), though the optimizations will not be. From the Linux kernel: struct boot_params boot_params __attribute__((aligned(16))); #if !__GNUC__ #define __attribute__(x) #endif I think that's a better way to do it than ASM (which totally hoses your portability).
      - pjmlp 12y ago
        For alignment yes, but that is not the case across all extensions. Some of them are no different than just writing plain Assembly.
- bhouston 12y ago
  Well, you can still use them in C/C++ code via: #include <mmintrin.h> // Intel MMX #include <xmmintrin.h> // Intel SSE #include <emmintrin.h> // Intel SSE2 #include <pmmintrin.h> // Intel SSE3 #include <tmmintrin.h> // Intel SSSE3 #include <smmintrin.h> // Intel SSE4.1 #include <nmmintrin.h> // Intel SSE4.2 #include <ammintrin.h> // Intel SSE4A #include <wmmintrin.h> // Intel AES #include <immintrin.h> // Intel AVX #include <zmmintrin.h> // Intel AVX-512 #include <arm_neon.h> // ARM NEON #include <mmintrin.h> // ARM WMMX And a full list of what is possible in Microsoft Visual C: http://msdn.microsoft.com/en-us/library/hh977022.aspx http://msdn.microsoft.com/en-us/library/hh977022.aspx Now the reason why they are not in ANSI C is that low-level CPU features (just like raw instructions) are not portable by nature.
  - pjmlp 12y ago
    Where are the headers for out-of-order execution, cache levels, execution pipelines, ...? Additionally not all C compilers provide such headers.
    - bluetomcat 12y ago
      Even if one programs in assembly, such features are a "black box" because instruction sets rarely provide any instructions for controlling these things, apart from issuing cache prefetching hints which are not mandatory for the CPU to honour.
    - bhouston 12y ago
      There are a lot of prefetch instructions and write-barriers in the available intrinsics: http://msdn.microsoft.com/en-us/library/hh977022.aspx http://msdn.microsoft.com/en-us/library/hh977022.aspx I do not think one has full control over OOO execution on x86-64 processors. Also I do not believe one has control over the execution pipeline even in assembly, although I do not know exactly what you mean by that, so it could just be a misunderstanding.
      - pjmlp 12y ago
        My whole point is that this isn't C any longer, it is Assembly.
acallan 12y ago
The author is incorrect in the section about memory fences. x86 has strong memory ordering [1], which means that writes always appear in program order with respect to other cores. Use a memory fence to guarantee that reads and writes are memory bus visible. The example that the author gives does not apply to x86. [1]There are memory types that do not have strong memory ordering, and if you use non-temporal instructions for streaming SIMD, SFENCE/LFENCE/MFENCE are useful.
- martincmartin 12y ago
  The authors point still holds if writes might not appear in program order on other CPU sockets, not just other cores within the same socket. Do you have a reference for the strong memory ordering on x86? I'd like to read more about it.
  - rayiner 12y ago
    x86 enforces (essentially) total store order across all sockets: http://www.cl.cam.ac.uk/~pes20/weakmemory/index3.html http://www.cl.cam.ac.uk/~pes20/weakmemory/index3.html. The barriers are still useful for kernel code because other processors on the machine usually don't participate in the cache-coherency protocol.
- Animats 12y ago
  Modern X86 machines do not quite have strong memory ordering. Loads may be reordered with older stores to different locations. This breaks some of the overly clever "lock free" algorithms: http://bartoszmilewski.com/2008/11/05/who-ordered-memory-fences-on-an-x86/ http://bartoszmilewski.com/2008/11/05/who-ordered-memory-fen... http://preshing.com/20120515/memory-reordering-caught-in-the-act/ http://preshing.com/20120515/memory-reordering-caught-in-the... This whole area is very touchy and easy to get wrong. Further reading: https://software.intel.com/en-us/articles/tsx-anti-patterns-in-lock-elision-code https://software.intel.com/en-us/articles/tsx-anti-patterns-...
majke 12y ago
Another exciting thing with the new PCI-Express standard is the Direct Cache Access, especially useful for high speed networking: http://web.stanford.edu/group/comparch/papers/huggahalli05.pdf http://web.stanford.edu/group/comparch/papers/huggahalli05.p...
- donavanm 12y ago
  Speaking of, I think we'll see nvme being the new hotness this year. NVME is going to bring crazy parallelism and queue depth to block devices. Finally we'll be able to exploit flashs inherent parallel access in a sane way. http://en.m.wikipedia.org/wiki/NVM_Express http://en.m.wikipedia.org/wiki/NVM_Express
  - ksk 12y ago
    Anandtech already did a review.. http://www.anandtech.com/show/8147/the-intel-ssd-dc-p3700-review-part-2-nvme-on-client-workloads/2 http://www.anandtech.com/show/8147/the-intel-ssd-dc-p3700-re...
bluetomcat 12y ago
The constant widening of the SIMD registers is an interesting trend to watch. MMX started off as 64-bit, and now AVX-512 is 8 times that. With many cores and a couple of SIMD units in each core, aren't CPUs becoming ever more suitable to handle graphic workloads that were once reasonable only for GPUs?
- deleted 12y ago
  [deleted]
- vardump 12y ago
  GPUs are still about 5x ahead compared to CPUs. CPUs lack texture lookup hardware, that's the biggest catch right now. AVX2 gives gather support (one instruction, up to 8x independent loads to a single vector register), but currently on Haswell there's no performance benefit compared to doing the memory loads individually. GPU memory (GDDR5) has also many times more bandwidth at expense of 10x higher latency compared to CPU. Currently you put the workload that requires low latency on a CPU and embarrassingly parallel "non-branchy" workload on a GPU. Both CPUs and GPUs gain about 20% performance per year per core. I'm no expert on CPU and GPU memory architectures, but both seem to be headed towards integrated 3D-stacked eDRAM memory. This can provide up to several TBps reasonably low latency bandwidth.
- Symmetry 12y ago
  Everything vardump said is true, but additionally there's the consideration of what isn't there as well as what is there. By not having an out of order execution setup or a broad forwarding network a GPU core is able to be much smaller than a CPU core of equivalent throughput. By "core" here I mean something capable of independently issuing instructions and executing them, not calling each execution unit a "core" the way Nvidia does (though with SIMT that isn't as crazy as it might look at first). That means you can pack more cores on a chip and use less power for a given amount of work, since power consumption is roughly proportional to the number of transistors used. If we eventually start doing graphics on CPUs it will because we move to ray tracing or other sorts of algorithms that GPUs are bad at.
  - 12y ago
avian 12y ago
> If we set _foo to 0 and have two threads that both execute incl (_foo) 10000 times each, incrementing the same location with a single instruction 20000 times, is guaranteed not to exceed 20000, but it could (theoretically) be as low as 2. Can someone explain the scenario where this test would result in _foo = 2? The lowest theoretical value I can understand is foo_ = 10000 (all 10000 incls from thread 1 are executed between one pair of load and store in thread 2 and hence lost).
- ColinDabritz 12y ago
  You're almost there! Basically that swapping you outlined occurs twice, once on each side. 1. Both threads read '0', beginning their first incl instruction (A: 0, B: 0, Store: 0) 2. Thread A completes execution of all BUT ONE incl, writing it's second to last value to the store (A: 9999, B: 0, Store: 9999) 3. Thread B increments it's '0' to '1' (A: 9999, B: 1, Store: 9999) 4. Thread B writes it's '1' to the store (A: 9999, B: 1, Store: 1) 5. Thread A begins it's final incl and reads the '1' that thread B just stored (A: 1, B: 1, Store: 1) 6. Thread B executes all remaining incl instructions, writing its final value to the store (A: 1, B:10000, Store: 10000) 7. Thread A continues it's final incl and increments it's '1' to '2' (A: 2, B:10000, Store: 10000) 8. Thread A stores it's final '2' result (A: 2, B:10000, Store: 2) 9. All instructions have executed, and the result is '2' It is, of course, completely theory and extremely unlikely under real use as the authors graph shows, but it's one of those threading 'gotchas' that lead to occasionally unpredictable results.
pedrocr 12y ago
>If we set _foo to 0 and have two threads that both execute incl (_foo) 10000 times each, incrementing the same location with a single instruction 20000 times, is guaranteed not to exceed 20000, but it could (theoretically) be as low as 2. Initially it seemed to me the theoretical minimum should be 10000 (as the practical minimum seems to be). But it does indeed seem possible to get 2: 1) Both threads load 0 2) Thread 1 increments 9999 times and stores 3) Thread 2 increments and stores 1 5) Both threads load 1 5) Thread 2 increments 9999 times and stores 6) Thread 1 increments and stores 2 Is there anything in the x86 coherency setup that would disallow this and make 10000 the actual minimum?
- JoshTriplett 12y ago
  Unless you use a "lock inc", increment is semantically a read-modify-write, so I believe those steps can technically happen on x86. However, since a thread cannot be interrupted in the middle of an instruction, it's very unlikely that 9999 operations can happen in one thread while another executes an increment. If you wrote it as a mov, an add, and another mov, all it would take is a context switch at exactly the right time. On the other hand, 20000 is unlikely as well, if the threads have any concurrency at all.
- logfromblammo 12y ago
  I think not. The only limit seems to be the number of times one CPU can increment while the other is between load and store operations. If you could manipulate your two cores with enough dexterity, you could force a 2 result, but without enough fine-grained control, the practical limit is going to be determined by the number of increments that can be "wasted" by sandwiching them between the load and store operations of the other thread. You would essentially need to stop and start each CPU at four very specific operations. The practical results show that "load + add + store + load + add + store" on one thread probably never happens during a single "add" on the other thread. You would need that to happen at least once to get below 10000. Otherwise, each increment can waste no more than one increment for the other thread, and you end up with at least 10000. The experimental numbers are probably indicative of how long the add portion of INCL takes in relation to the whole thing.
martincmartin 12y ago
In the section on rdtsc, the author should really mention that there's a new, serializing version called rdtscp, so you should prefer that.
- uxcn 12y ago
  rdtscp also consumes an extra register though.
gshrikant 12y ago
For a somewhat dated read for an introductory/high-level survey of the state of CPU design up to 2001, I find [1] to be quite informative. Of course, having been written in the frequency scaling era some of the 'predictions' are way off the mark. Nevertheless, I find it a good resource for someone looking to get a bird's eye view of the 30+ years development in the field. [1] http://www.ece.cmu.edu/~ece447/s13/lib/exe/fetch.php?media=00964437.pdf http://www.ece.cmu.edu/~ece447/s13/lib/exe/fetch.php?media=0...
antiuniverse 12y ago
If you enjoyed this article, you might also want to check out Agner Fog's optimization manuals and blog (which I decided were probably worth a separate submission): https://news.ycombinator.com/item?id=8874206 https://news.ycombinator.com/item?id=8874206
mopo2000 12y ago
Simd, speculative execution, caches and virtualization in cpus date back to the 60s. See IBM Stretch, Illiac, CP/CMS etc. See eg http://people.cs.clemson.edu/~mark/stretch.html http://people.cs.clemson.edu/~mark/stretch.html
uxcn 12y ago
One of the other weird things is that the general purpose chips are supposed to be getting large chunks of memory behind the northbridge. If I'm not mistaken, it's supposed to be used as another level in the cache hierarchy, which could be another boon to things that leverage concurrency.
zurn 12y ago
These features debuted way before the 80s, arguably with the exception of out of order execution. Take any feature discussed and its wikipedia history section will tell you about CPUs and computers in the 60s that first had them. Caches, SIMD (called vectors back then), speculative execution, branch prediction, virtual machines, virtual memory, accelerated IO bypassing the CPU... all from the 60s.
- pjmlp 12y ago
  Just like everything else in computing. We are still trying to catch up with the Xerox PARC model of live coding, for example. However just a selected few could touch said hardware and you could buy a very nice house with what they used to cost.
amelius 12y ago
Nice improvements. But they seem only marginal. Yes, speed may have gone up by a certain factor (which has remained surprisingly stable in the last decade, or so it seems). On the other hand, programming complexity has gone way up, while predictability of performance has been lost.
cowardlydragon 12y ago
Doesn't this come down to: #1: Fit in cache. #2: Try to multithread all the rest is marginal

pjmlp 12y ago

All features that make C look high level nowadays, contrary to what many think. None of them are exposed on ANSI C.

agumonkey 12y ago
Are there any 'finished' language giving access to the whole machinery ?
- pjmlp 12y ago
  As far as I know, no. Hence why even if C looks like a portable high level Assembler, there are many modern CPU features not available. Even most common extensions just cover part of them.
  - selckin 12y ago
    So who uses these features then?
    - agumonkey 12y ago
      Wild guess: libs (intel cilk?), compiler intrisics through reified assembly.
    - bhouston 12y ago
      I've used a lot of SSE2 and prefetch instincs in Visual C++ for math / data heavy high performance computing applications. It can make a fairly big different -- like more than 50% performance improvements for key parts of the code.
      - vardump 12y ago
        That depends heavily on with CPU and memory subsystem combination you're running it on. You need to know how far to prefetch and how to order your data in memory.
    - kabdib 12y ago
      Games on consoles, where the hardware architecture is frozen and you can optimize to the metal all you want. Not all titles do this, since most titles are cross-platform and you're not going to do tons of optimization for a specific one unless there is a huge payoff.
      - tarpherder 12y ago
        It is done on PC as well, but we can't assume very recent instruction sets. We'll have to do with semi recent instruction sets.
    - TallGuyShort 12y ago
      Specialized HPC workloads
    - lgeek 12y ago
      Many of the capabilities described in the article are just hardware features which don't need any control from application developers, but their effectiveness can be improved if the developers are aware of them (e.g. caches, TLBs). Others properties are exploited automatically by compilers (e.g. instruction scheduling for improving throughput in a superscalar processor) Instruction set extensions (which I believe might be what you were thinking of) are used via intrinsics and assembly programming by pretty much any performance-oriented code. For example look at the dozens of architecture-specific implementations of glibc functions: x86-64[0], arm[1]. [0] https://sourceware.org/git/?p=glibc.git;a=tree;f=sysdeps/x86_64/multiarch https://sourceware.org/git/?p=glibc.git;a=tree;f=sysdeps/x86... [1] https://sourceware.org/git/?p=glibc.git;a=tree;f=sysdeps/arm/armv7/multiarch https://sourceware.org/git/?p=glibc.git;a=tree;f=sysdeps/arm...
  - bhouston 12y ago
    > Even most common extensions just cover part of them. The headers I linked to here: https://news.ycombinator.com/item?id=8873764 https://news.ycombinator.com/item?id=8873764 Cover most useful Intel CPU extensions. And you can use them easily in code, it just sucks when you have to do CPU feature detection and have different code paths for different CPUs.
    - bluetomcat 12y ago
      > it just sucks when you have to do CPU feature detection and have different code paths for different CPUs. In the Linux kernel they went so far as to actually replacing instructions at boot-time at known locations, based on the capabilities of the CPU the code is executing on. This prevents them from maintaining many paths in the compiled code.
    - pjmlp 12y ago
      Yes, but as I answered they are compiler specific and cover only part of the CPU.
bluetomcat 12y ago
ANSI C doesn't expose any intrinsics, but once you get used to GCC extensions and attributes, it starts to feel a lot more "low-level". Think of __builtin_expect, __builtin_prefetch, alignment attributes, etc.
- pjmlp 12y ago
  That is compiler specific without any portability guarantee to another compiler, you can just write Assembly instead, for what is worth.
  - bluetomcat 12y ago
    It's not quite like writing assembly because these intrinsics are meant to translate to the "best" instructions for the particular ISA that the code is targeted for. You are able to compile the same chunk of code for two different architectures, for example. If you were to write inline assembly, you would be forced to have a separate chunk of code for each architecture and surround them with #ifdefs.
    - pjmlp 12y ago
      Assuming the same compiler is used.
  - falcolas 12y ago
    The code can be made portable with a few simple macros (which is why they are implemented they way they are in GCC), though the optimizations will not be. From the Linux kernel: struct boot_params boot_params __attribute__((aligned(16))); #if !__GNUC__ #define __attribute__(x) #endif I think that's a better way to do it than ASM (which totally hoses your portability).
    - pjmlp 12y ago
      For alignment yes, but that is not the case across all extensions. Some of them are no different than just writing plain Assembly.
bhouston 12y ago
Well, you can still use them in C/C++ code via: #include <mmintrin.h> // Intel MMX #include <xmmintrin.h> // Intel SSE #include <emmintrin.h> // Intel SSE2 #include <pmmintrin.h> // Intel SSE3 #include <tmmintrin.h> // Intel SSSE3 #include <smmintrin.h> // Intel SSE4.1 #include <nmmintrin.h> // Intel SSE4.2 #include <ammintrin.h> // Intel SSE4A #include <wmmintrin.h> // Intel AES #include <immintrin.h> // Intel AVX #include <zmmintrin.h> // Intel AVX-512 #include <arm_neon.h> // ARM NEON #include <mmintrin.h> // ARM WMMX And a full list of what is possible in Microsoft Visual C: http://msdn.microsoft.com/en-us/library/hh977022.aspx http://msdn.microsoft.com/en-us/library/hh977022.aspx Now the reason why they are not in ANSI C is that low-level CPU features (just like raw instructions) are not portable by nature.
- pjmlp 12y ago
  Where are the headers for out-of-order execution, cache levels, execution pipelines, ...? Additionally not all C compilers provide such headers.
  - bluetomcat 12y ago
    Even if one programs in assembly, such features are a "black box" because instruction sets rarely provide any instructions for controlling these things, apart from issuing cache prefetching hints which are not mandatory for the CPU to honour.
  - bhouston 12y ago
    There are a lot of prefetch instructions and write-barriers in the available intrinsics: http://msdn.microsoft.com/en-us/library/hh977022.aspx http://msdn.microsoft.com/en-us/library/hh977022.aspx I do not think one has full control over OOO execution on x86-64 processors. Also I do not believe one has control over the execution pipeline even in assembly, although I do not know exactly what you mean by that, so it could just be a misunderstanding.
    - pjmlp 12y ago
      My whole point is that this isn't C any longer, it is Assembly.

The author is incorrect in the section about memory fences. x86 has strong memory ordering [1], which means that writes always appear in program order with respect to other cores. Use a memory fence to guarantee that reads and writes are memory bus visible. The example that the author gives does not apply to x86. [1]There are memory types that do not have strong memory ordering, and if you use non-temporal instructions for streaming SIMD, SFENCE/LFENCE/MFENCE are useful.

martincmartin 12y ago
The authors point still holds if writes might not appear in program order on other CPU sockets, not just other cores within the same socket. Do you have a reference for the strong memory ordering on x86? I'd like to read more about it.
- rayiner 12y ago
  x86 enforces (essentially) total store order across all sockets: http://www.cl.cam.ac.uk/~pes20/weakmemory/index3.html http://www.cl.cam.ac.uk/~pes20/weakmemory/index3.html. The barriers are still useful for kernel code because other processors on the machine usually don't participate in the cache-coherency protocol.
Animats 12y ago
Modern X86 machines do not quite have strong memory ordering. Loads may be reordered with older stores to different locations. This breaks some of the overly clever "lock free" algorithms: http://bartoszmilewski.com/2008/11/05/who-ordered-memory-fences-on-an-x86/ http://bartoszmilewski.com/2008/11/05/who-ordered-memory-fen... http://preshing.com/20120515/memory-reordering-caught-in-the-act/ http://preshing.com/20120515/memory-reordering-caught-in-the... This whole area is very touchy and easy to get wrong. Further reading: https://software.intel.com/en-us/articles/tsx-anti-patterns-in-lock-elision-code https://software.intel.com/en-us/articles/tsx-anti-patterns-...

majke 12y ago

Another exciting thing with the new PCI-Express standard is the Direct Cache Access, especially useful for high speed networking: http://web.stanford.edu/group/comparch/papers/huggahalli05.pdf http://web.stanford.edu/group/comparch/papers/huggahalli05.p...

donavanm 12y ago
Speaking of, I think we'll see nvme being the new hotness this year. NVME is going to bring crazy parallelism and queue depth to block devices. Finally we'll be able to exploit flashs inherent parallel access in a sane way. http://en.m.wikipedia.org/wiki/NVM_Express http://en.m.wikipedia.org/wiki/NVM_Express
- ksk 12y ago
  Anandtech already did a review.. http://www.anandtech.com/show/8147/the-intel-ssd-dc-p3700-review-part-2-nvme-on-client-workloads/2 http://www.anandtech.com/show/8147/the-intel-ssd-dc-p3700-re...

bluetomcat 12y ago

The constant widening of the SIMD registers is an interesting trend to watch. MMX started off as 64-bit, and now AVX-512 is 8 times that. With many cores and a couple of SIMD units in each core, aren't CPUs becoming ever more suitable to handle graphic workloads that were once reasonable only for GPUs?

deleted 12y ago
[deleted]
vardump 12y ago
GPUs are still about 5x ahead compared to CPUs. CPUs lack texture lookup hardware, that's the biggest catch right now. AVX2 gives gather support (one instruction, up to 8x independent loads to a single vector register), but currently on Haswell there's no performance benefit compared to doing the memory loads individually. GPU memory (GDDR5) has also many times more bandwidth at expense of 10x higher latency compared to CPU. Currently you put the workload that requires low latency on a CPU and embarrassingly parallel "non-branchy" workload on a GPU. Both CPUs and GPUs gain about 20% performance per year per core. I'm no expert on CPU and GPU memory architectures, but both seem to be headed towards integrated 3D-stacked eDRAM memory. This can provide up to several TBps reasonably low latency bandwidth.
Symmetry 12y ago
Everything vardump said is true, but additionally there's the consideration of what isn't there as well as what is there. By not having an out of order execution setup or a broad forwarding network a GPU core is able to be much smaller than a CPU core of equivalent throughput. By "core" here I mean something capable of independently issuing instructions and executing them, not calling each execution unit a "core" the way Nvidia does (though with SIMT that isn't as crazy as it might look at first). That means you can pack more cores on a chip and use less power for a given amount of work, since power consumption is roughly proportional to the number of transistors used. If we eventually start doing graphics on CPUs it will because we move to ray tracing or other sorts of algorithms that GPUs are bad at.
- 12y ago

avian 12y ago

> If we set _foo to 0 and have two threads that both execute incl (_foo) 10000 times each, incrementing the same location with a single instruction 20000 times, is guaranteed not to exceed 20000, but it could (theoretically) be as low as 2. Can someone explain the scenario where this test would result in _foo = 2? The lowest theoretical value I can understand is foo_ = 10000 (all 10000 incls from thread 1 are executed between one pair of load and store in thread 2 and hence lost).

ColinDabritz 12y ago
You're almost there! Basically that swapping you outlined occurs twice, once on each side. 1. Both threads read '0', beginning their first incl instruction (A: 0, B: 0, Store: 0) 2. Thread A completes execution of all BUT ONE incl, writing it's second to last value to the store (A: 9999, B: 0, Store: 9999) 3. Thread B increments it's '0' to '1' (A: 9999, B: 1, Store: 9999) 4. Thread B writes it's '1' to the store (A: 9999, B: 1, Store: 1) 5. Thread A begins it's final incl and reads the '1' that thread B just stored (A: 1, B: 1, Store: 1) 6. Thread B executes all remaining incl instructions, writing its final value to the store (A: 1, B:10000, Store: 10000) 7. Thread A continues it's final incl and increments it's '1' to '2' (A: 2, B:10000, Store: 10000) 8. Thread A stores it's final '2' result (A: 2, B:10000, Store: 2) 9. All instructions have executed, and the result is '2' It is, of course, completely theory and extremely unlikely under real use as the authors graph shows, but it's one of those threading 'gotchas' that lead to occasionally unpredictable results.

pedrocr 12y ago

>If we set _foo to 0 and have two threads that both execute incl (_foo) 10000 times each, incrementing the same location with a single instruction 20000 times, is guaranteed not to exceed 20000, but it could (theoretically) be as low as 2. Initially it seemed to me the theoretical minimum should be 10000 (as the practical minimum seems to be). But it does indeed seem possible to get 2: 1) Both threads load 0 2) Thread 1 increments 9999 times and stores 3) Thread 2 increments and stores 1 5) Both threads load 1 5) Thread 2 increments 9999 times and stores 6) Thread 1 increments and stores 2 Is there anything in the x86 coherency setup that would disallow this and make 10000 the actual minimum?

JoshTriplett 12y ago
Unless you use a "lock inc", increment is semantically a read-modify-write, so I believe those steps can technically happen on x86. However, since a thread cannot be interrupted in the middle of an instruction, it's very unlikely that 9999 operations can happen in one thread while another executes an increment. If you wrote it as a mov, an add, and another mov, all it would take is a context switch at exactly the right time. On the other hand, 20000 is unlikely as well, if the threads have any concurrency at all.
logfromblammo 12y ago
I think not. The only limit seems to be the number of times one CPU can increment while the other is between load and store operations. If you could manipulate your two cores with enough dexterity, you could force a 2 result, but without enough fine-grained control, the practical limit is going to be determined by the number of increments that can be "wasted" by sandwiching them between the load and store operations of the other thread. You would essentially need to stop and start each CPU at four very specific operations. The practical results show that "load + add + store + load + add + store" on one thread probably never happens during a single "add" on the other thread. You would need that to happen at least once to get below 10000. Otherwise, each increment can waste no more than one increment for the other thread, and you end up with at least 10000. The experimental numbers are probably indicative of how long the add portion of INCL takes in relation to the whole thing.

martincmartin 12y ago

In the section on rdtsc, the author should really mention that there's a new, serializing version called rdtscp, so you should prefer that.

uxcn 12y ago
rdtscp also consumes an extra register though.

gshrikant 12y ago

For a somewhat dated read for an introductory/high-level survey of the state of CPU design up to 2001, I find [1] to be quite informative. Of course, having been written in the frequency scaling era some of the 'predictions' are way off the mark. Nevertheless, I find it a good resource for someone looking to get a bird's eye view of the 30+ years development in the field. [1] http://www.ece.cmu.edu/~ece447/s13/lib/exe/fetch.php?media=00964437.pdf http://www.ece.cmu.edu/~ece447/s13/lib/exe/fetch.php?media=0...

antiuniverse 12y ago

If you enjoyed this article, you might also want to check out Agner Fog's optimization manuals and blog (which I decided were probably worth a separate submission): https://news.ycombinator.com/item?id=8874206 https://news.ycombinator.com/item?id=8874206

mopo2000 12y ago

Simd, speculative execution, caches and virtualization in cpus date back to the 60s. See IBM Stretch, Illiac, CP/CMS etc. See eg http://people.cs.clemson.edu/~mark/stretch.html http://people.cs.clemson.edu/~mark/stretch.html

uxcn 12y ago

One of the other weird things is that the general purpose chips are supposed to be getting large chunks of memory behind the northbridge. If I'm not mistaken, it's supposed to be used as another level in the cache hierarchy, which could be another boon to things that leverage concurrency.

zurn 12y ago

These features debuted way before the 80s, arguably with the exception of out of order execution. Take any feature discussed and its wikipedia history section will tell you about CPUs and computers in the 60s that first had them. Caches, SIMD (called vectors back then), speculative execution, branch prediction, virtual machines, virtual memory, accelerated IO bypassing the CPU... all from the 60s.

pjmlp 12y ago
Just like everything else in computing. We are still trying to catch up with the Xerox PARC model of live coding, for example. However just a selected few could touch said hardware and you could buy a very nice house with what they used to cost.

amelius 12y ago

Nice improvements. But they seem only marginal. Yes, speed may have gone up by a certain factor (which has remained surprisingly stable in the last decade, or so it seems). On the other hand, programming complexity has gone way up, while predictability of performance has been lost.

cowardlydragon 12y ago

Doesn't this come down to: #1: Fit in cache. #2: Try to multithread all the rest is marginal