7 ms·

1. Eliminate cache coherency protocols (replacing it with cache manipulation/inter-CPU communication instructions) 2. Eliminate virtual memory (replacing it wi

by catern 10y ago

1. Eliminate cache coherency protocols (replacing it with cache manipulation/inter-CPU communication instructions) 2. Eliminate virtual memory (replacing it with nothing) I'm not a CPU designer, but my understanding is that removing features allows for a denser/faster CPU. Well, these are two features that a suitably high-level language has no need for, because a high-level language doesn't expose "memory" to the programmer. Edit: Though, I 100% agree with what I believe is the core point of the author. We should not implement high-level features in hardware. In fact we should implement as little as possible in hardware, moving as much as possible into software. If Intel would let third parties generate microcode for their CPUs, we could move a lot further in that direction...

Symmetry 10y ago
Usually replacing software with dedicated hardware tends to speed things up and I don't think it's obvious that using explicit communication would actually speed things up. Most memory operations don't require any form of sharing and having the sharing that's required happen automatically seems efficient. Getting rid of virtual memory is potentially a big win, especially for architectures where you can't make the L1D cache virtually indexed but physically tagged. And in general there are a lot of special cases you don't even have to think about if different memory addresses can't alias to the same memory. You do lose out on a lot of software tricks there, though.
- chrisseaton 10y ago
  > Most memory operations don't require any form of sharing Maybe that was the point? Most of the time it isn't needed, but you are still paying for the logic to detect when it does, and you are paying for false sharing when the automagic gets it wrong.
dman 10y ago
Finally I run into someone who shares my exact pet peeves!
_yosefk 10y ago
#2 saves a little but not much and precludes unsafe low-level code completely. #1 I think impairs many HLLs, certainly every multithreaded imperative shared memory ones (and IMO nothing is close to these in terms of efficiency on multicore); which languages can work well without cache coherence? (I worked in C++ on multicore with no hw coherence btw. Quite the cruel and unusual punishment.)
- catern 10y ago
  >#2 saves a little but not much and precludes unsafe low-level code completely. Sure, in the same way most people today don't run their entire application in ring 0, but instead run on top of a kernel. Privilege levels like that are another thing that high level languages don't need. A compiler or JIT can generate low-level code just fine. >certainly every multithreaded imperative shared memory ones (and IMO nothing is close to these in terms of efficiency on multicore) You're begging the question here. Why is shared memory multithreading the fastest way available? It's because cache coherence is heavily optimized in hardware. But, of course, cache coherence is built on shared-nothing message-passing under the hood. So why not just use message-passing directly? Plenty of languages where that's the native primitive.
  - Rusky 10y ago
    > cache coherence is built on shared-nothing message-passing under the hood Only sometimes, and only for nearer caches. Data still often gets shared by both sources reading and writing a shared memory location (whether a farther-out cache or main memory or disk).
  - _yosefk 10y ago
    It is not obvious to me that a compiler or JITter can generate low level code that is (A) free of bugs (compilers are notoriously buggy, as in hundreds of real bugs found by a single fuzzer like CSmith) and I don't want those bugs to hose my filesystem as it could on DOS machines, and (B) always as efficient as lower-level code. MS for instance did interesting experiments with removing virtual memory and I'm not saying it's necessarily a bad idea, just that it saves you very little and the real-world risks are higher than the theoretical risks. Shared memory multithreading is fastest because explicitly communicating all of your tasks' inputs and outputs without any caching will transfer more data than using caches where several tasks can read the same thing from the same place. Cache coherence is not only built on shared nothing-message-passing under the hood, it's built on caches. If you cache your messages/accesses/whatever somewhere, you'll have coherence problems, if you don't, you'll have efficiency problems. Just because you have a language with a native primitive doesn't mean this language doesn't throw efficiency out the window by building things around this primitive. Say, plenty of languages have cons cells for all the wrong reasons (I think Clojure is the one language that escaped this in its lineage) and still cons sells are fundamentally inefficient no matter what you do in hardware, this part I elaborated on in TFA. Just like having a language with a primitive solving the halting problem doesn't solve the halting problem, you can't have a language with arbitrary primitives and a magic machine making the sw+hw system as efficient as any alternative.
mycall 10y ago
> If Intel would let third parties generate microcode for their CPUs... I can see Intel add Altera FPGA features into some future generation CPUs.
trsohmers 10y ago
Boy, do I have a new processor architecture for you. For the past 3 years my company (REX Computing), has been working on a new architecture that not only covers both of your points, but removes hardware managed caching entirely and replaces it with software managed (physically addressed) scratchpad memory. By removing all of the unnecessary logic (MMU, TLB, everything required for virtual addressing, prefetching, coherency, etc) associated with the onchip memory system, we can fit more memory on the chip itself, have lower latency to that memory, and use a lot less power. As we fully expose the memory system to our software tools, we can make very good decisions at compile time that replicate most of the features you get in having a hardware managed caching system. We have silicon back, and are now working with early customers in showing a 10 to 25x energy efficiency improvement for high performance workloads. I gave a talk at Stanford last week that covers the hardware architecture and shows off our development hardware: https://www.youtube.com/watch?v=ki6jVXZM2XU https://www.youtube.com/watch?v=ki6jVXZM2XU
- i336_ 10y ago
  Wow. (Not op) Definitely watching the video later when I have time. Thoughts from my initial impression: 1. Please put benchmarks up on HN when you can :) 2. Would be awesome if you could get this into the hands of low level kernel developers who can tinker with this and see what Linux is like on it - if it's capable of doing that much? (Need to watch the video!) 3. Conservativism FTW (IMO) when thinking about offers [that come in after you've done (1) :P]
  - trsohmers 10y ago
    1: Around the 30:40 mark in the video I transition to the live hardware demo, where I learned to never do live demos. While it did not work there in person, I do show pictures from it working earlier in the day along with the results from our 1024-point in place FFT assembly test. It was written and tested Monday and Tuesday of last week (finished the night before the presentation), so it is not fully optimized, but we are getting a very good 25 double precision GFLOPs/watt. As a comparison, Intel and NVIDIA's advertised FP64 numbers are around 8 to 12 GFLOPs/watt while they are on a more advanced process node (They are on 14/16nm, while our test silicon was on 28nm). We're in the process of getting cross platform benchmarks (HPL, HPCG, FFTW, Coremark, etc) up and running, but I'm hoping we'll be posting results shortly. 2. Since Linux 4.2, there has been a mainline port for STM32 which does not have a MMU, so porting linux is technically possible, though not on our priority list. Something like uCLinux would probably be easier, but not as useful. I have no doubt our cores would be able to handle it, it is just that our current customers already expect their software to be running on bare metal.
hvidgaard 10y ago
Re. 2, it only makes sense for specialized workloads. Doing away with it, means the OS needs to keep track of who are allowed to access what memory, and that carries a far greater performance penalty.
pjc50 10y ago
> Eliminate virtual memory (replacing it with nothing) That would require all code running on the system to be run through a security verifier. The only non-research language that's even attempted this at the system is Java. And you can get Java bytecode smartcards that do this. It also precludes running Windows or Linux on it, reducing your market to the embedded market only straight out of the gate.
- qznc 10y ago
  You could use the bus (NoC) for isolation instead of the MMU. For example, take away the capability to access memory from a core, while others can still communicate with it somehow.

by catern 10y ago

Symmetry 10y ago

Usually replacing software with dedicated hardware tends to speed things up and I don't think it's obvious that using explicit communication would actually speed things up. Most memory operations don't require any form of sharing and having the sharing that's required happen automatically seems efficient. Getting rid of virtual memory is potentially a big win, especially for architectures where you can't make the L1D cache virtually indexed but physically tagged. And in general there are a lot of special cases you don't even have to think about if different memory addresses can't alias to the same memory. You do lose out on a lot of software tricks there, though.

chrisseaton 10y ago
> Most memory operations don't require any form of sharing Maybe that was the point? Most of the time it isn't needed, but you are still paying for the logic to detect when it does, and you are paying for false sharing when the automagic gets it wrong.

dman 10y ago

Finally I run into someone who shares my exact pet peeves!

_yosefk 10y ago

#2 saves a little but not much and precludes unsafe low-level code completely. #1 I think impairs many HLLs, certainly every multithreaded imperative shared memory ones (and IMO nothing is close to these in terms of efficiency on multicore); which languages can work well without cache coherence? (I worked in C++ on multicore with no hw coherence btw. Quite the cruel and unusual punishment.)

catern 10y ago
>#2 saves a little but not much and precludes unsafe low-level code completely. Sure, in the same way most people today don't run their entire application in ring 0, but instead run on top of a kernel. Privilege levels like that are another thing that high level languages don't need. A compiler or JIT can generate low-level code just fine. >certainly every multithreaded imperative shared memory ones (and IMO nothing is close to these in terms of efficiency on multicore) You're begging the question here. Why is shared memory multithreading the fastest way available? It's because cache coherence is heavily optimized in hardware. But, of course, cache coherence is built on shared-nothing message-passing under the hood. So why not just use message-passing directly? Plenty of languages where that's the native primitive.
- Rusky 10y ago
  > cache coherence is built on shared-nothing message-passing under the hood Only sometimes, and only for nearer caches. Data still often gets shared by both sources reading and writing a shared memory location (whether a farther-out cache or main memory or disk).
- _yosefk 10y ago
  It is not obvious to me that a compiler or JITter can generate low level code that is (A) free of bugs (compilers are notoriously buggy, as in hundreds of real bugs found by a single fuzzer like CSmith) and I don't want those bugs to hose my filesystem as it could on DOS machines, and (B) always as efficient as lower-level code. MS for instance did interesting experiments with removing virtual memory and I'm not saying it's necessarily a bad idea, just that it saves you very little and the real-world risks are higher than the theoretical risks. Shared memory multithreading is fastest because explicitly communicating all of your tasks' inputs and outputs without any caching will transfer more data than using caches where several tasks can read the same thing from the same place. Cache coherence is not only built on shared nothing-message-passing under the hood, it's built on caches. If you cache your messages/accesses/whatever somewhere, you'll have coherence problems, if you don't, you'll have efficiency problems. Just because you have a language with a native primitive doesn't mean this language doesn't throw efficiency out the window by building things around this primitive. Say, plenty of languages have cons cells for all the wrong reasons (I think Clojure is the one language that escaped this in its lineage) and still cons sells are fundamentally inefficient no matter what you do in hardware, this part I elaborated on in TFA. Just like having a language with a primitive solving the halting problem doesn't solve the halting problem, you can't have a language with arbitrary primitives and a magic machine making the sw+hw system as efficient as any alternative.

mycall 10y ago

> If Intel would let third parties generate microcode for their CPUs... I can see Intel add Altera FPGA features into some future generation CPUs.

trsohmers 10y ago

Boy, do I have a new processor architecture for you. For the past 3 years my company (REX Computing), has been working on a new architecture that not only covers both of your points, but removes hardware managed caching entirely and replaces it with software managed (physically addressed) scratchpad memory. By removing all of the unnecessary logic (MMU, TLB, everything required for virtual addressing, prefetching, coherency, etc) associated with the onchip memory system, we can fit more memory on the chip itself, have lower latency to that memory, and use a lot less power. As we fully expose the memory system to our software tools, we can make very good decisions at compile time that replicate most of the features you get in having a hardware managed caching system. We have silicon back, and are now working with early customers in showing a 10 to 25x energy efficiency improvement for high performance workloads. I gave a talk at Stanford last week that covers the hardware architecture and shows off our development hardware: https://www.youtube.com/watch?v=ki6jVXZM2XU https://www.youtube.com/watch?v=ki6jVXZM2XU

i336_ 10y ago
Wow. (Not op) Definitely watching the video later when I have time. Thoughts from my initial impression: 1. Please put benchmarks up on HN when you can :) 2. Would be awesome if you could get this into the hands of low level kernel developers who can tinker with this and see what Linux is like on it - if it's capable of doing that much? (Need to watch the video!) 3. Conservativism FTW (IMO) when thinking about offers [that come in after you've done (1) :P]
- trsohmers 10y ago
  1: Around the 30:40 mark in the video I transition to the live hardware demo, where I learned to never do live demos. While it did not work there in person, I do show pictures from it working earlier in the day along with the results from our 1024-point in place FFT assembly test. It was written and tested Monday and Tuesday of last week (finished the night before the presentation), so it is not fully optimized, but we are getting a very good 25 double precision GFLOPs/watt. As a comparison, Intel and NVIDIA's advertised FP64 numbers are around 8 to 12 GFLOPs/watt while they are on a more advanced process node (They are on 14/16nm, while our test silicon was on 28nm). We're in the process of getting cross platform benchmarks (HPL, HPCG, FFTW, Coremark, etc) up and running, but I'm hoping we'll be posting results shortly. 2. Since Linux 4.2, there has been a mainline port for STM32 which does not have a MMU, so porting linux is technically possible, though not on our priority list. Something like uCLinux would probably be easier, but not as useful. I have no doubt our cores would be able to handle it, it is just that our current customers already expect their software to be running on bare metal.

hvidgaard 10y ago

Re. 2, it only makes sense for specialized workloads. Doing away with it, means the OS needs to keep track of who are allowed to access what memory, and that carries a far greater performance penalty.

pjc50 10y ago

> Eliminate virtual memory (replacing it with nothing) That would require all code running on the system to be run through a security verifier. The only non-research language that's even attempted this at the system is Java. And you can get Java bytecode smartcards that do this. It also precludes running Windows or Linux on it, reducing your market to the embedded market only straight out of the gate.

qznc 10y ago
You could use the bus (NoC) for isolation instead of the MMU. For example, take away the capability to access memory from a core, while others can still communicate with it somehow.