4 ms·

I'd say exploiting parallelism is not the only way at all. Parallelism is only one way to compute differently. Specialization of hardware to specific workloads

I'd say exploiting parallelism is not the only way at all. Parallelism is only one way to compute differently. Specialization of hardware to specific workloads will explode in the next years as we can't rely anymore on Moore's law. This will happen on RISC-V, IMHO. We already have these: * Rendering, medium precision mathematics: GPU * Low precision mathematics: TPU * Software Defined Networking: Microsoft is deploying FPGAs, AWS has its own hardware We could have: * Databases: Projections, hashing, sorting in hardware. * Dynamic runtimes: Hardware implemented memory models, HW assisted GC, code caches and user-level interrupts for the JITs. Here is the J extension RISC-V working group: [1] etc. Also, why not have the usual hot paths in Node.js|Spring Framework|Django directly etched into hardware? HW http header parsing surely could bring benefit to them all. ---- Of course language and programmers will have to adapt, but in a lot of cases the runtimes will take care of it automatically. [1] https://groups.google.com/a/groups.riscv.org/forum/#!msg/hw-dev/itbdrL_zdrc/t4B5fdbgAAAJ https://groups.google.com/a/groups.riscv.org/forum/#!msg/hw-...

microcolonel 9y ago
Currently working on software for RISC-V and I can definitely see some opportunities here. One thing that's becoming apparent though, is that some level of abstraction needs to be available in order for many of these things to become useful. RISC-V has a very exciting Vector processing extension, which is intended to replace packed SIMD in most cases, but some software systems assume packed SIMD is the only way to get more FP performance. For example, WebAssembly specifically exposes SIMD primitives, which means that it may be necessary to work backwards from those SIMD primitives to make use of a true vector machine. I think many people simply underestimate the cost of adopting a new programming model. > Also, why not have the usual hot paths in Node.js|Spring Framework|Django directly etched into hardware? HW http header parsing surely could bring benefit to them all. Well, in all the listed cases here, the CPU is not the bottleneck on throughput. As far as I can tell problem with HTTP is not that headers take too long to parse, it's that memory is still too slow, and context switches cost us precious time. The problem with Node is not that the hardware doesn't adequately model the semantics, it's that dynamic, weak typing makes it hard for any system (software or hardware) to understand what type things are. Update: The J extension seems interesting, and I've read some research (not thoroughly) recently showing considerable power and time savings from hardware GC primitives. I'm excited to see what goes on in that committee.
- BenoitP 9y ago
  > I'm excited to see what goes on in that committee. I am too. So far, this post on the general RISC-V mailing list and a few videos online talk about it. I'd love to have some other sources of information on their progress. Also, I've heard that the RISC-V foundation is actively seeking collaboration for Java. There is some work on having RISC-V backends in HotSpot and JikesRVM, but so far it is limited to interpretation IIRC. The fact that Oracle is not jumping at it and pouring hundred of millions into it is beyond me.
  - microcolonel 9y ago
    > There is some work on having RISC-V backends in HotSpot and JikesRVM, but so far it is limited to interpretation IIRC. Well, JikesRVM is a proper JIT. Palmer Dabbelt from SiFive has worked on a HotSpot port before (for a different platform). I'm currently working on a V8 port. The availability of platform software and language environments is obviously of paramount importance, since it'll shape the remaining first impressions of the architecture. > The fact that Oracle is not jumping at it and pouring hundred of millions into it is beyond me. Well, it's a lot of work, and they have their SPARC investment to continue.
rbanffy 9y ago
Still, if you have only two floats to add and make a decision based on the result, a GPU or a TPU will not help you. Hashing, maybe sorting, blitting and some math could be performed at the on-module DRAM controller level even, without data crossing over the slow DDR4 bus or mangling the CPU caches. I'd love, in fact, to explore such an architecture in a simulator. What would happen to CPU performance if, say, hashes could be computed without reading the data, memory be cleared without zeroes hitting the bus or some SIMD operations be conducted on the memory. Edit: clarify the processing could be done on the module side of the memory bus.
- microcolonel 9y ago
  > memory be cleared without zeroes In some specific cases, such as when the Linux kernel maps memory into your process, this is exactly what happens. When you write the page, it faults and clears it on demand; but I don't think there would be a considerable benefit to doing this at a finer granularity.
  - rbanffy 9y ago
    Ideally, you would prefer to postpone the actual writing to memory to the moment the cached version is evicted from on-chip caches. When you write that data to RAM, it'll take a lot of CPU cycles during which even external memory fetches will end up being delayed. And if you want to write all zeros from that page when all you are using are the first 10 bytes, it's a lot of cycles that are going to waste. Being sure the memory was actually zeroed out when you commit those bytes to DDR could save a relatively large time for the memory buses.
- saltcured 9y ago
  Are you aware of previous work on processor-in-memory architecture research? They considered things like this, but I think it stalled out due to semiconductor process and economic practicality. This article mostly mentions a UC Berkeley effort: https://en.wikipedia.org/wiki/Computational_RAM https://en.wikipedia.org/wiki/Computational_RAM This page mentions others: http://www.ai.mit.edu/projects/aries/course/notes/pim.html http://www.ai.mit.edu/projects/aries/course/notes/pim.html
jacksmith21006 9y ago
There is a great paper from Jeff Dean on using the TPUs for more traditional CS functions. https://research.google.com/pubs/pub46518.html https://research.google.com/pubs/pub46518.html The Case for Learned Index Structures - Research at Google
madez 9y ago
GPU's and Google's TPU's are only capable of certain albeit very important aspects of numerical mathematics. There are other areas of mathematics that have numerical aspects and thus precision, but aren't linear algebra.

microcolonel 9y ago

Currently working on software for RISC-V and I can definitely see some opportunities here. One thing that's becoming apparent though, is that some level of abstraction needs to be available in order for many of these things to become useful. RISC-V has a very exciting Vector processing extension, which is intended to replace packed SIMD in most cases, but some software systems assume packed SIMD is the only way to get more FP performance. For example, WebAssembly specifically exposes SIMD primitives, which means that it may be necessary to work backwards from those SIMD primitives to make use of a true vector machine. I think many people simply underestimate the cost of adopting a new programming model. > Also, why not have the usual hot paths in Node.js|Spring Framework|Django directly etched into hardware? HW http header parsing surely could bring benefit to them all. Well, in all the listed cases here, the CPU is not the bottleneck on throughput. As far as I can tell problem with HTTP is not that headers take too long to parse, it's that memory is still too slow, and context switches cost us precious time. The problem with Node is not that the hardware doesn't adequately model the semantics, it's that dynamic, weak typing makes it hard for any system (software or hardware) to understand what type things are. Update: The J extension seems interesting, and I've read some research (not thoroughly) recently showing considerable power and time savings from hardware GC primitives. I'm excited to see what goes on in that committee.

BenoitP 9y ago
> I'm excited to see what goes on in that committee. I am too. So far, this post on the general RISC-V mailing list and a few videos online talk about it. I'd love to have some other sources of information on their progress. Also, I've heard that the RISC-V foundation is actively seeking collaboration for Java. There is some work on having RISC-V backends in HotSpot and JikesRVM, but so far it is limited to interpretation IIRC. The fact that Oracle is not jumping at it and pouring hundred of millions into it is beyond me.
- microcolonel 9y ago
  > There is some work on having RISC-V backends in HotSpot and JikesRVM, but so far it is limited to interpretation IIRC. Well, JikesRVM is a proper JIT. Palmer Dabbelt from SiFive has worked on a HotSpot port before (for a different platform). I'm currently working on a V8 port. The availability of platform software and language environments is obviously of paramount importance, since it'll shape the remaining first impressions of the architecture. > The fact that Oracle is not jumping at it and pouring hundred of millions into it is beyond me. Well, it's a lot of work, and they have their SPARC investment to continue.

rbanffy 9y ago

Still, if you have only two floats to add and make a decision based on the result, a GPU or a TPU will not help you. Hashing, maybe sorting, blitting and some math could be performed at the on-module DRAM controller level even, without data crossing over the slow DDR4 bus or mangling the CPU caches. I'd love, in fact, to explore such an architecture in a simulator. What would happen to CPU performance if, say, hashes could be computed without reading the data, memory be cleared without zeroes hitting the bus or some SIMD operations be conducted on the memory. Edit: clarify the processing could be done on the module side of the memory bus.

microcolonel 9y ago
> memory be cleared without zeroes In some specific cases, such as when the Linux kernel maps memory into your process, this is exactly what happens. When you write the page, it faults and clears it on demand; but I don't think there would be a considerable benefit to doing this at a finer granularity.
- rbanffy 9y ago
  Ideally, you would prefer to postpone the actual writing to memory to the moment the cached version is evicted from on-chip caches. When you write that data to RAM, it'll take a lot of CPU cycles during which even external memory fetches will end up being delayed. And if you want to write all zeros from that page when all you are using are the first 10 bytes, it's a lot of cycles that are going to waste. Being sure the memory was actually zeroed out when you commit those bytes to DDR could save a relatively large time for the memory buses.
saltcured 9y ago
Are you aware of previous work on processor-in-memory architecture research? They considered things like this, but I think it stalled out due to semiconductor process and economic practicality. This article mostly mentions a UC Berkeley effort: https://en.wikipedia.org/wiki/Computational_RAM https://en.wikipedia.org/wiki/Computational_RAM This page mentions others: http://www.ai.mit.edu/projects/aries/course/notes/pim.html http://www.ai.mit.edu/projects/aries/course/notes/pim.html

jacksmith21006 9y ago

There is a great paper from Jeff Dean on using the TPUs for more traditional CS functions. https://research.google.com/pubs/pub46518.html https://research.google.com/pubs/pub46518.html The Case for Learned Index Structures - Research at Google

madez 9y ago

GPU's and Google's TPU's are only capable of certain albeit very important aspects of numerical mathematics. There are other areas of mathematics that have numerical aspects and thus precision, but aren't linear algebra.