4 ms·

Do the AI cores use a standardized API? Is it an extension to the RISC-V set? Is the API similar to normal CPU multithreading, or more like GPU kernels or SIMD?

by ummonk 3y ago

sneakerblack 3y ago
> Occamy has a lightweight 32-bit CPU core that acts more as a control chip, and it is responsible for rerouting tasks to the AI cores, which are extensions to the instruction set architecture. Seems like the latter
monocasa 3y ago
It's RISC-V, both the control core and the compute cores. Has a few extensions on the compute side for loading the register file from DMA and chaining FPU ops with no core front end babysitting. Honestly reminds me a lot of GPU architectures, though not quite as wide per compute core (so not as wide compared to a SM or wavefront). Probably legitimately makes for a very nice HPC chip all else being equal.
- dpeckett 3y ago
  I hope someone picks up the architecture and brings it to the wider market. Would be great to see some opensource hardware AI accelerator options. Looking more closely at snitch, unfortunately it looks like the two largest contributors have already left the project. Perhaps they can keep momentum but industry poaching remains an issue for ESA.
- fer 3y ago
  It reminds me more of the (now abandoned) Xeon Phi line of processors.
- Symmetry 3y ago
  Parts of it reminded me of the Cell too.
sanxiyn 3y ago
It is a set of extensions to the RISC-V instruction set. Extensions (Xssr, Xfrep, Xdma) are documented here: https://pulp-platform.github.io/snitch/rm/custom_instructions/ https://pulp-platform.github.io/snitch/rm/custom_instruction...
panick21_ 3y ago
RISC-V does not yet have an official 'AI' instruction set extension. There is a working group thinking about different extensions to help AI. RISC-V has Vector instruction that do help accelerate many workloads but they are far more general. Many RISC-V companies have their own AI instructions, there is no standard as this is still a research topic. There is also no standard for GPU, there is a graphics workgroup as well.
- galangalalgol 3y ago
  My understanding was that risc-v intends to support vector processing, not simd to avoid instruction bloat and for portable binaries. There is already a letter reserved for packed simd in the isa naming convention though (P). Are there actually boards I can buy that support either? Will llvm generate either? I look forward to seeing how it works.
  - brucehoult 3y ago
    There are no chips/boards implementing the standard P extension as it is nowhere near finalised. The base for the proposed P extension came from Andes who have been shipping their custom packed SIMD ISA on their own NDS32 and then RISC-V cores for a number of years. There are currently no chips or boards implementing RVV 1.0. There seems to be a prospect for a board with a single core 1.6 GHz dual-issue in-order core (C908) implementing RVV 1.0 for $30-$40 perhaps towards the end of the year. https://twitter.com/SipeedIO/status/1654055669774036993 https://twitter.com/SipeedIO/status/1654055669774036993 There have been chips and boards implementing the mid-2019 RVV draft 0.7.1 since the first Allwinner D1 board with single 1.0 GHz 64 bit C906 core in April 2021 (and a lot more since). In the last few months we have seen three new chips implementing this vector ISA: - Buffalo Lab BL808 with a 480 MHz C906 (plus a couple of 32 bit cores) and 64 MB of PSRAM: Pine64 Ox64, Sipeed M1s - THead TH1520 with 4x 2.0 GHz OoO C910 cores similar to Arm A72 in e.g. Pi 4: Lichee Pi 4A, Roma laptop (supposedly), other as yet unannounced boards coming soon - Sophon SG2042 with 64x 2.0 GHz OoO C910 cores, 64 MB L3 cache, 32x PCIe gen 4, four DDR channels. I've been using one of these remotely via ssh. The first retail machine with it, the Milk-V "Pioneer" is available for preorder in China now, reportedly Crowdsupply soon. RVV 0.7.1 is very similar to 1.0, but there were a few changes in between that make them incompatible (which is expected because draft specs were never expected to be widely implemented .. but here we are). SOME carefully-written code is binary compatible between them e.g. memcpy(), but in general a little usually trivial conversion is necessary.

Do the AI cores use a standardized API? Is it an extension to the RISC-V set? Is the API similar to normal CPU multithreading, or more like GPU kernels or SIMD?

by ummonk 3y ago

Do the AI cores use a standardized API? Is it an extension to the RISC-V set? Is the API similar to normal CPU multithreading, or more like GPU kernels or SIMD?

sneakerblack 3y ago

> Occamy has a lightweight 32-bit CPU core that acts more as a control chip, and it is responsible for rerouting tasks to the AI cores, which are extensions to the instruction set architecture. Seems like the latter

monocasa 3y ago

It's RISC-V, both the control core and the compute cores. Has a few extensions on the compute side for loading the register file from DMA and chaining FPU ops with no core front end babysitting. Honestly reminds me a lot of GPU architectures, though not quite as wide per compute core (so not as wide compared to a SM or wavefront). Probably legitimately makes for a very nice HPC chip all else being equal.

dpeckett 3y ago
I hope someone picks up the architecture and brings it to the wider market. Would be great to see some opensource hardware AI accelerator options. Looking more closely at snitch, unfortunately it looks like the two largest contributors have already left the project. Perhaps they can keep momentum but industry poaching remains an issue for ESA.
fer 3y ago
It reminds me more of the (now abandoned) Xeon Phi line of processors.
Symmetry 3y ago
Parts of it reminded me of the Cell too.

sanxiyn 3y ago

It is a set of extensions to the RISC-V instruction set. Extensions (Xssr, Xfrep, Xdma) are documented here: https://pulp-platform.github.io/snitch/rm/custom_instructions/ https://pulp-platform.github.io/snitch/rm/custom_instruction...

panick21_ 3y ago

RISC-V does not yet have an official 'AI' instruction set extension. There is a working group thinking about different extensions to help AI. RISC-V has Vector instruction that do help accelerate many workloads but they are far more general. Many RISC-V companies have their own AI instructions, there is no standard as this is still a research topic. There is also no standard for GPU, there is a graphics workgroup as well.

galangalalgol 3y ago
My understanding was that risc-v intends to support vector processing, not simd to avoid instruction bloat and for portable binaries. There is already a letter reserved for packed simd in the isa naming convention though (P). Are there actually boards I can buy that support either? Will llvm generate either? I look forward to seeing how it works.
- brucehoult 3y ago
  There are no chips/boards implementing the standard P extension as it is nowhere near finalised. The base for the proposed P extension came from Andes who have been shipping their custom packed SIMD ISA on their own NDS32 and then RISC-V cores for a number of years. There are currently no chips or boards implementing RVV 1.0. There seems to be a prospect for a board with a single core 1.6 GHz dual-issue in-order core (C908) implementing RVV 1.0 for $30-$40 perhaps towards the end of the year. https://twitter.com/SipeedIO/status/1654055669774036993 https://twitter.com/SipeedIO/status/1654055669774036993 There have been chips and boards implementing the mid-2019 RVV draft 0.7.1 since the first Allwinner D1 board with single 1.0 GHz 64 bit C906 core in April 2021 (and a lot more since). In the last few months we have seen three new chips implementing this vector ISA: - Buffalo Lab BL808 with a 480 MHz C906 (plus a couple of 32 bit cores) and 64 MB of PSRAM: Pine64 Ox64, Sipeed M1s - THead TH1520 with 4x 2.0 GHz OoO C910 cores similar to Arm A72 in e.g. Pi 4: Lichee Pi 4A, Roma laptop (supposedly), other as yet unannounced boards coming soon - Sophon SG2042 with 64x 2.0 GHz OoO C910 cores, 64 MB L3 cache, 32x PCIe gen 4, four DDR channels. I've been using one of these remotely via ssh. The first retail machine with it, the Milk-V "Pioneer" is available for preorder in China now, reportedly Crowdsupply soon. RVV 0.7.1 is very similar to 1.0, but there were a few changes in between that make them incompatible (which is expected because draft specs were never expected to be widely implemented .. but here we are). SOME carefully-written code is binary compatible between them e.g. memcpy(), but in general a little usually trivial conversion is necessary.