5 ms·

RISC-V is getting MSIs

129 points by azhenley 4y ago

gavinray 4y ago
What are these MSI things useful for? (The article shows how to interact with it, but not what you can do with it) I know nothing about hardware
- jareklupinski 4y ago
  ordinarily, you would have to dedicate a physical pin on your device to listen for an interrupt signal from another device. MSI allows you do use existing data busses, like PCI, to send and listen for events or matched patterns on that bus, removing the need for a physical pin to signal an interrupt (in some cases)
  - toast0 4y ago
    Note that using the existing data connection (PCI-e is point to point, not a bus anymore) helps solve some race conditions. DMA writes from a device to memory may be buffered by bridges, but interrupt pins are likely not; so if the device wrote to memory and signals that to the host via an interrupt pin, the interrupt may arrive before the write. A message signaled interrupt uses the same mechanism as the DMA write, if the DMA write is buffered, the interrupt write will wait behind it.
- gbin 4y ago
  Interrupts are a way for peripherals to signal to the CPU something happened (packet is received, audio buffer has been played and I need more data, etc. the application is device dependent). Once a cpu receives an interrupt it jumps to a predetermined interrupt service routine the OS sets up to address the situation the peripheral signaled (usually ultimately in a driver). Traditionally those are carried by individual discreet connections to the CPU ie. a set of wires signaling an interrupt level. MSI is a way to virtualize this and have interrupts in band with the rest of the bus (Ie. Within the address space of the bus) instead of needing specific wires for each one.
fithisux 4y ago
RISC_V is getting better and better. Now we need a raspberry-pi like thing with all these goodness.
amelius 4y ago
What is the max latency of these signals?
- monocasa 4y ago
  Depends on the system architecture. But a lot of complex systems have been message signaled internally for a very long time, hypertransport based systems being a prominent early example in the desktop/server space.
  - eqvinox 4y ago
    Indeed there are no interrupt pins in PCI express at all, you only have a choice between special "Legacy/Compatibility" Assert_INTx message packets, or plain Memory Writes for MSI/MSI-X. But both are carried as in-band PCIe packets on the high speed lanes.
synergy20 4y ago
Somehow I always assumed RISC-V already had MSI for its PCI-Express use cases, which is essential to me.
- bonzini 4y ago
  Yes, the RISC-V AIA has been a standard for some months now.
  - phendrenad2 4y ago
    Out of curiosity, if anyone can answer: There are many RISC-V CPU cores. Are there implementations of full systems, including PCI-E host, on an FPGA platform?
  - p_l 4y ago
    The article seems to claim that AIA is still a work-in-progress thing?
    - pavehawk2007 4y ago
      AIA is still a work in progress. The last draft release was 18 days ago, and it is at version 0.3.0-31. Verify here: https://github.com/riscv/riscv-aia/releases https://github.com/riscv/riscv-aia/releases As the article mentions, there was a major change between 30 and 31 where the SETEIENUM/SETEIPNUM as well as the CLREIENUM/CLREIPNUM CSRs were removed. The issue on GitHub reads that they were removed since the architecture review board asked for them to be removed. A manufacturer can use a draft spec, but it may be rendered outdated sooner rather than later.
  - eqvinox 4y ago
    That is shockingly young, like the GP I had assumed MSI would've been designed in when the PLIC was designed (i.e. ≤ 2019). The PLIC doc says something about "Interrupt Gateways", did those not make it?
Taniwha 4y ago
Presumably the addresses for this is configurable right? Some systems put DRAM at 0. If a system has a fixed address it's probably best to describe in the spec how to discover it in the device tree
a-dub 4y ago
tangential question: if i wanted to poke around with the risc-v isa (like add an instruction) and then wanted to add support to an emulator, compiler and operating system for that instruction, is there an emulator, compiler and operating system that are best suited for this sort of isa experimentation?
- aseipp 4y ago
  If you just want to test the idea without any major complexity, like "I just want to add an instruction for int8 dot products" or whatever, you can write your own emulator for RV32IMC, in M-mode, in a few hundred lines of most modern programming languages. There are a bunch of test suites to check the base ISA works, and you can use a C compiler to target bare metal, and use GCC's .insn directive to embed custom opcodes inline. Run a bunch of tests this way. It's actually very easy to control the whole stack in cases like this. Once you get into actually needing to boot a system like Linux, you're going to need more support from the emulator, and at that level you'd either want to look into extending the Spike emulator, or QEMU itself. Like, features that interact with different privilege levels or whatever are just complex, there isn't any way around just working on the source directly. I realize that's not a very good answer. There are simpler kernels and whatnot you could also probably use. I mean, I would hope so. Honestly, a lot of it depends on the specifics of what you're looking at trying to implement. Simple instructions that work in M/U mode are the best place to start. I suggest trying to write your own emulator first. It'd surprise you at how easy the base RISC-V IMC ISA is, and you can add as many custom instructions as you like, and you'll have a C compiler (and Rust too, actually) to start with.
- trasz 4y ago
  The CHERI team had the same problem: https://github.com/CTSRD-CHERI/cheribuild https://github.com/CTSRD-CHERI/cheribuild. You'll need something along the lines of "./cheribuild.py sdk-riscv64 freebsd-riscv64 disk-image-riscv64 run-riscv64", bit of disk space, and a free couple of hours for it to finish building everything; at the end you'll get a login prompt for the guest.
- rwmj 4y ago
  QEMU TCG is not that difficult to understand, and has quite comprehensive support for RV64 already, so you could add a new instruction there. As for compiler/kernel support, that's probably a lot more difficult, but it's hard to say what would be involved (eg. deep changes, or simple __asm__ statements) without knowing what kind of feature you'd be trying to add.

129 points by azhenley 4y ago

What are these MSI things useful for? (The article shows how to interact with it, but not what you can do with it) I know nothing about hardware

jareklupinski 4y ago
ordinarily, you would have to dedicate a physical pin on your device to listen for an interrupt signal from another device. MSI allows you do use existing data busses, like PCI, to send and listen for events or matched patterns on that bus, removing the need for a physical pin to signal an interrupt (in some cases)
- toast0 4y ago
  Note that using the existing data connection (PCI-e is point to point, not a bus anymore) helps solve some race conditions. DMA writes from a device to memory may be buffered by bridges, but interrupt pins are likely not; so if the device wrote to memory and signals that to the host via an interrupt pin, the interrupt may arrive before the write. A message signaled interrupt uses the same mechanism as the DMA write, if the DMA write is buffered, the interrupt write will wait behind it.
gbin 4y ago
Interrupts are a way for peripherals to signal to the CPU something happened (packet is received, audio buffer has been played and I need more data, etc. the application is device dependent). Once a cpu receives an interrupt it jumps to a predetermined interrupt service routine the OS sets up to address the situation the peripheral signaled (usually ultimately in a driver). Traditionally those are carried by individual discreet connections to the CPU ie. a set of wires signaling an interrupt level. MSI is a way to virtualize this and have interrupts in band with the rest of the bus (Ie. Within the address space of the bus) instead of needing specific wires for each one.

fithisux 4y ago

RISC_V is getting better and better. Now we need a raspberry-pi like thing with all these goodness.

amelius 4y ago

What is the max latency of these signals?

monocasa 4y ago
Depends on the system architecture. But a lot of complex systems have been message signaled internally for a very long time, hypertransport based systems being a prominent early example in the desktop/server space.
- eqvinox 4y ago
  Indeed there are no interrupt pins in PCI express at all, you only have a choice between special "Legacy/Compatibility" Assert_INTx message packets, or plain Memory Writes for MSI/MSI-X. But both are carried as in-band PCIe packets on the high speed lanes.

synergy20 4y ago

Somehow I always assumed RISC-V already had MSI for its PCI-Express use cases, which is essential to me.

bonzini 4y ago
Yes, the RISC-V AIA has been a standard for some months now.
- phendrenad2 4y ago
  Out of curiosity, if anyone can answer: There are many RISC-V CPU cores. Are there implementations of full systems, including PCI-E host, on an FPGA platform?
- p_l 4y ago
  The article seems to claim that AIA is still a work-in-progress thing?
  - pavehawk2007 4y ago
    AIA is still a work in progress. The last draft release was 18 days ago, and it is at version 0.3.0-31. Verify here: https://github.com/riscv/riscv-aia/releases https://github.com/riscv/riscv-aia/releases As the article mentions, there was a major change between 30 and 31 where the SETEIENUM/SETEIPNUM as well as the CLREIENUM/CLREIPNUM CSRs were removed. The issue on GitHub reads that they were removed since the architecture review board asked for them to be removed. A manufacturer can use a draft spec, but it may be rendered outdated sooner rather than later.
- eqvinox 4y ago
  That is shockingly young, like the GP I had assumed MSI would've been designed in when the PLIC was designed (i.e. ≤ 2019). The PLIC doc says something about "Interrupt Gateways", did those not make it?

Taniwha 4y ago

Presumably the addresses for this is configurable right? Some systems put DRAM at 0. If a system has a fixed address it's probably best to describe in the spec how to discover it in the device tree

a-dub 4y ago

tangential question: if i wanted to poke around with the risc-v isa (like add an instruction) and then wanted to add support to an emulator, compiler and operating system for that instruction, is there an emulator, compiler and operating system that are best suited for this sort of isa experimentation?

aseipp 4y ago
If you just want to test the idea without any major complexity, like "I just want to add an instruction for int8 dot products" or whatever, you can write your own emulator for RV32IMC, in M-mode, in a few hundred lines of most modern programming languages. There are a bunch of test suites to check the base ISA works, and you can use a C compiler to target bare metal, and use GCC's .insn directive to embed custom opcodes inline. Run a bunch of tests this way. It's actually very easy to control the whole stack in cases like this. Once you get into actually needing to boot a system like Linux, you're going to need more support from the emulator, and at that level you'd either want to look into extending the Spike emulator, or QEMU itself. Like, features that interact with different privilege levels or whatever are just complex, there isn't any way around just working on the source directly. I realize that's not a very good answer. There are simpler kernels and whatnot you could also probably use. I mean, I would hope so. Honestly, a lot of it depends on the specifics of what you're looking at trying to implement. Simple instructions that work in M/U mode are the best place to start. I suggest trying to write your own emulator first. It'd surprise you at how easy the base RISC-V IMC ISA is, and you can add as many custom instructions as you like, and you'll have a C compiler (and Rust too, actually) to start with.
trasz 4y ago
The CHERI team had the same problem: https://github.com/CTSRD-CHERI/cheribuild https://github.com/CTSRD-CHERI/cheribuild. You'll need something along the lines of "./cheribuild.py sdk-riscv64 freebsd-riscv64 disk-image-riscv64 run-riscv64", bit of disk space, and a free couple of hours for it to finish building everything; at the end you'll get a login prompt for the guest.
rwmj 4y ago
QEMU TCG is not that difficult to understand, and has quite comprehensive support for RV64 already, so you could add a new instruction there. As for compiler/kernel support, that's probably a lot more difficult, but it's hard to say what would be involved (eg. deep changes, or simple __asm__ statements) without knowing what kind of feature you'd be trying to add.