4 ms·

Woah there, my dude. Let's try to understand a simple model first. A CPU can access memory. When a CPU performs loads & stores it initiates transactions contai

Woah there, my dude. Let's try to understand a simple model first. A CPU can access memory. When a CPU performs loads & stores it initiates transactions containing the address of the memory. Therefore, it is a bus master--it initiates transactions. A slave accepts transactions and services them. The interconnect routes those transactions to the appropriate hardware, e.g. the DDR controller, based on the system address map. Let's add a CPU, interconnect, and 2GB of DRAM memory: +-------+ | CPU | +---m---+ | +---s--------------------+ | Interconnect | +-------m----------------+ | +----s-----------+ | DDR controller | +----------------+ System Address Map: 0x8000_0000 - 0x0000_0000 DDR controller So, a memory access to 0x0004_0000 is going to DRAM memory storage. Let's add a GPU. +-------+ +-------+ | CPU | | GPU | +---m---+ +---s---+ | | +---s------------m-------+ | Interconnect | +-------m----------------+ | +----s-----------+ | DDR controller | +----------------+ System Address Map: 0x9000_0000 - 0x8000_0000 GPU 0x8000_0000 - 0x0000_0000 DDR controller Now the CPU can perform loads & stores from/to the GPU. The CPU can read/write registers in the GPU. But that's only one-way communication. Let's make the GPU a bus master as well: +-------+ +-------+ | CPU | | GPU | +---m---+ +--s-m--+ | | | +---s-----------m-s-----+ | Interconnect | +-------m----------------+ | +----s-----------+ | DDR controller | +----------------+ System Address Map: 0x9000_0000 - 0x8000_0000 GPU 0x8000_0000 - 0x0000_0000 DDR controller Now, the GPU can not only receive transactions, but it can also initiate transactions. Which also means it has access to DRAM memory too. But this is still only one-way communication (CPU->GPU). How can the GPU communicate to the CPU? Well, both have access to DRAM memory. The CPU can store information in DRAM memory (0x8000_0000 - 0x0000_0000) and then write to a register in the GPU (0x9000_0000 - 0x8000_0000) to inform the GPU that the information is ready. The GPU then reads that information from DRAM memory. In the other direction, the GPU can store information in DRAM memory, and then send an interrupt to the CPU to inform the CPU that the information is ready. The CPU then reads that information from DRAM memory. An alternative to using interrupts is to have the CPU poll. The GPU stores information in DRAM memory and then sets some bit in DRAM memory. The CPU polls on this bit in DRAM memory, and when it changes, the CPU knows that it can read the information in DRAM memory that was previously written by the GPU. Hope this helps. It's very fun stuff!

pedrolins 5y ago
Yeah, your explanation really hits the nail in regards to what I was trying to understand - MMIO coupled with all of that bus dynamic of a master and slave going on. It's clear to me now that my knowledge gap resides in not knowing enough about interconnects. Thanks a lot! I do wonder, why aren't interconnects more emphasized in the courses I took? All I've seen was just oversimplified pictures of the process. Your explanation goes just enough into the lower-level aspects of the process to allow me to piece it.
- brooksbp 5y ago
  > Yeah, your explanation really hits the nail in regards to what I was trying to understand :) There's a lot of knowledge, and acronyms, and BS out there. E.g. there is no need to discuss PCIe here. It's much easier, and enjoyable, to cultivate a simple understanding of fundamentals. Build up from there. Reduce it down to your own simple model. Interconnects focus on the transfer of data between components in the system. Topics like topology, switching/routing, and performance come into play. But, for the purposes of the simple model described above, all you really need to grasp is topology. I.e., how are things connected and where is data flowing? The memory model is another extension to the simple model described above. Both the CPU and the GPU have access to DRAM memory (shared memory). The CPU can send transactions to the GPU, and the GPU can interrupt the CPU. These are all different paths thru the system. But, remember that we described a very specific order of events that need to happen for shared memory communication between CPU and GPU. E.g. (1) the CPU sends transactions to the DDR controller to store some information in DRAM memory, and then (2) sends a transaction to the GPU to inform the GPU that it can now (3) send transactions to the DDR controller to retrieve that information from DRAM memory. But what if (1) and (3) happen much faster than (2)? The GPU will get old data, not the new data that was written by the CPU. Managing these order of events in the system is what the memory model is all about. What if shared memory exists not only in DRAM memory but also in caches elsewhere in the system? Edit: Back to your question of "how a CPU tells a GPU to start executing a program"... In the simple model, you could imagine something like: (1) CPU stores the shader program in DRAM memory. (2) CPU writes a GPU register informing the GPU of what address the shader program is located at in DRAM memory. (3) CPU also informs the GPU of the size of the shader program. (4) GPU loads the shader program from DRAM memory. (5) GPU starts executing shader program.

pedrolins 5y ago

Yeah, your explanation really hits the nail in regards to what I was trying to understand - MMIO coupled with all of that bus dynamic of a master and slave going on. It's clear to me now that my knowledge gap resides in not knowing enough about interconnects. Thanks a lot! I do wonder, why aren't interconnects more emphasized in the courses I took? All I've seen was just oversimplified pictures of the process. Your explanation goes just enough into the lower-level aspects of the process to allow me to piece it.

brooksbp 5y ago
> Yeah, your explanation really hits the nail in regards to what I was trying to understand :) There's a lot of knowledge, and acronyms, and BS out there. E.g. there is no need to discuss PCIe here. It's much easier, and enjoyable, to cultivate a simple understanding of fundamentals. Build up from there. Reduce it down to your own simple model. Interconnects focus on the transfer of data between components in the system. Topics like topology, switching/routing, and performance come into play. But, for the purposes of the simple model described above, all you really need to grasp is topology. I.e., how are things connected and where is data flowing? The memory model is another extension to the simple model described above. Both the CPU and the GPU have access to DRAM memory (shared memory). The CPU can send transactions to the GPU, and the GPU can interrupt the CPU. These are all different paths thru the system. But, remember that we described a very specific order of events that need to happen for shared memory communication between CPU and GPU. E.g. (1) the CPU sends transactions to the DDR controller to store some information in DRAM memory, and then (2) sends a transaction to the GPU to inform the GPU that it can now (3) send transactions to the DDR controller to retrieve that information from DRAM memory. But what if (1) and (3) happen much faster than (2)? The GPU will get old data, not the new data that was written by the CPU. Managing these order of events in the system is what the memory model is all about. What if shared memory exists not only in DRAM memory but also in caches elsewhere in the system? Edit: Back to your question of "how a CPU tells a GPU to start executing a program"... In the simple model, you could imagine something like: (1) CPU stores the shader program in DRAM memory. (2) CPU writes a GPU register informing the GPU of what address the shader program is located at in DRAM memory. (3) CPU also informs the GPU of the size of the shader program. (4) GPU loads the shader program from DRAM memory. (5) GPU starts executing shader program.