5 ms·

eGPU: A 750 MHz Class Soft GPGPU for FPGA

137 points by matt_d 3y ago

avmich 3y ago
Wonder it this could help to alleviate the momentary shortage of GPUs on the market.
- ZiiS 3y ago
  10 year old entry level GPUs have 100 750Mhz cores
  - monocasa 3y ago
    'Cores' are really overstated in GPUs. CUDA cores are really SIMD lanes and if you counted it the same way as a CPU does, you'd get somewhere in the dozens of cores range even for modern GPUs.
    - xigency 3y ago
      That seems backwards to me. Sure, a GPU core is less general, but in terms of concurrent execution, memory bandwidth, and FLOPS I would expect hundreds to thousands of cores for all new GPU offerings. Apple’s double-digit GPU core counts for instance sound extremely understated.
      - monocasa 3y ago
        It's not. The best comparison is the SM count for Nvidia hardware, or the wavefront count for AMD hardware. So a 4070 has 46 cores as you'd count them on a CPU.
      - imtringued 3y ago
        If you have a truck that can carry 64 boxes, then according to Nvidia marketing each box would be considered its own core even though the boxes all go the same way. Meanwhile if you had 64 passengers cars, then each of them would be able to drive a different route at the same time and if you counted the seats you would actually end up with 320 "Nvidia cores".
    - codedokode 3y ago
      A proper method is counting ALUs instead of vague "cores".
    - deaddodo 3y ago
      If you want to count "cores" in a GPU, the best equivalency is streaming multiprocessors. While it's true that they're in the dozens, strictly speaking, we've got GPUs with well over 100 now (the RTX 4090, for instance) and 64+ is a pretty common configuration.
  - kristianp 3y ago
    From the abstract: "The eGPU architecture is a streaming multiprocessor (SM) machine with 512 threads. Each SM contains 16 scalar processors (SP)."
- deleted 3y ago
  [deleted]
- monocasa 3y ago
  If it's on an FPGA then it doesn't really compete with GPUs you can buy from just about any perspective other than openness.
- latchkey 3y ago
  I don't think this will be momentary. Reality is that there have been shortages of GPUs for a long time now and demand isn't going down. People are signing 3 year contracts with lambda now.
stefanpie 3y ago
One group at Georgia Tech in our building has also been working on open source GPU designs that can also target FPGAs and interoperate with RISCV. They have several publications on the work they have built up. Thought I might share since it’s not referenced in the submission paper. https://vortex.cc.gatech.edu/ https://vortex.cc.gatech.edu/
- mepian 3y ago
  They still haven't published the source code for their Skybox project, I wonder why. Unless I missed it in their repository? https://github.com/vortexgpgpu https://github.com/vortexgpgpu
  - stefanpie 3y ago
    Not sure, my best advice would be to just email the authors if you are interested. Seems like they presented to ASPLOS so maybe the students disappeared over the summer for internships or vacation.
dragontamer 3y ago
For a GPU circuit, it basically comes down to the number of hardware multipliers on the FPGA, does it not? I remember synthesizing a 16-bit Wallace tree in a lab exercise back in college. I think that single multiplier used up 70% of my LUTs. You only will get massive amounts of hardware parallel multipliers if the underlying circuit has a ton of hardware multipliers (Like Xilinx's VLIW SIMD AI chips) ------- At all computer sizes, a GPU probably will have more multiply circuits than an equivalent cost FPGA, with exception of maybe those AI chips from Xilinx (where the individual cores are basically presynthesized with hardcoded ISA). Ex: at under 500mW power usage you probably will prefer some ARM NEON SIMD or TI DSP / VLIW. At cell phone levels you'd prefer a cell phone GPU, and at desktop/server levels you'd prefer a desktop GPU.
- danhor 3y ago
  > At all computer sizes, a GPU probably will have more multiply circuits than an equivalent cost FPGA Very likely yes, but FPGAs often have hundreds to thousands of hardware multipliers, as part of the DSP blocks. Here for example newer AMD FPGAs: https://eu.mouser.com/datasheet/2/903/ds890_ultrascale_overview-1591529.pdf https://eu.mouser.com/datasheet/2/903/ds890_ultrascale_overv...
  - pkaye 3y ago
    How much would that FPGA cost?
  - UncleOxidant 3y ago
    The FPGAs with enough multipliers to be competitive against an actual GPU are going to be quite a bit more expensive than a GPU aren't they?
  - mathisfun123 3y ago
    I wish people would stop quoting marketing material as some kind representation of what they know. You're giving completely the wrong impression about dsp slices - it is absolutely not 1 dsp slice per FP operator at any precision that you would want to do floating point arithmetic. It's definitely at least 2 plus a whole bunch of LUTs (~500) for FP16 with 4 stages or something like that. And if you want faster (fewer stages) then you need more slices. On alveo u280, which is an ultrascale part, I have never been able to effectively utilize more than ~4000 dsp slices (out of 9024) for 5,4 mults and that cost basically 99% of clbs in SLR1 and SLR2. And even then, disconnected FPUs are completely meaningless without a datapath implementing eg matmul and boy oh boy do you have no clue what you're in for there. Takeaway: it's pointless to compare raw specsheet numbers when everything comes down to datapath.
Lramseyer 3y ago
Full Disclosure, I work for an FPGA company. The mind blowing part of all of this is the fact that they were able to close timing at 771MHz. That is insanely fast for an FPGA. For perspective, most modern FPGAs run their designs at around 300MHz* While most of the heavy lifting in this design use hardened components like DSPs and FPUs, it's still very impressive to see! What I didn't see talked about much was how memory is loaded in and out of the processor. I'm curious to see what the memory bandwidth numbers look like as well as the resource utilization of the higher level routing. *For most hardware designs that aren't things like CPUs and GPUs, you don't always need a super high clock speed. You have a lot more flexibility to compute in space rather than in time (think more threads running slower.) The pros and cons of such tradeoffs are a bit of a complicated topic, but should at least be noted.
- mathisfun123 3y ago
  > The mind blowing part of all of this is the fact that they were able to close timing at 771MHz It's true but I mean this is Intel in-house research right? If they can't get absolute peak fmax on their own parts that would be a really bad look right? Plus these stratix parts have hard FP blocks (not just DSPs) so they're basically mostly scheduling stuff rather building the whole datapath. But admittedly I haven't read the paper... >Full Disclosure, I work for an FPGA company I currently do too (as an intern, maybe even the same one as you) and I haven't looked very hard but I'm sure we have similar fmax achieving projects (maybe even GPUs since we're fighting hard to compete with Nvidia...).
  - aseipp 3y ago
    I think it's more that the compile was unconstrained, so no floorplanning or handholding synthesis anything (aside from the usual bullshit to e.g. make sure DSPs map correctly). Which is pretty good; yeah, the architecture is also good and you'd expect that at minimum from a team like this, but they also didn't have to beat the tool into submission either which seems nice. IME, until fairly recently at least, Quartus loved to get fiddly on even the most basic designs when using Stratix class parts where you wanted to rely on, say, hyper-registers or retiming. Like a thing that should clearly be pipelined just... doesn't, and falls over from a gust of wind. It seems to have gotten quite a bit better in the last few versions, though, so seeing an unconstrained compile go this high is pretty good I think. Too bad they don't seem to have released their code, though.
gsmecher 3y ago
Also discussed here: https://old.reddit.com/r/FPGA/comments/15fnb6u/egpu_a_750_mhz_class_soft_gpgpu_for_fpga/ https://old.reddit.com/r/FPGA/comments/15fnb6u/egpu_a_750_mh...
unwind 3y ago
Uh, non-native question: what is the word "class" doing in the title? Is a hyphen missing, so it should be "750 MHz-class"? I searched the linked page but the word only appears in the title, sans hyphen.
- WantonQuantum 3y ago
  Yes, the title is saying that the "soft GPU" being executed on the FPGA is running like (in the same "class" as) a 750 MHz GPU.
kristianp 3y ago
This page isn't loading for me. Mirrored at: https://archive.is/jewT4 https://archive.is/jewT4
tasty_freeze 3y ago
The "G" in "GPU" or the second "G" in "GPGPU" stands for "graphics". The paper describes a compute-oriented processor that doesn't appear to contain any gfx-specific hardware. For instance, in conventional GPUs, there is dedicated hardware for rasterizing the pixels covered by a triangle; there is dedicated hardware for doing texture map calculations and sample filtering. The instruction set disclosed in the paper doesn't contain anything to help accelerate those tasks. With enough code I'm sure one could get it to draw something, but it will be notably inefficient while doing so.
- iamtedd 3y ago
  Also, the 'e' in eGPU usually means that the card is housed externally to the rest of the computer - in its own enclosure. They don't actually explain anywhere in their paper what 'eGPU' means. Is it 'emulated'? I thought circuits implemented in FPGAs are not considered emulation?
  - hmry 3y ago
    > This paper describes the architecture and implementation of a high performance embedded GPU I'm tinking the e means embedded
deleted 3y ago
[deleted]
DrNosferatu 3y ago
Peaked my interest, but then: - No (price or non-price) comparison/benchmark with any physical GPU of any sort; - Seems to only be able to do FFTs; - To be truly useful (or "General" - hey, they do mention _G_PGPU), it would have to be able to run OpenCL code; - And on that note, what's the point, since (many) FPGAs can run OpenCL code directly? Is there something I'm missing?

137 points by matt_d 3y ago

avmich 3y ago

Wonder it this could help to alleviate the momentary shortage of GPUs on the market.

ZiiS 3y ago
10 year old entry level GPUs have 100 750Mhz cores
- monocasa 3y ago
  'Cores' are really overstated in GPUs. CUDA cores are really SIMD lanes and if you counted it the same way as a CPU does, you'd get somewhere in the dozens of cores range even for modern GPUs.
  - xigency 3y ago
    That seems backwards to me. Sure, a GPU core is less general, but in terms of concurrent execution, memory bandwidth, and FLOPS I would expect hundreds to thousands of cores for all new GPU offerings. Apple’s double-digit GPU core counts for instance sound extremely understated.
    - monocasa 3y ago
      It's not. The best comparison is the SM count for Nvidia hardware, or the wavefront count for AMD hardware. So a 4070 has 46 cores as you'd count them on a CPU.
    - imtringued 3y ago
      If you have a truck that can carry 64 boxes, then according to Nvidia marketing each box would be considered its own core even though the boxes all go the same way. Meanwhile if you had 64 passengers cars, then each of them would be able to drive a different route at the same time and if you counted the seats you would actually end up with 320 "Nvidia cores".
  - codedokode 3y ago
    A proper method is counting ALUs instead of vague "cores".
  - deaddodo 3y ago
    If you want to count "cores" in a GPU, the best equivalency is streaming multiprocessors. While it's true that they're in the dozens, strictly speaking, we've got GPUs with well over 100 now (the RTX 4090, for instance) and 64+ is a pretty common configuration.
- kristianp 3y ago
  From the abstract: "The eGPU architecture is a streaming multiprocessor (SM) machine with 512 threads. Each SM contains 16 scalar processors (SP)."
deleted 3y ago
[deleted]
monocasa 3y ago
If it's on an FPGA then it doesn't really compete with GPUs you can buy from just about any perspective other than openness.
latchkey 3y ago
I don't think this will be momentary. Reality is that there have been shortages of GPUs for a long time now and demand isn't going down. People are signing 3 year contracts with lambda now.

stefanpie 3y ago

One group at Georgia Tech in our building has also been working on open source GPU designs that can also target FPGAs and interoperate with RISCV. They have several publications on the work they have built up. Thought I might share since it’s not referenced in the submission paper. https://vortex.cc.gatech.edu/ https://vortex.cc.gatech.edu/

mepian 3y ago
They still haven't published the source code for their Skybox project, I wonder why. Unless I missed it in their repository? https://github.com/vortexgpgpu https://github.com/vortexgpgpu
- stefanpie 3y ago
  Not sure, my best advice would be to just email the authors if you are interested. Seems like they presented to ASPLOS so maybe the students disappeared over the summer for internships or vacation.

dragontamer 3y ago

For a GPU circuit, it basically comes down to the number of hardware multipliers on the FPGA, does it not? I remember synthesizing a 16-bit Wallace tree in a lab exercise back in college. I think that single multiplier used up 70% of my LUTs. You only will get massive amounts of hardware parallel multipliers if the underlying circuit has a ton of hardware multipliers (Like Xilinx's VLIW SIMD AI chips) ------- At all computer sizes, a GPU probably will have more multiply circuits than an equivalent cost FPGA, with exception of maybe those AI chips from Xilinx (where the individual cores are basically presynthesized with hardcoded ISA). Ex: at under 500mW power usage you probably will prefer some ARM NEON SIMD or TI DSP / VLIW. At cell phone levels you'd prefer a cell phone GPU, and at desktop/server levels you'd prefer a desktop GPU.

danhor 3y ago
> At all computer sizes, a GPU probably will have more multiply circuits than an equivalent cost FPGA Very likely yes, but FPGAs often have hundreds to thousands of hardware multipliers, as part of the DSP blocks. Here for example newer AMD FPGAs: https://eu.mouser.com/datasheet/2/903/ds890_ultrascale_overview-1591529.pdf https://eu.mouser.com/datasheet/2/903/ds890_ultrascale_overv...
- pkaye 3y ago
  How much would that FPGA cost?
- UncleOxidant 3y ago
  The FPGAs with enough multipliers to be competitive against an actual GPU are going to be quite a bit more expensive than a GPU aren't they?
- mathisfun123 3y ago
  I wish people would stop quoting marketing material as some kind representation of what they know. You're giving completely the wrong impression about dsp slices - it is absolutely not 1 dsp slice per FP operator at any precision that you would want to do floating point arithmetic. It's definitely at least 2 plus a whole bunch of LUTs (~500) for FP16 with 4 stages or something like that. And if you want faster (fewer stages) then you need more slices. On alveo u280, which is an ultrascale part, I have never been able to effectively utilize more than ~4000 dsp slices (out of 9024) for 5,4 mults and that cost basically 99% of clbs in SLR1 and SLR2. And even then, disconnected FPUs are completely meaningless without a datapath implementing eg matmul and boy oh boy do you have no clue what you're in for there. Takeaway: it's pointless to compare raw specsheet numbers when everything comes down to datapath.

Lramseyer 3y ago

Full Disclosure, I work for an FPGA company. The mind blowing part of all of this is the fact that they were able to close timing at 771MHz. That is insanely fast for an FPGA. For perspective, most modern FPGAs run their designs at around 300MHz* While most of the heavy lifting in this design use hardened components like DSPs and FPUs, it's still very impressive to see! What I didn't see talked about much was how memory is loaded in and out of the processor. I'm curious to see what the memory bandwidth numbers look like as well as the resource utilization of the higher level routing. *For most hardware designs that aren't things like CPUs and GPUs, you don't always need a super high clock speed. You have a lot more flexibility to compute in space rather than in time (think more threads running slower.) The pros and cons of such tradeoffs are a bit of a complicated topic, but should at least be noted.

mathisfun123 3y ago
> The mind blowing part of all of this is the fact that they were able to close timing at 771MHz It's true but I mean this is Intel in-house research right? If they can't get absolute peak fmax on their own parts that would be a really bad look right? Plus these stratix parts have hard FP blocks (not just DSPs) so they're basically mostly scheduling stuff rather building the whole datapath. But admittedly I haven't read the paper... >Full Disclosure, I work for an FPGA company I currently do too (as an intern, maybe even the same one as you) and I haven't looked very hard but I'm sure we have similar fmax achieving projects (maybe even GPUs since we're fighting hard to compete with Nvidia...).
- aseipp 3y ago
  I think it's more that the compile was unconstrained, so no floorplanning or handholding synthesis anything (aside from the usual bullshit to e.g. make sure DSPs map correctly). Which is pretty good; yeah, the architecture is also good and you'd expect that at minimum from a team like this, but they also didn't have to beat the tool into submission either which seems nice. IME, until fairly recently at least, Quartus loved to get fiddly on even the most basic designs when using Stratix class parts where you wanted to rely on, say, hyper-registers or retiming. Like a thing that should clearly be pipelined just... doesn't, and falls over from a gust of wind. It seems to have gotten quite a bit better in the last few versions, though, so seeing an unconstrained compile go this high is pretty good I think. Too bad they don't seem to have released their code, though.

gsmecher 3y ago

Also discussed here: https://old.reddit.com/r/FPGA/comments/15fnb6u/egpu_a_750_mhz_class_soft_gpgpu_for_fpga/ https://old.reddit.com/r/FPGA/comments/15fnb6u/egpu_a_750_mh...

unwind 3y ago

Uh, non-native question: what is the word "class" doing in the title? Is a hyphen missing, so it should be "750 MHz-class"? I searched the linked page but the word only appears in the title, sans hyphen.

WantonQuantum 3y ago
Yes, the title is saying that the "soft GPU" being executed on the FPGA is running like (in the same "class" as) a 750 MHz GPU.

kristianp 3y ago

This page isn't loading for me. Mirrored at: https://archive.is/jewT4 https://archive.is/jewT4

tasty_freeze 3y ago

The "G" in "GPU" or the second "G" in "GPGPU" stands for "graphics". The paper describes a compute-oriented processor that doesn't appear to contain any gfx-specific hardware. For instance, in conventional GPUs, there is dedicated hardware for rasterizing the pixels covered by a triangle; there is dedicated hardware for doing texture map calculations and sample filtering. The instruction set disclosed in the paper doesn't contain anything to help accelerate those tasks. With enough code I'm sure one could get it to draw something, but it will be notably inefficient while doing so.

iamtedd 3y ago
Also, the 'e' in eGPU usually means that the card is housed externally to the rest of the computer - in its own enclosure. They don't actually explain anywhere in their paper what 'eGPU' means. Is it 'emulated'? I thought circuits implemented in FPGAs are not considered emulation?
- hmry 3y ago
  > This paper describes the architecture and implementation of a high performance embedded GPU I'm tinking the e means embedded

deleted 3y ago

[deleted]

DrNosferatu 3y ago

Peaked my interest, but then: - No (price or non-price) comparison/benchmark with any physical GPU of any sort; - Seems to only be able to do FFTs; - To be truly useful (or "General" - hey, they do mention _G_PGPU), it would have to be able to run OpenCL code; - And on that note, what's the point, since (many) FPGAs can run OpenCL code directly? Is there something I'm missing?