12 ms·

FuryGpu – Custom PCIe FPGA GPU

446 points by argulane 3y ago

notorandit 3y ago
It needs to be very fancy to write text in light gray on white. I am not sure your product will be a success. I am sure you web design skills need a good overhaul.
- nicolas_17 2y ago
  It's not a "product" that will be "sold" or has intention of being "successful" in a commercial sense.
KallDrexx 3y ago
This is my dream! The last year I've been working on a 2d focused GPU for I/O constrained microcontrollers (https://github.com/KallDrexx/microgpu https://github.com/KallDrexx/microgpu). I've been able to utilize this to get user interfaces on slow SPI machines to render on large displays, and it's been fascinating to work on. But seeing the limitation of processor pipelines I've had the thought for a while that FPGAs could make this faster. I've recently gotten some low end FPGAs to start learning to try and turn my microgpu from an ESP32 based one to an FPGA one. I don't know if I"ll ever get to this level due to kids and free time constraints, but man, I would love to get even a hundredth of this level.
- Chabsff 3y ago
  You probably know this already, but for anyone else curious about going down that road: For this type of use, it's definitely worth it to constrain yourself to FPGAs with dedicated high-bandwidth transceivers. A "basic" 1080p RGB signal at 60hz requires some high-frequency signal processing that's really hard to contend with in pure FPGA-land.
  - KallDrexx 3y ago
    That's good to know actually. I'm still very very early in my FPGA adaption (learning the fpga basics) and I am intending to start with standard 640x480 VGA before expanding.
detuur 3y ago
I can't believe that this is the closest we have to a compact, stand-alone GPU option. There's nothing like a M.2 format GPU out there. All I want is a stand-alone M.2 GPU with modest performance, something on the level of embedded GPUs like Intel UHD Graphics, AMD Radeon, or Qualcomm's Adreno. I have an idea for a small embedded product which needs a lot of compute and networking, but only very modest graphical capabilities. The NXP Layerscape LX2160A [1] would be perfect, but I have to pass on it because it doesn't come with an embedded GPU. I just want a small GPU! [1]: https://www.nxp.com/products/processors-and-microcontrollers/arm-processors/layerscape-processors/layerscape-lx2160a-lx2120a-lx2080a-processors:LX2160A https://www.nxp.com/products/processors-and-microcontrollers...
- magixx 3y ago
  What about MXM GPUs that used to be found in gaming laptops? I know the standard is very niche and thus expensive ($400 for a 3080M used on ebay) but it does exists and you could convert them to PCI-E and thus m.2
- t-3 3y ago
  Maybe a little bit too low-powered for you, but: https://www.matrixorbital.com/ftdi-eve https://www.matrixorbital.com/ftdi-eve
- cpgxiii 3y ago
  There's at least one m.2 GPU based on the Silicon Motion SM750 controller made by Asrock Rack. Similar products exist for mPCIe form factor. Performance is nowhere near a modern iGPU, because an iGPU has access to all of the system memory and caches and power budget, and a simple m.2 device has node of that. Even low-end PCIe GPUs (single slot, half-length/half-height) struggle to outperform better iGPUs and really only make sense when you have to use them for basic display functionality.
PfhorSlayer 3y ago
So, this is my project! Was somewhat hoping to wait until there was a bit more content up on the site before it started doing the rounds, but here we are! :) To answer what seems to be the most common question I get asked about this, I am intending on open-sourcing the entire stack (PCB schematic/layout, all the HDL, Windows WDDM drivers, API runtime drivers, and Quake ported to use the API) at some point, but there are a number of legal issues that need to be cleared (with respect to my job) and I need to decide the rest of the particulars (license, etc.) - this stuff is not what I do for a living, but it's tangentially-related enough that I need to cover my ass. The first commit for this project was on August 22, 2021. It's been a bit over two and a half years I've been working on this, and while I didn't write anything up during that process, there are a fair number of videos in my YouTube FuryGpu playlist (https://www.youtube.com/playlist?list=PL4FPA1MeZF440A9CFfMJ7F6YoZ1Jvxu5j https://www.youtube.com/playlist?list=PL4FPA1MeZF440A9CFfMJ7...) that can kind of give you an idea of how things progressed. The next set of blog posts that are in the works concern the PCIe interface. It'll probably be a multi-part series starting at the PCB schematic/layout and moving through the FPGA design and ending with the Windows drivers. No timeline on when that'll be done, though. After having written just that post on how the Texture Units work, I've got even more respect for those that can write up technical stuff like that with any sort of timing consistency. I'll answer the remaining questions in the threads where they were asked. Thanks for the interest!
- michaelt 3y ago
  Googling the Xilinx Zynq UltraScale+ it seems kinda expensive. Of course plenty of hobbies let people spend thousands (or more) so there's nothing wrong with that if you've got the money. But is it the end target for your project? Or do you have ambitions to go beyond that?
  - kanetw 3y ago
    The Kria SOM in use here is like $300.
  - PfhorSlayer 3y ago
    Let's be clear here, this is a toy. Beyond being a fun project to work on that could maybe get my foot in the door were I ever to decide to change careers and move into hardware design, this is not going to change the GPU landscape or compete with any of the commercial players. What it might do is pave the way for others to do interesting things in this space. A board with all of the video hardware that you can plug into a computer with all the infrastructure available to play around with accelerating graphics could be a fun, if extremely niche, product. That would also require a *significant* time and money investment from me, and that's not something I necessarily want to deal with. When this is eventually open-sourced, those who really are interested could make their own boards. One thing to note that is that while the US+ line is generally quite expensive (the higher end parts sit in the five-figures range for a one-off purchase! No one actually buying these is paying that price, but still!), the Kria SOMs are quite cheap in comparison. They've got a reasonably-powerful Zynq US+ for about $400, or just $350ish the dev boards (which do not expose some of the high-speed interfaces like PCIe). I'm starting to sound like a Xilinx shill given how many times I've re-stated this, but for anyone serious about getting into this kind of thing, those devboards are an amazing deal.
raphlinus 3y ago
Very cool project, and I love to see more work in this space. Something else to look at is the Vortex project from Georgia Tech[1]. Rather than recapitulating the fixed-function past of GPU design, I think it looks toward the future, as it's at heart a highly parallel computer, based on RISC-V with some extensions to handle GPU workloads better. The boards it runs on are a few thousand dollars, so it's not exactly a hobbyist friendly, but it certainly is more accessible than closed, proprietary development. There's a 2.0 release that just landed a few months ago. [1]: https://vortex.cc.gatech.edu/ https://vortex.cc.gatech.edu/
userbinator 3y ago
Supporting hardware features equivalent to a high-end graphics card of the mid 1990s I see no one else has asked this question yet, so I will: How VGA-compatible is it? Would I be able to e.g. plug it into any PC with a PCIe slot, boot to DOS and play DOOM with it?
bobharris 2y ago
beyond amazing. i've dreamt of this. so inspiring. it reminds me of alot of time i spent thinking about this: https://rcl.ece.iastate.edu/sites/default/files/papers/SteJon11A.pdf https://rcl.ece.iastate.edu/sites/default/files/papers/SteJo... i actually wrote one of the professors asking for more info. didn't get a reply. my dream EE class I never got to take.
allanrbo 2y ago
What an inspiring passion project! Very ambitious first Verilog project.
anon115 2y ago
can you run valorant on it?
snvzz 3y ago
Pipeline seems retro, but far better than nothing. There's no open hardware GPU to speak of. Depending on license (can't find information?), this could be the first, and a starting point for more.
- mips_r4300i 3y ago
  Ticket2Ride Number9 is a fixed function GPU from the late 90s that was completely open sourced under GPL
- Hazematman 3y ago
  There's also Nyuzi which is more GPGPU focused https://github.com/jbush001/NyuziProcessor https://github.com/jbush001/NyuziProcessor, but the author also experimented with having it do 3D graphics.
- userbinator 3y ago
  Depends what you mean by "GPU". https://github.com/schlae/graphics-gremlin https://github.com/schlae/graphics-gremlin is an MDA/CGA compatible adapter https://github.com/OmarMongy/VGA https://github.com/OmarMongy/VGA is a VGA core https://github.com/archlabo/Frix https://github.com/archlabo/Frix is a whole IBM PC compatible SoC, including a VGA.
- deleted 3y ago
  [deleted]
- crote 3y ago
  It all depends on your definition of "open", of course. As far as I know there is no open-source toolchain for any remotely-recent FPGA, so you're still stick with proprietary (paid?) tooling to actually modify it. You're pretty much out of luck if you need more than an iCE40 UP5k.
  - rwmj 3y ago
    At least some Xilinx 7-series FPGAs have been reverse engineered: https://yosyshq.readthedocs.io/projects/yosys/en/latest/cmd/synth_xilinx.html https://yosyshq.readthedocs.io/projects/yosys/en/latest/cmd/...
jamesu 3y ago
Similarly there is this: https://github.com/ToNi3141/Rasterix https://github.com/ToNi3141/Rasterix Would be neat if someone made an FPGA GPU which had a shader pipeline honestly.
- danbruc 3y ago
  If you are going to that effort, you might also want a decent resolution. Say we aim for one megapixel (720p) and 30 frames per second, then we have to calculate 27.7 megapixel per second. If you get your FPGA to run at 500 MHz, that gives you 18 clock cycles per pixel. So you would probably want something like 100 cores keeping in mind that we also have to run vertex shaders. We also need quick access to a sizable amount of memory and I am not sure if one can get away with integer respectively fixed point arithmetics or whether floating point arithemtics is pretty much necessary. Another complication that I would expect is that it is probably much easier to build a long execution pipeline if you are implementing a fixed function pipeline as compared to a programmable processor. Things like out-of-order execution are probably best off-loaded to the compiler in order to keep the design simpler and more compact. So my guess is that it would be quite challenging to implement a modern GPU in an affordable FPGA if you want more than a proof of concept.
  - d_tr 3y ago
    There's a new board by Trenz with a Versal chip which can do 440 GFLOPS just with the DSP58 slices (the lowest speed grade) and it costs under 1000 Euros, but you also need to buy a Vivado license currently. Cheaper boards are definitely possible since there are smaller parts in that family, but they need to offer support for some of them in the free version of Vivado...
  - PfhorSlayer 3y ago
    You've nailed the problem directly on the head. For hitting 60Hz in FuryGpu, I actually render at 640x360 and then pixel-double (well, pixel->quad) the output to the full 720p. Even with my GPU cores running at 400MHz and the texture units at 480MHz with fully fixed-function pipelines, it can still struggle to keep up at times. I do not doubt that a shader core could be built, but I have reservations about the ability to run it fast enough or have as many of them as would be needed to get similar performance out of them. FuryGpu does its front-end (everything up through primitive assembly) in full fp32. Because that's just a simple fixed modelview-projection matrix transform it can be done relatively quickly, but having every single vertex/pixel able to run full fp32 shader instructions requires the ability to cover instruction latency with additional data sets - it gets complicated, fast!
gchadwick 3y ago
Cool! I found the hello blog here illuminating to understand the creators intentions: https://www.furygpu.com/blog/hello https://www.furygpu.com/blog/hello As I read it, it's just a fun hobby project for them first and foremost and looks like they're intending to write a whole bunch more about how they built it. It's certainly an impressive piece of work, in particular as they've got the full stack working, a windows driver implementing a custom graphics API and then quake running on top of that. A shame they've not got some DX/GL support but I can certainly understand why they went the custom API route. I wonder if they'll open source the design?
- PfhorSlayer 3y ago
  I'm in the process of actually trying to work out what would be feasible performance-wise if I were to spent the considerable effort to add the features required for base D3D support. It's not looking good, unfortunately. Beyond just "shaders", there are a significant amount of other requirements that even just the OS's window manager needs to function at all. It's all built up on 20+ years of evolving tech and for the normal players in this space (AMD, Nvidia, Intel, Imagination, etc.) it's always been an iterative process.
deleted 3y ago
[deleted]
iAkashPaul 3y ago
FPGAs for native FP4 will change the entire landscape
- blacklion 3y ago
  Entire landscape of open graphic chips? Not every GPU should be used to train or infer so-called AI. Please, stop, we need some hardware to put images on the screens.
- imtringued 2y ago
  How? NPUs are going to be included in every PC in 2025. The only differentiators will be how much SRAM and memory bandwidth you have or whether you use processing in memory or not. AMD is already shipping APUs with 16 TOPS or 4 TFLOPS (bfloat16) and that is more than enough for inference considering the limited memory bandwidth. Strix Halo will have around 12 TFLOPS (bfloat16) and four memory channels. llama.cpp already supports 4 bit quantization. They unpack the quantization back to bfloat16 at runtime for better accuracy. The best use case for an FPGA I have seen so far was to pair it with SK Hynix's AI GDDR and even that could be replaced by an even cheaper inference chip specializing in multi board communication and as many memory channels as possible.
- luma 3y ago
  How so?
  - CamperBob2 3y ago
    4-bit values (or 6-bit values, nowadays) values are interesting because they're small enough to address a single LUT, which is the lowest-level atomic element of an FPGA. That gives them major advantages in the timing and resource-usage departments.
  - iAkashPaul 3y ago
    Reduced memory requirements, dropping higher precision IP blocks for starters
- jsheard 3y ago
spuz 3y ago
This looks like an incredible achievement. I'd love to see some photos of the physical device. I'm also slightly confused about which FGPA module is being used. The blog mentions the Xylinx Kria SoMs but if you follow the links to the specs of those modules, you see they have ARM SoCs rather than Xylinx FGPAs. The whole world of FGPAs is pretty unfamiliar to me so maybe I'm missing something. https://www.amd.com/en/products/system-on-modules/kria/k26/k26c-commercial.html https://www.amd.com/en/products/system-on-modules/kria/k26/k...
- PfhorSlayer 3y ago
  You're in luck! https://imgur.com/a/BE0h9cZ https://imgur.com/a/BE0h9cZ As mentioned in the rest of this thread, the Kria SoMs are FPGA fabric with hardened ARM cores running the show. Beyond just being what was available (for oh so cheap, the Kria devboards are like $350!), these devices also include things like hardened DisplayPort IP attached to the ARM cores allowing me to offload things like video output and audio to the firmware. A previous version of this project was running on a Zynq 7020, for which I needed to write my own HDMI stuff that, while not super complicated, takes up a fair amount of logic and also gets way more complex if it needs to be configurable.
- crote 3y ago
  > you see they have ARM SoCs rather than Xylinx FGPAs It's a mixed chip: FPGA and traditional SoC glued together. This mean you don't have a softcore MCU taking up precious FPGA resources just to do some basic management tasks.
  - spuz 3y ago
    Ah that makes sense. It's slightly ironic then that the ARM SoC includes a Mali GPU which presumably easily outperforms what can be achieved with the FGPA.
  - chrsw 3y ago
    I didn't see any mention of what the software on the Zynq's ARM core is doing, which made me wonder why use Zynq at all.
wpwpwpw 3y ago
Excellent job. Would be amazing if this became an open source hardware project.
sylware 3y ago
Hopefully their hardware programming model is going full hardware circular command/interrupt buffers (even for GPU register programming). It is how it is done on AMD GPU, that said I have no idea what is the nvidia hardware programming model.
codedokode 3y ago
"UltraScale" in name assumes ultra price? FPGAs seem to be an expensive toy.
- PfhorSlayer 3y ago
  In general, yes. However, the Kria series are amazingly good deals for what you get - a quite powerful Zynq US+ part and a dev board for like $350.
- varispeed 3y ago
  Ages ago I bought TinyFPGA, which is like £40 and I was able to synthesize RISC-V cpu on it. It was fun.
- mattalex 3y ago
  Not in the grand scheme of things: you can get fpga dev boards for $50 that are already useable for this type of thing (you can go even lower, but those aren't really useable for "CPU like" operation and are closer to "a whole lot of logic gates in a single chip"). Of course the "industry grade" solutions pack significantly more of a punch, but they can also be had for <$500.
- nxobject 3y ago
  It's worth mentioning that it's easy enough to find absurdly cheap (~$20) early-generation dev boards for Zynq FPGAs with embedded ARM cores on Aliexpress, shucked from obsolete Bitcoin miners [1]. Interfaces include SD, Ethernet, 3 banks of GPIO. [1] https://github.com/xjtuecho/EBAZ4205 https://github.com/xjtuecho/EBAZ4205
  - deleted 3y ago
    [deleted]
  - thrtythreeforty 3y ago
    Zynq is deeply annoying to work with, though. Unfortunately the hard ARM core bootloads the FPGA fabric, rather than the other way around (or having the option to initialize both separately). This means you have to muck with software on the target to update FPGA bitstreams.
MalphasWats 3y ago
It's incredible how influential Ben Eater's breadboard computer series has been in hobby electronics. I've been similarly inspired to try to design my own "retro" CPU. I desperately want something as easy to plug into things as the 6502, but with jussst a little more capability - few more registers, hardware division, that sort of thing. It's a really daunting task. I always end up coming back to just use an MCU and be done with it, and then I hit the How To Generate Graphics problem.
- MenhirMike 3y ago
  I was about to recommend the Parallax Propeller (the first one that's available in DIP format), but arguably, that one is way more complex to program for (and also significantly more powerful, and at that point you might as well look into an ESP32 and that is "just use an MCU" :)) And yeah, video output is a significant issue because of the required bandwidth for digital outputs (unless you're okay with composite or VGA outputs, I guess they can still be done with readily available chips?). The recent Commander X16 settled for an FPGA for this.
  - MalphasWats 3y ago
    I feel like the CX16 lost its way about a week after the project started and it suddenly became an expensive FPGA-based blob. But at the same time, I'm not sure what other option there is for a project like that. I always got the impression that David sort of got railroaded by the other members of the team that wanted to keep adding features and MOAR POWAH, and didn't have a huge amount of choice because those features quickly scoped out of his own areas of knowledge.
    - MenhirMike 3y ago
      I think so too - it must have been a great learning experience for him though, but for me, the idea of "The best C64-like computer that ever existed" died pretty quickly. He also did run into a similar problem that I ran into when I tried something like that as well: Sound Chips. Building a system around a Yamaha FM Synthesizer is perfect, but I found as well that most of the chips out there are broken, fake, or both and that no one else makes them anymore. Which makes sense because if you want a sound chip in this day, you use an AC97 or HD Audio codec and call it a day, but that goes against that spirit. I think that the spirit on hobby electronics is really found in FPGAs these days instead of rarer and rarer DIP parts. Which is a bit sad, but I guess that's just the passage of time. I wonder if that's how some people felt in the 70s when CPUs replaced many distinct layouts, or if they rejoiced and embraced it instead. I've given up trying to build a system on a breadboard and think that MiSTer is the modern equivalent of that.
bloatfish 3y ago
This is insane! As a hobby hardware designer myself, I can imagine how much work must have gone into reaching this stage. Well done!
nxobject 3y ago
I hope the author goes into some detail about how he implements the PCIe interface! I doubt I'll ever do hardware work at that level of sophistication, but for general cultural awareness I think it's worth looking under the hood of PCIe.
- PfhorSlayer 3y ago
  Next blog post will be covering exactly that! Probably going to do a multi-part series - first one will be the PCB schematic/layout, then the FPGA interfaces and testing, followed by Windows drivers.
- alexforencich 2y ago
  It uses https://github.com/alexforencich/verilog-pcie https://github.com/alexforencich/verilog-pcie on top of the Xilinx PCIe hard IP core, which provides everything below the transaction layer.
- gorkish 3y ago
  The FPGA he is using has native pcie so usually all you get on this front is an interface to a vendor proprietary ip block. The state of open interfaces in FPGA land is abysmal. I think the best I’ve seen fully open source is a gigabit MAC
  - 0xcde4c3db 3y ago
    There is an open-source DisplayPort transmitter [1] that apparently supports multiple 2.7 Gbps lanes (albeit using family-specific SERDES/differential transceiver blocks, but I doubt that's avoidable at these speeds). This isn't PCIe, but it's also surprisingly close to PCIe 1.0 (2.5 Gbps/lane, and IIRC they use the same 8b/10b code and scrambling algorithm). [1] https://github.com/hamsternz/FPGA_DisplayPort https://github.com/hamsternz/FPGA_DisplayPort
  - alexforencich 2y ago
    The GPU uses this: https://github.com/alexforencich/verilog-pcie https://github.com/alexforencich/verilog-pcie . And there is an open-source 100G NIC here, including open source 10G/25G MACs: https://github.com/corundum/corundum https://github.com/corundum/corundum

446 points by argulane 3y ago

It needs to be very fancy to write text in light gray on white. I am not sure your product will be a success. I am sure you web design skills need a good overhaul.

nicolas_17 2y ago
It's not a "product" that will be "sold" or has intention of being "successful" in a commercial sense.

KallDrexx 3y ago

This is my dream! The last year I've been working on a 2d focused GPU for I/O constrained microcontrollers (https://github.com/KallDrexx/microgpu https://github.com/KallDrexx/microgpu). I've been able to utilize this to get user interfaces on slow SPI machines to render on large displays, and it's been fascinating to work on. But seeing the limitation of processor pipelines I've had the thought for a while that FPGAs could make this faster. I've recently gotten some low end FPGAs to start learning to try and turn my microgpu from an ESP32 based one to an FPGA one. I don't know if I"ll ever get to this level due to kids and free time constraints, but man, I would love to get even a hundredth of this level.

Chabsff 3y ago
You probably know this already, but for anyone else curious about going down that road: For this type of use, it's definitely worth it to constrain yourself to FPGAs with dedicated high-bandwidth transceivers. A "basic" 1080p RGB signal at 60hz requires some high-frequency signal processing that's really hard to contend with in pure FPGA-land.
- KallDrexx 3y ago
  That's good to know actually. I'm still very very early in my FPGA adaption (learning the fpga basics) and I am intending to start with standard 640x480 VGA before expanding.

detuur 3y ago

I can't believe that this is the closest we have to a compact, stand-alone GPU option. There's nothing like a M.2 format GPU out there. All I want is a stand-alone M.2 GPU with modest performance, something on the level of embedded GPUs like Intel UHD Graphics, AMD Radeon, or Qualcomm's Adreno. I have an idea for a small embedded product which needs a lot of compute and networking, but only very modest graphical capabilities. The NXP Layerscape LX2160A [1] would be perfect, but I have to pass on it because it doesn't come with an embedded GPU. I just want a small GPU! [1]: https://www.nxp.com/products/processors-and-microcontrollers/arm-processors/layerscape-processors/layerscape-lx2160a-lx2120a-lx2080a-processors:LX2160A https://www.nxp.com/products/processors-and-microcontrollers...

magixx 3y ago
What about MXM GPUs that used to be found in gaming laptops? I know the standard is very niche and thus expensive ($400 for a 3080M used on ebay) but it does exists and you could convert them to PCI-E and thus m.2
t-3 3y ago
Maybe a little bit too low-powered for you, but: https://www.matrixorbital.com/ftdi-eve https://www.matrixorbital.com/ftdi-eve
cpgxiii 3y ago
There's at least one m.2 GPU based on the Silicon Motion SM750 controller made by Asrock Rack. Similar products exist for mPCIe form factor. Performance is nowhere near a modern iGPU, because an iGPU has access to all of the system memory and caches and power budget, and a simple m.2 device has node of that. Even low-end PCIe GPUs (single slot, half-length/half-height) struggle to outperform better iGPUs and really only make sense when you have to use them for basic display functionality.

PfhorSlayer 3y ago

So, this is my project! Was somewhat hoping to wait until there was a bit more content up on the site before it started doing the rounds, but here we are! :) To answer what seems to be the most common question I get asked about this, I am intending on open-sourcing the entire stack (PCB schematic/layout, all the HDL, Windows WDDM drivers, API runtime drivers, and Quake ported to use the API) at some point, but there are a number of legal issues that need to be cleared (with respect to my job) and I need to decide the rest of the particulars (license, etc.) - this stuff is not what I do for a living, but it's tangentially-related enough that I need to cover my ass. The first commit for this project was on August 22, 2021. It's been a bit over two and a half years I've been working on this, and while I didn't write anything up during that process, there are a fair number of videos in my YouTube FuryGpu playlist (https://www.youtube.com/playlist?list=PL4FPA1MeZF440A9CFfMJ7F6YoZ1Jvxu5j https://www.youtube.com/playlist?list=PL4FPA1MeZF440A9CFfMJ7...) that can kind of give you an idea of how things progressed. The next set of blog posts that are in the works concern the PCIe interface. It'll probably be a multi-part series starting at the PCB schematic/layout and moving through the FPGA design and ending with the Windows drivers. No timeline on when that'll be done, though. After having written just that post on how the Texture Units work, I've got even more respect for those that can write up technical stuff like that with any sort of timing consistency. I'll answer the remaining questions in the threads where they were asked. Thanks for the interest!

michaelt 3y ago
Googling the Xilinx Zynq UltraScale+ it seems kinda expensive. Of course plenty of hobbies let people spend thousands (or more) so there's nothing wrong with that if you've got the money. But is it the end target for your project? Or do you have ambitions to go beyond that?
- kanetw 3y ago
  The Kria SOM in use here is like $300.
- PfhorSlayer 3y ago
  Let's be clear here, this is a toy. Beyond being a fun project to work on that could maybe get my foot in the door were I ever to decide to change careers and move into hardware design, this is not going to change the GPU landscape or compete with any of the commercial players. What it might do is pave the way for others to do interesting things in this space. A board with all of the video hardware that you can plug into a computer with all the infrastructure available to play around with accelerating graphics could be a fun, if extremely niche, product. That would also require a *significant* time and money investment from me, and that's not something I necessarily want to deal with. When this is eventually open-sourced, those who really are interested could make their own boards. One thing to note that is that while the US+ line is generally quite expensive (the higher end parts sit in the five-figures range for a one-off purchase! No one actually buying these is paying that price, but still!), the Kria SOMs are quite cheap in comparison. They've got a reasonably-powerful Zynq US+ for about $400, or just $350ish the dev boards (which do not expose some of the high-speed interfaces like PCIe). I'm starting to sound like a Xilinx shill given how many times I've re-stated this, but for anyone serious about getting into this kind of thing, those devboards are an amazing deal.

raphlinus 3y ago

Very cool project, and I love to see more work in this space. Something else to look at is the Vortex project from Georgia Tech[1]. Rather than recapitulating the fixed-function past of GPU design, I think it looks toward the future, as it's at heart a highly parallel computer, based on RISC-V with some extensions to handle GPU workloads better. The boards it runs on are a few thousand dollars, so it's not exactly a hobbyist friendly, but it certainly is more accessible than closed, proprietary development. There's a 2.0 release that just landed a few months ago. [1]: https://vortex.cc.gatech.edu/ https://vortex.cc.gatech.edu/

userbinator 3y ago

Supporting hardware features equivalent to a high-end graphics card of the mid 1990s I see no one else has asked this question yet, so I will: How VGA-compatible is it? Would I be able to e.g. plug it into any PC with a PCIe slot, boot to DOS and play DOOM with it?

bobharris 2y ago

beyond amazing. i've dreamt of this. so inspiring. it reminds me of alot of time i spent thinking about this: https://rcl.ece.iastate.edu/sites/default/files/papers/SteJon11A.pdf https://rcl.ece.iastate.edu/sites/default/files/papers/SteJo... i actually wrote one of the professors asking for more info. didn't get a reply. my dream EE class I never got to take.

allanrbo 2y ago

What an inspiring passion project! Very ambitious first Verilog project.

anon115 2y ago

can you run valorant on it?

snvzz 3y ago

Pipeline seems retro, but far better than nothing. There's no open hardware GPU to speak of. Depending on license (can't find information?), this could be the first, and a starting point for more.

mips_r4300i 3y ago
Ticket2Ride Number9 is a fixed function GPU from the late 90s that was completely open sourced under GPL
Hazematman 3y ago
There's also Nyuzi which is more GPGPU focused https://github.com/jbush001/NyuziProcessor https://github.com/jbush001/NyuziProcessor, but the author also experimented with having it do 3D graphics.
userbinator 3y ago
Depends what you mean by "GPU". https://github.com/schlae/graphics-gremlin https://github.com/schlae/graphics-gremlin is an MDA/CGA compatible adapter https://github.com/OmarMongy/VGA https://github.com/OmarMongy/VGA is a VGA core https://github.com/archlabo/Frix https://github.com/archlabo/Frix is a whole IBM PC compatible SoC, including a VGA.
deleted 3y ago
[deleted]
crote 3y ago
It all depends on your definition of "open", of course. As far as I know there is no open-source toolchain for any remotely-recent FPGA, so you're still stick with proprietary (paid?) tooling to actually modify it. You're pretty much out of luck if you need more than an iCE40 UP5k.
- rwmj 3y ago
  At least some Xilinx 7-series FPGAs have been reverse engineered: https://yosyshq.readthedocs.io/projects/yosys/en/latest/cmd/synth_xilinx.html https://yosyshq.readthedocs.io/projects/yosys/en/latest/cmd/...

jamesu 3y ago

Similarly there is this: https://github.com/ToNi3141/Rasterix https://github.com/ToNi3141/Rasterix Would be neat if someone made an FPGA GPU which had a shader pipeline honestly.

danbruc 3y ago
If you are going to that effort, you might also want a decent resolution. Say we aim for one megapixel (720p) and 30 frames per second, then we have to calculate 27.7 megapixel per second. If you get your FPGA to run at 500 MHz, that gives you 18 clock cycles per pixel. So you would probably want something like 100 cores keeping in mind that we also have to run vertex shaders. We also need quick access to a sizable amount of memory and I am not sure if one can get away with integer respectively fixed point arithmetics or whether floating point arithemtics is pretty much necessary. Another complication that I would expect is that it is probably much easier to build a long execution pipeline if you are implementing a fixed function pipeline as compared to a programmable processor. Things like out-of-order execution are probably best off-loaded to the compiler in order to keep the design simpler and more compact. So my guess is that it would be quite challenging to implement a modern GPU in an affordable FPGA if you want more than a proof of concept.
- d_tr 3y ago
  There's a new board by Trenz with a Versal chip which can do 440 GFLOPS just with the DSP58 slices (the lowest speed grade) and it costs under 1000 Euros, but you also need to buy a Vivado license currently. Cheaper boards are definitely possible since there are smaller parts in that family, but they need to offer support for some of them in the free version of Vivado...
- PfhorSlayer 3y ago
  You've nailed the problem directly on the head. For hitting 60Hz in FuryGpu, I actually render at 640x360 and then pixel-double (well, pixel->quad) the output to the full 720p. Even with my GPU cores running at 400MHz and the texture units at 480MHz with fully fixed-function pipelines, it can still struggle to keep up at times. I do not doubt that a shader core could be built, but I have reservations about the ability to run it fast enough or have as many of them as would be needed to get similar performance out of them. FuryGpu does its front-end (everything up through primitive assembly) in full fp32. Because that's just a simple fixed modelview-projection matrix transform it can be done relatively quickly, but having every single vertex/pixel able to run full fp32 shader instructions requires the ability to cover instruction latency with additional data sets - it gets complicated, fast!

gchadwick 3y ago

Cool! I found the hello blog here illuminating to understand the creators intentions: https://www.furygpu.com/blog/hello https://www.furygpu.com/blog/hello As I read it, it's just a fun hobby project for them first and foremost and looks like they're intending to write a whole bunch more about how they built it. It's certainly an impressive piece of work, in particular as they've got the full stack working, a windows driver implementing a custom graphics API and then quake running on top of that. A shame they've not got some DX/GL support but I can certainly understand why they went the custom API route. I wonder if they'll open source the design?

PfhorSlayer 3y ago
I'm in the process of actually trying to work out what would be feasible performance-wise if I were to spent the considerable effort to add the features required for base D3D support. It's not looking good, unfortunately. Beyond just "shaders", there are a significant amount of other requirements that even just the OS's window manager needs to function at all. It's all built up on 20+ years of evolving tech and for the normal players in this space (AMD, Nvidia, Intel, Imagination, etc.) it's always been an iterative process.

deleted 3y ago

[deleted]

iAkashPaul 3y ago

FPGAs for native FP4 will change the entire landscape

blacklion 3y ago
Entire landscape of open graphic chips? Not every GPU should be used to train or infer so-called AI. Please, stop, we need some hardware to put images on the screens.
imtringued 2y ago
How? NPUs are going to be included in every PC in 2025. The only differentiators will be how much SRAM and memory bandwidth you have or whether you use processing in memory or not. AMD is already shipping APUs with 16 TOPS or 4 TFLOPS (bfloat16) and that is more than enough for inference considering the limited memory bandwidth. Strix Halo will have around 12 TFLOPS (bfloat16) and four memory channels. llama.cpp already supports 4 bit quantization. They unpack the quantization back to bfloat16 at runtime for better accuracy. The best use case for an FPGA I have seen so far was to pair it with SK Hynix's AI GDDR and even that could be replaced by an even cheaper inference chip specializing in multi board communication and as many memory channels as possible.
luma 3y ago
How so?
- CamperBob2 3y ago
  4-bit values (or 6-bit values, nowadays) values are interesting because they're small enough to address a single LUT, which is the lowest-level atomic element of an FPGA. That gives them major advantages in the timing and resource-usage departments.
- iAkashPaul 3y ago
  Reduced memory requirements, dropping higher precision IP blocks for starters
jsheard 3y ago

spuz 3y ago

This looks like an incredible achievement. I'd love to see some photos of the physical device. I'm also slightly confused about which FGPA module is being used. The blog mentions the Xylinx Kria SoMs but if you follow the links to the specs of those modules, you see they have ARM SoCs rather than Xylinx FGPAs. The whole world of FGPAs is pretty unfamiliar to me so maybe I'm missing something. https://www.amd.com/en/products/system-on-modules/kria/k26/k26c-commercial.html https://www.amd.com/en/products/system-on-modules/kria/k26/k...

PfhorSlayer 3y ago
You're in luck! https://imgur.com/a/BE0h9cZ https://imgur.com/a/BE0h9cZ As mentioned in the rest of this thread, the Kria SoMs are FPGA fabric with hardened ARM cores running the show. Beyond just being what was available (for oh so cheap, the Kria devboards are like $350!), these devices also include things like hardened DisplayPort IP attached to the ARM cores allowing me to offload things like video output and audio to the firmware. A previous version of this project was running on a Zynq 7020, for which I needed to write my own HDMI stuff that, while not super complicated, takes up a fair amount of logic and also gets way more complex if it needs to be configurable.
crote 3y ago
> you see they have ARM SoCs rather than Xylinx FGPAs It's a mixed chip: FPGA and traditional SoC glued together. This mean you don't have a softcore MCU taking up precious FPGA resources just to do some basic management tasks.
- spuz 3y ago
  Ah that makes sense. It's slightly ironic then that the ARM SoC includes a Mali GPU which presumably easily outperforms what can be achieved with the FGPA.
- chrsw 3y ago
  I didn't see any mention of what the software on the Zynq's ARM core is doing, which made me wonder why use Zynq at all.

wpwpwpw 3y ago

Excellent job. Would be amazing if this became an open source hardware project.

sylware 3y ago

Hopefully their hardware programming model is going full hardware circular command/interrupt buffers (even for GPU register programming). It is how it is done on AMD GPU, that said I have no idea what is the nvidia hardware programming model.

codedokode 3y ago

"UltraScale" in name assumes ultra price? FPGAs seem to be an expensive toy.

PfhorSlayer 3y ago
In general, yes. However, the Kria series are amazingly good deals for what you get - a quite powerful Zynq US+ part and a dev board for like $350.
varispeed 3y ago
Ages ago I bought TinyFPGA, which is like £40 and I was able to synthesize RISC-V cpu on it. It was fun.
mattalex 3y ago
Not in the grand scheme of things: you can get fpga dev boards for $50 that are already useable for this type of thing (you can go even lower, but those aren't really useable for "CPU like" operation and are closer to "a whole lot of logic gates in a single chip"). Of course the "industry grade" solutions pack significantly more of a punch, but they can also be had for <$500.
nxobject 3y ago
It's worth mentioning that it's easy enough to find absurdly cheap (~$20) early-generation dev boards for Zynq FPGAs with embedded ARM cores on Aliexpress, shucked from obsolete Bitcoin miners [1]. Interfaces include SD, Ethernet, 3 banks of GPIO. [1] https://github.com/xjtuecho/EBAZ4205 https://github.com/xjtuecho/EBAZ4205
- deleted 3y ago
  [deleted]
- thrtythreeforty 3y ago
  Zynq is deeply annoying to work with, though. Unfortunately the hard ARM core bootloads the FPGA fabric, rather than the other way around (or having the option to initialize both separately). This means you have to muck with software on the target to update FPGA bitstreams.

MalphasWats 3y ago

It's incredible how influential Ben Eater's breadboard computer series has been in hobby electronics. I've been similarly inspired to try to design my own "retro" CPU. I desperately want something as easy to plug into things as the 6502, but with jussst a little more capability - few more registers, hardware division, that sort of thing. It's a really daunting task. I always end up coming back to just use an MCU and be done with it, and then I hit the How To Generate Graphics problem.

MenhirMike 3y ago
I was about to recommend the Parallax Propeller (the first one that's available in DIP format), but arguably, that one is way more complex to program for (and also significantly more powerful, and at that point you might as well look into an ESP32 and that is "just use an MCU" :)) And yeah, video output is a significant issue because of the required bandwidth for digital outputs (unless you're okay with composite or VGA outputs, I guess they can still be done with readily available chips?). The recent Commander X16 settled for an FPGA for this.
- MalphasWats 3y ago
  I feel like the CX16 lost its way about a week after the project started and it suddenly became an expensive FPGA-based blob. But at the same time, I'm not sure what other option there is for a project like that. I always got the impression that David sort of got railroaded by the other members of the team that wanted to keep adding features and MOAR POWAH, and didn't have a huge amount of choice because those features quickly scoped out of his own areas of knowledge.
  - MenhirMike 3y ago
    I think so too - it must have been a great learning experience for him though, but for me, the idea of "The best C64-like computer that ever existed" died pretty quickly. He also did run into a similar problem that I ran into when I tried something like that as well: Sound Chips. Building a system around a Yamaha FM Synthesizer is perfect, but I found as well that most of the chips out there are broken, fake, or both and that no one else makes them anymore. Which makes sense because if you want a sound chip in this day, you use an AC97 or HD Audio codec and call it a day, but that goes against that spirit. I think that the spirit on hobby electronics is really found in FPGAs these days instead of rarer and rarer DIP parts. Which is a bit sad, but I guess that's just the passage of time. I wonder if that's how some people felt in the 70s when CPUs replaced many distinct layouts, or if they rejoiced and embraced it instead. I've given up trying to build a system on a breadboard and think that MiSTer is the modern equivalent of that.

bloatfish 3y ago

This is insane! As a hobby hardware designer myself, I can imagine how much work must have gone into reaching this stage. Well done!

nxobject 3y ago

I hope the author goes into some detail about how he implements the PCIe interface! I doubt I'll ever do hardware work at that level of sophistication, but for general cultural awareness I think it's worth looking under the hood of PCIe.

PfhorSlayer 3y ago
Next blog post will be covering exactly that! Probably going to do a multi-part series - first one will be the PCB schematic/layout, then the FPGA interfaces and testing, followed by Windows drivers.
alexforencich 2y ago
It uses https://github.com/alexforencich/verilog-pcie https://github.com/alexforencich/verilog-pcie on top of the Xilinx PCIe hard IP core, which provides everything below the transaction layer.
gorkish 3y ago
The FPGA he is using has native pcie so usually all you get on this front is an interface to a vendor proprietary ip block. The state of open interfaces in FPGA land is abysmal. I think the best I’ve seen fully open source is a gigabit MAC
- 0xcde4c3db 3y ago
  There is an open-source DisplayPort transmitter [1] that apparently supports multiple 2.7 Gbps lanes (albeit using family-specific SERDES/differential transceiver blocks, but I doubt that's avoidable at these speeds). This isn't PCIe, but it's also surprisingly close to PCIe 1.0 (2.5 Gbps/lane, and IIRC they use the same 8b/10b code and scrambling algorithm). [1] https://github.com/hamsternz/FPGA_DisplayPort https://github.com/hamsternz/FPGA_DisplayPort
- alexforencich 2y ago
  The GPU uses this: https://github.com/alexforencich/verilog-pcie https://github.com/alexforencich/verilog-pcie . And there is an open-source 100G NIC here, including open source 10G/25G MACs: https://github.com/corundum/corundum https://github.com/corundum/corundum