9 ms·

SuperRT – Realtime Raytracing on the SNES

340 points by mkeeter 6y ago

csharptwdec19 6y ago
Very cool. > Thus the full image can only be refreshed once every two frames, effectively limiting the maximum framerate to 30FPS - although the test scene runs at closer to 20FPS (primarily due to some bottlenecks with the logic on the SNES side at present). Getting around this would make this even more interesting. Like most other console PPUs of it's day, due to bandwidth/etc the normal 'flow' is to have a background and a series of 'character' sprites preloaded into VRAM and then shuffled around (this is why in some 'busy' games you would see clipping or game elements disappear on early consoles; too many sprite objects for the PPU to keep track of.) To that same end, I'd be curious to see what could be done like this on Genesis/Mega Drive hardware
raidicy 6y ago
This project seems like absolute magic to me.
- sgrove 6y ago
  In that case, you might like one of tom7's many wonnderful videos [0], "Reverse emulating the NES to give it SUPER POWERS!" where he hacks a real NES cartridge with a raspberry pi with an emulator, then figures out how to map all of the data into NES sprites, etc. As usual for his videos, it's absolutely wonderful. [0] https://www.youtube.com/watch?v=ar9WRwCiSr0 https://www.youtube.com/watch?v=ar9WRwCiSr0
  - farias0 6y ago
    This is one of my favorite videos on all YouTube. I know in retrospect it seems obvious, but when it showed that it was all running on a NES it blew my mind. This video felt like transforming nerd hackery into an art form.
- wyldfire 6y ago
  It's funny that you mention that. I, too, often have this reaction to really interesting, cool projects and accomplishments. But over the years, I've learned to realize that the stuff that seems magical is not beyond my reach. Look no further than this article itself: > The idea originated when I was trying to think of an interesting idea for a project to help me learn Verilog and FPGA design, and the notion of building a simple raytracer came to mind (partly inspired by a scarily smart friend of mine who is building his own GPU). The way this author describes his friend who is working on a challenging design is 'scarily smart.' IMO "that other skill domain" seems intimidating from here but if you start small and are willing to put in the time, you can do it. The people who work on biochemistry, mechanical engineering, auto repair, digital hardware -- they're all just people, after all.
MaxBarraclough 6y ago
* on an expansion card. This is not real-time raytracing on standard SNES hardware, as the title implies. Still a very impressive project, of course.
- deleted 6y ago
  [deleted]
- rzzzt 6y ago
  In mice! Is Star Fox with the Super FX chip also considered cheating?
- r3nruturnEr 6y ago
  Where in the title does it say that it's a standard SNES?
- ivanbakel 6y ago
  >This is not real-time raytracing on standard SNES hardware, as the title implies. I'm not sure the title implies that. Performant raytracing on SNES hardware would be (to my understanding) pure magic - SNES modes may have been good, but none of them were that good. Just reimplementing raytracing in software on a SNES would also not really be that interesting technically. It can already do any kind of calculation, if slowly. Expansion cards are established ways to improve the SNES's hardware capabilities, as many cartridges did, especially toward the end of the console's lifetime. It's not dishonest to describe using one as doing something "on the SNES" - especially when part of the writeup is about technical challenges interfacing the expansion chip with the actual console.
  - kbenson 6y ago
    It sort of is. The SNES was sold as a specific unit with specific licensed cartridges, and has a well known set of expansion chops shipped in cartridges. I think it's pretty well understood that because of this when someone says "on the SNES" they mean using the existing hardware, possibly plus existing chips shipped at the time. The idea conveyed is that this could have been done at the time if only the expertise and effort were put into it. Sort of like the Commodore demo scene. What they can accomplish with the original hardware is amazing. If they were relying on additional hardware to do so, it's less interesting (but may still be interesting, and an accomplishment, as this is). A more accurate title based on how people might think of this might be "SuperRT – Hacking SNES hardware to support realtime raytracing" (and even then you're going to get people assuming it's all original hardware).
CyberRabbi 6y ago
Saw a comment on YouTube about implementing this expansion chip as an sd2snes firmware. If the expansion chip interface was standardized would be cool to see Homebrew using this. Kind of like byuu’s MSU-1 chip.
croes 6y ago
Take this Nvidia
Grieverheart 6y ago
I also have a DE10-nano for running MiSTer cores and I'd like to learn to program the FPGA. Do you have any recommendations on getting started?
- nullobject 6y ago
  Sure. I wrote the Tecmo and CAVE cores for MiSTer, and it's a long road to a complex core like an arcade game, or in this case, a ray tracer. If you're just starting out, then you've gotta begin with the basics: blinking a few LEDs, etc. From there you can progress to more and more complex circuits as you learn to code in VHDL or Verilog. I'd recommend buying a couple of 2nd hand books on VHDL and/or Verilog to get familiar with the syntax. There are some good free books too: http://freerangefactory.org/pdf/df344hdh4h8kjfh3500ft2/free_range_vhdl.pdf http://freerangefactory.org/pdf/df344hdh4h8kjfh3500ft2/free_... This little book is great if you're interested in how to build video game hardware for FPGA: https://gumroad.com/8bitworkshop#JGZkq https://gumroad.com/8bitworkshop#JGZkq I also have a repo of examples I wrote for the DE10 Nano: https://github.com/nullobject/de10-nano-examples https://github.com/nullobject/de10-nano-examples
  - Grieverheart 6y ago
    Thanks! I have already read the 8bitworkshop one, but I was on vacation back then so I couldn't get my hands dirty. The samples are also definitely going to help. I do like the approach of using something like Verilator, though. The barrier to entry for actually doing stuff on the board seems too large for me at the moment -- Need to install and learn software (Quartus), and my everyday computer is a Mac so I have to use a slow VM!
bovermyer 6y ago
Some people are just orders of magnitude more intelligent than I ever was. This is truly impressive.
- echelon 6y ago
  Or have time, patience, and energy to follow through on an interest.
  - dimator 6y ago
    This is the "growth mindset" interpretation, and it's really a super power if you believe it.
  - bovermyer 6y ago
    This isn't like woodcarving, which is fairly intuitive and requires only a lot of practice. FPGA design, and graphics programming generally, requires higher-order math. I have always struggled mightily with trigonometry; something about it just doesn't click in my brain. Calculus I can handle, but spatial math breaks down in my head. I could probably figure it out given enough time, but it would take much more time than it would take someone whose brain is aligned for it.
xwdv 6y ago
Are there any cpu cycles left to make some kind of decent game? Say, Super Mario 64 or Star Fox?
- Kranar 6y ago
  Decent game? Probably not. Based on the article the demo runs at 20 FPS completely saturating the SNES' CPU with a hard limit of 30 FPS due to bandwidth constraints. At 20 FPS you're already pretty laggy; dropping the FPS further to manage game logic will likely make playing the game an unpleasant experience.
1-6 6y ago
---“SNES” and “Super Nintendo” are trademarks of Nintendo Co Ltd. This is a hobby project and completely unassociated with Nintendo.--- But I think Nintendo would gladly like to accept your suggestions :-)
- dirtnugget 6y ago
  Well they do tend to sue
nielsbot 6y ago
Super fun! Would love to see the output on an honest-to-goodness CRT.
kelvin0 6y ago
Wow, this is stunning! Now if I forgo sleep for the next 5 years, I might be able to finally finish that hacking/reverse engineer project I've been putting off ...
Rebelgecko 6y ago
This is pretty neat. Is anyone familiar with the limitations of expansion cards? Like, would it be possible to put a flash drive + an x86 CPU and have the SNES pull from a frame buffer so you can play Crysis on it?
Voltage 6y ago
Very cool indeed. Seems like a good platform to apply the demoscene trick of "adaptive subsampling" to greatly improve performance at the cost of a little image quality. It takes advantage of the fact that pixels that are close together in screenspace are likely to be similar enough to interpolate from it's neighbours where possible. It focuses the computation time on areas where neighbouring pixels are very different, such as object boundaries, shadow boundaries and large lighting differences. https://web.archive.org/web/20101123123238/http://exceed.hu/h7/subsample.htm https://web.archive.org/web/20101123123238/http://exceed.hu/...
Andrex 6y ago
One of my biggest dream retro projects like this would be taking the cartridge expansion idea and using it to integrate a Wi-Fi radio and full online play connected to a central server. (I originally thought of it for GameBoy, but a Super NES cart would offer a lot more flexibility...) You can see the bones of such technology in real products: https://en.wikipedia.org/wiki/Satellaview https://en.wikipedia.org/wiki/Satellaview http://nectaris.tg-16.com/GB-KISS-LINK-FAQ-2-hudson-gameboy-nectaris.html http://nectaris.tg-16.com/GB-KISS-LINK-FAQ-2-hudson-gameboy-... https://shop.insidegadgets.com/product/wireless-gameboy-controller/ https://shop.insidegadgets.com/product/wireless-gameboy-cont...
- flatiron 6y ago
  https://www.youtube.com/watch?v=ar9WRwCiSr0 https://www.youtube.com/watch?v=ar9WRwCiSr0 Raspberry pi in a NES cart. No reason you couldn’t add netplay to this.
  - Andrex 6y ago
    That would be a fantastic place to start, thank you! ...I hope NES programming is similar to GB/Z80, since as far as retro gaming tools go that's all I know right now, haha.
- jf 6y ago
  What would you get from this ideal cart that you wouldn’t get from a flash cart? Online play?
  - Andrex 6y ago
    The online game's ROM and cartridge hardware including the Wi-Fi radio would have to be made together. As the Wi-Fi hardware inside the cartridge would be custom, the ROM would have to contain instructions that would not be emulatable by a flash cart (or any other means.) Edit- Missed "online play?" in your comment. Yes, that's the idea, new homebrew games with actual working online, made for systems released 25-30 years ago.
desktopninja 6y ago
Old tech still rules!
djmips 6y ago
Or in other words, Realtime Raytracing on an FPGA and using the SNES and an awkward display.
- djmips 6y ago
  Just rub your project in retro vibes and it becomes 'remarkable'. But I am not thaat jaded. It's cool. ;-) In fact, there is an original history on the SNES of using coprocessors in the cartridge to enhance the rather underpowered 65816.

340 points by mkeeter 6y ago

Very cool. > Thus the full image can only be refreshed once every two frames, effectively limiting the maximum framerate to 30FPS - although the test scene runs at closer to 20FPS (primarily due to some bottlenecks with the logic on the SNES side at present). Getting around this would make this even more interesting. Like most other console PPUs of it's day, due to bandwidth/etc the normal 'flow' is to have a background and a series of 'character' sprites preloaded into VRAM and then shuffled around (this is why in some 'busy' games you would see clipping or game elements disappear on early consoles; too many sprite objects for the PPU to keep track of.) To that same end, I'd be curious to see what could be done like this on Genesis/Mega Drive hardware

raidicy 6y ago

This project seems like absolute magic to me.

sgrove 6y ago
In that case, you might like one of tom7's many wonnderful videos [0], "Reverse emulating the NES to give it SUPER POWERS!" where he hacks a real NES cartridge with a raspberry pi with an emulator, then figures out how to map all of the data into NES sprites, etc. As usual for his videos, it's absolutely wonderful. [0] https://www.youtube.com/watch?v=ar9WRwCiSr0 https://www.youtube.com/watch?v=ar9WRwCiSr0
- farias0 6y ago
  This is one of my favorite videos on all YouTube. I know in retrospect it seems obvious, but when it showed that it was all running on a NES it blew my mind. This video felt like transforming nerd hackery into an art form.
wyldfire 6y ago
It's funny that you mention that. I, too, often have this reaction to really interesting, cool projects and accomplishments. But over the years, I've learned to realize that the stuff that seems magical is not beyond my reach. Look no further than this article itself: > The idea originated when I was trying to think of an interesting idea for a project to help me learn Verilog and FPGA design, and the notion of building a simple raytracer came to mind (partly inspired by a scarily smart friend of mine who is building his own GPU). The way this author describes his friend who is working on a challenging design is 'scarily smart.' IMO "that other skill domain" seems intimidating from here but if you start small and are willing to put in the time, you can do it. The people who work on biochemistry, mechanical engineering, auto repair, digital hardware -- they're all just people, after all.

MaxBarraclough 6y ago

* on an expansion card. This is not real-time raytracing on standard SNES hardware, as the title implies. Still a very impressive project, of course.

deleted 6y ago
[deleted]
rzzzt 6y ago
In mice! Is Star Fox with the Super FX chip also considered cheating?
r3nruturnEr 6y ago
Where in the title does it say that it's a standard SNES?
ivanbakel 6y ago
>This is not real-time raytracing on standard SNES hardware, as the title implies. I'm not sure the title implies that. Performant raytracing on SNES hardware would be (to my understanding) pure magic - SNES modes may have been good, but none of them were that good. Just reimplementing raytracing in software on a SNES would also not really be that interesting technically. It can already do any kind of calculation, if slowly. Expansion cards are established ways to improve the SNES's hardware capabilities, as many cartridges did, especially toward the end of the console's lifetime. It's not dishonest to describe using one as doing something "on the SNES" - especially when part of the writeup is about technical challenges interfacing the expansion chip with the actual console.
- kbenson 6y ago
  It sort of is. The SNES was sold as a specific unit with specific licensed cartridges, and has a well known set of expansion chops shipped in cartridges. I think it's pretty well understood that because of this when someone says "on the SNES" they mean using the existing hardware, possibly plus existing chips shipped at the time. The idea conveyed is that this could have been done at the time if only the expertise and effort were put into it. Sort of like the Commodore demo scene. What they can accomplish with the original hardware is amazing. If they were relying on additional hardware to do so, it's less interesting (but may still be interesting, and an accomplishment, as this is). A more accurate title based on how people might think of this might be "SuperRT – Hacking SNES hardware to support realtime raytracing" (and even then you're going to get people assuming it's all original hardware).

CyberRabbi 6y ago

Saw a comment on YouTube about implementing this expansion chip as an sd2snes firmware. If the expansion chip interface was standardized would be cool to see Homebrew using this. Kind of like byuu’s MSU-1 chip.

croes 6y ago

Take this Nvidia

Grieverheart 6y ago

I also have a DE10-nano for running MiSTer cores and I'd like to learn to program the FPGA. Do you have any recommendations on getting started?

nullobject 6y ago
Sure. I wrote the Tecmo and CAVE cores for MiSTer, and it's a long road to a complex core like an arcade game, or in this case, a ray tracer. If you're just starting out, then you've gotta begin with the basics: blinking a few LEDs, etc. From there you can progress to more and more complex circuits as you learn to code in VHDL or Verilog. I'd recommend buying a couple of 2nd hand books on VHDL and/or Verilog to get familiar with the syntax. There are some good free books too: http://freerangefactory.org/pdf/df344hdh4h8kjfh3500ft2/free_range_vhdl.pdf http://freerangefactory.org/pdf/df344hdh4h8kjfh3500ft2/free_... This little book is great if you're interested in how to build video game hardware for FPGA: https://gumroad.com/8bitworkshop#JGZkq https://gumroad.com/8bitworkshop#JGZkq I also have a repo of examples I wrote for the DE10 Nano: https://github.com/nullobject/de10-nano-examples https://github.com/nullobject/de10-nano-examples
- Grieverheart 6y ago
  Thanks! I have already read the 8bitworkshop one, but I was on vacation back then so I couldn't get my hands dirty. The samples are also definitely going to help. I do like the approach of using something like Verilator, though. The barrier to entry for actually doing stuff on the board seems too large for me at the moment -- Need to install and learn software (Quartus), and my everyday computer is a Mac so I have to use a slow VM!

bovermyer 6y ago

Some people are just orders of magnitude more intelligent than I ever was. This is truly impressive.

echelon 6y ago
Or have time, patience, and energy to follow through on an interest.
- dimator 6y ago
  This is the "growth mindset" interpretation, and it's really a super power if you believe it.
- bovermyer 6y ago
  This isn't like woodcarving, which is fairly intuitive and requires only a lot of practice. FPGA design, and graphics programming generally, requires higher-order math. I have always struggled mightily with trigonometry; something about it just doesn't click in my brain. Calculus I can handle, but spatial math breaks down in my head. I could probably figure it out given enough time, but it would take much more time than it would take someone whose brain is aligned for it.

xwdv 6y ago

Are there any cpu cycles left to make some kind of decent game? Say, Super Mario 64 or Star Fox?

Kranar 6y ago
Decent game? Probably not. Based on the article the demo runs at 20 FPS completely saturating the SNES' CPU with a hard limit of 30 FPS due to bandwidth constraints. At 20 FPS you're already pretty laggy; dropping the FPS further to manage game logic will likely make playing the game an unpleasant experience.

1-6 6y ago

---“SNES” and “Super Nintendo” are trademarks of Nintendo Co Ltd. This is a hobby project and completely unassociated with Nintendo.--- But I think Nintendo would gladly like to accept your suggestions :-)

dirtnugget 6y ago
Well they do tend to sue

nielsbot 6y ago

Super fun! Would love to see the output on an honest-to-goodness CRT.

kelvin0 6y ago

Wow, this is stunning! Now if I forgo sleep for the next 5 years, I might be able to finally finish that hacking/reverse engineer project I've been putting off ...

Rebelgecko 6y ago

This is pretty neat. Is anyone familiar with the limitations of expansion cards? Like, would it be possible to put a flash drive + an x86 CPU and have the SNES pull from a frame buffer so you can play Crysis on it?

Voltage 6y ago

Very cool indeed. Seems like a good platform to apply the demoscene trick of "adaptive subsampling" to greatly improve performance at the cost of a little image quality. It takes advantage of the fact that pixels that are close together in screenspace are likely to be similar enough to interpolate from it's neighbours where possible. It focuses the computation time on areas where neighbouring pixels are very different, such as object boundaries, shadow boundaries and large lighting differences. https://web.archive.org/web/20101123123238/http://exceed.hu/h7/subsample.htm https://web.archive.org/web/20101123123238/http://exceed.hu/...

Andrex 6y ago

One of my biggest dream retro projects like this would be taking the cartridge expansion idea and using it to integrate a Wi-Fi radio and full online play connected to a central server. (I originally thought of it for GameBoy, but a Super NES cart would offer a lot more flexibility...) You can see the bones of such technology in real products: https://en.wikipedia.org/wiki/Satellaview https://en.wikipedia.org/wiki/Satellaview http://nectaris.tg-16.com/GB-KISS-LINK-FAQ-2-hudson-gameboy-nectaris.html http://nectaris.tg-16.com/GB-KISS-LINK-FAQ-2-hudson-gameboy-... https://shop.insidegadgets.com/product/wireless-gameboy-controller/ https://shop.insidegadgets.com/product/wireless-gameboy-cont...

flatiron 6y ago
https://www.youtube.com/watch?v=ar9WRwCiSr0 https://www.youtube.com/watch?v=ar9WRwCiSr0 Raspberry pi in a NES cart. No reason you couldn’t add netplay to this.
- Andrex 6y ago
  That would be a fantastic place to start, thank you! ...I hope NES programming is similar to GB/Z80, since as far as retro gaming tools go that's all I know right now, haha.
jf 6y ago
What would you get from this ideal cart that you wouldn’t get from a flash cart? Online play?
- Andrex 6y ago
  The online game's ROM and cartridge hardware including the Wi-Fi radio would have to be made together. As the Wi-Fi hardware inside the cartridge would be custom, the ROM would have to contain instructions that would not be emulatable by a flash cart (or any other means.) Edit- Missed "online play?" in your comment. Yes, that's the idea, new homebrew games with actual working online, made for systems released 25-30 years ago.

desktopninja 6y ago

Old tech still rules!

djmips 6y ago

Or in other words, Realtime Raytracing on an FPGA and using the SNES and an awkward display.

djmips 6y ago
Just rub your project in retro vibes and it becomes 'remarkable'. But I am not thaat jaded. It's cool. ;-) In fact, there is an original history on the SNES of using coprocessors in the cartridge to enhance the rather underpowered 65816.