6 ms·

Can someone explain the advantage of increasing the native resolution vs upscaling after rendering? For something like a photograph, you want as high a 'native

Can someone explain the advantage of increasing the native resolution vs upscaling after rendering? For something like a photograph, you want as high a 'native resolution' as possible, so that there is as much information as possible in the original image. But for old video games, the assets and textures are still the same. Is this a bit like taking a high megapixel photo of a low quality print? Or is my understanding wrong? I'm sure things like AA work better in the native renderer. Are there other advantages?

SECProto 6y ago
There are some projects that swap out textures for higher res versions. But I think the bigger advantage is for distant objects - if you zoom in the goldeneye screenshot, you can see the face texture of the guy at the other end of the hall. If the regular native resolution had been used and upscaled afterwards, you wouldn't see the face at all, just a face-coloured blur
Polylactic_acid 6y ago
I don't know the details of this project but I have seen emulators use processes more complex than a basic image upscale to get really nice results out of low resolution textures. I think its similar to those anime upscailing tools where its trained on a dataset of textures and is able to redraw them at higher resolutions.
- saagarjha 6y ago
  Sadly, those techniques aren’t nearly fast enough to work for emulation.
  - Polylactic_acid 6y ago
    Could they not run once on all of the textures and then save them to a file for use in game?
  - rossy 6y ago
    You'd be surprised at what image processing algorithms are fast enough to work in realtime when they have optimised GPU implementations. Anime4K is one of those CNN-based anime upscaling tools, and it can run in realtime as a shader in mpv. (I'm not a fan of those upscaling tools myself, but I don't see why they couldn't be used in emulators, for the people who like them.)
- louhike 6y ago
  This emulator prioritizes accuracy, not fancy graphical options, that's why they lack this kind of features.
Hydraulix989 6y ago
The N64's "GPU" is the Reality Display Processor (RDP) chip which is specialized hardware built by Silicon Graphics specifically for the N64 console. The RDP runs a custom program that is part of each game that tells it how to take high-level commands from the game code and render them. For this accurate N64 emulation, it is being emulated at a lower level than before. Previously, N64 emulators just used the commands sent to the RDP to tell the host system GPU what to do with something higher-level like OpenGL or DirectX (of course, this meant a lot of game-specific "hacks" in the emulator), rather than emulating the RDP itself by sending lower-level commands directly from the RDP to the host GPU with something like Vulkan. This is so-called High-Level Emulation (HLE), and it's a massive shortcut to emulating the whole RDP -- which meant that N64 could even be "emulated" on a PC from 1999. Lower-level emulation of the RDP itself has been recently made possible, and now it can also now be "up-scaled" to arbitrary resolutions -- instead of just sticking to high level emulation and telling OpenGL or DirectX to render to a larger resolution -- or even worse, scaling the rasterized rendered output frames by treating them as images. In practice, Mario 64 was just telling the RDP to render triangles anyways so it mapped nicely to OpenGL for the HLE case, but for more "accurate" emulation (this is like getting from 95% to 99%), the RDP itself needs to be emulated as well for things like the Perfect Dark drone camera mentioned elsewhere.
- midnightclubbed 6y ago
  To be pedantic the RDP was the rasterizer (fixed function) which was fed by the RSP processor. The RSP ran the display-list commands from the cpu/memory (very similar to a Vulkan command list) and did vertex transforms/clipping before feeding its output to the RDP. Most games used one of a few Nintendo provided RSP programs although later in the machine's lifetime they opened up the RSP compiler and tools to developers.
  - Hydraulix989 6y ago
    Yes, I was not going to go into that much detail :)
p1necone 6y ago
For 2d stuff you'd be correct - but with 3d you can scale the polygons infinitely just like vector art. This works especially well on the N64 because it has a lot of games that make heavy use of flat shading (see Super Mario 64, but Perfect Dark isn't one of these - it uses a lot of pretty low res textures). I'm a (sort of) purist who has an emulation (hence the sort of) PC hooked up to a 240p CRT TV for older games, but N64 running at higher res does look pretty nice in some games, and anything looks better than native 240p output with blurry bilinear upscaling on an LCD.
- causality0 6y ago
  "240p" is not a meaningful statement when referring to CRTs. CRT phosphors are not pixels. For more on that: https://www.youtube.com/watch?v=Ea6tw-gulnQ https://www.youtube.com/watch?v=Ea6tw-gulnQ
  - p1necone 6y ago
    The 'p' in '240p' doesn't stand for pixels, it stands for progressive scan. Also 240p/480i/480p are the standard accepted terms for these low resolution video signals[0], nitpicking technical details as a 'gotcha' when people use standard terminology isn't helpful. [0] https://en.wikipedia.org/wiki/Low-definition_television https://en.wikipedia.org/wiki/Low-definition_television
  - toast0 6y ago
    Phosphors may not be pixels, but 240p doesn't say anything about pixels. The number tells us how many lines, and the p tells us that each screenful of lines covers the whole picture (the p is for progressive, vs i for interlaced). The whole phrase 240p CRT TV tells us it's a normalish NTSC tv, not a hi-res tv with fancier electronics to work with digital tv and which would likely have more processing delays.
    - p1necone 6y ago
      Interestingly the '240p' signal sent out by video game consoles of that era is really a hack, as 240p wasn't a standard signal supported by TVs of the time. It's actually a 480i signal with the timing fiddled with so that the alternate lines still strike the same part of the screen (this is why games from that era had such noticeable scanlines - the CRT beam is only lighting up alternate horizontal lines). This also means that a lot of more modern TVs (and even some upscalers marketed for retro gaming) do an extra terrible job of upscaling 240p signals because they run the same logic that they would if it was normal 480i, resulting in unnecessary flickering or dropped frames.
wtallis 6y ago
Often times, the native resolution of the console isn't high enough to show the full detail of the textures. This is most strikingly apparent on Super Nintendo Mode 7 graphics: https://arstechnica.com/gaming/2019/04/hd-emulation-mod-makes-mode-7-snes-games-look-like-new/ https://arstechnica.com/gaming/2019/04/hd-emulation-mod-make... PlayStation 1 games suffer from a similar problem, and rendering at higher resolution in an emulator also often reveals a surprisingly high polygon count for eg. character models that look like badly-drawn sprites at native resolution. > I'm sure things like AA work better in the native renderer. Rendering at higher resolution is the highest-quality form of AA possible.
BubRoss 6y ago
Rendering at a higher resolution makes a giant difference, even for low poly games. Old games were not only low res, but didn't have a lot of antialiasing either. On an emulator you can have resolution, antialiasing and better texture filtering.
dagmx 6y ago
Generally details will still be better the closer to native display resolution you can render rather than rendering to native source resolution and then up scaling. Even if the textures are blurry (they can be swapped too), the polygon details will be easier to see.
aasasd 6y ago
The difference is especially noticeable on distant objects in games where you need to see ahead: e.g. in Gran Turismo you can't tell which way chevron turn signs are pointing unless you have higher internal resolution. (Not suggesting that one should drive by signs in GT, but they're still fine as a cue.)
_0ffh 6y ago
>the [...] textures are still the same I wonder if someone has already thought of using one of the available pixel art upscalers to improve the texture resolutions. If not, it's probably only a matter of time.
- 7786655 6y ago
  Yes, they have. You see this on a lot of Super Mario 64 videos. I personally think it looks awful.
proverbialbunny 6y ago
With ML the textures can be upscaled too. Though, no one has attempted to do this as an emulator plugin yet as far as I know. Image superscaling is a well known problem with real world solutions so, it's only a matter of time and interest. ML will draw in details that never existed to begin with. It's quite amazing, but most versions of image superscaling that exists today is trained on drawn art, like from Deviant Art, so the ML makes the images look subtly pastel. It's great if you want a 4k desktop background. Heck, I could make one if I cared enough. A friend of mine makes a popular emulator. Maybe she'd appreciate the functionality.

SECProto 6y ago

There are some projects that swap out textures for higher res versions. But I think the bigger advantage is for distant objects - if you zoom in the goldeneye screenshot, you can see the face texture of the guy at the other end of the hall. If the regular native resolution had been used and upscaled afterwards, you wouldn't see the face at all, just a face-coloured blur

Polylactic_acid 6y ago

I don't know the details of this project but I have seen emulators use processes more complex than a basic image upscale to get really nice results out of low resolution textures. I think its similar to those anime upscailing tools where its trained on a dataset of textures and is able to redraw them at higher resolutions.

saagarjha 6y ago
Sadly, those techniques aren’t nearly fast enough to work for emulation.
- Polylactic_acid 6y ago
  Could they not run once on all of the textures and then save them to a file for use in game?
- rossy 6y ago
  You'd be surprised at what image processing algorithms are fast enough to work in realtime when they have optimised GPU implementations. Anime4K is one of those CNN-based anime upscaling tools, and it can run in realtime as a shader in mpv. (I'm not a fan of those upscaling tools myself, but I don't see why they couldn't be used in emulators, for the people who like them.)
louhike 6y ago
This emulator prioritizes accuracy, not fancy graphical options, that's why they lack this kind of features.

Hydraulix989 6y ago

The N64's "GPU" is the Reality Display Processor (RDP) chip which is specialized hardware built by Silicon Graphics specifically for the N64 console. The RDP runs a custom program that is part of each game that tells it how to take high-level commands from the game code and render them. For this accurate N64 emulation, it is being emulated at a lower level than before. Previously, N64 emulators just used the commands sent to the RDP to tell the host system GPU what to do with something higher-level like OpenGL or DirectX (of course, this meant a lot of game-specific "hacks" in the emulator), rather than emulating the RDP itself by sending lower-level commands directly from the RDP to the host GPU with something like Vulkan. This is so-called High-Level Emulation (HLE), and it's a massive shortcut to emulating the whole RDP -- which meant that N64 could even be "emulated" on a PC from 1999. Lower-level emulation of the RDP itself has been recently made possible, and now it can also now be "up-scaled" to arbitrary resolutions -- instead of just sticking to high level emulation and telling OpenGL or DirectX to render to a larger resolution -- or even worse, scaling the rasterized rendered output frames by treating them as images. In practice, Mario 64 was just telling the RDP to render triangles anyways so it mapped nicely to OpenGL for the HLE case, but for more "accurate" emulation (this is like getting from 95% to 99%), the RDP itself needs to be emulated as well for things like the Perfect Dark drone camera mentioned elsewhere.

midnightclubbed 6y ago
To be pedantic the RDP was the rasterizer (fixed function) which was fed by the RSP processor. The RSP ran the display-list commands from the cpu/memory (very similar to a Vulkan command list) and did vertex transforms/clipping before feeding its output to the RDP. Most games used one of a few Nintendo provided RSP programs although later in the machine's lifetime they opened up the RSP compiler and tools to developers.
- Hydraulix989 6y ago
  Yes, I was not going to go into that much detail :)

p1necone 6y ago

For 2d stuff you'd be correct - but with 3d you can scale the polygons infinitely just like vector art. This works especially well on the N64 because it has a lot of games that make heavy use of flat shading (see Super Mario 64, but Perfect Dark isn't one of these - it uses a lot of pretty low res textures). I'm a (sort of) purist who has an emulation (hence the sort of) PC hooked up to a 240p CRT TV for older games, but N64 running at higher res does look pretty nice in some games, and anything looks better than native 240p output with blurry bilinear upscaling on an LCD.

causality0 6y ago
"240p" is not a meaningful statement when referring to CRTs. CRT phosphors are not pixels. For more on that: https://www.youtube.com/watch?v=Ea6tw-gulnQ https://www.youtube.com/watch?v=Ea6tw-gulnQ
- p1necone 6y ago
  The 'p' in '240p' doesn't stand for pixels, it stands for progressive scan. Also 240p/480i/480p are the standard accepted terms for these low resolution video signals[0], nitpicking technical details as a 'gotcha' when people use standard terminology isn't helpful. [0] https://en.wikipedia.org/wiki/Low-definition_television https://en.wikipedia.org/wiki/Low-definition_television
- toast0 6y ago
  Phosphors may not be pixels, but 240p doesn't say anything about pixels. The number tells us how many lines, and the p tells us that each screenful of lines covers the whole picture (the p is for progressive, vs i for interlaced). The whole phrase 240p CRT TV tells us it's a normalish NTSC tv, not a hi-res tv with fancier electronics to work with digital tv and which would likely have more processing delays.
  - p1necone 6y ago
    Interestingly the '240p' signal sent out by video game consoles of that era is really a hack, as 240p wasn't a standard signal supported by TVs of the time. It's actually a 480i signal with the timing fiddled with so that the alternate lines still strike the same part of the screen (this is why games from that era had such noticeable scanlines - the CRT beam is only lighting up alternate horizontal lines). This also means that a lot of more modern TVs (and even some upscalers marketed for retro gaming) do an extra terrible job of upscaling 240p signals because they run the same logic that they would if it was normal 480i, resulting in unnecessary flickering or dropped frames.

wtallis 6y ago

Often times, the native resolution of the console isn't high enough to show the full detail of the textures. This is most strikingly apparent on Super Nintendo Mode 7 graphics: https://arstechnica.com/gaming/2019/04/hd-emulation-mod-makes-mode-7-snes-games-look-like-new/ https://arstechnica.com/gaming/2019/04/hd-emulation-mod-make... PlayStation 1 games suffer from a similar problem, and rendering at higher resolution in an emulator also often reveals a surprisingly high polygon count for eg. character models that look like badly-drawn sprites at native resolution. > I'm sure things like AA work better in the native renderer. Rendering at higher resolution is the highest-quality form of AA possible.

BubRoss 6y ago

Rendering at a higher resolution makes a giant difference, even for low poly games. Old games were not only low res, but didn't have a lot of antialiasing either. On an emulator you can have resolution, antialiasing and better texture filtering.

dagmx 6y ago

Generally details will still be better the closer to native display resolution you can render rather than rendering to native source resolution and then up scaling. Even if the textures are blurry (they can be swapped too), the polygon details will be easier to see.

aasasd 6y ago

The difference is especially noticeable on distant objects in games where you need to see ahead: e.g. in Gran Turismo you can't tell which way chevron turn signs are pointing unless you have higher internal resolution. (Not suggesting that one should drive by signs in GT, but they're still fine as a cue.)

_0ffh 6y ago

>the [...] textures are still the same I wonder if someone has already thought of using one of the available pixel art upscalers to improve the texture resolutions. If not, it's probably only a matter of time.

7786655 6y ago
Yes, they have. You see this on a lot of Super Mario 64 videos. I personally think it looks awful.

proverbialbunny 6y ago

With ML the textures can be upscaled too. Though, no one has attempted to do this as an emulator plugin yet as far as I know. Image superscaling is a well known problem with real world solutions so, it's only a matter of time and interest. ML will draw in details that never existed to begin with. It's quite amazing, but most versions of image superscaling that exists today is trained on drawn art, like from Deviant Art, so the ML makes the images look subtly pastel. It's great if you want a 4k desktop background. Heck, I could make one if I cared enough. A friend of mine makes a popular emulator. Maybe she'd appreciate the functionality.