27 ms·

Linux in a Pixel Shader – A RISC-V Emulator for VRChat

428 points by PiMaker 5y ago

FranchuFranchu 5y ago
This is insane
- xena 5y ago
  I loaded it up in game and it proceeded to instantly make my VR experience go down to 10 FPS. I did manage to get a selfie of the Linux boot though: https://twitter.com/theprincessxena/status/1431042936045715457?s=21 https://twitter.com/theprincessxena/status/14310429360457154...
  - deleted 5y ago
    [deleted]
  - jimmySixDOF 5y ago
    so if you run VRC in 2D you could get Doom working ;}
    - Sohcahtoa82 5y ago
      There's no way someone hasn't implemented DOOM in a pixel shader already.
      - pema99 5y ago
        https://www.shadertoy.com/view/lldGDr https://www.shadertoy.com/view/lldGDr
      - jimmySixDOF 5y ago
        nice but I mean running Doom in the emulated linux instance so its a play within a play within a play .... ?
MrLeap 5y ago
This is outrageous. I'm making a text editor / game where you have a mechanical typewriter, and I've mused quite a few times about giving someone a beige computer running linux inside the game at some point, but it's all seemed unreasonable / impossible.. With this it's suddenly reasonable (if expensive performance wise) Thank you for this, haha.
- Asooka 5y ago
  When you have a full programming language at your disposal, you can compile the linux kernel for the same CPU as your game and run it paravirtualised, skipping all the performance cost of having a full emulated machine.
- exikyut 5y ago
  Unreasonable how? That sounds kinda interesting/fun. FWIW, given that you're running a game, and most people run games fullscreen, without anything else going, you can probably get away with not using virtualization, aka eating 100% CPU (or at least 100% of one core)... and even then, with most systems having >4 cores nowadays, you probably needn't worry at all. But just to clarify the corollary, this implementation is running at around 250kHz (!). You wouldn't viably run Linux like this in a practical context anytime soon.
  - zamadatix 5y ago
    > you can probably get away with not using virtualization, aka eating 100% CPU (or at least 100% of one core)... That's not how virtualization works, it's time sliced which is why you can have more vCPUs than physical CPUs. You can also limit the slice utilization to an arbitrary percentage or give it a priority level for scheduling.
    - messe 5y ago
      > That's not how virtualization works, it's time sliced which is why you can have more vCPUs than physical CPUs. You can also limit the slice utilization to an arbitrary percentage or give it a priority level for scheduling. At that point though, why bother virtualizing? If you're going for the beige PC look, why not go all the way and emulate an 80486 (or 80386 even) on an IBM PC clone in software running linux 0.01? Performance isn't going to matter too much, so you could limit it to as much or little CPU utilization as you want. Plus, virtualizing is going to potentially require additional permissions that users may not want to grant your game (or may not be available on the platform).
dleslie 5y ago
Thank-you _pi_, I loved this when I first saw it, and I love it now.
chii 5y ago
reminds me of this https://dolphin-emu.org/blog/2017/07/30/ubershaders/ https://dolphin-emu.org/blog/2017/07/30/ubershaders/
- Const-me 5y ago
  Interesting. If I would want to solve that problem, first thing I would have tried is bypassing text representation of these shaders. For a few things in the past I have generated, and dynamically patched, D3D11 shaders directly in DXBC byte code, without HLSL anywhere. The byte code format is even documented by Microsoft. Not the complete DXBC files though, but some people on the internets have reverse engineered the missing pieces.
  - TazeTSchnitzel 5y ago
    Now you have a new set of problems: getting the compiler that consumes the bytecode not to crash. :)
    - Const-me 5y ago
      I wouldn’t expect that. DXBC is not terribly complex. The instruction set only has 207 of them, each one is really simple. Take a look: https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl-sm4-asm https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/... https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/shader-model-5-assembly--directx-hlsl- https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/...
      - TazeTSchnitzel 5y ago
        That the instructions are simple doesn't mean that what the compiler does with them will be! It's still a long way from the actual hardware.
  - Asooka 5y ago
    Even if you use byte code, the driver will have to compile your shader for the target hardware. It won't be nearly as fast as what the game does on console - loading a blob in GPU memory. It will definitely be faster than compiling a text representation, but still not fast enough to skip the uebershader.
etaioinshrdlu 5y ago
Can I have it in WebGL now too? I wonder if the slow speed can be partially redeemed by having many emulated cores.
- arthur2e5 5y ago
  I can totally see a “show HN: webapp that turns your GPU into a discount xeon phi” appearing…
- pjmlp 5y ago
  No, because in all their wisdom, WebGL never supported compute shaders, Intel has tried multiple times to add it to the standard and now it has been officially dropped for WebGPU. So when the browsers bother to finally replace their 2011 GPU model by WebGPU (stable), which is still MVP 1.0 compared to Vulkan/Metal/DX 12 Ultimate, then you might have it.
  - contravariant 5y ago
    The shader in question isn't a compute shader.
    - pjmlp 5y ago
      I guess I misunderstood "Compute Shaders in VRChat" then.
      - zamadatix 5y ago
        It's about using the classic fragment shader (also sometimes referred to as pixel shader) + texture approach as a workaround for compute shaders not being available if the body wasn't clear.
        deleted 5y ago
        [deleted]
        pjmlp 5y ago
exikyut 5y ago
I nearly fell off my chair discovering this just under a week ago, and immediately posted it: https://news.ycombinator.com/item?id=28255821 https://news.ycombinator.com/item?id=28255821 I commented underneath: > More details and demonstration, naturally presented natively in VRChat: https://www.youtube.com/watch?v=G2u7NOpzcBQ&t=5052s https://www.youtube.com/watch?v=G2u7NOpzcBQ&t=5052s > Video found via https://twitter.com/_g1fan_/status/1427073177142939648 https://twitter.com/_g1fan_/status/1427073177142939648 > Source: https://twitter.com/fuopy/status/1427051048032620544 https://twitter.com/fuopy/status/1427051048032620544 > Just discovered Twitter doesn't let you copy a link to a retweet reference D:, but I discovered this on https://twitter.com/whitequark/ https://twitter.com/whitequark/ 's timeline.
2bitencryption 5y ago
Definitely watch the presentation by the developer in VR Chat: https://www.youtube.com/watch?v=G2u7NOpzcBQ https://www.youtube.com/watch?v=G2u7NOpzcBQ I haven't played VR Chat in a few years, and holy cow this makes me want to jump back in. Seeing fully captured motion avatars of people is SO immersive. It's absolutely insane how the experience of interacting with people can feel so natural even when they're on the other side of the planet. I just love watching "body language" of the presenter, it's such a precise rendering of what a "real life" presentation like this would be. Plus, you meet people like the writer of this blog post, who are so knowledgeable and interesting and fun to talk to.
- RamRodification 5y ago
  Why are the majority of avatars small children or furry-things? I'm really not one of those people who bashes nerdy or alternative culture but I can't deny that it weirds me out a bit. It's a clear barrier of entry for me.
  - maeln 5y ago
    StraszFilms made a video touching on this subject : https://www.youtube.com/watch?v=5v_Dl7i4Bcw https://www.youtube.com/watch?v=5v_Dl7i4Bcw ( he also made a very nice documentary about VRChat nightclubs https://www.youtube.com/watch?v=R1wUg9HCODU https://www.youtube.com/watch?v=R1wUg9HCODU )
  - caslon 5y ago
    Most of them aren't actually all that small, it's just that the perspective it's being filmed from (the narrator's head-camera) is on a stage, and VR warps things in your periphery pretty hard. There's also the fact that it's easier to be immersed if you aren't leaning heavily on realism; Oculus has a lot of research on this, actually. For example, if you have a photorealistic avatar but aren't doing eye-tracking, it's actually more jarring and discomforting than a cartoon avatar without it. That's why the initial avatars they did had force-obscured eyes and no color, and the new ones are literally cartoons: your mind can adapt to something very different, but it can't adapt to something that's just subtly different. However, some people do choose to do weird stuff. That's mostly a tech thing though. If you go to a normal place, you'll find a lot of actual kids and actual teenagers, and they're relatively normal. Very few people as weird as the tech community. Really, technical people in digital spaces kind of get a bit wild with their self-expression. Some theorize it's because most of them don't express themselves much in real life. I've tried this particular app only a handful of times (I don't like using proprietary software, but it's important to keep yourself updated with the state of it), and when I wasn't visiting explicitly tech-related things, it was pretty normal. Lots of copyright infringement of Nintendo characters, as you might expect.
HammadB 5y ago
Projects like this are so inspiring.
tuyiown 5y ago
ha, it reminds me of the Croquet Project https://en.wikipedia.org/wiki/Croquet_Project https://en.wikipedia.org/wiki/Croquet_Project
anthk 5y ago
Interesting. Also, I would like to test VR walks such as the one from Albarracín, but my GPU only supports GL 2.1.
mamanbd1 5y ago
good
hypertele-Xii 5y ago
...Every pixel is an Operating System? This is the kind of madness/genius that'll take us places.
- zamadatix 5y ago
  Every 2048x2048 state texture is the state of an operating system, including CPU and memory. The pixel shaders operate on this state to advance different parts of the state of the operating system as a whole.
valyagolev 5y ago
can't wait until i see kubernetes running on risc-v in a job description
p0nce 5y ago
Sounds like a headline optimized for HN success!
kevingadd 5y ago
"One of the many problems with shader code is that HLSL doesn’t support arrays in a meaningful way. Pointer math (and thus array indexing) just isn’t a thing on the GPU, so writing to a non-constant index of an array is impossible. To work around this, there are several places in the code with patterns like this:" This is weird, AFAIK pixel shaders have had arrays since DX10 and the OpenGL equivalent through Buffers (and also Unordered Access Views for reading/writing pixel data) https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl-to-load https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/... Are VRChat shaders stuck to Direct3D9/ps_3_0 level functionality?
- pema99 5y ago
  No, we can use SM 5.0. It's all DX11. It's not that the framework the game is written in doesn't support such features, we just have no way to bind UAVs for example. What you can do on the CPU is very very limited, while GPU code is completely unrestricted, so we are at the mercy of whatever Unity lets you do on GPU with minimal setup.
ncmncm 5y ago
This is a thing that should not ("in the name of all that is holy") have been done; but being done, cannot but be admired.

428 points by PiMaker 5y ago

FranchuFranchu 5y ago

This is insane

xena 5y ago
I loaded it up in game and it proceeded to instantly make my VR experience go down to 10 FPS. I did manage to get a selfie of the Linux boot though: https://twitter.com/theprincessxena/status/1431042936045715457?s=21 https://twitter.com/theprincessxena/status/14310429360457154...
- deleted 5y ago
  [deleted]
- jimmySixDOF 5y ago
  so if you run VRC in 2D you could get Doom working ;}
  - Sohcahtoa82 5y ago
    There's no way someone hasn't implemented DOOM in a pixel shader already.
    - pema99 5y ago
      https://www.shadertoy.com/view/lldGDr https://www.shadertoy.com/view/lldGDr
    - jimmySixDOF 5y ago
      nice but I mean running Doom in the emulated linux instance so its a play within a play within a play .... ?

MrLeap 5y ago

This is outrageous. I'm making a text editor / game where you have a mechanical typewriter, and I've mused quite a few times about giving someone a beige computer running linux inside the game at some point, but it's all seemed unreasonable / impossible.. With this it's suddenly reasonable (if expensive performance wise) Thank you for this, haha.

Asooka 5y ago
When you have a full programming language at your disposal, you can compile the linux kernel for the same CPU as your game and run it paravirtualised, skipping all the performance cost of having a full emulated machine.
exikyut 5y ago
Unreasonable how? That sounds kinda interesting/fun. FWIW, given that you're running a game, and most people run games fullscreen, without anything else going, you can probably get away with not using virtualization, aka eating 100% CPU (or at least 100% of one core)... and even then, with most systems having >4 cores nowadays, you probably needn't worry at all. But just to clarify the corollary, this implementation is running at around 250kHz (!). You wouldn't viably run Linux like this in a practical context anytime soon.
- zamadatix 5y ago
  > you can probably get away with not using virtualization, aka eating 100% CPU (or at least 100% of one core)... That's not how virtualization works, it's time sliced which is why you can have more vCPUs than physical CPUs. You can also limit the slice utilization to an arbitrary percentage or give it a priority level for scheduling.
  - messe 5y ago
    > That's not how virtualization works, it's time sliced which is why you can have more vCPUs than physical CPUs. You can also limit the slice utilization to an arbitrary percentage or give it a priority level for scheduling. At that point though, why bother virtualizing? If you're going for the beige PC look, why not go all the way and emulate an 80486 (or 80386 even) on an IBM PC clone in software running linux 0.01? Performance isn't going to matter too much, so you could limit it to as much or little CPU utilization as you want. Plus, virtualizing is going to potentially require additional permissions that users may not want to grant your game (or may not be available on the platform).

dleslie 5y ago

Thank-you _pi_, I loved this when I first saw it, and I love it now.

chii 5y ago

reminds me of this https://dolphin-emu.org/blog/2017/07/30/ubershaders/ https://dolphin-emu.org/blog/2017/07/30/ubershaders/

Const-me 5y ago
Interesting. If I would want to solve that problem, first thing I would have tried is bypassing text representation of these shaders. For a few things in the past I have generated, and dynamically patched, D3D11 shaders directly in DXBC byte code, without HLSL anywhere. The byte code format is even documented by Microsoft. Not the complete DXBC files though, but some people on the internets have reverse engineered the missing pieces.
- TazeTSchnitzel 5y ago
  Now you have a new set of problems: getting the compiler that consumes the bytecode not to crash. :)
  - Const-me 5y ago
    I wouldn’t expect that. DXBC is not terribly complex. The instruction set only has 207 of them, each one is really simple. Take a look: https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl-sm4-asm https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/... https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/shader-model-5-assembly--directx-hlsl- https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/...
    - TazeTSchnitzel 5y ago
      That the instructions are simple doesn't mean that what the compiler does with them will be! It's still a long way from the actual hardware.
- Asooka 5y ago
  Even if you use byte code, the driver will have to compile your shader for the target hardware. It won't be nearly as fast as what the game does on console - loading a blob in GPU memory. It will definitely be faster than compiling a text representation, but still not fast enough to skip the uebershader.

etaioinshrdlu 5y ago

Can I have it in WebGL now too? I wonder if the slow speed can be partially redeemed by having many emulated cores.

arthur2e5 5y ago
I can totally see a “show HN: webapp that turns your GPU into a discount xeon phi” appearing…
pjmlp 5y ago
No, because in all their wisdom, WebGL never supported compute shaders, Intel has tried multiple times to add it to the standard and now it has been officially dropped for WebGPU. So when the browsers bother to finally replace their 2011 GPU model by WebGPU (stable), which is still MVP 1.0 compared to Vulkan/Metal/DX 12 Ultimate, then you might have it.
- contravariant 5y ago
  The shader in question isn't a compute shader.
  - pjmlp 5y ago
    I guess I misunderstood "Compute Shaders in VRChat" then.
    - zamadatix 5y ago
      It's about using the classic fragment shader (also sometimes referred to as pixel shader) + texture approach as a workaround for compute shaders not being available if the body wasn't clear.
      - deleted 5y ago
        [deleted]
      - pjmlp 5y ago

exikyut 5y ago

I nearly fell off my chair discovering this just under a week ago, and immediately posted it: https://news.ycombinator.com/item?id=28255821 https://news.ycombinator.com/item?id=28255821 I commented underneath: > More details and demonstration, naturally presented natively in VRChat: https://www.youtube.com/watch?v=G2u7NOpzcBQ&t=5052s https://www.youtube.com/watch?v=G2u7NOpzcBQ&t=5052s > Video found via https://twitter.com/_g1fan_/status/1427073177142939648 https://twitter.com/_g1fan_/status/1427073177142939648 > Source: https://twitter.com/fuopy/status/1427051048032620544 https://twitter.com/fuopy/status/1427051048032620544 > Just discovered Twitter doesn't let you copy a link to a retweet reference D:, but I discovered this on https://twitter.com/whitequark/ https://twitter.com/whitequark/ 's timeline.

2bitencryption 5y ago

Definitely watch the presentation by the developer in VR Chat: https://www.youtube.com/watch?v=G2u7NOpzcBQ https://www.youtube.com/watch?v=G2u7NOpzcBQ I haven't played VR Chat in a few years, and holy cow this makes me want to jump back in. Seeing fully captured motion avatars of people is SO immersive. It's absolutely insane how the experience of interacting with people can feel so natural even when they're on the other side of the planet. I just love watching "body language" of the presenter, it's such a precise rendering of what a "real life" presentation like this would be. Plus, you meet people like the writer of this blog post, who are so knowledgeable and interesting and fun to talk to.

RamRodification 5y ago
Why are the majority of avatars small children or furry-things? I'm really not one of those people who bashes nerdy or alternative culture but I can't deny that it weirds me out a bit. It's a clear barrier of entry for me.
- maeln 5y ago
  StraszFilms made a video touching on this subject : https://www.youtube.com/watch?v=5v_Dl7i4Bcw https://www.youtube.com/watch?v=5v_Dl7i4Bcw ( he also made a very nice documentary about VRChat nightclubs https://www.youtube.com/watch?v=R1wUg9HCODU https://www.youtube.com/watch?v=R1wUg9HCODU )
- caslon 5y ago
  Most of them aren't actually all that small, it's just that the perspective it's being filmed from (the narrator's head-camera) is on a stage, and VR warps things in your periphery pretty hard. There's also the fact that it's easier to be immersed if you aren't leaning heavily on realism; Oculus has a lot of research on this, actually. For example, if you have a photorealistic avatar but aren't doing eye-tracking, it's actually more jarring and discomforting than a cartoon avatar without it. That's why the initial avatars they did had force-obscured eyes and no color, and the new ones are literally cartoons: your mind can adapt to something very different, but it can't adapt to something that's just subtly different. However, some people do choose to do weird stuff. That's mostly a tech thing though. If you go to a normal place, you'll find a lot of actual kids and actual teenagers, and they're relatively normal. Very few people as weird as the tech community. Really, technical people in digital spaces kind of get a bit wild with their self-expression. Some theorize it's because most of them don't express themselves much in real life. I've tried this particular app only a handful of times (I don't like using proprietary software, but it's important to keep yourself updated with the state of it), and when I wasn't visiting explicitly tech-related things, it was pretty normal. Lots of copyright infringement of Nintendo characters, as you might expect.

HammadB 5y ago

Projects like this are so inspiring.

tuyiown 5y ago

ha, it reminds me of the Croquet Project https://en.wikipedia.org/wiki/Croquet_Project https://en.wikipedia.org/wiki/Croquet_Project

anthk 5y ago

Interesting. Also, I would like to test VR walks such as the one from Albarracín, but my GPU only supports GL 2.1.

mamanbd1 5y ago

good

hypertele-Xii 5y ago

...Every pixel is an Operating System? This is the kind of madness/genius that'll take us places.

zamadatix 5y ago
Every 2048x2048 state texture is the state of an operating system, including CPU and memory. The pixel shaders operate on this state to advance different parts of the state of the operating system as a whole.

valyagolev 5y ago

can't wait until i see kubernetes running on risc-v in a job description

p0nce 5y ago

Sounds like a headline optimized for HN success!

kevingadd 5y ago

"One of the many problems with shader code is that HLSL doesn’t support arrays in a meaningful way. Pointer math (and thus array indexing) just isn’t a thing on the GPU, so writing to a non-constant index of an array is impossible. To work around this, there are several places in the code with patterns like this:" This is weird, AFAIK pixel shaders have had arrays since DX10 and the OpenGL equivalent through Buffers (and also Unordered Access Views for reading/writing pixel data) https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl-to-load https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/... Are VRChat shaders stuck to Direct3D9/ps_3_0 level functionality?

pema99 5y ago
No, we can use SM 5.0. It's all DX11. It's not that the framework the game is written in doesn't support such features, we just have no way to bind UAVs for example. What you can do on the CPU is very very limited, while GPU code is completely unrestricted, so we are at the mercy of whatever Unity lets you do on GPU with minimal setup.

ncmncm 5y ago

This is a thing that should not ("in the name of all that is holy") have been done; but being done, cannot but be admired.