8 ms·

Show HN: Metashade – a Pythonic GPU shading/compute EDSL

47 points by ppenenko 2y ago

Superficially, it may look similar to Nvidia's Warp or OpenAI's Triton, but instead of transpiling a subset of Python to the target language, it implements dynamic codegen in pure Python. These slides discuss the existing GPU programming solutions and make a case for Metashade's approach: https://docs.google.com/presentation/d/e/2PACX-1vQtYIwXIkMnVC6TzWTKPAtZIA6_xeUCQc8Mvyziu0qy7HDUduz_onsJ5TabxTuuVQ/pub?start=false&loop=false&delayms=3000 https://docs.google.com/presentation/d/e/2PACX-1vQtYIwXIkMnV...

2genders1206 2y ago
[flagged]
2genders13469 2y ago
[flagged]
SEXMCNIGGA7323 2y ago
[flagged]
2genders23779 2y ago
[flagged]
sexmc12681 2y ago
[flagged]
sexmc33282 2y ago
[flagged]
2genders25297 2y ago
[flagged]
2genders23289 2y ago
[flagged]
2genders14678 2y ago
[flagged]
sexmc47940 2y ago
[flagged]
sexmc31107 2y ago
[flagged]
sexmc17627 2y ago
[flagged]
SEXMCNIGGA16009 2y ago
[dead]
SEXMCNIGGA36472 2y ago
[dead]
SEXMCNIGGA36557 2y ago
[flagged]
SEXMCNIGGA28386 2y ago
[dead]
SEXMCNIGGA5453 2y ago
[flagged]
SEXMCNIGGA14392 2y ago
[dead]
2genders28507 2y ago
[dead]
2genders8765 2y ago
[dead]
2genders14637 2y ago
[dead]
2genders26709 2y ago
[dead]
2genders5056 2y ago
[dead]
2genders777 2y ago
[dead]
2genders22238 2y ago
[dead]
2genders9531 2y ago
[flagged]
2genders22942 2y ago
[flagged]
2genders47148 2y ago
[flagged]
2genders32561 2y ago
[flagged]
2genders47272 2y ago
[flagged]
2genders839 2y ago
[flagged]
2genders26728 2y ago
[flagged]
2genders31161 2y ago
[flagged]
2genders37574 2y ago
[flagged]
2genders38930 2y ago
[flagged]
2genders3648 2y ago
[flagged]
2genders30187 2y ago
[flagged]
2genders2516 2y ago
[flagged]
2genders44794 2y ago
[flagged]
2genders34771 2y ago
[flagged]
2genders49246 2y ago
[flagged]
SEXMCNIGGA45535 2y ago
[flagged]
2genders28563 2y ago
[flagged]
2genders40320 2y ago
[flagged]
2genders44424 2y ago
[flagged]
2genders20975 2y ago
[flagged]
2genders12074 2y ago
[flagged]
2genders45764 2y ago
[flagged]
2genders3617 2y ago
[flagged]
2genders9639 2y ago
[flagged]
SEXMCNIGGA29133 2y ago
[flagged]
SEXMCNIGGA28322 2y ago
[dead]
SEXMCNIGGA32816 2y ago
[flagged]
SEXMCNIGGA28854 2y ago
[dead]
SEXMCNIGGA43215 2y ago
[dead]
SEXMCNIGGA3291 2y ago
[dead]
SEXMCNIGGA47335 2y ago
[dead]
SEXMCNIGGA6701 2y ago
[dead]
SEXMCNIGGA19574 2y ago
[dead]
SEXMCNIGGA26902 2y ago
[dead]
SEXMCNIGGA20543 2y ago
[dead]
indianmilf7474 2y ago
[flagged]
indianmilf23186 2y ago
[flagged]
indianmilf40012 2y ago
[flagged]
sexmc22464 2y ago
[flagged]
sexmc25878 2y ago
[flagged]
sexmc7261 2y ago
[flagged]
sexmc34063 2y ago
[flagged]
sexmc44307 2y ago
[flagged]
sexmc17979 2y ago
[flagged]
sexmc114 2y ago
[flagged]
sexmc22797 2y ago
[flagged]
sexmc34509 2y ago
[flagged]
sexmc33999 2y ago
[flagged]
leguminous 2y ago
I was using Taichi recently to prototype some fragment shaders (I just mocked out texelFetch, texture, etc and wrote a for loop over each pixel). I found it convenient to be able to run on the CPU and also not have to deal with any OpenGL or Vulkan setup. One thing that was frustrating was that I couldn't template a function with another function (as far as I know). That lead to some copied and pasted code when I had multiple implementations that I wanted to compare. Can Metashade do that?
- ppenenko 2y ago
  Agreed that the ability to just run on the CPU is valuable. Metashade doesn't support that yet and its codegen syntax doesn't look like regular Python code (everything codegen-related is prefixed with `sh.` etc.) but it's certainly possible to write a generator that would just execute the code "in the immediate mode" or generate C/C++ code for the CPU. Regarding templating functions with functions - in Metashade you can just specialize the generated code however you see fit, with Python as the meta language. E.g. Python's `if` statements can act like `#ifdef`s or `if constexpr`, and you can certainly pass around callables to parameterize behavior.
raytopia 2y ago
Super interesting project. Been looking into template generation and uber shaders to make coding shaders easier for the game engine Ursina. I'm going to bring this to their attention. Would it be easy to add another target like glsl?
- ppenenko 2y ago
  Thanks! Multiple targets is definitely the plan, and GLSL is my next priority. Metashade can currently generate HLSL for the DX12 version of https://github.com/ppenenko/glTFSample/tree/metashade_demo https://github.com/ppenenko/glTFSample/tree/metashade_demo, and there's also a Vulkan version of that demo using GLSL. So implementing GLSL generation for that would be a great proof of concept and a test bed: HLSL and GLSL generated from single source and producing identical rendering results. The diagram on slide 31 of my presentation shows how implementation is currently inherited between the Metashade packages. The future GLSL generators should inherit common functionality from the "rtsl" package, just like the existing HLSL generators. However, I expect heavy refactoring to be necessary because I was initially targeting just one language and so not all code is implemented at the appropriate level. BTW, here's a poll where you can vote for a target language you'd like to see prioritized: https://github.com/ppenenko/metashade/discussions/17 https://github.com/ppenenko/metashade/discussions/17
pixelpoet 2y ago
Hi Pavlo, nice to see you here and thanks for the presentation at Maxon :)
- ppenenko 2y ago
  Cheers Thomas!
kookamamie 2y ago
Wouldn't jinja templates would have sufficed for this? I'm not sure it's a good idea to create a whole new language just for getting templating capabilities to shaders.
- ppenenko 2y ago
  No, I don't believe Jinja would suffice. For starters, how would you abstract out the syntax differences between, say, HLSL and WGSL? Jinja's approach seems to be taking the syntax of the target language and embedding templating into it. But with Metashade, we're replacing the different syntaxes of individual target languages with that of Python. Since this approach is largely agnostic of the target language syntax, new targets can be added in the future without rewriting target-independent, polymorphic code implementing rendering or compute techniques.

47 points by ppenenko 2y ago

2genders1206 2y ago

[flagged]

2genders13469 2y ago

[flagged]

SEXMCNIGGA7323 2y ago

[flagged]

2genders23779 2y ago

[flagged]

sexmc12681 2y ago

[flagged]

sexmc33282 2y ago

[flagged]

2genders25297 2y ago

[flagged]

2genders23289 2y ago

[flagged]

2genders14678 2y ago

[flagged]

sexmc47940 2y ago

[flagged]

sexmc31107 2y ago

[flagged]

sexmc17627 2y ago

[flagged]

SEXMCNIGGA16009 2y ago

[dead]

SEXMCNIGGA36472 2y ago

[dead]

SEXMCNIGGA36557 2y ago

[flagged]

SEXMCNIGGA28386 2y ago

[dead]

SEXMCNIGGA5453 2y ago

[flagged]

SEXMCNIGGA14392 2y ago

[dead]

2genders28507 2y ago

[dead]

2genders8765 2y ago

[dead]

2genders14637 2y ago

[dead]

2genders26709 2y ago

[dead]

2genders5056 2y ago

[dead]

2genders777 2y ago

[dead]

2genders22238 2y ago

[dead]

2genders9531 2y ago

[flagged]

2genders22942 2y ago

[flagged]

2genders47148 2y ago

[flagged]

2genders32561 2y ago

[flagged]

2genders47272 2y ago

[flagged]

2genders839 2y ago

[flagged]

2genders26728 2y ago

[flagged]

2genders31161 2y ago

[flagged]

2genders37574 2y ago

[flagged]

2genders38930 2y ago

[flagged]

2genders3648 2y ago

[flagged]

2genders30187 2y ago

[flagged]

2genders2516 2y ago

[flagged]

2genders44794 2y ago

[flagged]

2genders34771 2y ago

[flagged]

2genders49246 2y ago

[flagged]

SEXMCNIGGA45535 2y ago

[flagged]

2genders28563 2y ago

[flagged]

2genders40320 2y ago

[flagged]

2genders44424 2y ago

[flagged]

2genders20975 2y ago

[flagged]

2genders12074 2y ago

[flagged]

2genders45764 2y ago

[flagged]

2genders3617 2y ago

[flagged]

2genders9639 2y ago

[flagged]

SEXMCNIGGA29133 2y ago

[flagged]

SEXMCNIGGA28322 2y ago

[dead]

SEXMCNIGGA32816 2y ago

[flagged]

SEXMCNIGGA28854 2y ago

[dead]

SEXMCNIGGA43215 2y ago

[dead]

SEXMCNIGGA3291 2y ago

[dead]

SEXMCNIGGA47335 2y ago

[dead]

SEXMCNIGGA6701 2y ago

[dead]

SEXMCNIGGA19574 2y ago

[dead]

SEXMCNIGGA26902 2y ago

[dead]

SEXMCNIGGA20543 2y ago

[dead]

indianmilf7474 2y ago

[flagged]

indianmilf23186 2y ago

[flagged]

indianmilf40012 2y ago

[flagged]

sexmc22464 2y ago

[flagged]

sexmc25878 2y ago

[flagged]

sexmc7261 2y ago

[flagged]

sexmc34063 2y ago

[flagged]

sexmc44307 2y ago

[flagged]

sexmc17979 2y ago

[flagged]

sexmc114 2y ago

[flagged]

sexmc22797 2y ago

[flagged]

sexmc34509 2y ago

[flagged]

sexmc33999 2y ago

[flagged]

leguminous 2y ago

I was using Taichi recently to prototype some fragment shaders (I just mocked out texelFetch, texture, etc and wrote a for loop over each pixel). I found it convenient to be able to run on the CPU and also not have to deal with any OpenGL or Vulkan setup. One thing that was frustrating was that I couldn't template a function with another function (as far as I know). That lead to some copied and pasted code when I had multiple implementations that I wanted to compare. Can Metashade do that?

ppenenko 2y ago
Agreed that the ability to just run on the CPU is valuable. Metashade doesn't support that yet and its codegen syntax doesn't look like regular Python code (everything codegen-related is prefixed with `sh.` etc.) but it's certainly possible to write a generator that would just execute the code "in the immediate mode" or generate C/C++ code for the CPU. Regarding templating functions with functions - in Metashade you can just specialize the generated code however you see fit, with Python as the meta language. E.g. Python's `if` statements can act like `#ifdef`s or `if constexpr`, and you can certainly pass around callables to parameterize behavior.

raytopia 2y ago

Super interesting project. Been looking into template generation and uber shaders to make coding shaders easier for the game engine Ursina. I'm going to bring this to their attention. Would it be easy to add another target like glsl?

ppenenko 2y ago
Thanks! Multiple targets is definitely the plan, and GLSL is my next priority. Metashade can currently generate HLSL for the DX12 version of https://github.com/ppenenko/glTFSample/tree/metashade_demo https://github.com/ppenenko/glTFSample/tree/metashade_demo, and there's also a Vulkan version of that demo using GLSL. So implementing GLSL generation for that would be a great proof of concept and a test bed: HLSL and GLSL generated from single source and producing identical rendering results. The diagram on slide 31 of my presentation shows how implementation is currently inherited between the Metashade packages. The future GLSL generators should inherit common functionality from the "rtsl" package, just like the existing HLSL generators. However, I expect heavy refactoring to be necessary because I was initially targeting just one language and so not all code is implemented at the appropriate level. BTW, here's a poll where you can vote for a target language you'd like to see prioritized: https://github.com/ppenenko/metashade/discussions/17 https://github.com/ppenenko/metashade/discussions/17

pixelpoet 2y ago

Hi Pavlo, nice to see you here and thanks for the presentation at Maxon :)

ppenenko 2y ago
Cheers Thomas!

kookamamie 2y ago

Wouldn't jinja templates would have sufficed for this? I'm not sure it's a good idea to create a whole new language just for getting templating capabilities to shaders.

ppenenko 2y ago
No, I don't believe Jinja would suffice. For starters, how would you abstract out the syntax differences between, say, HLSL and WGSL? Jinja's approach seems to be taking the syntax of the target language and embedding templating into it. But with Metashade, we're replacing the different syntaxes of individual target languages with that of Python. Since this approach is largely agnostic of the target language syntax, new targets can be added in the future without rewriting target-independent, polymorphic code implementing rendering or compute techniques.