6 ms·

Show HN: A physically-based GPU ray tracer written in Julia

198 points by simondanisch 8mo ago

We ported pbrt-v4 to Julia and built it into a Makie backend. Any Makie plot can now be rendered with physically-based path tracing. Julia compiles user-defined physics directly into GPU kernels, so anyone can extend the ray tracer with new materials and media - a black hole with gravitational lensing is ~200 lines of Julia. Runs on AMD, NVIDIA, and CPU via KernelAbstractions.jl, with Metal coming soon. Demo scenes: github.com/SimonDanisch/RayDemo

amelius 8mo ago
Is the material description part of the language the same as in PBRT? I'm asking because I had a lot of trouble trying to describe interfaces between materials, only to find out that what I wanted to do was not possible in PBRT without modifying the code. Apparently, in PBRT a material can only have one other material touching it. So, for example rendering a glass filled with water and ice is not possible without hacks. From a user's point of view this is a bit of a let-down, of course. Context: https://news.ycombinator.com/item?id=45668543 https://news.ycombinator.com/item?id=45668543
- simondanisch 8mo ago
  Nope, we made a complete high level Julia interface and I plan to have the Makie API be the main user facing scene description, which can be more descriptive than pbrt I think!
  - amelius 8mo ago
    Ok. Did you see this: https://blog.yiningkarlli.com/2019/05/nested-dielectrics.html https://blog.yiningkarlli.com/2019/05/nested-dielectrics.htm... And I'm curious how you solve it.
    - simondanisch 8mo ago
      Sorry, I was on my phone. This doesn't seem to be a problem of the description language, but rather how the integrator and materials work internally, so this works the same way in Julia currently. I do think though, that its more approachable to add experimental features like this in the Julia version. Would certainly be an interesting project! I do want to over time get further away from the pbrt-v4 architecture and get to something much more modular and easy to extend. I feel like the overlaps resolve should happen at scene creation time, to not have an expensive priority stack at raytracing time - then it would be just a matter of better tracking the media at boundary crossing. But haven't really thought this through of course ;)
      - amelius 8mo ago
        I think it was a problem with the language as well as how they handle it internally. It was basically the algorithm that dictates how the language works, and consequently there was no way to have one material touch more than one other material. But I might misremember. Anyway, I'm looking at this from the user's perspective. I wanted to do some physics-based ray-tracing with lenses and pbrt is what I ended up trying. As such, I really needed the multi-material aspect to work correctly. Also, it would be nice to be able to describe surfaces using a z=f(x,y) kind of formulation, or a way to place a hook in the renderer.
        simondanisch 8mo ago
        It's definitely an architectural problem as well. I do wonder if we could extend that though, without too much trouble for the general architecture - after all, the material does not necessarily need to represent all the outside materials and instead the ray only needs to be able to go from one medium to another. I'm happy to chat about possible extensions in that direction, although to be fair I wont have much time in the next weeks to sit down on anything like this. But, I do really hope that this can become a playground for ray tracing experiments in general!
        amelius 8mo ago
        I think maybe the easiest way to tackle the problem is to have the language describe surfaces instead of solid objects, and let every surface have a normal and two materials. This might be the most natural representation for a ray tracer.
LoganDark 8mo ago
On iOS Safari the videos are fullscreening themselves as I scroll. I've seen this on other blogs before but I don't know what causes it. Super annoying
- simondanisch 8mo ago
  Ugh, yeah I had some super weird bugs like this in safari, still haven't found the source :(
  - embedding-shape 8mo ago
    Don't quote me on this, but I think there is a "playsinline" / "webkit-playsinline" attribute for the video element you need to add to avoid that, + if it's autoplay you need to set "muted" too. I've also had this happen and I think both/either of those solved it last time.
krastanov 8mo ago
As an aside, it is really interesting to see a computational package that, while supporting multiple GPU vendors, was first vetted on AMD, not NVidia. It is encouraging to see ROCM finally shaking off its reputation for poor support.
- simondanisch 8mo ago
  well, I do hate vendor lock in with a passion ;) But yeah, a lot did happen, this likely wouldn't have been possible one or two years ago!
- mr_octopus 7mo ago
  The vendor-agnostic GPU approach via KernelAbstractions is great to see. The Vulkan compute path is underrated for this — it runs on AMD, NVIDIA, and Intel without needing ROCm or CUDA, just whatever driver ships with the GPU. Re: the compilation latency discussion — it's a real tension. JIT gives you expressiveness but kills startup. AOT gives you instant start but limits flexibility. Interesting that most GPU languages went JIT when the GPU itself runs pre-compiled SPIR-V/PTX anyway.
NoboruWataya 8mo ago
I don't hear nearly as much about Julia as I used to. A few years ago the view was that it was about to replace Python as the language of choice for data science. Seems like that didn't happen?
- simondanisch 8mo ago
  I think the hype has slowed down, but all growth statistics haven't. Personally, I think Julia is the only language where I can implement something like Makie without running into a maintenance nightmare, and with Julia GPU programming is actually fun and high level and composes well, which I miss in most other languages. So, I dont really care about it replacing python or not. I do think for replacing python Julia will need to solve compilation latency, shipping AOT binaries and maybe interpret more of the glue code, which currently introduces quite a lot of compilation overhead without much gains in terms of performance.
- bobajeff 8mo ago
  As someone who currently uses dabbles in both. That prediction seems a bit unrealistic. Julia is a fantastic language but it has some trade offs that need to be considered. Probably the most well known is `time to first x`. Julia like Python is used comfortably in notebooks but loading libraries can take a minute, compared to Python where it happens right away. It may lead you to not reach for it when you want to do quick testing of something especially plotting. You can mitigate this somewhat by loading all the libraries you'll ever need at startup (preferably long before you are ready to experiment) but that assumes you already know what libraries you'll need for what you're wanting to try.
  - simondanisch 8mo ago
    What prediction? Maybe I need to rephrase what I said: My prediction is, that if Julia ever wants to have a shot at replacing Python, it absolutely has to solve the first time to first x problem! That's what I mean by shipping fully ahead of time compiled binaries and interpreting more glue code - which both have the potential to solve the first time to x problem.
blueaquilae 8mo ago
That's an impressive accomplishment and a fantastic tool to explore.
johnbatman 8mo ago
[dead]
bobajeff 8mo ago
It's says: >the reference implementation from Physically Based Rendering (Pharr, Jakob, Humphreys) I'd like to know a little about the process you went through for the port. That book * sounds like an excellent resource to start from but what was it like using it and the code? * https://pbrt.org/ https://pbrt.org/
- simondanisch 8mo ago
  I've done lots of manually refactoring of the initial Prototype in Trace.jl (by Anton Smirnov, who I think ported an earlier version of the pbrt book). This helped familiarizing myself with the math and infrastructure and the general problems a raytracer faces and lay the ground work for the general architecture and what to pay attention to for fast GPU execution. One key insight was, that its possible to not need to have an UberMaterial, but instead use a MultiTypeSet for storing different materials and lights, which allows fast and concretely typed iterations. Then I found that pbrt moved away from the initial design and I used claude code to port large parts of the new C++ code to Julia. This lead to a pretty bad port and I had lots of back and forth to fix bugs, improve the GPU acceleration, make the code more concise and "Julian" and correct the AIs mistakes and bogus design decisions ;) This polish isn't really over yet, but it works well enough and is fast enough for a beta release!
the_harpia_io 8mo ago
[flagged]
- simondanisch 8mo ago
  Well I'm a bit of an AMD "fanboy" and really dislike NVIDIA's vendor lock in. I'm not sure what you mean by dynamic dispatch across GPU backends - nothing should be dynamic there and most easier primitives map quite nicely between vendors (e.g. local memory, work groups etc). To be honest, the BVH/TLAS has been pretty simple in comparison to the wavefront infrastructure. We haven't done anything fancy yet, but the performance is still really good. I'm sure there are still lots of things we can do to improve performance, but right now I've concentrated on getting something usable out. Right now, we're mostly matching pbrt-v4 performance, but I couldn't compare to their NVIDIA only GPU acceleration without an NVIDIA gpu. I can just say that the performance is MUCH better than what I initially aimed for and it feels equally usable as some of the state of the art renderers I've been using. A 1:1 comparison is still missing though, since it's not easy to do a good comparison without comparing apples to oranges (already mapping materials and light types from one render to another is not trivial).
  - the_harpia_io 8mo ago
    [flagged]
    - simondanisch 8mo ago
      To be fair I was suprised too. But I made a relatively simple straight port from the AMD rays sdk plus some input from the pbrt-v4 CPU bvh code and it just worked relatively well out of the box... This is the main intersection function which is quite simple: https://github.com/JuliaGeometry/Raycore.jl/blob/sd/multitype-vec/src/instanced-bvh.jl#L1611-L1728 https://github.com/JuliaGeometry/Raycore.jl/blob/sd/multityp... I'm not even using local memory, since it was already fast enough ;) But I think we can still do quite a lot, large parts of the construction code are still very messy, and I want to polish and modularize the code over time.
builderhq_io 8mo ago
[flagged]
zamalek 7mo ago
> Cross-vendor GPU support: A single codebase runs on AMD, NVIDIA, and CPU via KernelAbstractions.jl This is why I wish Julia were the language for ML and sci comp in general, but Python is sucking all of the air out of the room.
- jampekka 7mo ago
  Maybe because Python can reasonably used to make actual applications instead of just notebooks or REPL sessions.
  - yellowapple 7mo ago
    What's stopping Julia from being reasonably usable to make actual applications? It's been awhile since I've touched it, but I ain't seeing a whole lot in the way of obstacles there — just less inertia.
    - zamalek 7mo ago
      It's actually better suited IMO, being a compiled language. I'm not sure how anyone could consider the current train wreck of getting python code just to run "actual applications." uv is great and all, but many of these "actual applications" don't use it.
    - fc417fc802 7mo ago
      Presumably inertia and ecosystem size (but that's a follow on of inertia). When Julia came out Python already had traction for ~most things. Keep in mind that it went with 1 based indexes to make the switch easy for Matlab types. I'm not sure if that was a good or bad move for the long term. I'm sure it got some people to move who otherwise wouldn't have but conversely there are also people like me who rejected it outright as a result (after suffering at the hands of 1 based indexing in Matlab I will never touch those again if I have any say in the matter). I've considered switching to it a few times since seeing that they added variable indexes but Python works well enough. Honestly if I were going to the trouble of switching I'd much rather use Common Lisp or R5RS. The nearest miss for me is probably Chicken, where you can seamlessly inline C code but (fatally) not C++ templates. If I ever encounter "Chicken, except Rust" I will probably switch to that for most things.
the__alchemist 7mo ago
The molecule and MD trajectory renders look great and an easy API! I have been doing this in rust, but it's a full program vs something scriptable like this. The images and animations on this page also look a hell of a lot better than what I cobbled together in WGPU.
juleiie 7mo ago
Interesting name. Strange feeling to use language/tech named same as you for some reason and this is a name that isn’t even niche or quirky but like second or third most popular It’s like calling a framework Mike
- Twisol 7mo ago
  I have a friend who named their custom-built languages "Monica" and "Joe". It's surprisingly common for homegrown languages, I think.
FacelessJim 7mo ago
That post is 10 years old, stale, with all issues resolved and more. Waving around an outdated blogpost as if it would automatically invalidate everything is just silly at this point.
- FacelessJim 7mo ago
  This was supposed to be under a different post. Not the main article. Whops.
tylermw 7mo ago
Nice work! It's always fun to see a new renderer in the wild.
tigoo 7mo ago
super cool!
avadodin 7mo ago
There was a meta article a few days back, but HN should do something to rescue ShowHN posts that are neither AIslop nor humanslop because people here seem to report them by default. They were paid to do this cool thing and possibly to post it here as well but if you dislike "capitalist pigs", maybe you shouldn't browse yc.

198 points by simondanisch 8mo ago

amelius 8mo ago

Is the material description part of the language the same as in PBRT? I'm asking because I had a lot of trouble trying to describe interfaces between materials, only to find out that what I wanted to do was not possible in PBRT without modifying the code. Apparently, in PBRT a material can only have one other material touching it. So, for example rendering a glass filled with water and ice is not possible without hacks. From a user's point of view this is a bit of a let-down, of course. Context: https://news.ycombinator.com/item?id=45668543 https://news.ycombinator.com/item?id=45668543

simondanisch 8mo ago
Nope, we made a complete high level Julia interface and I plan to have the Makie API be the main user facing scene description, which can be more descriptive than pbrt I think!
- amelius 8mo ago
  Ok. Did you see this: https://blog.yiningkarlli.com/2019/05/nested-dielectrics.html https://blog.yiningkarlli.com/2019/05/nested-dielectrics.htm... And I'm curious how you solve it.
  - simondanisch 8mo ago
    Sorry, I was on my phone. This doesn't seem to be a problem of the description language, but rather how the integrator and materials work internally, so this works the same way in Julia currently. I do think though, that its more approachable to add experimental features like this in the Julia version. Would certainly be an interesting project! I do want to over time get further away from the pbrt-v4 architecture and get to something much more modular and easy to extend. I feel like the overlaps resolve should happen at scene creation time, to not have an expensive priority stack at raytracing time - then it would be just a matter of better tracking the media at boundary crossing. But haven't really thought this through of course ;)
    - amelius 8mo ago
      I think it was a problem with the language as well as how they handle it internally. It was basically the algorithm that dictates how the language works, and consequently there was no way to have one material touch more than one other material. But I might misremember. Anyway, I'm looking at this from the user's perspective. I wanted to do some physics-based ray-tracing with lenses and pbrt is what I ended up trying. As such, I really needed the multi-material aspect to work correctly. Also, it would be nice to be able to describe surfaces using a z=f(x,y) kind of formulation, or a way to place a hook in the renderer.
      - simondanisch 8mo ago
        It's definitely an architectural problem as well. I do wonder if we could extend that though, without too much trouble for the general architecture - after all, the material does not necessarily need to represent all the outside materials and instead the ray only needs to be able to go from one medium to another. I'm happy to chat about possible extensions in that direction, although to be fair I wont have much time in the next weeks to sit down on anything like this. But, I do really hope that this can become a playground for ray tracing experiments in general!
        amelius 8mo ago
        I think maybe the easiest way to tackle the problem is to have the language describe surfaces instead of solid objects, and let every surface have a normal and two materials. This might be the most natural representation for a ray tracer.

LoganDark 8mo ago

On iOS Safari the videos are fullscreening themselves as I scroll. I've seen this on other blogs before but I don't know what causes it. Super annoying

simondanisch 8mo ago
Ugh, yeah I had some super weird bugs like this in safari, still haven't found the source :(
- embedding-shape 8mo ago
  Don't quote me on this, but I think there is a "playsinline" / "webkit-playsinline" attribute for the video element you need to add to avoid that, + if it's autoplay you need to set "muted" too. I've also had this happen and I think both/either of those solved it last time.

krastanov 8mo ago

As an aside, it is really interesting to see a computational package that, while supporting multiple GPU vendors, was first vetted on AMD, not NVidia. It is encouraging to see ROCM finally shaking off its reputation for poor support.

simondanisch 8mo ago
well, I do hate vendor lock in with a passion ;) But yeah, a lot did happen, this likely wouldn't have been possible one or two years ago!
mr_octopus 7mo ago
The vendor-agnostic GPU approach via KernelAbstractions is great to see. The Vulkan compute path is underrated for this — it runs on AMD, NVIDIA, and Intel without needing ROCm or CUDA, just whatever driver ships with the GPU. Re: the compilation latency discussion — it's a real tension. JIT gives you expressiveness but kills startup. AOT gives you instant start but limits flexibility. Interesting that most GPU languages went JIT when the GPU itself runs pre-compiled SPIR-V/PTX anyway.

NoboruWataya 8mo ago

I don't hear nearly as much about Julia as I used to. A few years ago the view was that it was about to replace Python as the language of choice for data science. Seems like that didn't happen?

simondanisch 8mo ago
I think the hype has slowed down, but all growth statistics haven't. Personally, I think Julia is the only language where I can implement something like Makie without running into a maintenance nightmare, and with Julia GPU programming is actually fun and high level and composes well, which I miss in most other languages. So, I dont really care about it replacing python or not. I do think for replacing python Julia will need to solve compilation latency, shipping AOT binaries and maybe interpret more of the glue code, which currently introduces quite a lot of compilation overhead without much gains in terms of performance.
bobajeff 8mo ago
As someone who currently uses dabbles in both. That prediction seems a bit unrealistic. Julia is a fantastic language but it has some trade offs that need to be considered. Probably the most well known is `time to first x`. Julia like Python is used comfortably in notebooks but loading libraries can take a minute, compared to Python where it happens right away. It may lead you to not reach for it when you want to do quick testing of something especially plotting. You can mitigate this somewhat by loading all the libraries you'll ever need at startup (preferably long before you are ready to experiment) but that assumes you already know what libraries you'll need for what you're wanting to try.
- simondanisch 8mo ago
  What prediction? Maybe I need to rephrase what I said: My prediction is, that if Julia ever wants to have a shot at replacing Python, it absolutely has to solve the first time to first x problem! That's what I mean by shipping fully ahead of time compiled binaries and interpreting more glue code - which both have the potential to solve the first time to x problem.

blueaquilae 8mo ago

That's an impressive accomplishment and a fantastic tool to explore.

johnbatman 8mo ago

[dead]

bobajeff 8mo ago

It's says: >the reference implementation from Physically Based Rendering (Pharr, Jakob, Humphreys) I'd like to know a little about the process you went through for the port. That book * sounds like an excellent resource to start from but what was it like using it and the code? * https://pbrt.org/ https://pbrt.org/

simondanisch 8mo ago
I've done lots of manually refactoring of the initial Prototype in Trace.jl (by Anton Smirnov, who I think ported an earlier version of the pbrt book). This helped familiarizing myself with the math and infrastructure and the general problems a raytracer faces and lay the ground work for the general architecture and what to pay attention to for fast GPU execution. One key insight was, that its possible to not need to have an UberMaterial, but instead use a MultiTypeSet for storing different materials and lights, which allows fast and concretely typed iterations. Then I found that pbrt moved away from the initial design and I used claude code to port large parts of the new C++ code to Julia. This lead to a pretty bad port and I had lots of back and forth to fix bugs, improve the GPU acceleration, make the code more concise and "Julian" and correct the AIs mistakes and bogus design decisions ;) This polish isn't really over yet, but it works well enough and is fast enough for a beta release!

the_harpia_io 8mo ago

[flagged]

simondanisch 8mo ago
Well I'm a bit of an AMD "fanboy" and really dislike NVIDIA's vendor lock in. I'm not sure what you mean by dynamic dispatch across GPU backends - nothing should be dynamic there and most easier primitives map quite nicely between vendors (e.g. local memory, work groups etc). To be honest, the BVH/TLAS has been pretty simple in comparison to the wavefront infrastructure. We haven't done anything fancy yet, but the performance is still really good. I'm sure there are still lots of things we can do to improve performance, but right now I've concentrated on getting something usable out. Right now, we're mostly matching pbrt-v4 performance, but I couldn't compare to their NVIDIA only GPU acceleration without an NVIDIA gpu. I can just say that the performance is MUCH better than what I initially aimed for and it feels equally usable as some of the state of the art renderers I've been using. A 1:1 comparison is still missing though, since it's not easy to do a good comparison without comparing apples to oranges (already mapping materials and light types from one render to another is not trivial).
- the_harpia_io 8mo ago
  [flagged]
  - simondanisch 8mo ago
    To be fair I was suprised too. But I made a relatively simple straight port from the AMD rays sdk plus some input from the pbrt-v4 CPU bvh code and it just worked relatively well out of the box... This is the main intersection function which is quite simple: https://github.com/JuliaGeometry/Raycore.jl/blob/sd/multitype-vec/src/instanced-bvh.jl#L1611-L1728 https://github.com/JuliaGeometry/Raycore.jl/blob/sd/multityp... I'm not even using local memory, since it was already fast enough ;) But I think we can still do quite a lot, large parts of the construction code are still very messy, and I want to polish and modularize the code over time.

builderhq_io 8mo ago

[flagged]

zamalek 7mo ago

> Cross-vendor GPU support: A single codebase runs on AMD, NVIDIA, and CPU via KernelAbstractions.jl This is why I wish Julia were the language for ML and sci comp in general, but Python is sucking all of the air out of the room.

jampekka 7mo ago
Maybe because Python can reasonably used to make actual applications instead of just notebooks or REPL sessions.
- yellowapple 7mo ago
  What's stopping Julia from being reasonably usable to make actual applications? It's been awhile since I've touched it, but I ain't seeing a whole lot in the way of obstacles there — just less inertia.
  - zamalek 7mo ago
    It's actually better suited IMO, being a compiled language. I'm not sure how anyone could consider the current train wreck of getting python code just to run "actual applications." uv is great and all, but many of these "actual applications" don't use it.
  - fc417fc802 7mo ago
    Presumably inertia and ecosystem size (but that's a follow on of inertia). When Julia came out Python already had traction for ~most things. Keep in mind that it went with 1 based indexes to make the switch easy for Matlab types. I'm not sure if that was a good or bad move for the long term. I'm sure it got some people to move who otherwise wouldn't have but conversely there are also people like me who rejected it outright as a result (after suffering at the hands of 1 based indexing in Matlab I will never touch those again if I have any say in the matter). I've considered switching to it a few times since seeing that they added variable indexes but Python works well enough. Honestly if I were going to the trouble of switching I'd much rather use Common Lisp or R5RS. The nearest miss for me is probably Chicken, where you can seamlessly inline C code but (fatally) not C++ templates. If I ever encounter "Chicken, except Rust" I will probably switch to that for most things.

the__alchemist 7mo ago

The molecule and MD trajectory renders look great and an easy API! I have been doing this in rust, but it's a full program vs something scriptable like this. The images and animations on this page also look a hell of a lot better than what I cobbled together in WGPU.

juleiie 7mo ago

Interesting name. Strange feeling to use language/tech named same as you for some reason and this is a name that isn’t even niche or quirky but like second or third most popular It’s like calling a framework Mike

Twisol 7mo ago
I have a friend who named their custom-built languages "Monica" and "Joe". It's surprisingly common for homegrown languages, I think.

FacelessJim 7mo ago

That post is 10 years old, stale, with all issues resolved and more. Waving around an outdated blogpost as if it would automatically invalidate everything is just silly at this point.

FacelessJim 7mo ago
This was supposed to be under a different post. Not the main article. Whops.

tylermw 7mo ago

Nice work! It's always fun to see a new renderer in the wild.

tigoo 7mo ago

super cool!

avadodin 7mo ago

There was a meta article a few days back, but HN should do something to rescue ShowHN posts that are neither AIslop nor humanslop because people here seem to report them by default. They were paid to do this cool thing and possibly to post it here as well but if you dislike "capitalist pigs", maybe you shouldn't browse yc.