7 ms·

John Carmack's Comment on Hardware Ray Tracing

153 points by lispython 14y ago

hayksaakian 14y ago
Its so cool that someone as high up in the games industry as carmack actually knows his technical shit.
- damian2000 14y ago
  He might be high up now, but he started off as a programmer. He co-founded id software and (along with others) developed Wolfenstein 3D, Doom, etc. http://en.wikipedia.org/wiki/Id_Software http://en.wikipedia.org/wiki/Id_Software
  - enneff 14y ago
    He still is a programmer. He's the lead programmer at id and writes code for Armadillo Aerospace.
    - damian2000 14y ago
      yeah, he can probably do whatever he wants, the fact he's still programming means he obviously enjoys it.
  - boyter 14y ago
    Not only that but I believe he is still "in the trenches" and cuts industry leading code to this day.
  - ibrahima 14y ago
    If you follow him on Twitter, he posts almost exclusively highly technical content.
  - hayksaakian 14y ago
    I was aware of this, but Mark Z. Also started in the programming, today he's mister CEO, too important for code.
- james4k 14y ago
  He's known for knowing his shit. :)
  - damian2000 14y ago
    another example ... "I can send an IP packet to Europe faster than I can send a pixel to the screen. How f’d up is that?" http://superuser.com/questions/419070/transatlantic-ping-faster-than-sending-a-pixel-to-the-screen/419167#419167 http://superuser.com/questions/419070/transatlantic-ping-fas...
electrograv 14y ago
Excellent response by Carmack. "Ray Tracing" has always been an overhyped or misunderstood technology, at least in my experience. Because of the impressive lighting effects it's famous for producing, people view it as some "holy grail" of superior computer graphics technology that we just need to optimize a bit for use in games. As Carmack described, those highly realistic ray traced renders come at a price: billions of ray calculations. You don't get all that pixel-perfect refraction, etc. for free (and probably never will - at least not with ray tracing.) He also explains that for most people, this (e.g. pixel perfect refraction) really doesn't matter, when rasterization techniques exist that achieve extremely high quality approximations at a fraction of the computational cost. Conversely though, ray tracing and related concepts (ray casting) are not at all without value. Many modern video games today actually use a sort of hybrid rasterization / ray casting approach. Macrostructure objects are rasterized the normal way, while smaller high resolution detail patterns (like a brick wall, or stones in the ground) are "added" to surfaces via per-pixel displacement mapping, which is ray casting at its core. This is one of the few cases where you can take advantage of the "log2" efficiency Carmack mentioned -- in a highly specialized implementation without a huge time constant.
- ChuckMcM 14y ago
  Pretty much agree, the only thing that we might disagree on is the 'billions' number, while I expect good ray tracing to trace billions of rays, if transistors are cheap enough this becomes more interesting. The typical HDMI 1080p display is 2 megapixels (or megatexels) if you have a rendering engine with 2,073,600 cores each of which is looking at a billion ray 'view' based on where it sits in the scene, its easier to set up the scene and light it. That presumes that you can do those cores like DRAM instead of like current processors of course. My point is that cheap transistors keep amazing me and whenever I say "Oh that will never happen" some fool goes and shows me I'm wrong. I'm trying to imagine how complex a 'core' would be that computed the incident rays on a single pixel. Then figuring out how big that is in a 18nm process technology and then trying to see if I can fit 2M on a reasonable size die. My head exploded sadly.
  - yk 14y ago
chii 14y ago
Having no real knowledge of ray tracing (or rendering for that matter), can someone tell me why we can't be moving towards a massive cluster of gpu (in the order of 10,000) ray tracing a scene in parellel? I imagine each ray trace call is quite independent of any other. SOooner or later, cpu (and gpu) will have many cores available for such form of rendering. Is it because of the chicken/egg problem?
- gsg 14y ago
  Ray tracing enjoys a ridiculous amount of inherent parallelism. That's not much of an advantage though, since rasterisers also enjoy a ridiculous amount of inherent parallelism. Large scale "render farms" that exploit both already exist and are in commercial use in the VFX industry.
  - danwills 14y ago
    I work in the VFX industry, and I agree. We often need to make a choice between a rasterisation-like method - which in VFX usually means micropolygon rendering like Pixar's REYES algorithm - or a raytracing-based method, like SideFX Mantra's Phsically-Based-Rendering. It is also possible to hybridize the two, which usually means casting rays in order to shade micropolygons. Both approaches have their own advantages. There are some tools that allow you to render a frame quickly by using many hosts to parallelise the task. This is usually only practical when the data that generates the frame is relatively small, in other words not gigabytes of fluid simulation voxels. It's also not realtime, by any stretch! A renderfarm is usually not technologically any different to local rendering on a workstation. We generally don't parallelize across hosts within a single frame, but at least you can render lots of frames simultaneously! Another thing that doesn't seem to have been mentioned on this post is that raytracing can be accelerated massively (where applicable) by using instancing. A single object can be used many times in a scene, only differing in its transformation. This allows the geometry to be stored in ram once and re-used, incoming rays that are incident on an instance's bounding box can simply be transformed into the local space of the instance. Of course this is of no help in an extremely complex scene full of unique objects, but in practice you can make great savings (and create very complex scenes that are cheap to raytrace) this way.
jamieb 14y ago
Here's a commercial, realtime raytracing tool. Runs happily on a dual-core macbook pro. No custom card needed. Yes, it runs better on a $20,000 64-thread Tigerton, but it will run usably on a laptop. http://www.youtube.com/watch?v=yUiLBmioRI4 http://www.youtube.com/watch?v=yUiLBmioRI4 Carmack is right about games (surprise!). I can't imagine "Imagination Technologies" is pushing the Caustic R2500 at games.
- samwillis 14y ago
  We use KeyShot at work and its an amazing package, its the industry standard now for rendering product design CAD. The demo video showing the Caustic card running is about equivalent to KeyShot running on a $3000 PC. They have a way to go until they convince me that they have anything better than what KeyShot is capable of.
ianb 14y ago
"For example, all surfaces that are shaded with interpolated normal will have an unnatural shadow discontinuity at the silhouette edges with single shadow ray traces." For some reason I really want to understand what this sentence means. I don't know why it jumped out at me, maybe because it seems both accessible and arcane, I want some path to even just tour the arcane concerns of someone so deep into a (this) particular domain.
- daeken 14y ago
  In English: if you're only using one ray for shading, then you'll have areas that are completely in shadow directly next to areas that are completely not in shadow; what we call hard shadows. Soft shadows are a better approximation of reality, because they show the shades of grey between "covered" and "uncovered", but they require more ray intersections (unless you're using sphere tracing!).
  - dualogy 14y ago
    Sphere tracing is awesome. So many effects can be so easily and cheaply approximated, apart from soft shadows also subsurface scattering, bloom/glare etc. Downside, while great for geometry describable by distance fields, it's quite useless with polygonal assets...
- deleted 14y ago
  [deleted]
- kybernetikos 14y ago
  Typically when ray tracing you follow one (or more) rays from the eye through a pixel on the screen until it hits something. Then you have to decide what that point on the surface of thing it hits should look like. Usually this breaks down into a combination of how much ambient light you're simulating, the material properties of the surface, the results of any reflection/refraction rays you fire off, and the effect of light sources on it. The simplest way to calculate the light sources is to follow another ray to each of your light sources, and if there's nothing in the way, you add the intensity of that light to the pixel. The problem with this simple approach is that something is either blocking your path to the light source or it isn't, which creates absolutely sharp shadows because it's simulating the lights as if all the light's brightness is emanating from a single infinitesimal point. In real life, lights tend not to be like that. To get realistic looking shadows with ray tracing you have to send multiple rays to different parts of each of your lights, adding a portion of the lights brightness each time (you're essentially doing a monte carlo integration over the area of the light). The more you do, the better looking shadow edges you can get, but also the more effort you have to spend in calculating. Raytracing is rather fun because you can get really interesting results with relatively little code and it's very visual - you actually see your bugs. Some people do simple ray tracers as katas. I did a basic one as a way of learning Scala. If you're interested in having a look it's here (very simple of course - I do treat my light sources as points): https://github.com/kybernetikos/ScalaTrace/wiki/ScalaTrace https://github.com/kybernetikos/ScalaTrace/wiki/ScalaTrace
ohwp 14y ago
I think path tracing is one of the most promising methods for realtime ray tracing. Brigade is one example: http://igad.nhtv.nl/~bikker/ http://igad.nhtv.nl/~bikker/ Here an in-game example: http://www.youtube.com/watch?v=6_DrgiwLABk http://www.youtube.com/watch?v=6_DrgiwLABk And a nice blog with posts about Brigade, Octane and others: http://raytracey.blogspot.nl/ http://raytracey.blogspot.nl/
gavanwoolery 14y ago
I think he is not wrong at all, but personally, I've been on both sides of the debate, and then I kind of formed the opinion that it's not worth debating about. The one thing I would strongly, strongly push for is general purpose hardware (we are kind of headed in that direction with OpenCL and CUDA, but industry-wide interest in these things is really low). Don't force me to use rasterization or raytracing (or even triangles), just give me a really powerful, flexible parallel processor...even if that means sacrificing a bit of speed. I think above all Carmack would appreciate such hardware...there was a kind of beauty in his old code you don't find any more -- you can do lots of tricks with GPUs, but overall it is a much more limited, less creative programming experience.
- Peaker 14y ago
  Typically limitations aid creativity, not hamper it.
  - gavanwoolery 14y ago
    Limitations force you to come up with creative solutions to problems that should have been easier in the first place. :) But sometimes limitations force you to focus on the important things (if you could render absolutely anything, you could spend forever building an engine instead of a game).
    - Peaker 14y ago
      I think sometimes the creative solutions end up being better than the simple "obvious" solutions that were apparent before the limitations were put in place.
- duaneb 14y ago
  > just give me a really powerful, flexible parallel processor Isn't that exactly what a GPU is? They aren't terribly fantastic at traditional computation, but then again we're talking parallelism here.
  - 14y ago
Keyframe 14y ago
As anyone who ever wrote a raytracer would know (or find out soon): Primary rays cache. Secondary rays trash! Brazil guys wrote a decent raytracer (entirely based on Glassner's book) and are really competent devs in general (both of them). Current state of the art, performance oriented, offline renderer is Arnold by SolidAngle, with more smart guys behind it. I think we will see convergence of offline and online RT soon enough. There are already steps being made in that direction, in offline world, towards RT previews.
javanix 14y ago
I wish I could sit down and talk with him for an hour or so. I'm a Unix programmer but I suspect I could learn as much from him about software design as from weeks of talking with just about anyone else.
- jerf 14y ago
  Here's a link you may enjoy: http://news.ycombinator.com/item?id=3388290 http://news.ycombinator.com/item?id=3388290 , which goes to http://www.altdevblogaday.com/2011/12/24/static-code-analysis/ http://www.altdevblogaday.com/2011/12/24/static-code-analysi... .
- asdfs 14y ago
  Check out his QuakeCon keynote. It's basically him talking for an hour about technical things that interest him.
MrJagil 14y ago
This reminds me of http://en.wikipedia.org/wiki/Euclideon http://en.wikipedia.org/wiki/Euclideon ... Not trying to draw a comparison, i just find these "future of gaming" technologies interesting, vapor or not.
krosaen 14y ago
Within his comment is a great example / reminder to consider constant factors when doing your 'big-oh' analysis of different approaches to a problem. """ Because ray tracing involves a log2 scale of the number of primitives, while rasterization is linear, it appears that highly complex scenes will render faster with ray tracing, but it turns out that the constant factors are so different that no dataset that fits in memory actually crosses the time order threshold. """

153 points by lispython 14y ago

hayksaakian 14y ago

Its so cool that someone as high up in the games industry as carmack actually knows his technical shit.

damian2000 14y ago
He might be high up now, but he started off as a programmer. He co-founded id software and (along with others) developed Wolfenstein 3D, Doom, etc. http://en.wikipedia.org/wiki/Id_Software http://en.wikipedia.org/wiki/Id_Software
- enneff 14y ago
  He still is a programmer. He's the lead programmer at id and writes code for Armadillo Aerospace.
  - damian2000 14y ago
    yeah, he can probably do whatever he wants, the fact he's still programming means he obviously enjoys it.
- boyter 14y ago
  Not only that but I believe he is still "in the trenches" and cuts industry leading code to this day.
- ibrahima 14y ago
  If you follow him on Twitter, he posts almost exclusively highly technical content.
- hayksaakian 14y ago
  I was aware of this, but Mark Z. Also started in the programming, today he's mister CEO, too important for code.
james4k 14y ago
He's known for knowing his shit. :)
- damian2000 14y ago
  another example ... "I can send an IP packet to Europe faster than I can send a pixel to the screen. How f’d up is that?" http://superuser.com/questions/419070/transatlantic-ping-faster-than-sending-a-pixel-to-the-screen/419167#419167 http://superuser.com/questions/419070/transatlantic-ping-fas...

electrograv 14y ago

Excellent response by Carmack. "Ray Tracing" has always been an overhyped or misunderstood technology, at least in my experience. Because of the impressive lighting effects it's famous for producing, people view it as some "holy grail" of superior computer graphics technology that we just need to optimize a bit for use in games. As Carmack described, those highly realistic ray traced renders come at a price: billions of ray calculations. You don't get all that pixel-perfect refraction, etc. for free (and probably never will - at least not with ray tracing.) He also explains that for most people, this (e.g. pixel perfect refraction) really doesn't matter, when rasterization techniques exist that achieve extremely high quality approximations at a fraction of the computational cost. Conversely though, ray tracing and related concepts (ray casting) are not at all without value. Many modern video games today actually use a sort of hybrid rasterization / ray casting approach. Macrostructure objects are rasterized the normal way, while smaller high resolution detail patterns (like a brick wall, or stones in the ground) are "added" to surfaces via per-pixel displacement mapping, which is ray casting at its core. This is one of the few cases where you can take advantage of the "log2" efficiency Carmack mentioned -- in a highly specialized implementation without a huge time constant.

ChuckMcM 14y ago
Pretty much agree, the only thing that we might disagree on is the 'billions' number, while I expect good ray tracing to trace billions of rays, if transistors are cheap enough this becomes more interesting. The typical HDMI 1080p display is 2 megapixels (or megatexels) if you have a rendering engine with 2,073,600 cores each of which is looking at a billion ray 'view' based on where it sits in the scene, its easier to set up the scene and light it. That presumes that you can do those cores like DRAM instead of like current processors of course. My point is that cheap transistors keep amazing me and whenever I say "Oh that will never happen" some fool goes and shows me I'm wrong. I'm trying to imagine how complex a 'core' would be that computed the incident rays on a single pixel. Then figuring out how big that is in a 18nm process technology and then trying to see if I can fit 2M on a reasonable size die. My head exploded sadly.
- yk 14y ago

chii 14y ago

Having no real knowledge of ray tracing (or rendering for that matter), can someone tell me why we can't be moving towards a massive cluster of gpu (in the order of 10,000) ray tracing a scene in parellel? I imagine each ray trace call is quite independent of any other. SOooner or later, cpu (and gpu) will have many cores available for such form of rendering. Is it because of the chicken/egg problem?

gsg 14y ago
Ray tracing enjoys a ridiculous amount of inherent parallelism. That's not much of an advantage though, since rasterisers also enjoy a ridiculous amount of inherent parallelism. Large scale "render farms" that exploit both already exist and are in commercial use in the VFX industry.
- danwills 14y ago
  I work in the VFX industry, and I agree. We often need to make a choice between a rasterisation-like method - which in VFX usually means micropolygon rendering like Pixar's REYES algorithm - or a raytracing-based method, like SideFX Mantra's Phsically-Based-Rendering. It is also possible to hybridize the two, which usually means casting rays in order to shade micropolygons. Both approaches have their own advantages. There are some tools that allow you to render a frame quickly by using many hosts to parallelise the task. This is usually only practical when the data that generates the frame is relatively small, in other words not gigabytes of fluid simulation voxels. It's also not realtime, by any stretch! A renderfarm is usually not technologically any different to local rendering on a workstation. We generally don't parallelize across hosts within a single frame, but at least you can render lots of frames simultaneously! Another thing that doesn't seem to have been mentioned on this post is that raytracing can be accelerated massively (where applicable) by using instancing. A single object can be used many times in a scene, only differing in its transformation. This allows the geometry to be stored in ram once and re-used, incoming rays that are incident on an instance's bounding box can simply be transformed into the local space of the instance. Of course this is of no help in an extremely complex scene full of unique objects, but in practice you can make great savings (and create very complex scenes that are cheap to raytrace) this way.

jamieb 14y ago

Here's a commercial, realtime raytracing tool. Runs happily on a dual-core macbook pro. No custom card needed. Yes, it runs better on a $20,000 64-thread Tigerton, but it will run usably on a laptop. http://www.youtube.com/watch?v=yUiLBmioRI4 http://www.youtube.com/watch?v=yUiLBmioRI4 Carmack is right about games (surprise!). I can't imagine "Imagination Technologies" is pushing the Caustic R2500 at games.

samwillis 14y ago
We use KeyShot at work and its an amazing package, its the industry standard now for rendering product design CAD. The demo video showing the Caustic card running is about equivalent to KeyShot running on a $3000 PC. They have a way to go until they convince me that they have anything better than what KeyShot is capable of.

ianb 14y ago

"For example, all surfaces that are shaded with interpolated normal will have an unnatural shadow discontinuity at the silhouette edges with single shadow ray traces." For some reason I really want to understand what this sentence means. I don't know why it jumped out at me, maybe because it seems both accessible and arcane, I want some path to even just tour the arcane concerns of someone so deep into a (this) particular domain.

daeken 14y ago
In English: if you're only using one ray for shading, then you'll have areas that are completely in shadow directly next to areas that are completely not in shadow; what we call hard shadows. Soft shadows are a better approximation of reality, because they show the shades of grey between "covered" and "uncovered", but they require more ray intersections (unless you're using sphere tracing!).
- dualogy 14y ago
  Sphere tracing is awesome. So many effects can be so easily and cheaply approximated, apart from soft shadows also subsurface scattering, bloom/glare etc. Downside, while great for geometry describable by distance fields, it's quite useless with polygonal assets...
deleted 14y ago
[deleted]
kybernetikos 14y ago
Typically when ray tracing you follow one (or more) rays from the eye through a pixel on the screen until it hits something. Then you have to decide what that point on the surface of thing it hits should look like. Usually this breaks down into a combination of how much ambient light you're simulating, the material properties of the surface, the results of any reflection/refraction rays you fire off, and the effect of light sources on it. The simplest way to calculate the light sources is to follow another ray to each of your light sources, and if there's nothing in the way, you add the intensity of that light to the pixel. The problem with this simple approach is that something is either blocking your path to the light source or it isn't, which creates absolutely sharp shadows because it's simulating the lights as if all the light's brightness is emanating from a single infinitesimal point. In real life, lights tend not to be like that. To get realistic looking shadows with ray tracing you have to send multiple rays to different parts of each of your lights, adding a portion of the lights brightness each time (you're essentially doing a monte carlo integration over the area of the light). The more you do, the better looking shadow edges you can get, but also the more effort you have to spend in calculating. Raytracing is rather fun because you can get really interesting results with relatively little code and it's very visual - you actually see your bugs. Some people do simple ray tracers as katas. I did a basic one as a way of learning Scala. If you're interested in having a look it's here (very simple of course - I do treat my light sources as points): https://github.com/kybernetikos/ScalaTrace/wiki/ScalaTrace https://github.com/kybernetikos/ScalaTrace/wiki/ScalaTrace

ohwp 14y ago

I think path tracing is one of the most promising methods for realtime ray tracing. Brigade is one example: http://igad.nhtv.nl/~bikker/ http://igad.nhtv.nl/~bikker/ Here an in-game example: http://www.youtube.com/watch?v=6_DrgiwLABk http://www.youtube.com/watch?v=6_DrgiwLABk And a nice blog with posts about Brigade, Octane and others: http://raytracey.blogspot.nl/ http://raytracey.blogspot.nl/

gavanwoolery 14y ago

I think he is not wrong at all, but personally, I've been on both sides of the debate, and then I kind of formed the opinion that it's not worth debating about. The one thing I would strongly, strongly push for is general purpose hardware (we are kind of headed in that direction with OpenCL and CUDA, but industry-wide interest in these things is really low). Don't force me to use rasterization or raytracing (or even triangles), just give me a really powerful, flexible parallel processor...even if that means sacrificing a bit of speed. I think above all Carmack would appreciate such hardware...there was a kind of beauty in his old code you don't find any more -- you can do lots of tricks with GPUs, but overall it is a much more limited, less creative programming experience.

Peaker 14y ago
Typically limitations aid creativity, not hamper it.
- gavanwoolery 14y ago
  Limitations force you to come up with creative solutions to problems that should have been easier in the first place. :) But sometimes limitations force you to focus on the important things (if you could render absolutely anything, you could spend forever building an engine instead of a game).
  - Peaker 14y ago
    I think sometimes the creative solutions end up being better than the simple "obvious" solutions that were apparent before the limitations were put in place.
duaneb 14y ago
> just give me a really powerful, flexible parallel processor Isn't that exactly what a GPU is? They aren't terribly fantastic at traditional computation, but then again we're talking parallelism here.
- 14y ago

Keyframe 14y ago

As anyone who ever wrote a raytracer would know (or find out soon): Primary rays cache. Secondary rays trash! Brazil guys wrote a decent raytracer (entirely based on Glassner's book) and are really competent devs in general (both of them). Current state of the art, performance oriented, offline renderer is Arnold by SolidAngle, with more smart guys behind it. I think we will see convergence of offline and online RT soon enough. There are already steps being made in that direction, in offline world, towards RT previews.

javanix 14y ago

I wish I could sit down and talk with him for an hour or so. I'm a Unix programmer but I suspect I could learn as much from him about software design as from weeks of talking with just about anyone else.

jerf 14y ago
Here's a link you may enjoy: http://news.ycombinator.com/item?id=3388290 http://news.ycombinator.com/item?id=3388290 , which goes to http://www.altdevblogaday.com/2011/12/24/static-code-analysis/ http://www.altdevblogaday.com/2011/12/24/static-code-analysi... .
asdfs 14y ago
Check out his QuakeCon keynote. It's basically him talking for an hour about technical things that interest him.

MrJagil 14y ago

This reminds me of http://en.wikipedia.org/wiki/Euclideon http://en.wikipedia.org/wiki/Euclideon ... Not trying to draw a comparison, i just find these "future of gaming" technologies interesting, vapor or not.

krosaen 14y ago

Within his comment is a great example / reminder to consider constant factors when doing your 'big-oh' analysis of different approaches to a problem. """ Because ray tracing involves a log2 scale of the number of primitives, while rasterization is linear, it appears that highly complex scenes will render faster with ray tracing, but it turns out that the constant factors are so different that no dataset that fits in memory actually crosses the time order threshold. """