4 ms·

I think graphics technology has already "arrived" for the most part. There's tons of stuff that surpasses what Crytek did that blew away the industry 16 years a

I think graphics technology has already "arrived" for the most part. There's tons of stuff that surpasses what Crytek did that blew away the industry 16 years ago, but a lot of that wasn't just technology, it's that they had world-class artists. If you go back to that game today, there's a lot you can nitpick, but I think the biggest problem that remains today is that it's still extremely labor intensive to make assets. At Planimeter, I worry more about asset creation turnaround time than anything else. I worry about it more than the tech we write, I worry about it more than fiction writing, or audio engineering. Just nothing else compares. It's the most expensive part of any game development pipeline, and yet industry-wide accessible photogrammetry is half-backed, and it also doesn't help hobbyists who do 2.5D or 2D work. So yeah, this neural engine stuff is superphenominal, for people who care about PBR workflows and photorealistic pipelines. This is obviously the most computationally complex work that can be addressed today, anyway, which I appreciate. But the people who can actually access and harness that tech is just so absolutely tiny. I guess I just care a lot about this one particular sector of the industry where you have these world-class bedroom professionals who become studio professionals and Unreal and Nvidia are probably the only orgs in the world who cater to them, but for some reason I have to put in a lot of effort to articulate why what we have today, despite being so much more powerful than what we've had 20 years ago is less accessible and less functional and less empowering than what we had then. I think it's primarily the labor factor in artwork, but also accessible engine tech today is actually worse than what we were using then, simply because no one really uses the id Tech family of engines anymore besides the most modern incarnations of them, and even to this day no class of engine compares to old versions of id Tech. Not even Unreal, by a long shot, despite having industry leading rendering capabilities. Everything else about that engine is half-baked or unusable.

mmcconnell1618 3y ago
I wonder how much AI models can help. If they are trained on enough real world content, they should be able to produce rough models based on descriptions: "Generate a tropical island about 5km in width by 10km length with an active volcano in the center" and then then the artists adjust/massage as needed.
- T-A 3y ago
  https://analyticsindiamag.com/raja-koduri-gives-a-sneak-peek-into-his-new-startup/ https://analyticsindiamag.com/raja-koduri-gives-a-sneak-peek...
- imiric 3y ago
  Right, we're already there, and the tech is quickly improving. I'm not sure why GP thinks this will remain the most expensive process of game development. Generating assets using AI will drastically cut costs and turnaround time. I'm eager to see these tools generate assets in real-time. Imagine being able to interact with an endless possibility of scenes. Games would ship with a generative model, and entire levels could be created on the fly, ensuring every play through is as different as the authors want it to be. Exciting stuff.
  - andrewmcwatters 3y ago
    I never said it will remain the most expensive part; that's what I hope more engineering effort will go into. The graphics capabilities we use today are already commonplace and well written about. Fast content creation isn't, or the tools are sparse and locked behind small, costly proprietary silos.
- xh-dude 3y ago
  I’m not sure text prompts offer the kind of plasticity needed for artists to really do their thing. Frank Gehry used to do really chaotic abstract drawings and hand them off to aerospace CAD operators to turn them into something that could be engineered and machined and assembled. Lots of people could do the drawings and have magic mushroom facades but Gehry understood the resulting interior spaces as well … I think it’s there is a ton of potential for great tools employing the past decade or so of ML/AI but ISTM it should empower the process rather than replace it.
Animats 3y ago
Good point. I would have liked to see examples of human skin, and of cloth. That's hard and important. Rendering the perfect cheese grater and glazed pot is nice, but really, not that important. Most of the hard problems in graphics today involve scale. Epic's Nanite texture compression system is impressive, but about 60% of the work has to be done in CPUs because GPUs don't have the right stuff for it. And Nanite is still just for rigid objects. Crowds of individually dressed people are still tough to render fast. NVidia put in ray-tracing hardware into GPUs. It's not used much in games. NVidia was angry with reviewers who just ignored their ray-tracing hardware in reviews. The reviewers were not wrong. And could we please have good hardware support for order-independent translucency? That should be standard. Then we can get rid of depth sorting of faces, which never works right.
- mrtranscendence 3y ago
  > NVidia put in ray-tracing hardware into GPUs. It's not used much in games. Well, it's true that the proportion of games that have ray tracing -- over the space of all games released -- is small. But most triple-A games seem to have ray tracing these days. Of the last four such titles I've played three had ray tracing (Resident Evil 4, Jedi Survivor, Hogwarts Legacy). I think that's only going to become more true in the future, as IMO it does materially make an image quality difference.
  - ehnto 3y ago
    They had the technology but not the power, so most games still only had bits and pieces using ray tracing and were not globally illuminated by it. Not to mention you have to make the game work on old equipment too. That means all the downsides of old lighting techniques remained and none of the art efficiencies are realized. We are at the third generation of cards with RTX and there are almost no games using ray traced global illumination, and even if they do they still have to offer rasterized lighting for older hardware.
    - yazzku 3y ago
      What do you mean "almost no games using ray traced global illumination"? The games mentioned above and many others are using ray-traced shadows, reflections, and sometimes one bounce of (diffuse) light. Not full path tracing, of course, but these techniques are still considered "ray-traced global illumination".
sufehmi 3y ago
Sorry I've read the last paragraph multiple times, but still couldn't figure out whether you're saying that id Tech is the best, or the worst...? I think you're saying that id Tech engine, even the old ones, are the best by a long shot; but some words in that paragraph seem to contradict that. And despite being a fan of John Carmack's works, I understand that the Unreal engines have its own advantages.
- jasonwatkinspdx 3y ago
  Yeah, I only worked in the business shortly but I would dispute that statement as well. Back at the time licensing from iD was basically "here's a cd, never call us again." Meanwhile Unreal, at a much more approachable price, also came with support contacts, a private email list and irc channel, etc. It was like night and day between the two. In my role as a level designer I vastly preferred UnrealEd. The workflow was more interactive, and the BSP implementation way more robust against leaks. There were flaws to Unreal... the initial versions of UnrealScript were not great. But again, there was simply no comparing the two imo.
syntheweave 3y ago
Easing the creation task for professional uses is definitely "in the pipeline". If you're making a 2D game, diffusion models are already capable enough to act as a shortcut for detailed painting. It's stuff with generating optimized meshes, animation, scene lighting, other details that is still in the research stage. In many ways it's more straightforward and gives a better demo to address the "render the entire scene" case than to go asset-by-asset and work out a better UX for that than "do everything by hand". Even though the latter is what's really being called for.
llamaLord 3y ago
This... 100% this. I've dabbled in India game making for a while and have no issue creating the relevant systems etc to make a game work because with good software engineering practises and modern off-the-shelf game engines you can scale that stuff like nobody's business. What you still can't scale... Art assets... Especially if you're not creatively talented in that way. There's a reason so many AAA games these days have staff that are like 80% artists, 10% developers, 10% everything else.
- KennyBlanken 3y ago
  Pretty sure all the artists are in contracted studios or contractors themselves. At least in F2P, the 'staff' are 80% artists because cosmetics are ludicrously profitable.
elihu 3y ago
I think part of the point of moving to ray tracing and physically correct global illumination is that if the technology stack works right you don't actually have to work very hard to get your assets to look good. Let's take for example Second Life. The content is almost all user-generated, and when it originally came out it looked a lot worse than other games that were available at the time. The difference is mostly: since the content is user generated, you don't have some army of artists touching up the textures and pre-baking the lighting to make everything look at least plausibly realistic. What Second Life was missing was the technology to compute global illumination in real time against arbitrary content so that the user-generated assets look as good as if some artist had specially lit and textured each one. (Maybe even better, because computers are more thorough and exact than humans -- assuming exactness is what you want. Artists sometimes take creative liberties because unrealism sometimes looks better.) I haven't paid attention to Second Life in a long time; if a few minutes looking at random youtube videos is any indication, it seems they haven't really made much progress on this front. I think eventually companies will figure this out. I haven't paid attention to Facebook's Metaverse project because I expect it to be a flop (for non-technical reasons), but maybe they're banking on the graphics technology being there to make user-generated content look not-fake. I think that part might be achievable now with modest hardware. As far as this new paper goes, I'm not sure if it makes things easier or harder for content creators. It seems like it makes the rendering stack more complex and harder to understand, but if someone can supply came creators with awesome textures and they don't have to care where they came from or how they work then maybe that's fine.
bl4ckm0r3 3y ago
How do you think AI and tools like midjourney could improve this asset creation process (if you think they can be used in this scenario)?
- andrewmcwatters 3y ago
  I think AI can be a force multiplier for asset turnaround time by helping artists who don’t have particular preferences for vision make generally real-world recognizable objects quickly and build meaningful libraries of models to help build world building sets. Theres also non-AI work that can help dynamically fill scenes with world meshes to help create seamless transitions between spaces as a part of overall terrain development. A lot of this tech will be proprietary in nature, and it may be a long time, think decades, if ever that we get to use it as general creators. It means that the gap between studios and bedroom professionals who used to create the mods that became full fledged intellectual properties that dominated for years will widen and there will be fewer opportunities for individual authors and small teams to become the next generation of production studios. This will happen while large organizations enjoy extreme technical leverage while not having many great ideas to execute on simply because there will be fewer fiction writers and fewer studios who have concentrated technologies. I currently don’t see much room for the next Zoid to create the next CTF gamemode. I don’t really see anything meaningful come out of community contributions to games anymore ever since the introduction of the UGC model which has superseded the traditional modding model. Instead of modders using game software mechanisms to create something new and interesting organically, UGC structures in games are explicitly designed so that creators can only do nominal things like create hats and skins for the most part. It’s shifted the mentally from, “Here are the tools we used to create the game,” to “Here’s what you’re allowed to create and put on a marketplace so we can take a cut of profits.” It’s gross, it sucks, and it’s another example of rug-pulling. Future generations will completely miss out on the previous paradigm because there’s so much focus entrenched on user-generated content versus modding.
oliwarner 3y ago
For gaming you're absolutely right. It's interactive so the threshold for immersive is that much lower. But imagine the impact on rendered art, VFX, etc. How long until you can ask ChatGPT for a model of a teapot with shaders for blue glaze? The labour involved from inception to render is going to be cut down to a tiny fraction of what's required today.
HellDunkel 3y ago
A long rant about the state of things in game engine techn and quite offtopic. I could not agree less. The id engine was hardly usable for anyone except a few studios with big pockets. With modern game engines you get great documentation and affordable tech. Unreal even comes with free photoreal assets and open source.
- deleted 3y ago
  [deleted]

mmcconnell1618 3y ago

I wonder how much AI models can help. If they are trained on enough real world content, they should be able to produce rough models based on descriptions: "Generate a tropical island about 5km in width by 10km length with an active volcano in the center" and then then the artists adjust/massage as needed.

T-A 3y ago
https://analyticsindiamag.com/raja-koduri-gives-a-sneak-peek-into-his-new-startup/ https://analyticsindiamag.com/raja-koduri-gives-a-sneak-peek...
imiric 3y ago
Right, we're already there, and the tech is quickly improving. I'm not sure why GP thinks this will remain the most expensive process of game development. Generating assets using AI will drastically cut costs and turnaround time. I'm eager to see these tools generate assets in real-time. Imagine being able to interact with an endless possibility of scenes. Games would ship with a generative model, and entire levels could be created on the fly, ensuring every play through is as different as the authors want it to be. Exciting stuff.
- andrewmcwatters 3y ago
  I never said it will remain the most expensive part; that's what I hope more engineering effort will go into. The graphics capabilities we use today are already commonplace and well written about. Fast content creation isn't, or the tools are sparse and locked behind small, costly proprietary silos.
xh-dude 3y ago
I’m not sure text prompts offer the kind of plasticity needed for artists to really do their thing. Frank Gehry used to do really chaotic abstract drawings and hand them off to aerospace CAD operators to turn them into something that could be engineered and machined and assembled. Lots of people could do the drawings and have magic mushroom facades but Gehry understood the resulting interior spaces as well … I think it’s there is a ton of potential for great tools employing the past decade or so of ML/AI but ISTM it should empower the process rather than replace it.

Animats 3y ago

Good point. I would have liked to see examples of human skin, and of cloth. That's hard and important. Rendering the perfect cheese grater and glazed pot is nice, but really, not that important. Most of the hard problems in graphics today involve scale. Epic's Nanite texture compression system is impressive, but about 60% of the work has to be done in CPUs because GPUs don't have the right stuff for it. And Nanite is still just for rigid objects. Crowds of individually dressed people are still tough to render fast. NVidia put in ray-tracing hardware into GPUs. It's not used much in games. NVidia was angry with reviewers who just ignored their ray-tracing hardware in reviews. The reviewers were not wrong. And could we please have good hardware support for order-independent translucency? That should be standard. Then we can get rid of depth sorting of faces, which never works right.

mrtranscendence 3y ago
> NVidia put in ray-tracing hardware into GPUs. It's not used much in games. Well, it's true that the proportion of games that have ray tracing -- over the space of all games released -- is small. But most triple-A games seem to have ray tracing these days. Of the last four such titles I've played three had ray tracing (Resident Evil 4, Jedi Survivor, Hogwarts Legacy). I think that's only going to become more true in the future, as IMO it does materially make an image quality difference.
- ehnto 3y ago
  They had the technology but not the power, so most games still only had bits and pieces using ray tracing and were not globally illuminated by it. Not to mention you have to make the game work on old equipment too. That means all the downsides of old lighting techniques remained and none of the art efficiencies are realized. We are at the third generation of cards with RTX and there are almost no games using ray traced global illumination, and even if they do they still have to offer rasterized lighting for older hardware.
  - yazzku 3y ago
    What do you mean "almost no games using ray traced global illumination"? The games mentioned above and many others are using ray-traced shadows, reflections, and sometimes one bounce of (diffuse) light. Not full path tracing, of course, but these techniques are still considered "ray-traced global illumination".

sufehmi 3y ago

Sorry I've read the last paragraph multiple times, but still couldn't figure out whether you're saying that id Tech is the best, or the worst...? I think you're saying that id Tech engine, even the old ones, are the best by a long shot; but some words in that paragraph seem to contradict that. And despite being a fan of John Carmack's works, I understand that the Unreal engines have its own advantages.

jasonwatkinspdx 3y ago
Yeah, I only worked in the business shortly but I would dispute that statement as well. Back at the time licensing from iD was basically "here's a cd, never call us again." Meanwhile Unreal, at a much more approachable price, also came with support contacts, a private email list and irc channel, etc. It was like night and day between the two. In my role as a level designer I vastly preferred UnrealEd. The workflow was more interactive, and the BSP implementation way more robust against leaks. There were flaws to Unreal... the initial versions of UnrealScript were not great. But again, there was simply no comparing the two imo.

syntheweave 3y ago

Easing the creation task for professional uses is definitely "in the pipeline". If you're making a 2D game, diffusion models are already capable enough to act as a shortcut for detailed painting. It's stuff with generating optimized meshes, animation, scene lighting, other details that is still in the research stage. In many ways it's more straightforward and gives a better demo to address the "render the entire scene" case than to go asset-by-asset and work out a better UX for that than "do everything by hand". Even though the latter is what's really being called for.

llamaLord 3y ago

This... 100% this. I've dabbled in India game making for a while and have no issue creating the relevant systems etc to make a game work because with good software engineering practises and modern off-the-shelf game engines you can scale that stuff like nobody's business. What you still can't scale... Art assets... Especially if you're not creatively talented in that way. There's a reason so many AAA games these days have staff that are like 80% artists, 10% developers, 10% everything else.

KennyBlanken 3y ago
Pretty sure all the artists are in contracted studios or contractors themselves. At least in F2P, the 'staff' are 80% artists because cosmetics are ludicrously profitable.

elihu 3y ago

I think part of the point of moving to ray tracing and physically correct global illumination is that if the technology stack works right you don't actually have to work very hard to get your assets to look good. Let's take for example Second Life. The content is almost all user-generated, and when it originally came out it looked a lot worse than other games that were available at the time. The difference is mostly: since the content is user generated, you don't have some army of artists touching up the textures and pre-baking the lighting to make everything look at least plausibly realistic. What Second Life was missing was the technology to compute global illumination in real time against arbitrary content so that the user-generated assets look as good as if some artist had specially lit and textured each one. (Maybe even better, because computers are more thorough and exact than humans -- assuming exactness is what you want. Artists sometimes take creative liberties because unrealism sometimes looks better.) I haven't paid attention to Second Life in a long time; if a few minutes looking at random youtube videos is any indication, it seems they haven't really made much progress on this front. I think eventually companies will figure this out. I haven't paid attention to Facebook's Metaverse project because I expect it to be a flop (for non-technical reasons), but maybe they're banking on the graphics technology being there to make user-generated content look not-fake. I think that part might be achievable now with modest hardware. As far as this new paper goes, I'm not sure if it makes things easier or harder for content creators. It seems like it makes the rendering stack more complex and harder to understand, but if someone can supply came creators with awesome textures and they don't have to care where they came from or how they work then maybe that's fine.

bl4ckm0r3 3y ago

How do you think AI and tools like midjourney could improve this asset creation process (if you think they can be used in this scenario)?

andrewmcwatters 3y ago
I think AI can be a force multiplier for asset turnaround time by helping artists who don’t have particular preferences for vision make generally real-world recognizable objects quickly and build meaningful libraries of models to help build world building sets. Theres also non-AI work that can help dynamically fill scenes with world meshes to help create seamless transitions between spaces as a part of overall terrain development. A lot of this tech will be proprietary in nature, and it may be a long time, think decades, if ever that we get to use it as general creators. It means that the gap between studios and bedroom professionals who used to create the mods that became full fledged intellectual properties that dominated for years will widen and there will be fewer opportunities for individual authors and small teams to become the next generation of production studios. This will happen while large organizations enjoy extreme technical leverage while not having many great ideas to execute on simply because there will be fewer fiction writers and fewer studios who have concentrated technologies. I currently don’t see much room for the next Zoid to create the next CTF gamemode. I don’t really see anything meaningful come out of community contributions to games anymore ever since the introduction of the UGC model which has superseded the traditional modding model. Instead of modders using game software mechanisms to create something new and interesting organically, UGC structures in games are explicitly designed so that creators can only do nominal things like create hats and skins for the most part. It’s shifted the mentally from, “Here are the tools we used to create the game,” to “Here’s what you’re allowed to create and put on a marketplace so we can take a cut of profits.” It’s gross, it sucks, and it’s another example of rug-pulling. Future generations will completely miss out on the previous paradigm because there’s so much focus entrenched on user-generated content versus modding.

oliwarner 3y ago

For gaming you're absolutely right. It's interactive so the threshold for immersive is that much lower. But imagine the impact on rendered art, VFX, etc. How long until you can ask ChatGPT for a model of a teapot with shaders for blue glaze? The labour involved from inception to render is going to be cut down to a tiny fraction of what's required today.

HellDunkel 3y ago

A long rant about the state of things in game engine techn and quite offtopic. I could not agree less. The id engine was hardly usable for anyone except a few studios with big pockets. With modern game engines you get great documentation and affordable tech. Unreal even comes with free photoreal assets and open source.

deleted 3y ago
[deleted]