6 ms·

GPU synchronization in Godot 4.3 is getting a major upgrade

159 points by __natty__ 3y ago

Vt71fcAqt7 3y ago
It's great that Godot keeps improving. AFAIK it is still mostly for indie devs though. I don't know why valve isn't open sourcing Source 2. They barely even make games anymore and it would make them money in the long term if it means games could be made cheaper and better. UE5 on the other hand is leaps and bounds greater than both Source 2 and Godot judging from the few games that have released with it so far. There's also O3DE which last I checked isn't ready.
- krapp 3y ago
  The AAA industry is eating itself right now. Indie seems like a good place to be. Also, indie games are a billion dollar market. Maybe it's time to stop using it as a synonym for "not serious." The consumer doesn't care that Undertale was made with Game Maker. >There's also O3DE which last I checked isn't ready. Literally no one cares about O3DE.
  - freedomben 3y ago
    I don't think "indie" is synonymous with "not serious," at least if people use it that way I think they're wrong and I think there a lot of people who use it the same way I do. To me "indie" is usually more a reflection of budget, which heavily impacts graphical sophistication, size of the game (as in how many levels, how many hours of play, etc), and usually means the price of the game will be between $5 and $30. AAA I expect $60 and 100+ hours of playable content. Indie game I expect 5 to 6 hours and I'm pleasantly surprised when it's more. When it comes to choice of game engine, that usually means they prefer less flexibility but more simplicity and "free stuff" from the framework, whereas a AAA game might prefer maximum flexibility, but that comes with complexity. As in all things it's a budget (time & money) tradeoff, not necessarily an "are we serious" tradeoff.
    - filleduchaos 3y ago
      > AAA I expect $60 and 100+ hours of playable content. Indie game I expect 5 to 6 hours and I'm pleasantly surprised when it's more. My experience is much more the opposite, with plenty of indie games providing many hours of actual gameplay (including replayability) and plenty of AAA games inflating their runtime with long cutscenes and other fluff (or milking people for all they're worth through live service).
      - freedomben 3y ago
        Thank you yes, fair point. the cutscenes can get quite outrageous, and I refuse to play the games that do in app purchases and the like, so I have an unrepresentative sample in my experience.
- jsheard 3y ago
  Valve doesn't try to make general purpose engines, they make engines that suit their own games, so unless you happen to be making a game which is shaped very much like HL:Alyx or CS2 you would probably be underwhelmed with Source 2. That includes platform support, Source 2 doesn't officially support any of the consoles because Valve hasn't needed it to, having only ever shipped it on PC and (briefly) Android/iOS.
  - snoutie 3y ago
    Then again there is s&box which I would not consider being close to either, well the games that can be made with s&box at least. But I agree that it would probably be hard to build something like Teardown inside the Source 2 Engine. On the other hand: Teardown does not use Unity either.
    - jsheard 3y ago
      S&box licensing Source 2 is a bit of an enigma to me, I get the impression that they're having to rewrite huge swaths of the engine so I don't know what they're getting out of it. They've gone to the lengths of integrating C# scripting from scratch rather than using the scripting facilities that Source 2 comes with. Nobody else has licensed Source 2 in the 4 years since s&box adopted it, so they're the lone outlier in any case. Respawns use of Source 1 was a similar story, they ended up rewriting practically everything in the course of developing Titanfall and Apex Legends, to the point that it's almost unrecognisable as Source at this point.
      - jna_sh 3y ago
        I don’t think there is (or needs to be) much more to it than Garry’s personal history and success with Source
      - meibo 3y ago
        Garry has publicly expressed regret over choosing Source 2 for this reason, but they might be in too deep now. Generally, the tooling is really great, so I guess that's the primary benefit.
- pests 3y ago
  Valves doesn't make games anymore because they are making piles of money running a platform.
  - rowanG077 3y ago
    That and the skin trade is literally a multi billion dollar business. It boggles the mind.
- andrewmcwatters 3y ago
  Valve does not have a great reputation with its licensees anymore or its former mod developer community. This is probably a large part of it.
- genocidicbunny 3y ago
  > AFAIK it is still mostly for indie devs though There was a time when Unity was the 'indie game engine' as well. Give Godot some time (especially since it's progress seems to be picking up steam.) > UE5 on the other hand is leaps and bounds greater UE5 has the benefit of standing on the shoulders of giants, not to mention giant-sized pockets of cash to spend on developing it. UE has been around since 1998, so we're talking a quarter of a century of development leading up to what UE5 is today. It also has the added benefit of having been used for a huge number of different games, so there's an additional level of refinement in the toolset that it offers, since it's had to adapt to suit so many games.
  - Narishma 3y ago
    > There was a time when Unity was the 'indie game engine' as well. That's still the case, depending on how you define indie. I'm not aware of any major AAA game made with Unity.
    - genocidicbunny 3y ago
      I don't know if it's quite considered AAA, but Cities Skylines was made in Unity, and that was a pretty big game.
    - naet 3y ago
      Not sure how you define AAA, but a lot of popular games have been made with Unity. Some major studios have used Unity for mobile games related to bigger properties too. Some Unity games I know of: Genshin Impact (maybe the closest to "AAA") Rust Fall Guys Escape from Tarkov Cuphead Fallout Shelter The Elder Scrolls: Legends Magic: The Gathering Arena Super Mario Run Mario Kart Tour Hollow Knight Untitled Goose Game
snoutie 3y ago
I like the dedication behind godot. Since unity can be a hassle to get to work on linux i might try out their solution. On a tangential note: I am also excited to see whether Embark Studios will open source their engine once it's ready.
- cyber_kinetist 3y ago
  I thought they were still using Unreal Engine for their games (notably The Finals?)
  - Jeaye 3y ago
    It's my understanding that The Finals was made using an in-house engine built in Rust.
    - cyber_kinetist 3y ago
      Nope it's Unreal Engine 5 (see https://www.reddit.com/r/rust/comments/wvyvxx/how_much_rust_was_used_for_the_finals_an_upcoming/ https://www.reddit.com/r/rust/comments/wvyvxx/how_much_rust_... for an official engineer response) And reasonably so, since it will take quite some time to build an entire triple-A-grade game engine from scratch (not just in Rust, but with any language! Though building it in Rust will make things much harder since they need to first build entire ecosystems from scratch...) I think Embark Studios ultimately do want to use Rust for their projects and are pursuing small R&D projects to go in that direction (rust-gpu and also their Kajiya renderer) - but they're first and foremost a game company that needs to release games for profit, so made the pragmatic decision of using UE5 at the meantime.
vunderba 3y ago
I really really really want to use Godot, but sadly, the support for web exports just isn't there in Godot 4: - Can't use C# at all - Has issues running on MacOS / iOS Meanwhile as much as Unity irritates me, they're working on even better web platform support in Unity 6 and smaller runtime bundled deployments. https://unity.com/solutions/web https://unity.com/solutions/web I think if I was predominantly focused on traditional console/platform gaming it would be a different story.
- ImPleadThe5th 3y ago
  I thought godot does have c# support? Isn't that what godot mono is?
  - EGG_CREAM 3y ago
    I had the same thought as you, but then I saw this: >Currently, projects written in C# cannot be exported to the web platform. To use C# on that platform, consider Godot 3 instead. https://docs.godotengine.org/en/stable/tutorials/scripting/c_sharp/index.html https://docs.godotengine.org/en/stable/tutorials/scripting/c...
    - ImPleadThe5th 3y ago
      Ahh I see. That's quite disappointing. I know nothing about it, but I wonder if web assembly could help out here.
- nusl 3y ago
  C# is a language but not a limiting factor to making a game. GDScript is for the most part fine to work with. Issues on other platforms is also something that will improve over time. Unity has not shown good faith in the least.
  - vunderba 3y ago
    I'm not so confident that it will be fixed. It's been a known issue for nearly a year now: "Godot 4's HTML5 exports currently cannot run on macOS and iOS due to upstream bugs with SharedArrayBuffer and WebGL 2.0. We recommend using macOS and iOS native export functionality instead, as it will also result in better performance."
dogprez 3y ago
> The order of execution of the recorded commands inside a command buffer is NOT guaranteed to complete in the order they were submitted: the GPU can reorder these commands in whatever order it thinks is best to complete the job as quickly as possible. It's my understanding that commands inside of a command buffer are guaranteed to complete in order. The synchronization must happen when you are `vkQueueSubmit`ing multiple command buffers, no? I think that's what they meant to say?
- unclad5968 3y ago
  Commands are guaranteed to start in the order they are inserted into the buffer but not guaranteed to complete in that order. Per kronos: > Commands are also guaranteed to start in the exact order they were inserted, but because they can run in parallel, there is no guarantee that the commands will complete in that same order https://www.khronos.org/blog/understanding-vulkan-synchronization https://www.khronos.org/blog/understanding-vulkan-synchroniz...
andrewmcwatters 3y ago
I've only done a rough minimal implementation of Vulkan in the past. I thought Direct3D 12 was basically Vulkan.
- pjmlp 3y ago
  Still more development friendly than Vulkan, in API, IDE tooling , GPU debugging, shading capabilities.
  - andrewmcwatters 3y ago
    After I implemented it, I really got the feeling that I should have started with Direct3D 12.
zx8080 3y ago
I'm out of games context, so what is "web export"? Is it something which Flash has been before?
- nusl 3y ago
  Compile to WASM or HTML5 to run in a browser.
Animats 3y ago
Good to hear that Godot is addressing this. The Rust graphics crates are bottlenecking on buffer management. I'm seeing the render thread become compute-bound on a static scene, with the GPU under 25% busy. Stack is Rend3/WGPU/Vulkan/X11/Linux. The latest round of WGPU improvements now has copying content into the GPU memory concurrent with rendering. That level of synchronization has been fixed. But it's still too slow. I dread having to learn how things work down at that level. By the way, what's the thread situation in WASM now? Previously, it was multiprocess with some shared memory, not real threads. Various real threading proposals were floating around. Did that happen yet?
cyber_kinetist 3y ago
This adds more to the evidence that Vulkan / DX12 seems like a failed API design, when too many graphics engineers are reinventing the wheel by building their render-graph API on top of these (propietary triple-A game engines, Unreal Engine, and now Godot...) Instead of real-time graphics APIs providing all these low-level manual synchronization primitives and cumbersome PSOs, maybe they should just provide an official Render Graph API instead? Provide all of the operations and its dependencies in an acyclic graph up-front, and the driver handles synchronization automatically in the most performant manner tailored to the hardware. I guess there needed some trial-and-error in the gamedev world for about a decade to really nail down a nice to use but also performant graphics API design. Vulkan being originated from the Mantle API from AMD didn't help - since it was a low-level console API mainly accustomed to AMD's GPUs and really didn't seem like it would fit for a more "general-purpose" API spanning a huge range of hardware and can stand the test of time. And with Microsoft's DX12 hastely copying from AMD's initial design it also has all the same issues (The irony is that DX11 is still the best graphics API you can use in gamedev in Windows in terms of ergonomics and even performance - seeing many trying to dauntingly build a DX12 backend and end up performing worse than DX11...) Nowadays I'm obversing that the industry has known these issues for a while and are experimenting with alternative API designs... there are some experiental render-graph extensions available in both DX12 / Vulkan (albeit in a limited fashion): - DX12's Work Graph API as preview (https://devblogs.microsoft.com/directx/d3d12-work-graphs-pre https://devblogs.microsoft.com/directx/d3d12-work-graphs-pre...) - Vulkan's VK_AMDX_shader_enqueue extension (https://gpuopen.com/gpu-work-graphs-in-vulkan/ https://gpuopen.com/gpu-work-graphs-in-vulkan/)
- mmis1000 3y ago
  I think Vulkan / DX12 is explicitly designed to make making another abstraction on it possible? It's possible to make a full app on it without any assistant, but it's not the easiest way to use it. For example, you may make a opengl emulation layer on directX, but not the other way around. And dx12/vulkan is designed to be the underlying one (While hide some hardware specified gotcha).
  - cyber_kinetist 3y ago

159 points by __natty__ 3y ago

It's great that Godot keeps improving. AFAIK it is still mostly for indie devs though. I don't know why valve isn't open sourcing Source 2. They barely even make games anymore and it would make them money in the long term if it means games could be made cheaper and better. UE5 on the other hand is leaps and bounds greater than both Source 2 and Godot judging from the few games that have released with it so far. There's also O3DE which last I checked isn't ready.

krapp 3y ago
The AAA industry is eating itself right now. Indie seems like a good place to be. Also, indie games are a billion dollar market. Maybe it's time to stop using it as a synonym for "not serious." The consumer doesn't care that Undertale was made with Game Maker. >There's also O3DE which last I checked isn't ready. Literally no one cares about O3DE.
- freedomben 3y ago
  I don't think "indie" is synonymous with "not serious," at least if people use it that way I think they're wrong and I think there a lot of people who use it the same way I do. To me "indie" is usually more a reflection of budget, which heavily impacts graphical sophistication, size of the game (as in how many levels, how many hours of play, etc), and usually means the price of the game will be between $5 and $30. AAA I expect $60 and 100+ hours of playable content. Indie game I expect 5 to 6 hours and I'm pleasantly surprised when it's more. When it comes to choice of game engine, that usually means they prefer less flexibility but more simplicity and "free stuff" from the framework, whereas a AAA game might prefer maximum flexibility, but that comes with complexity. As in all things it's a budget (time & money) tradeoff, not necessarily an "are we serious" tradeoff.
  - filleduchaos 3y ago
    > AAA I expect $60 and 100+ hours of playable content. Indie game I expect 5 to 6 hours and I'm pleasantly surprised when it's more. My experience is much more the opposite, with plenty of indie games providing many hours of actual gameplay (including replayability) and plenty of AAA games inflating their runtime with long cutscenes and other fluff (or milking people for all they're worth through live service).
    - freedomben 3y ago
      Thank you yes, fair point. the cutscenes can get quite outrageous, and I refuse to play the games that do in app purchases and the like, so I have an unrepresentative sample in my experience.
jsheard 3y ago
Valve doesn't try to make general purpose engines, they make engines that suit their own games, so unless you happen to be making a game which is shaped very much like HL:Alyx or CS2 you would probably be underwhelmed with Source 2. That includes platform support, Source 2 doesn't officially support any of the consoles because Valve hasn't needed it to, having only ever shipped it on PC and (briefly) Android/iOS.
- snoutie 3y ago
  Then again there is s&box which I would not consider being close to either, well the games that can be made with s&box at least. But I agree that it would probably be hard to build something like Teardown inside the Source 2 Engine. On the other hand: Teardown does not use Unity either.
  - jsheard 3y ago
    S&box licensing Source 2 is a bit of an enigma to me, I get the impression that they're having to rewrite huge swaths of the engine so I don't know what they're getting out of it. They've gone to the lengths of integrating C# scripting from scratch rather than using the scripting facilities that Source 2 comes with. Nobody else has licensed Source 2 in the 4 years since s&box adopted it, so they're the lone outlier in any case. Respawns use of Source 1 was a similar story, they ended up rewriting practically everything in the course of developing Titanfall and Apex Legends, to the point that it's almost unrecognisable as Source at this point.
    - jna_sh 3y ago
      I don’t think there is (or needs to be) much more to it than Garry’s personal history and success with Source
    - meibo 3y ago
      Garry has publicly expressed regret over choosing Source 2 for this reason, but they might be in too deep now. Generally, the tooling is really great, so I guess that's the primary benefit.
pests 3y ago
Valves doesn't make games anymore because they are making piles of money running a platform.
- rowanG077 3y ago
  That and the skin trade is literally a multi billion dollar business. It boggles the mind.
andrewmcwatters 3y ago
Valve does not have a great reputation with its licensees anymore or its former mod developer community. This is probably a large part of it.
genocidicbunny 3y ago
> AFAIK it is still mostly for indie devs though There was a time when Unity was the 'indie game engine' as well. Give Godot some time (especially since it's progress seems to be picking up steam.) > UE5 on the other hand is leaps and bounds greater UE5 has the benefit of standing on the shoulders of giants, not to mention giant-sized pockets of cash to spend on developing it. UE has been around since 1998, so we're talking a quarter of a century of development leading up to what UE5 is today. It also has the added benefit of having been used for a huge number of different games, so there's an additional level of refinement in the toolset that it offers, since it's had to adapt to suit so many games.
- Narishma 3y ago
  > There was a time when Unity was the 'indie game engine' as well. That's still the case, depending on how you define indie. I'm not aware of any major AAA game made with Unity.
  - genocidicbunny 3y ago
    I don't know if it's quite considered AAA, but Cities Skylines was made in Unity, and that was a pretty big game.
  - naet 3y ago
    Not sure how you define AAA, but a lot of popular games have been made with Unity. Some major studios have used Unity for mobile games related to bigger properties too. Some Unity games I know of: Genshin Impact (maybe the closest to "AAA") Rust Fall Guys Escape from Tarkov Cuphead Fallout Shelter The Elder Scrolls: Legends Magic: The Gathering Arena Super Mario Run Mario Kart Tour Hollow Knight Untitled Goose Game

snoutie 3y ago

I like the dedication behind godot. Since unity can be a hassle to get to work on linux i might try out their solution. On a tangential note: I am also excited to see whether Embark Studios will open source their engine once it's ready.

cyber_kinetist 3y ago
I thought they were still using Unreal Engine for their games (notably The Finals?)
- Jeaye 3y ago
  It's my understanding that The Finals was made using an in-house engine built in Rust.
  - cyber_kinetist 3y ago
    Nope it's Unreal Engine 5 (see https://www.reddit.com/r/rust/comments/wvyvxx/how_much_rust_was_used_for_the_finals_an_upcoming/ https://www.reddit.com/r/rust/comments/wvyvxx/how_much_rust_... for an official engineer response) And reasonably so, since it will take quite some time to build an entire triple-A-grade game engine from scratch (not just in Rust, but with any language! Though building it in Rust will make things much harder since they need to first build entire ecosystems from scratch...) I think Embark Studios ultimately do want to use Rust for their projects and are pursuing small R&D projects to go in that direction (rust-gpu and also their Kajiya renderer) - but they're first and foremost a game company that needs to release games for profit, so made the pragmatic decision of using UE5 at the meantime.

vunderba 3y ago

I really really really want to use Godot, but sadly, the support for web exports just isn't there in Godot 4: - Can't use C# at all - Has issues running on MacOS / iOS Meanwhile as much as Unity irritates me, they're working on even better web platform support in Unity 6 and smaller runtime bundled deployments. https://unity.com/solutions/web https://unity.com/solutions/web I think if I was predominantly focused on traditional console/platform gaming it would be a different story.

ImPleadThe5th 3y ago
I thought godot does have c# support? Isn't that what godot mono is?
- EGG_CREAM 3y ago
  I had the same thought as you, but then I saw this: >Currently, projects written in C# cannot be exported to the web platform. To use C# on that platform, consider Godot 3 instead. https://docs.godotengine.org/en/stable/tutorials/scripting/c_sharp/index.html https://docs.godotengine.org/en/stable/tutorials/scripting/c...
  - ImPleadThe5th 3y ago
    Ahh I see. That's quite disappointing. I know nothing about it, but I wonder if web assembly could help out here.
nusl 3y ago
C# is a language but not a limiting factor to making a game. GDScript is for the most part fine to work with. Issues on other platforms is also something that will improve over time. Unity has not shown good faith in the least.
- vunderba 3y ago
  I'm not so confident that it will be fixed. It's been a known issue for nearly a year now: "Godot 4's HTML5 exports currently cannot run on macOS and iOS due to upstream bugs with SharedArrayBuffer and WebGL 2.0. We recommend using macOS and iOS native export functionality instead, as it will also result in better performance."

dogprez 3y ago

> The order of execution of the recorded commands inside a command buffer is NOT guaranteed to complete in the order they were submitted: the GPU can reorder these commands in whatever order it thinks is best to complete the job as quickly as possible. It's my understanding that commands inside of a command buffer are guaranteed to complete in order. The synchronization must happen when you are `vkQueueSubmit`ing multiple command buffers, no? I think that's what they meant to say?

unclad5968 3y ago
Commands are guaranteed to start in the order they are inserted into the buffer but not guaranteed to complete in that order. Per kronos: > Commands are also guaranteed to start in the exact order they were inserted, but because they can run in parallel, there is no guarantee that the commands will complete in that same order https://www.khronos.org/blog/understanding-vulkan-synchronization https://www.khronos.org/blog/understanding-vulkan-synchroniz...

andrewmcwatters 3y ago

I've only done a rough minimal implementation of Vulkan in the past. I thought Direct3D 12 was basically Vulkan.

pjmlp 3y ago
Still more development friendly than Vulkan, in API, IDE tooling , GPU debugging, shading capabilities.
- andrewmcwatters 3y ago
  After I implemented it, I really got the feeling that I should have started with Direct3D 12.

zx8080 3y ago

I'm out of games context, so what is "web export"? Is it something which Flash has been before?

nusl 3y ago
Compile to WASM or HTML5 to run in a browser.

Animats 3y ago

Good to hear that Godot is addressing this. The Rust graphics crates are bottlenecking on buffer management. I'm seeing the render thread become compute-bound on a static scene, with the GPU under 25% busy. Stack is Rend3/WGPU/Vulkan/X11/Linux. The latest round of WGPU improvements now has copying content into the GPU memory concurrent with rendering. That level of synchronization has been fixed. But it's still too slow. I dread having to learn how things work down at that level. By the way, what's the thread situation in WASM now? Previously, it was multiprocess with some shared memory, not real threads. Various real threading proposals were floating around. Did that happen yet?

cyber_kinetist 3y ago

This adds more to the evidence that Vulkan / DX12 seems like a failed API design, when too many graphics engineers are reinventing the wheel by building their render-graph API on top of these (propietary triple-A game engines, Unreal Engine, and now Godot...) Instead of real-time graphics APIs providing all these low-level manual synchronization primitives and cumbersome PSOs, maybe they should just provide an official Render Graph API instead? Provide all of the operations and its dependencies in an acyclic graph up-front, and the driver handles synchronization automatically in the most performant manner tailored to the hardware. I guess there needed some trial-and-error in the gamedev world for about a decade to really nail down a nice to use but also performant graphics API design. Vulkan being originated from the Mantle API from AMD didn't help - since it was a low-level console API mainly accustomed to AMD's GPUs and really didn't seem like it would fit for a more "general-purpose" API spanning a huge range of hardware and can stand the test of time. And with Microsoft's DX12 hastely copying from AMD's initial design it also has all the same issues (The irony is that DX11 is still the best graphics API you can use in gamedev in Windows in terms of ergonomics and even performance - seeing many trying to dauntingly build a DX12 backend and end up performing worse than DX11...) Nowadays I'm obversing that the industry has known these issues for a while and are experimenting with alternative API designs... there are some experiental render-graph extensions available in both DX12 / Vulkan (albeit in a limited fashion): - DX12's Work Graph API as preview (https://devblogs.microsoft.com/directx/d3d12-work-graphs-pre https://devblogs.microsoft.com/directx/d3d12-work-graphs-pre...) - Vulkan's VK_AMDX_shader_enqueue extension (https://gpuopen.com/gpu-work-graphs-in-vulkan/ https://gpuopen.com/gpu-work-graphs-in-vulkan/)

mmis1000 3y ago
I think Vulkan / DX12 is explicitly designed to make making another abstraction on it possible? It's possible to make a full app on it without any assistant, but it's not the easiest way to use it. For example, you may make a opengl emulation layer on directX, but not the other way around. And dx12/vulkan is designed to be the underlying one (While hide some hardware specified gotcha).
- cyber_kinetist 3y ago