4 ms·
The data-oriented design process for game development
- jcelerier 4y agoSince the article talks a bit about this part of DoD - plugging my C++ library for automatic conversion of AoS layout to SoA: https://github.com/celtera/ahsohtoa https://github.com/celtera/ahsohtoa if it can be useful to anyone ; the entire code is pretty short and fully documented to explain the techniques used: https://github.com/celtera/ahsohtoa/blob/main/include/ahsohtoa/ahsohtoa.hpp https://github.com/celtera/ahsohtoa/blob/main/include/ahsoht...
- tyleo 4y agoWhy not just write it as SoA to begin with? I think the library is cool. It’s cool that this can be done with C++ and it’s cool that someone did it. However, when considering something like this for production I always think, “folks will understand things better if they have to type it themselves and can _see_ the structure in the source code.” Are there real production uses for this or do you consider it a toy?
- klik99 4y agoI’ve been doing a lot of stuff in the UE4/5 Slate framework lately which uses a declarative construction syntax - the amount of boilerplate to enable the required declarative syntax is frustrating and would love to have it automatically done. But truthfully I’m not a fan of doing this kind of declarative DSL within c++, it’s just not a pleasant experience and would rather use yaml/toml or embedded lua for the data driven interface
- dgb23 4y agoI think that’s generally a good point. I guess the utility here is to have a clearer model of “this stuff belongs to this object”. Reminds me a bit of relational projections and views. But at the same time it goes against a principle in this paradigm, to program closer to the machine and the data layout.
- jcelerier 4y ago> Why not just write it as SoA to begin with? To give my personal reason: I have absolutely never managed to find SoA style code readable at all aha > folks will understand things better if they have to type it themselves I guess it comes down to if you work in a top-down or bottom-up way to create an understanding of a codebase. Generally, I start by looking at the class names to get a general idea of the architecture and make some mental diagrams of it, and then I zoom in to the level of detail needed. I know other people work better by reading individual functions and working upward from there but it's definitely not my case.
- chrsig 4y agoThis definitely has my attention...I like that you parameterized the underlying storage. Dislike the dependency on boost :( Not having to have an extra code generation step is definitely nice though. The SoA -> AoS difficulty has driven me to julia for my personal projects, there's a library called StructArrays.jl[0] that is quite similar to your project here. [0] https://github.com/JuliaArrays/StructArrays.jl https://github.com/JuliaArrays/StructArrays.jl
- jcelerier 4y agoThe dependency on boost is not mandatory: - Either you can use standalone PFR (it does not really change anything as it's almost exactly the same code, just outside of namespace boost) but apparently that is an issue ahah - Or you can lobby the committee to accept P1061 ; there's already a compiler implementation based on clang-12: https://github.com/ricejasonf/llvm-project/tree/ricejasonf/p1061 https://github.com/ricejasonf/llvm-project/tree/ricejasonf/p... With it, the code becomes even simpler and would not need PFR at all ; a function such as std::size_t create() { [&]<std::size_t... N>(std::index_sequence<N...>) { (std::get<N>(vec).push_back({}), ...); } (indices{}); return size() - 1; } just becomes std::size_t create() { auto& [... v] = vec; ((v.push_back({}), ...); return size() - 1; }
- Warwolt 4y agoNice article! I've been curious about DoD for a while, but have found most existing articles and videos on the subject to be very vague and fuzzy. This more precise treatment of what the author means by DoD is welcome in my opinion.
- runevault 4y agoHopefully once Unity's ECS portion of DOTS gets stabilized we'll start seeing more and more practical examples of the design pattern, because Unity tutorials are rarely as architecture astronaut, focused on "this is how I did xyz and now my character jumps!"
- dgb23 4y ago> The core of DOD is not about optimization or making fast programs through hardware consideration; it is about organizing programs around a deep knowledge of data and its transformation. This is a very surprising take to say the least. I always understood data orientation in game dev as exactly that. Can experts chime in on this?
- DixieDev 4y agoIt's a strange sentence for sure. Organising your programs around data and its transformation happens to mean you are aware of how hardware organises, represents, and transforms data, and accommodate that to keep things running smoothly. I would say the purpose of DoD is most certainly to make fast programs, but guess the point of that quote is just to further emphasise that this is all about data.
- Warwolt 4y agoI think this is a very important distinction though, because if DOD is to be possible to consider as a programming paradigm you'd have to be able to state how a program is meant to be built up from scratch. An object oriented programmer would propose the program to be built up from objects, the functional programmer would propose to build with pure functions. I would image a DOD programmer would propose to build the program with data transforms, which is a more general thing than just "make program go fast". Of course making good use of hardware is easier in a paradigm that uses hardware level data transforms as its building block, but I don't think it makes any sense to make that the _defining_ feature of the paradigm.
- jstimpfle 4y agoThe emphasis is on the data, not the transforms. The code is often not the interesting part.
- peteradio 4y agoSaying you want to make a highly optimized fast program doesn't guide you how to actually do it. The pattern says "when we focused on the data layer, optimization and speed came more naturally".
- deleted 4y ago[deleted]
- evilturnip 4y agoAlthough the article focuses on Unity DOTS, Unreal Engine 5 introduced a fast and performant data-oriented system called Mass. Originally introduced for its particle system, the underlying system is a pure data-oriented framework that is supposed to be extremely fast. https://docs.unrealengine.com/5.0/en-US/overview-of-mass-entity-in-unreal-engine/ https://docs.unrealengine.com/5.0/en-US/overview-of-mass-ent...
- colbyhub 4y agoLately, I've been wanting to explore the DOD pattern in the world of full-stack web development to see if there would be similar benefits. Might put together a proof-of-concept this weekend! Has anyone else explored this?
- evilturnip 4y agoIf it's just to serve CRUD apps or run web sites, bottleneck is network time and browser rendering time, unless you're doing actual data processing. And if you are doing data processing, easier to use a data science library like Pandas for Python which implicitly have DOD built-in.
- wswope 4y agoI've been doing something in this vein for a big personal project, using this python library: https://nackjicholson.github.io/aiosql/ https://nackjicholson.github.io/aiosql/. In short, I'm using a run of the mill stack (Caddy/Gunicorn/Flask/Postgres) - but with the twist that all my core logic is defined in plaintext SQL files, which get bound into namespaced Python methods by aiosql. Routing, error handling, templating, etc. are all done in Python - but data manipulation and processing are outsourced to the DB level. All database object definitions are laid out in a massive, idempotent "init_db" method that gets called at launch, so I can essentially point the app at a fresh instance of Postgres and rebuild from scratch. The design is primarily driven by my personal distaste for ORMs, but I've found it extremely beneficial in terms of rigid typing, integrity checks, and performance.