5 ms·

Component Graph System (2017)

86 points by Kinrany 5y ago

syntaxfree 5y ago
I don’t know what ECS is. You should define the acronym…
- anentropic 5y ago
  Entity Component System commonly used in game programming
- mrspeaker 5y ago
  "Entity Component Systems" are common in game engine architecture, and there are a lot of variations on them - but they are a high-level architecture for making modular behavior in games. Generally they were thought of as "better than Object Oriented, because they very very heavily favor composition over inheritance." The idea is that everything in your game - like Players, NPCs, Bullets - are Entities. Entities are simply an ID. That's their only property. Nothing else. But there are Components - they have the data. For example, you might have a `PositionComponent` that has an `X Position` and `Y position` property. And maybe a `MassComponent` that has a property `Mass`. Somewhere in the framework (like a database table), a given Component will be related to a given Entity ID. The Entity therefore becomes a "bag of Components" through the relations. Finally there are Systems. They do stuff with the data. One system might be `GravitySystem`. It doesn't care directly about `Entity`s, but can query the relations. It might ask for anything that has BOTH a `PositionComponent` and a `MassComponent` (it doesn't know which Entity it is for - it only gets the Components). It takes any `PositionComponent`s it gets and applies gravity: `component.y += gravity`. The effect is that you can mix and match Components in weird and wonderful ways without having to hard-code behavior and rules for certain entities. You can obviously achieve this with other architectures too - but that's the primary goal of this one.
  - thom 5y ago
    Correct me if I’m wrong, but isn’t one of the other motivations to store your data in CPU/memory management friendly ways? It’s not purely about developer ergonomics.
    - mrspeaker 5y ago
      I think originally it was primarily about developer ergonomics - here's the first thing I ever read about them in 2007 where they tested it on Tony Hawk Pro Skater (http://cowboyprogramming.com/2007/01/05/evolve-your-heirachy/ http://cowboyprogramming.com/2007/01/05/evolve-your-heirachy...). The CPU/memory management has been an on-going discussion ever since! Being able to process domains as flat arrays helps with cache misses - but there is a lot of other overhead, depending on how you designed everything. Making it performant was one of the big problems initially, and it seems like the `Component Graph System` this thread is about trying to answer "how do systems efficiently process components?"... But I don't really get how it answers that actually.
      - gmueckl 5y ago
        As far as I understand it this is about the ECS equivalent of an INNER JOIN, that is, some variation on "iterate over all entities that possess instances of component A and B". Iterating over single component types in a system is trivial if they are separated into pools by types. When you have a system requiring more than one component, it isn't that easy to do better than iterating over all entities and checking whether they possess each required component. You can decide to maintain back references from components to their entities, but this means that you incur that management overhead pretty much globally. This essay says that doing away with entities altogether and just keeping direct links between your components is better. And it probably is when it comes to handling related components. However, I believe (without having actually tried it) that this proposal has its own drawbacks. it's harder to thoroughly clean up all components that form a single entity. My suspicion is that it's easier to produce bugs like keeping e.g. a stray collision shape or sound source around while the other parts of an entity are removed. All you need for that to happen is an incomplete graph traversal.
- jcelerier 5y ago
  there are two distinct definitions of ECS, the first one given in https://news.ycombinator.com/item?id=27856030 https://news.ycombinator.com/item?id=27856030 is the "(entity-component-system) architecture" where there are entities, components, and systems, as distinct objects in your software. The second is "(entity-component) system" or "entity-component architecture" where the behaviour goes inside components (for instance, Unity3D gameobjects with their components).
- nodejs_rulez_1 5y ago
  It's good old procedural programming. Systems (procedures) are mutating state (components) grouped by a common ID (entity).
  - bordercases 5y ago
    You could do it functionally, i.e. the systems are all reducers on partial states of the world.
  - leoc 5y ago
    I'm no expert on any of this but that doesn't seem right; is it? IIUC, most procedural programming is nearly as wedded to the "one entity, one master data structure, one address in memory" frame of mind as OO is. (And the same is true of most functional programming, come to that.)
deleted 5y ago
[deleted]
gameman144 5y ago
> And we decided that each system processes relevant entity data by iterating through entities first. Now we have another problem - growing the number of entities slows down every system, even the ones that are not related... Call me out if I'm wrong, but I don't know of any high-performance ECS framework that actually does this. Most frameworks maintain sorted collections of components with no gaps. For instance, if I want all entries with X,Y,Z, then I iterate those three arrays starting at their X,Y, Z section, and the order of components in that section of the arrays matches exactly (meaning we can just iterate through them all). This library seems like a really neat concept, but I think the ECS concepts that it's critiquing aren't actually used in many production-quality frameworks. Call me out if I'm just missing context here. Also, I'd love more details on how CGS is able to always iterate on flat vectors with no gaps. The only way I've seen to accomplish this is to do a decent amount of work curating components into multiple orderings, so if CGS is avoiding that, I am equal parts skeptical and very very excited (since that would be super useful!)
- hrh 5y ago
  What libraries/frameworks are state of the art?
  - jayd16 5y ago
    Unity's implementation is well documented and handles this.
  - xaedes 5y ago
    There is EnTT, an ECS as C++ library: EnTT is a header-only, tiny and easy to use library for game programming and much more written in modern C++. Among others, it's used in Minecraft by Mojang, ... https://github.com/skypjack/entt https://github.com/skypjack/entt
    - gameman144 5y ago
      EnTT is the one I was thinking of -- I'd highly recommend looking into its implementation, as it's both very well-documented and readable, and blazing fast. The fact that even this caliber of implementation has to handle multi-component cases by doing super careful curation of components is what makes me skeptical that CGS would be able to solve this situation more efficiently/easily. I would be very happy to be proven wrong here, though.
enbugger 5y ago
>Performance-wise, some overhead may come from reference-counting the components Well, that's what ECS gets rid of and mainly benefits from that. Doesn't returning it back mean going back to OOP? What's the point then?
- Kinrany 5y ago
  I believe the main benefit is a constrained form of dynamic typing. You don't want to lose compile-time type checking and intellisense, and you want to keep most of the performance benefits. But you want to be able to add data types without recompiling.
m0llusk 5y ago
> CGS is more flexible thus targeting a wider use case than generic ECS: it's a hierarchy of components instead of the flat entity-component duality ... Wasn't the flatness of ECS supposed to be a useful feature that simplifies things like iteration and searches?
Kinrany 5y ago
> There is no ~~spoon~~ entity. What was our entity like? A structure with pointers (IDs) to different components. Now, let's treat it as just another component! The whole ECS becomes simpler. To me this is the most interesting part. Game logic needs components with references to other entities and components anyway. And if there's some way of identifying components, you no longer need entities to have identities. And you can still apply all the optimizations as long as you treat some components as "entity" components. But only when you want to exchange simplicity for performance.
brainbag 5y ago
I haven't used it but this sounds similar to my understanding of how Godot's structure works. Is this a similar pattern?
pshirshov 5y ago
Sounds alike to distage?.. https://izumi.7mind.io/distage/ https://izumi.7mind.io/distage/

86 points by Kinrany 5y ago

syntaxfree 5y ago

I don’t know what ECS is. You should define the acronym…

anentropic 5y ago
Entity Component System commonly used in game programming
mrspeaker 5y ago
"Entity Component Systems" are common in game engine architecture, and there are a lot of variations on them - but they are a high-level architecture for making modular behavior in games. Generally they were thought of as "better than Object Oriented, because they very very heavily favor composition over inheritance." The idea is that everything in your game - like Players, NPCs, Bullets - are Entities. Entities are simply an ID. That's their only property. Nothing else. But there are Components - they have the data. For example, you might have a `PositionComponent` that has an `X Position` and `Y position` property. And maybe a `MassComponent` that has a property `Mass`. Somewhere in the framework (like a database table), a given Component will be related to a given Entity ID. The Entity therefore becomes a "bag of Components" through the relations. Finally there are Systems. They do stuff with the data. One system might be `GravitySystem`. It doesn't care directly about `Entity`s, but can query the relations. It might ask for anything that has BOTH a `PositionComponent` and a `MassComponent` (it doesn't know which Entity it is for - it only gets the Components). It takes any `PositionComponent`s it gets and applies gravity: `component.y += gravity`. The effect is that you can mix and match Components in weird and wonderful ways without having to hard-code behavior and rules for certain entities. You can obviously achieve this with other architectures too - but that's the primary goal of this one.
- thom 5y ago
  Correct me if I’m wrong, but isn’t one of the other motivations to store your data in CPU/memory management friendly ways? It’s not purely about developer ergonomics.
  - mrspeaker 5y ago
    I think originally it was primarily about developer ergonomics - here's the first thing I ever read about them in 2007 where they tested it on Tony Hawk Pro Skater (http://cowboyprogramming.com/2007/01/05/evolve-your-heirachy/ http://cowboyprogramming.com/2007/01/05/evolve-your-heirachy...). The CPU/memory management has been an on-going discussion ever since! Being able to process domains as flat arrays helps with cache misses - but there is a lot of other overhead, depending on how you designed everything. Making it performant was one of the big problems initially, and it seems like the `Component Graph System` this thread is about trying to answer "how do systems efficiently process components?"... But I don't really get how it answers that actually.
    - gmueckl 5y ago
      As far as I understand it this is about the ECS equivalent of an INNER JOIN, that is, some variation on "iterate over all entities that possess instances of component A and B". Iterating over single component types in a system is trivial if they are separated into pools by types. When you have a system requiring more than one component, it isn't that easy to do better than iterating over all entities and checking whether they possess each required component. You can decide to maintain back references from components to their entities, but this means that you incur that management overhead pretty much globally. This essay says that doing away with entities altogether and just keeping direct links between your components is better. And it probably is when it comes to handling related components. However, I believe (without having actually tried it) that this proposal has its own drawbacks. it's harder to thoroughly clean up all components that form a single entity. My suspicion is that it's easier to produce bugs like keeping e.g. a stray collision shape or sound source around while the other parts of an entity are removed. All you need for that to happen is an incomplete graph traversal.
jcelerier 5y ago
there are two distinct definitions of ECS, the first one given in https://news.ycombinator.com/item?id=27856030 https://news.ycombinator.com/item?id=27856030 is the "(entity-component-system) architecture" where there are entities, components, and systems, as distinct objects in your software. The second is "(entity-component) system" or "entity-component architecture" where the behaviour goes inside components (for instance, Unity3D gameobjects with their components).
nodejs_rulez_1 5y ago
It's good old procedural programming. Systems (procedures) are mutating state (components) grouped by a common ID (entity).
- bordercases 5y ago
  You could do it functionally, i.e. the systems are all reducers on partial states of the world.
- leoc 5y ago
  I'm no expert on any of this but that doesn't seem right; is it? IIUC, most procedural programming is nearly as wedded to the "one entity, one master data structure, one address in memory" frame of mind as OO is. (And the same is true of most functional programming, come to that.)

deleted 5y ago

[deleted]

gameman144 5y ago

> And we decided that each system processes relevant entity data by iterating through entities first. Now we have another problem - growing the number of entities slows down every system, even the ones that are not related... Call me out if I'm wrong, but I don't know of any high-performance ECS framework that actually does this. Most frameworks maintain sorted collections of components with no gaps. For instance, if I want all entries with X,Y,Z, then I iterate those three arrays starting at their X,Y, Z section, and the order of components in that section of the arrays matches exactly (meaning we can just iterate through them all). This library seems like a really neat concept, but I think the ECS concepts that it's critiquing aren't actually used in many production-quality frameworks. Call me out if I'm just missing context here. Also, I'd love more details on how CGS is able to always iterate on flat vectors with no gaps. The only way I've seen to accomplish this is to do a decent amount of work curating components into multiple orderings, so if CGS is avoiding that, I am equal parts skeptical and very very excited (since that would be super useful!)

hrh 5y ago
What libraries/frameworks are state of the art?
- jayd16 5y ago
  Unity's implementation is well documented and handles this.
- xaedes 5y ago
  There is EnTT, an ECS as C++ library: EnTT is a header-only, tiny and easy to use library for game programming and much more written in modern C++. Among others, it's used in Minecraft by Mojang, ... https://github.com/skypjack/entt https://github.com/skypjack/entt
  - gameman144 5y ago
    EnTT is the one I was thinking of -- I'd highly recommend looking into its implementation, as it's both very well-documented and readable, and blazing fast. The fact that even this caliber of implementation has to handle multi-component cases by doing super careful curation of components is what makes me skeptical that CGS would be able to solve this situation more efficiently/easily. I would be very happy to be proven wrong here, though.

enbugger 5y ago

>Performance-wise, some overhead may come from reference-counting the components Well, that's what ECS gets rid of and mainly benefits from that. Doesn't returning it back mean going back to OOP? What's the point then?

Kinrany 5y ago
I believe the main benefit is a constrained form of dynamic typing. You don't want to lose compile-time type checking and intellisense, and you want to keep most of the performance benefits. But you want to be able to add data types without recompiling.

m0llusk 5y ago

> CGS is more flexible thus targeting a wider use case than generic ECS: it's a hierarchy of components instead of the flat entity-component duality ... Wasn't the flatness of ECS supposed to be a useful feature that simplifies things like iteration and searches?

Kinrany 5y ago

> There is no ~~spoon~~ entity. What was our entity like? A structure with pointers (IDs) to different components. Now, let's treat it as just another component! The whole ECS becomes simpler. To me this is the most interesting part. Game logic needs components with references to other entities and components anyway. And if there's some way of identifying components, you no longer need entities to have identities. And you can still apply all the optimizations as long as you treat some components as "entity" components. But only when you want to exchange simplicity for performance.

brainbag 5y ago

I haven't used it but this sounds similar to my understanding of how Godot's structure works. Is this a similar pattern?

pshirshov 5y ago

Sounds alike to distage?.. https://izumi.7mind.io/distage/ https://izumi.7mind.io/distage/