6 ms·

Entity Component Systems and Data Oriented Design [pdf]

82 points by magoghm 8y ago

platz 8y ago
jonathan blow is not a big fan of ECS https://www.youtube.com/watch?v=4t1K66dMhWk https://www.youtube.com/watch?v=4t1K66dMhWk
- gameswithgo 8y ago
  It isn't appropriate for all games. Which is important to keep in mind. Sometimes people get crazy and do these really complex high performance ECS systems for a 2d game with 10 entities. Or a game with only 2 kinds of entities. Just unnecessary complexity in those cases.
  - platz 8y ago
    I am not a game dev, but from watching some of his stuff, I have a feeling he would invert that claim, and say ECS is fine for smaller games but for really involved systems with time-to-market pressures the framework is too limiting.
    - mcphage 8y ago
      > but for really involved systems with time-to-market pressures the framework is too limiting It was specifically designed and developed for involved systems.
    - jungler 8y ago
      Jon is also known for being hugely opinionated with only his direct personal experience backing him up. His basic thought on ECS is that it's "obvious". And if you have worked on an involved game a few times and tried to examine the performance bottlenecks of the architecture, it really is. There are different flavors of "how to ECS" that make tradeoffs between static compile-time composition(theoretical fastest ECS: an entire scene is fed into a compiler and it emits a custom bespoke memory layout and an API for it) and runtime dynamism(most flexible ECS: dynamic types, reflection, some code to automatically maintain indexes). But the basic principles of working directly with compositions of plain old data are ingrained throughout, and emerge naturally from trying to extend a simple game with simple data structures and an imperative code style, without resorting to OO inheritance(which was tried and discarded because properties inevitably crept upwards into the parent class leading to a memory-hungry "God Object").
      - platz 8y ago
        I don't think I'm claiming that Jon would advocate for OO inheritance. I also think he thinks that SoA instead of AoS is a good thing - no qualms there. I guess he would probably have the most comments on how to deal/access/manipulate that SoA at a higher level
- Impossible 8y ago
  Jonathan Blow's commentary isn't anti-ECS. It's more skeptical of Rust's borrow checker, more specifically he's pointing out how the original talk ends with bypassing the borrow checker and copying a pattern that would have equal safety in C or C++.
  - platz 8y ago
    That is true + there is an offhand quote he makes about ECS in general, but 99% is about the issue you mention
- dfsdfsdfsdfs 8y ago
  couldn't make it through as he says "right?" at the end of every sentence
plopz 8y ago
In ECS I don't understand where you put the collections (scene graphs, quad trees, etc) or what manages them. Does anyone have insight in this?
- kvark 8y ago
  The collections with any connections between elements are not exactly fitting the ECS model, which works best for independent components. When I was thinking about it back in the day, I came up with a Component Graph concept (implemented in [1]), which has the data locality similar to ECS but doesn't have a notion of systems. [1] https://github.com/kvark/froggy https://github.com/kvark/froggy
- osaariki 8y ago
  Like kvark said, they don't really fit the ECS model well. The standard answer would be to encapsulate them into a system (i.e. go outside the ECS). The way I see ECSs is that they are in-memory relational databases that only support equijoins (look up component matching entity ID) and optimize for table scans. The typical choice of an index structure is a contiguous ordered array, making for very fast lookups and scans. If insertion/deletion performance becomes an issue you can add one level of indirection and only keep a list of pointers to components ordered. However, like you indicated, games also need other kinds of queries, so scene graphs and quad trees should be supported as indices on the same level as the arrays and hash tables that are traditional for ECSs. I would love to see an in-memory database with all the features required by games. There are some very different tradeoffs to be made than in typical in-memory databases: - The most important queries are known up-front, since the simulation/rendering loop is fairly defined. - Transactions don't need to be durable. Just need to be able to checkpoint once in a while. - Lots of bulk updates from the simulation. - Query language needs to be well integrated into the application language to avoid overheads.
- meheleventyone 8y ago
  The main thing to realise is that not everything has to go into the ECS. This is a bit more clear with middleware like physics engines where you’re forced to integrate two views of the world. The game view and the physics view and need to translate and update between the two. Similarly with data in an ECS you can separately process spatial relationships either reusing the required data or keeping local copies as an optimisation.
ryanmarsh 8y ago
I really wish there was a video of this talk, I like this deck but I'm missing the color commentary.
- aras_p 8y ago
  There was a video, but since I talked in Lithuanian, I doubt if it would be useful to many people :)
netgusto 8y ago
An ECS for Go, with a TypeScript port: https://github.com/ByteArena/ecs https://github.com/ByteArena/ecs
- lytedev 8y ago
  For the past few days I've been messing with an ECS module for Elixir! My fork is here: https://github.com/lytedev/ecs https://github.com/lytedev/ecs Really simple with Elixir, but the naive implementation has some potential performance issues methinks.
twtw 8y ago
A response to the OOP part of this: https://www.gamedev.net/blogs/entry/2265481-oop-is-dead-long-live-oop/ https://www.gamedev.net/blogs/entry/2265481-oop-is-dead-long... I have no preference here, just posting it as an interesting further reading.
- platz 8y ago
  I agree that ECS is at its most fundamental a relational scheme. Not sure why that is controversial though..
zimablue 8y ago
There are some parallels here with data engineering/science I think, at least on the surface of "array of structs" => "struct of arrays" is another view on "row oriented" vs "column oriented" data. Killing the object hierarchy and moving to a more relational style (indexed dataframes). OO -> Systems, Systems seem like seperated functional programming.
- _pmf_ 8y ago
  I wonder why ECS is completely absent in non-game systems. It seems like a natural way for non-invasive extension.

82 points by magoghm 8y ago

jonathan blow is not a big fan of ECS https://www.youtube.com/watch?v=4t1K66dMhWk https://www.youtube.com/watch?v=4t1K66dMhWk

gameswithgo 8y ago
It isn't appropriate for all games. Which is important to keep in mind. Sometimes people get crazy and do these really complex high performance ECS systems for a 2d game with 10 entities. Or a game with only 2 kinds of entities. Just unnecessary complexity in those cases.
- platz 8y ago
  I am not a game dev, but from watching some of his stuff, I have a feeling he would invert that claim, and say ECS is fine for smaller games but for really involved systems with time-to-market pressures the framework is too limiting.
  - mcphage 8y ago
    > but for really involved systems with time-to-market pressures the framework is too limiting It was specifically designed and developed for involved systems.
  - jungler 8y ago
    Jon is also known for being hugely opinionated with only his direct personal experience backing him up. His basic thought on ECS is that it's "obvious". And if you have worked on an involved game a few times and tried to examine the performance bottlenecks of the architecture, it really is. There are different flavors of "how to ECS" that make tradeoffs between static compile-time composition(theoretical fastest ECS: an entire scene is fed into a compiler and it emits a custom bespoke memory layout and an API for it) and runtime dynamism(most flexible ECS: dynamic types, reflection, some code to automatically maintain indexes). But the basic principles of working directly with compositions of plain old data are ingrained throughout, and emerge naturally from trying to extend a simple game with simple data structures and an imperative code style, without resorting to OO inheritance(which was tried and discarded because properties inevitably crept upwards into the parent class leading to a memory-hungry "God Object").
    - platz 8y ago
      I don't think I'm claiming that Jon would advocate for OO inheritance. I also think he thinks that SoA instead of AoS is a good thing - no qualms there. I guess he would probably have the most comments on how to deal/access/manipulate that SoA at a higher level
Impossible 8y ago
Jonathan Blow's commentary isn't anti-ECS. It's more skeptical of Rust's borrow checker, more specifically he's pointing out how the original talk ends with bypassing the borrow checker and copying a pattern that would have equal safety in C or C++.
- platz 8y ago
  That is true + there is an offhand quote he makes about ECS in general, but 99% is about the issue you mention
dfsdfsdfsdfs 8y ago
couldn't make it through as he says "right?" at the end of every sentence

plopz 8y ago

In ECS I don't understand where you put the collections (scene graphs, quad trees, etc) or what manages them. Does anyone have insight in this?

kvark 8y ago
The collections with any connections between elements are not exactly fitting the ECS model, which works best for independent components. When I was thinking about it back in the day, I came up with a Component Graph concept (implemented in [1]), which has the data locality similar to ECS but doesn't have a notion of systems. [1] https://github.com/kvark/froggy https://github.com/kvark/froggy
osaariki 8y ago
Like kvark said, they don't really fit the ECS model well. The standard answer would be to encapsulate them into a system (i.e. go outside the ECS). The way I see ECSs is that they are in-memory relational databases that only support equijoins (look up component matching entity ID) and optimize for table scans. The typical choice of an index structure is a contiguous ordered array, making for very fast lookups and scans. If insertion/deletion performance becomes an issue you can add one level of indirection and only keep a list of pointers to components ordered. However, like you indicated, games also need other kinds of queries, so scene graphs and quad trees should be supported as indices on the same level as the arrays and hash tables that are traditional for ECSs. I would love to see an in-memory database with all the features required by games. There are some very different tradeoffs to be made than in typical in-memory databases: - The most important queries are known up-front, since the simulation/rendering loop is fairly defined. - Transactions don't need to be durable. Just need to be able to checkpoint once in a while. - Lots of bulk updates from the simulation. - Query language needs to be well integrated into the application language to avoid overheads.
meheleventyone 8y ago
The main thing to realise is that not everything has to go into the ECS. This is a bit more clear with middleware like physics engines where you’re forced to integrate two views of the world. The game view and the physics view and need to translate and update between the two. Similarly with data in an ECS you can separately process spatial relationships either reusing the required data or keeping local copies as an optimisation.

ryanmarsh 8y ago

I really wish there was a video of this talk, I like this deck but I'm missing the color commentary.

aras_p 8y ago
There was a video, but since I talked in Lithuanian, I doubt if it would be useful to many people :)

netgusto 8y ago

An ECS for Go, with a TypeScript port: https://github.com/ByteArena/ecs https://github.com/ByteArena/ecs

lytedev 8y ago
For the past few days I've been messing with an ECS module for Elixir! My fork is here: https://github.com/lytedev/ecs https://github.com/lytedev/ecs Really simple with Elixir, but the naive implementation has some potential performance issues methinks.

twtw 8y ago

A response to the OOP part of this: https://www.gamedev.net/blogs/entry/2265481-oop-is-dead-long-live-oop/ https://www.gamedev.net/blogs/entry/2265481-oop-is-dead-long... I have no preference here, just posting it as an interesting further reading.

platz 8y ago
I agree that ECS is at its most fundamental a relational scheme. Not sure why that is controversial though..

zimablue 8y ago

There are some parallels here with data engineering/science I think, at least on the surface of "array of structs" => "struct of arrays" is another view on "row oriented" vs "column oriented" data. Killing the object hierarchy and moving to a more relational style (indexed dataframes). OO -> Systems, Systems seem like seperated functional programming.

_pmf_ 8y ago
I wonder why ECS is completely absent in non-game systems. It seems like a natural way for non-invasive extension.